UmbraAudio/specs/14-two-phase-segmentation.md

# Spec: Two-Phase Segmentation (Pass 2)

## Overview

Pass 2 segments chapter text into utterances (narration and dialogue) and attributes speakers using a two-phase approach: a mechanical segmenter handles text splitting, and an LLM attributes speakers to all dialogue. This produces a 0.05% dialogue-as-narrator error rate (1/1,882 dialogue utterances) across all 35 chapters.

## Architecture

```
Raw chapter text
    │
    ▼
┌─────────────────────────────────────────────────┐
│  PHASE 1: MECHANICAL SEGMENTER                  │
│  mapping/segmenter.py                           │
│                                                 │
│  - Sentence splitting at .!? (not commas)       │
│  - Dialogue detection via quotes ("'""')        │
│  - Tag-based attribution (forward/backward)     │
│  - Scene break (<<<>>>) handling                 │
│  - Chapter title detection                      │
└─────────────────────────────────────────────────┘
    │
    ▼
┌─────────────────────────────────────────────────┐
│  PHASE 2: LLM SPEAKER ATTRIBUTION               │
│  analysis/llm_attributor.py                     │
│                                                 │
│  - Attributes ALL dialogue (overrides guesses)  │
│  - Resolves pronouns using context              │
│  - Uses character profiles from speakers.json   │
│  - Batches of 40 utterances for efficiency      │
│  - Normalizes names (aliases/first-word,        │
│    narrator lowercase, pronoun guard)           │
└─────────────────────────────────────────────────┘
    │
    ▼
┌─────────────────────────────────────────────────┐
│  OUTPUT: Utterances table                       │
│  4,339 utterances across 35 chapters            │
│  0.05% dialogue-as-narrator (1/1,882)           │
└─────────────────────────────────────────────────┘
```

## Components

### 1. Mechanical Segmenter (`mapping/segmenter.py`)

#### Class: `Segmenter`

```python
class Segmenter:
    def segment(
        self,
        text: str,
        chapter_id: int,
        character_names: list[str] | None = None,
    ) -> list[dict[str, Any]]
```

**Returns:**
```python
[{
    "index": int,
    "text": str,
    "type": "narration" | "dialogue",
    "character_name": str | None,
}, ...]
```

**Segmentation Pipeline:**

```
Raw text
    │
    ├── 1. _split_segments(text) → list[str]
    │       Character-by-character parser
    │       Detects quote boundaries and sentence endings
    │
    ├── 2. _build_utterances(segments) → list[dict]
    │       Tags each segment as "dialogue" or "narration"
    │       based on leading quote character
    │
    └── 3. _attribute_speakers(utterances, character_names) → list[dict]
            Forward-tracking speaker attribution
            Regex matching against character names
```

**Quote handling:** a quoted dialogue span is extracted whole and kept as one utterance even
with multiple sentences. `_restore_quote_wrap` re-wraps it with typographic double quotes and
only treats double-quote closes (`"`, `\u201d`) as outer closes — trailing **inner
single-quotes** (`’`) are preserved (e.g. `...named ‘Nanny.’`). Previously the inner text was
split back into sentences and each was re-quoted, which fragmented multi-sentence speech and
mangled inner single-quotes.

**Quote Detection:**
- Open: `"`, `'`, `\u201c` (left double), `\u2018` (left single)
- Close: `"`, `'`, `\u201d` (right double), `\u2019` (right single)
- Matching pairs: `"` → `"`, `\u201c` → `\u201d`, etc.

**Sentence Splitting:**
- Split at `.`, `!`, `?` followed by whitespace/quote/dash
- Minimum 2 words per segment (avoids fragments)
- Quote boundaries always create segment breaks

**Speaker Attribution Algorithm:**

```python
def _attribute_speakers(self, utterances, character_names) -> list[dict]
```

1. **Tag Detection**: Find `NAME said/asked/replied` patterns
2. **Forward Tracking**: When tag found, set `next_speaker = NAME`
3. **Backward Reference**: When no tag, look back for `said NAME` patterns
4. **Name Normalization**: Case-insensitive matching, alias resolution

**Speech Verbs (42 total):**
```
said, asked, replied, whispered, shouted, muttered, called,
exclaimed, answered, began, continued, added, finished, noted,
stated, remarked, yelled, screamed, laughed, sighed, says,
tells, told, orders, ordered, demands, demanded, insists,
insisted, begs, begged, pleads, pleaded, commands, commanded,
snapped, snarled, hissed, growled, barked, roared, croaked,
grunted, murmured, purred, intoned, declared, announced,
proclaimed, responds, repeats, mentions, offers
```

### 2. LLM Speaker Attribution (`analysis/llm_attributor.py`)

#### Class: `LLMAttributor`

```python
class LLMAttributor:
    def __init__(self, config: LLMConfig, model: str | None = None) -> None
    
    def attribute_speakers(
        self,
        utterances: list[dict[str, Any]],
        chapter_title: str,
        chapter_id: int,
    ) -> list[dict[str, Any]]
```

**Key Insight:** The LLM attributes **all** dialogue utterances. The mechanical attribution from the segmenter is preserved only as a `current_speaker` hint (Rule 8) — the LLM's verdict takes precedence so that ambiguous or incorrectly guessed lines can be corrected. This differs from the original design where only `speaker_name == "narrator"` dialogue was attributed.

**Attribution Flow:**

1. **Filter**: Collect all dialogue utterances (type == "dialogue")
2. **Batch**: Group into batches of 40 utterances
3. **Context**: Split into scenes (`<<<>>>`); include per-scene narration names + last-12 narration lines as context
4. **LLM Call**: Send batch with character profiles and scene context to LLM
5. **Parse Response**: Extract JSON array with index → speaker mapping
6. **Post-Process**: Strip speech verb suffixes ("says", "said", etc.); run `_normalize_speaker` (aliases + first-word-of-primary → primary; narrator/the-narrator → lowercase `narrator`; pronouns → `narrator` guard)
7. **Update**: Apply LLM verdicts to all dialogue utterances
8. **Fallback**: Re-attribute any dialogue still un-attributed via speech-tag regex on the 3 preceding utterances, then speaker carry-forward (scene breaks reset the carried speaker)

**LLM Prompt (inline):**
```
You are a dialogue attribution expert for audiobook production.

Given utterances that need speaker attribution, identify who is speaking each line.

## Character Profiles
{character_profiles}

## Chapter Title
{chapter_title}

## Scene Context
This chapter has {scene_count} scene(s) separated by scene breaks (<<<>>>).
You are attributing dialogue for: {scene_label}

{scene_speakers_context}

## Utterances to Attribute
{utterances_json}

## Rules
1. Each utterance is already correctly split — do NOT merge or split them
2. EVERY utterance MUST be attributed to a speaker. Return EXACTLY one entry per utterance index. Do not skip any.
3. ALL these utterances are dialogue (in quotes) — they are spoken by a CHARACTER, almost never "Narrator". Only use "Narrator" if the line is truly un-attributable.
3b. STRONG HINT: when a preceding narration line says a character IS telling/explaining/speaking to someone (e.g. "Nanny is telling the princess about what will happen."), the following quoted lines are that character's dialogue, until another character is indicated.
3c. A multi-line speech block from one character continues until dialogue tags, action, or the responder's lines indicate a change.
3d. A single quoted dialogue utterance may contain MULTIPLE sentences (e.g. one paragraph of speech) — every sentence within it belongs to the SAME speaker. Do not split the speaker across sentences within one utterance.
4. Use dialogue tags (X says/replies/asks) to identify speakers
5. Use context: who is present, who was speaking before, who is addressed
6. Resolve pronouns: "she says" refers to the most recently mentioned female character
7. If a character has aliases, use their EXACT full primary_name shown in the profiles (e.g. "Kainda Walters Foster Murphy", NOT "Kainda" or "Princess Kainda")
8. The "current_speaker" field shows who the mechanical segmenter guessed — use it as a hint but correct if wrong
9. In a conversation, speakers usually alternate. If one line is Character A, the next line is likely a different character (often the one addressed).
10. CRITICAL — only return the EXACT utterance indices provided in the input. Do NOT invent or hallucinate extra indices.
11. CRITICAL — every returned speaker must be the ACTUAL character speaking. Assigning the wrong speaker ruins the audiobook.
12. AI-1 is a specific AI character voice. Only attribute dialogue to AI-1 if the text explicitly indicates AI-1 is speaking. Do NOT default undetermined utterances to AI-1.

## Scene Attribution Rules (CRITICAL)
- Chapters can contain multiple scenes separated by scene breaks (<<<>>>)
- Each scene is an independent unit. Speakers identified in one scene may NOT appear in other scenes
- Within a scene, once a speaker is identified, subsequent dialogue by the same person continues until another speaker is identified
- "captain" in one scene may refer to a different person than "captain" in another scene — there can be multiple characters with the same title
- Only attribute a speaker to dialogue if there is evidence within THAT SCENE that they are speaking
- If a scene introduces new characters via narration, those characters are candidates for dialogue in that scene
- A speaker who appears in scene 1 may be entirely absent from scene 2 — do NOT carry speakers across scene boundaries
- When ambiguous ("he says", "she says"), resolve using the most recent speaker mention within the SAME SCENE

## Output Format
Return ONLY a JSON array of objects, one per utterance, each with:
- "index": the utterance index (number)
- "speaker": the correct speaker name (use primary_name from profiles, MUST be a character name, NOT "Narrator")

Return ONLY the JSON array, no other text.
```

**Performance:**
- ~5 seconds per batch of 40 utterances with `google/gemma-4-12b-qat` (~34 tok/s on the remote LM Studio server)
- The model must be loaded with `context_length: 8192` (default 262k-token context requests ~44.9 GB of memory and blocks loading)
- ~20-30 minutes for all 35 chapters; every dialogue utterance is processed (not just unattributed ones)

### 3. Scene Break Handling

**Scene Breaks (`<<<>>>`):**
- 51 scene breaks across the book
- Marked as special utterances: `{"speaker": "scene_break", "text": "<<<>>>"}`
- Within a scene: track which characters are present
- At a scene break: re-evaluate present characters
- Chapter context carries across scene breaks

**LLM Context:**
- Full chapter text with `<<<>>>` preserved
- LLM maintains context across scene breaks
- Can alter character presence between scenes

### 4. Sentence Splitting Rules

**Correct Behavior:**
- Split at sentence endings: `.` `!` `?` (narration only)
- Do NOT split at commas
- Narration: each sentence = separate utterance
- Quoted dialogue: the whole quoted span = ONE utterance, even when it contains multiple sentences (all sentences share the same speaker — Rule 3d)

**Examples:**
```
"I'm on a yacht." → ["I'm on a yacht."]
"That is not something..." → ["That is not something..."]
"Help! Open the latch." → ["Help! Open the latch."]  # one utterance, one speaker
```

### 5. Speaker Name Resolution

Every LLM-returned speaker passes through `_normalize_speaker` before being stored:

**Aliases → Primary Names:**
- "Princess", "Kainda", "the princess" → "Kainda Walters Foster Murphy"
- "Captain", "Captain Wake" → "Willamina Wake"
- "Hunter", "Ship AI", "The Deal Hunter" → "Deal Hunter" (the ship's AI — one speaker)

**First-word-of-primary mapping:** the first word of each primary name maps to the full
primary name when it is not already an alias (e.g. "kainda" → "Kainda Walters Foster Murphy").

**Narrator lowercase:** "narrator" / "the narrator" → canonical lowercase `narrator` (the DB
and generation code check this exact case).

**Pronoun guard:** pronouns ("she", "he", "they", "her", etc.) → `narrator`; a bare pronoun
is an LLM failure, never a character.

**LLM Post-Processing:**
- Strip speech verb suffixes: "Kainda says" → "Kainda"
- Handle group speakers: "Management Group", "Guard Detail Members"
- Handle AI speakers: "Deal Hunter" (alias "Hunter"/"Ship AI"), "AI-1"
- Unknown names are kept as-is and surfaced on the orphaned-utterances page for review

## Output

Utterances are saved to the `utterances` table in `project.db`:
```sql
INSERT INTO utterances (
    chapter_id,
    utterance_index,
    text,
    type,  -- "narration" | "dialogue"
    speaker_name  -- "narrator" | "Princess Kainda" | "scene_break" | etc.
) VALUES (?, ?, ?, ?, ?)
```

## Quality Metrics

| Metric | Value |
|--------|-------|
| Total utterances | 4,339 |
| Dialogue utterances | 1,882 |
| Narration utterances | 2,453 |
| Dialogue-as-narrator errors | 1 (0.05%) |
| Scene breaks | 51 |
| Chapters processed | 35 |

## Error Analysis

**Remaining 0.05% Error:**
- 1 dialogue utterance still attributed to narrator (ch35 "About the author" back matter — expected, no speaker is present)
- Edge cases where the LLM and mechanical segmenter both fail
- Acceptable for production use

**Common Failure Modes:**
1. Missing dialogue tags with no context
2. Multiple speakers in rapid succession
3. Internal thoughts (should be narrator, sometimes misattributed)
4. Indirect speech ("she said that...") vs direct speech
5. Bare pronoun speakers ("She") — caught by the pronoun guard, which maps them to `narrator`

## Validation

Run after any re-segment or re-attribute to confirm the quality gates hold:

```sql
-- Dialogue left as narrator (expect only the ch35 back-matter exception)
SELECT chapter_id, COUNT(*) FROM utterances
WHERE type = 'dialogue' AND lower(speaker_name) = 'narrator'
GROUP BY chapter_id;

-- Pronoun speakers (must be 0 — guard maps these to narrator)
SELECT COUNT(*) FROM utterances
WHERE type = 'dialogue' AND lower(speaker_name) IN
  ('i','me','my','you','your','he','him','his','she','her',
   'they','them','their','we','us','our','it','its','one');

-- Narrator case (must be lowercase 'narrator', never 'Narrator')
SELECT COUNT(*) FROM utterances WHERE speaker_name = 'Narrator';

-- Unknown speakers (cross-check against speakers.json aliases/first-words;
-- leftovers appear on the /orphaned-utterances page)
SELECT DISTINCT speaker_name FROM utterances
WHERE type = 'dialogue' AND lower(speaker_name) != 'narrator';
```

Additional checks:
- **ch13 golden reference**: 44 utterances — 15 Kainda / 15 Nanny / 14 narrator (verified against the manuscript, e.g. "Fifty light years" is Kainda, "Don't help Princess Kainda" is Nanny)
- **Multi-sentence quotes**: spot-check that a paragraph of quoted speech is ONE dialogue utterance with a single speaker (Rule 3d)
- **Alternation**: dialogue should alternate between the characters present; a long run of one speaker with no tag is suspicious

## Scripts

### resegment_all.py

```bash
# Run on all chapters with LLM attribution
python scripts/resegment_all.py --model google/gemma-4-12b-qat

# Run on specific chapters
python scripts/resegment_all.py --chapters 5,6,7 --model google/gemma-4-12b-qat

# Mechanical-only (no LLM)
python scripts/resegment_all.py --mechanical-only
```

### reattribute_all.py

Re-runs LLM speaker attribution on existing utterances **without re-segmenting** (preserves utterance IDs and mechanical segmentation; only updates `speaker_name`):

```bash
# All chapters (--sleep paces requests to the remote server)
python scripts/reattribute_all.py --model google/gemma-4-12b-qat --sleep 0.2

# Specific chapters
python scripts/reattribute_all.py --chapters 5,6,7 --model google/gemma-4-12b-qat
```

**Note:** before running attribution, ensure the remote LM Studio server has the model loaded with `context_length: 8192`:

```bash
curl -X POST http://192.168.123.14:1234/api/v1/models/load \
  -H 'Content-Type: application/json' \
  -d '{"model": "google/gemma-4-12b-qat", "context_length": 8192, "flash_attention": true, "echo_load_config": true}'
```

## Key Design Decisions

1. **Two-phase over one-phase**: Mechanical segmenter handles text splitting reliably; LLM only does attribution
2. **LLM attributes all dialogue**: The LLM's verdict overrides the mechanical `current_speaker` guess so ambiguous lines get corrected (not just the unattributed ones)
3. **Quotes kept whole**: Multi-sentence quoted speech is one utterance; splitting it loses the single-speaker evidence
4. **Inner-quote preservation**: `_restore_quote_wrap` only recognizes double-quote closes so trailing inner single-quotes (`’`) survive re-wrapping
5. **Name normalization**: Aliases and first-word-of-primary map to canonical primary names; narrator stays lowercase; pronouns map to narrator (no fabricated speakers)
6. **Deterministic fallback**: Speech-tag regex + speaker carry-forward guarantee un-attributed dialogue still gets a speaker
7. **Batch processing**: 40 utterances per LLM call for efficiency
8. **Scene break preservation**: `<<<>>>` markers maintained as special utterances; they reset the carried speaker
9. **google/gemma-4-12b-qat model**: ~34 tok/s on the remote LM Studio server, must be loaded with `context_length: 8192` (~44.9 GB context otherwise blocks loading)