UmbraAudio/specs/13-character-discovery.md

Spec: Analysis — LLM Character Discovery (Pass 1b)

Overview

Pass 1b extracts characters from chapter text using a 3-pass LLM analysis pipeline, builds character profiles with vocal traits and communication styles, validates alias relationships using text co-occurrence evidence, and assigns unique voices from the Edge-TTS voice library. The narrator is handled as a special case — not a character, but a dedicated voice.

Architecture: 3-Pass Discovery Pipeline

Chapters (source_text)
    │
    ▼
┌─────────────────────────────────────────────────┐
│  PASS 1: CHAPTER-BY-CHAPTER DISCOVERY           │
│  LLM identifies characters, aliases, traits     │
│  per chapter using CHARACTER_DISCOVERY_PROMPT    │
│                                                 │
│  - Known speakers passed forward                │
│  - Scene awareness (scene_index output)         │
│  - Alias evidence required                      │
│  - Unidentified speakers flagged                │
│                                                 │
│  Output: .umbra/discovery/discovery_pass1.json  │
└─────────────────────────────────────────────────┘
    │
    ▼
┌─────────────────────────────────────────────────┐
│  PASS 2: ALIAS VALIDATION                       │
│  Validates every claimed alias pair using        │
│  text co-occurrence evidence                    │
│                                                 │
│  - Scene-level co-occurrence check              │
│  - Two proper names in same scene = not alias   │
│  - Invalid aliases removed                      │
│  - Re-merges with cleaned alias lists           │
│                                                 │
│  Output: .umbra/discovery/discovery_pass2.json  │
└─────────────────────────────────────────────────┘
    │
    ▼
┌─────────────────────────────────────────────────┐
│  PASS 3: CROSS-CHAPTER RECONCILIATION           │
│  Final validation across all chapters           │
│                                                 │
│  - Flags suspicious merges                      │
│  - Lists unresolved unidentified speakers       │
│  - Emits validation log                         │
│                                                 │
│  Output: .umbra/discovery/discovery_pass3.json  │
└─────────────────────────────────────────────────┘
    │
    ▼
┌─────────────────────────────────────────────────┐
│  FINAL: speakers.json                           │
│  Merged, validated speaker list                 │
│  Narrator auto-added if missing                 │
└─────────────────────────────────────────────────┘

Components

1. Character Discovery Engine (analysis/character_discovery.py)

Class: CharacterDiscoverer

class CharacterDiscoverer:
    def __init__(self, llm_config: LLMConfig) -> None

    async def discover_speakers(
        self,
        chapters: list[dict[str, Any]],
        book_title: str = "",
        max_chars: int = 40000,
    ) -> DiscoveryResult

API Contract (preserved from prior version):

• Input: chapters list of dicts with source_text, title, chapter_index keys
• Output: DiscoveryResult with .speakers, .save(), .load() methods
• Callers: CLI (cli.py), Web routes (web/routes.py)

Pass 1: Chapter-by-Chapter Discovery

1. For each chapter, send text to LLM with CHARACTER_DISCOVERY_PROMPT.md
2. Previously discovered speakers passed as {known_speakers_section} to prevent duplicates
3. LLM output includes: speaking_characters, non_speaking_characters, validation_notes
4. Each character includes: scenes_appeared, unidentified, unidentified_reason, alias_evidence
5. Results accumulated into ChapterSpeakers list
6. After all chapters: merge across chapters via _merge_chapter_results()
7. Save to .umbra/discovery/discovery_pass1.json

Pass 2: Alias Validation

For each chapter's discovered characters:

1. Parse chapter text into scenes (by <<<>>> breaks or paragraph grouping)
2. For each claimed alias pair (primary_name, alias):
   • Proper Name Test: If both have distinct proper names and one is not contained in the other → invalid
   • Co-occurrence Test: If both names appear as speakers in the same scene → invalid
3. Remove invalid aliases from character data
4. Re-merge across chapters with cleaned alias lists
5. Save to .umbra/discovery/discovery_pass2.json

Pass 3: Cross-Chapter Reconciliation

1. Flag any speakers still marked as unidentified
2. Scan all speakers for suspicious merges (two distinct proper names in primary_name + aliases)
3. Emit validation log with all corrections and warnings
4. Save to .umbra/discovery/discovery_pass3.json

2. LLM Prompt (CHARACTER_DISCOVERY_PROMPT.md)

Key rules in the prompt:

• Rule 1 (Scene Awareness): Track which characters appear in which scenes
• Rule 5 (Alias Validation): Hard rules for valid vs invalid aliases:
  • VALID: Title + name (e.g., "Princess" = "Kainda")
  • INVALID: Two distinct proper names (e.g., "Kainda" ≠ "Captain Willamina Wake")
  • INVALID: Shared title only (e.g., "Captain" ≠ "Princess")
  • INVALID: Role description (e.g., "the guard" ≠ "John")
• Rule 6 (Unidentified Speaker Flagging): Flag speakers not introduced in the scene
• Rule 7 (Cross-Chapter Identity): Use known speakers list, create new entry when in doubt

Output schema additions:

• scenes_appeared: List of scene indices (1-based) where character appears
• unidentified: Boolean flag for unidentified speakers
• unidentified_reason: Explanation of why the speaker is unidentified
• alias_evidence: Textual evidence confirming each alias is valid
• validation_notes: Any concerns about the chapter's analysis accuracy

3. Scene Parsing (_parse_scenes)

Splits chapter text into scenes for co-occurrence validation:

1. Primary: Split on <<<>>> scene break markers
2. Fallback: Split on paragraph breaks (\n\s*\n) with 3000-char grouping
3. Always returns at least one scene (the full chapter text)

4. Alias Validation (validate_aliases)

def validate_aliases(
    self,
    chapter_text: str,
    chapter_title: str,
    characters: list[dict[str, Any]],
) -> tuple[list[dict[str, Any]], list[str]]

Returns:

• Cleaned character list with invalid aliases removed
• List of validation notes describing each correction

Co-occurrence detection:

1. Build regex from all known character names (longest-first matching)
2. For each scene, find which names appear in the text
3. If primary_name and alias both appear in the same scene as independent matches → invalid alias

5. Character Merge Algorithm

def _merge_chapter_results(
    self,
    chapter_results: list[ChapterSpeakers],
    book_title: str,
    total_words: int,
) -> list[Speaker]

Algorithm:

1. Collect: Gather all characters from all chapters
2. Dedup by Primary Name: Case-insensitive matching
3. Merge Aliases: Combine alias lists from different chapters
4. Merge Alias Evidence: Combine evidence from all chapters
5. Preserve Demographics: Keep first non-unknown value for each field
6. Merge Vocal Traits: Union of all vocal traits across chapters
7. Merge Communication Styles: Keep most detailed description
8. Context-Aware Dedup: _should_merge_by_context prevents merging speakers with distinct proper names
9. Sort by Frequency: Characters appearing in more chapters first

6. Speaker Data Model (Speaker dataclass)

@dataclass
class Speaker:
    primary_name: str
    aliases: list[str]
    demographics: Demographics
    vocal_traits: list[str]
    communication_style: CommunicationStyle
    is_narrator: bool
    is_group: bool
    description: str
    sample_dialogue: str
    chapters_appeared: list[int]
    unidentified: bool              # NEW: flagged if speaker not introduced
    unidentified_reason: str        # NEW: explanation of why
    alias_evidence: list[str]       # NEW: textual evidence for aliases

7. Temporary Storage Files

File	Contents
.umbra/discovery/discovery_pass1.json	Raw LLM discovery results, merged
.umbra/discovery/discovery_pass2.json	After alias validation
.umbra/discovery/discovery_pass3.json	After cross-chapter reconciliation
.umbra/speakers.json	Final output (overwritten on each run)

8. Narrator Handling

The narrator is not a character — it's a dedicated voice assignment:

• Voice: en-US-GuyNeural:-10 (soft-spoken male, deep radio speaker style)
• Auto-added by DiscoveryResult.save() if no is_narrator=True speaker exists
• Not included in character discovery
• Utterances with speaker_name = "narrator" use this voice

9. Voice Assignment (scripts/assign_voices.py)

def assign_voices(
    speakers: list[dict],
    voice_library: list[str],
) -> dict[str, str]

Voice Library:

• 27 distinct Edge-TTS base voices (US, UK, AU, IE, IN, CA)
• Gender-balanced: 14 male, 13 female
• Pitch shifting for additional variety (+48 to -33 semitones)

Assignment Rules:

1. Narrator: Fixed voice en-US-GuyNeural:-10
2. Characters: Assign from pool based on gender and vocal traits
3. Groups: Assign distinct voices (e.g., "Management Group" → en-US-AriaNeural)
4. Verification: Each voice tested with the Rainbow Passage (phonetically balanced paragraph covering all English phonemes).

10. Quality Gates

Character Discovery Quality:

• All speaking characters must have aliases documented
• Aliases must have alias_evidence entries confirming validity
• Unidentified speakers must have unidentified_reason filled
• Vocal traits must be specific (not "normal" or "average")
• Communication style must include register and any shifts
• No two co-occurring characters can share the same primary_name or alias

Alias Validation Quality:

• No claimed alias pair may contain two distinct proper names
• No claimed alias pair may co-occur as independent speakers in the same scene
• All alias removals logged in validation_log

Voice Assignment Quality:

• No two characters sharing dialogue proximity can have identical voices
• Verification samples must be generated for all assigned voices
• Web UI allows human review and reassignment

Database Updates

Stage	Table	Fields Set
Characters extracted	speakers.json	Full character profiles with alias_evidence
Voices assigned	voice_assignments.json	Character → voice mappings
Samples generated	.umbra/voice_tests/	WAV files for verification

Key Design Decisions

1. 3-pass pipeline over single-pass: Discovery → Validation → Reconciliation catches alias errors that single-pass cannot
2. Text co-occurrence validation: Mechanical check that "Kainda" and "Captain Willamina Wake" can't be aliases if both appear in the same scene
3. Unidentified speaker flagging: Explicitly marks speakers that appear without introduction, rather than silently misattributing
4. Alias evidence requirement: LLM must cite textual evidence for each alias claim, enabling audit
5. Context-aware dedup: _should_merge_by_context prevents merging speakers with distinct proper names even after case-insensitive match
6. Temporary JSON between passes: Enables debugging, re-runs, and manual inspection of each pipeline stage
7. Narrator is not a character: Dedicated voice, never participates in character discovery
8. Voice variety: 59 unique voices from 27 base voices via pitch shifting
9. Human review: All voice assignments go through web UI approval
10. Verification phrase: Standard test sentence ensures voice quality