UmbraAudio/specs/23-speaker-duplicate-detection.md

Spec 23 — Duplicate Speaker Detection and Approval-Safe Merging

• Status: implemented (scanner + merge wrapper + tests). Nothing applied to Last Dwarf yet — the scan is a report, and the merge is a deliberate act.
• Date: 2026-10-03
• Trigger: operator review of Last Dwarf, which has accumulated duplicate speakers from successive scans. "The last scan created several new speakers. I have created and approved voices for several characters. I don't want to lose those voices."
• Companions: specs/17 (speaker confidence, designations), specs/22 (releases).

---

1. The two requirements, and how each is enforced

requirement	how it is enforced
Do not lose an approved voice.	Survivor selection prefers an approved voice and a voice reference above everything else. merge_group() carries an approval across to the survivor instead of dropping it. And a group with two different voice references is blocked — never merged, because merging would silently discard one generated voice.
Better scan: eliminate duplicates among non-minor characters.	A silent twin is the evidence. A designation invented by one scan and re-invented by another exists only as a name with zero lines; if the name also matches a speaker who is heard, they are one person. That is the operator's single-chapter rule, made mechanical.

2. What the matching actually knows

String similarity is not enough. manifest.md has no opinion about who "the tallest" is, but the scan can encode what a human knows:

signal	meaning	example
determiner / numeral prefix stripped	same name, different lead-in	The Sergeant = Sergeant, One Man = The Man
ordinals normalized to digits	same enumeration, two styles	The first guard = Guard 1
superlatives stemmed	emphasis varies, identity does not	The biggest one = The Big One
activity modifiers	same person, described by what they are doing	The Girl = The Girl helping (0.95)
distinguishing modifiers — a VETO	different members of one crowd	The Oldest ≠ The Tallest, Guard 1 ≠ Guard 2
role nouns — a VETO	different roles that share a word	Soldier (Rider) ≠ One Rider
proper names need ratio ≥ 0.88	real names must be genuinely close	Anfisa ≠ Tanisha (0.77); Rielan ≈ Rieland (0.93)

The vetoes are the point. Two rules that were written and then removed are recorded here so nobody re-adds them:

• A shared token plus a narrower role noun rule proposed The Oldest = The Old Woman because both contain "old". They are different characters. Removed.
• A single shared token rule proposed Scum leader = The Leader. Removed; a shared word is not an identity.

A missed duplicate costs one manual merge. A wrong one silently rewrites a speaker. The scan is tuned to find fewer, not more.

3. Verdicts

verdict	meaning	action
auto	every member is a strong variant of the survivor, and the other members have zero lines	safe to apply unattended
review	plausible, but two people were actually heard, or the weakest member is only suggestive	a human decides
blocked	two members have approved voices, or two different voice references	never automatic — a human must choose which voice survives

4. Last Dwarf scan, 2026-10-03

96 facts (93 profiles + 3 spoken-only names), 62 with lines, 13 approved voices.

AUTO — 10 groups, apply-ready

survivor	duplicates	why
Scum country boy	scum country boy (7 utt)	case-only
Scum leader	scum leader (10 utt)	case-only
Guard 2	The second guard (3 utt)	determiner
Lieutenant	One Lieutenant	determiner
The Man	One Man (14 utt)	determiner
Soldier	One Soldier, The Soldier (20 utt)	determiner
Sergeant	The Sergeant (14 utt)	determiner
Walking oldest	The Oldest	activity modifier
Walking Tallest	The Tallest	activity modifier
Walking Youngest	The Youngest	activity modifier

REVIEW — 2 groups, your call

group	question
Guard 1 ← The first guard	both have lines (6 and 4). One chapter. Your single-chapter rule says probably the same.
The Big One ← The Biggest One	both silent; nothing to move, so merging is cosmetic.

BLOCKED — 4 groups, conflicting voices

group	conflict	the decision
Narrator ← Narratorold, OrigNarrator, narrator	three different ref WAVs; Narrator is approved	three stale narrator profiles, and 3 dialogue rows still labelled lowercase narrator
The Girl helping ← The Girl, One Girl Helping	two different ref WAVs	your example. Same person, two generated voices. Which voice survives?
The Officer ← Officer	two different ref WAVs	The Officer has 24 utt / 4 ch; Officer has 5 / 2. Probably one person.
Border Soldier ← Soldier (Rider)	two different ref WAVs; Border Soldier is approved	probably two different riders

5. Two real bugs the verification found

Run against a copy of Last Dwarf, not the live project. Both would have been invisible in a code review.

1. An approval would have vanished. orphan_resolution.merge_speakers folds aliases and re-points utterances but never touches voice_status.json. Merging an approved speaker into a target that had no ledger entry would have deleted the approval record silently. merge_group() now copies the approval to the survivor, removes the loser's entry, and reports approvals_carried.

2. 5,981 rows moved instead of 3. narrator is the labelling convention for ~5,978 narration rows as well as the 3 dialogue rows. The first re-point was unscoped, so merging narrator into Narrator would have renamed every narration row — invalidating thousands of clips, because every staleness marker records the speaker exactly. The re-point is now lower(type)='dialogue' only.

   Worth flagging separately: Last Dwarf uses lowercase narrator for 5,978 narration rows and Narrator for 9. That inconsistency is real and specs/16 §5a says scene breaks must be Narrator. Harmonising it is a deliberate decision with a large regeneration cost — not something a duplicate scan should do on its own.

6. Verified outcome on a copy

BEFORE  profiles=93  refs=72  approved=13
AFTER   profiles=84  refs=72  approved=13
utterances re-pointed: 23        (was 6001 before the dialogue-only fix)
type counts unchanged: True     dialogue 3918 / narration 6016 / scene_break 81
VOICE REF FILES  72 -> 72, removed 0
APPROVALS lost: 0

Nine profiles removed, no voice file touched, no approval lost, narration untouched.

7. How to use it

# Report only (default). Nothing is written.
python scripts/scan_speaker_duplicates.py --project "Last Dwarf"
python scripts/scan_speaker_duplicates.py --project "Last Dwarf" --verdict auto
python scripts/scan_speaker_duplicates.py --project "Last Dwarf" --json

# Apply one group (explicit, by name).
python scripts/scan_speaker_duplicates.py --project "Last Dwarf" \
  --apply "Sergeant <- The Sergeant"

# Re-point case-variant spoken names at their profile.
python scripts/scan_speaker_duplicates.py --project "Last Dwarf" --apply-unresolved

8. Consequences of applying, stated plainly

• The 23 re-pointed utterances become speaker-stale; their clips regenerate on the next chapter build. Nothing else regenerates.
• Merged profiles keep their voice references on disk. The loser's file is not deleted — the next voice-design pass for that book should decide whether to prune orphans.
• A merged speaker's chapters_appeared widens, and the loser's name becomes an alias, so the merge is reversible by editing one JSON file.

9. Applied 2026-10-03

The operator confirmed Narratorold and OrigNarrator are dead profiles. Applied with the new --force flag, which exists precisely for a decision a scan cannot make — it prints the overridden conflict before acting and never deletes a reference file:

FORCED: overriding conflicting voice references: Narratorold=7e8a91ac2ff2936e,
        OrigNarrator=27572e3f16e1cfc7
        those reference FILES are left untouched on disk.
  merged profile 'Narratorold' into 'Narrator'   (0 utt re-pointed)
  merged profile 'OrigNarrator' into 'Narrator'  (0 utt re-pointed)
  re-pointed spoken-only 'narrator' -> 'Narrator'  (3 dialogue utt)

	before	after
profiles	93	91
approved voices	13	13
voice ref files	72	72
dialogue rows labelled lowercase narrator	3	0
narration rows labelled lowercase narrator	5978	5978 (untouched)

The two dead names survive as aliases of Narrator (narrator, the narrator, Narratorold, OrigNarrator), so no stray utterance can resurrect a profile. Their two voice reference WAVs are left on disk — deleting a generated binary was not asked for.

The 3 re-pointed rows are a real attribution fix, not cosmetic: ch5 utt26, ch27 utt125 and ch42 utt290 were dialogue attributed to narrator. ch5 utt26 and ch27 utt125 read as narration mis-segmented as dialogue; ch42 utt290 is a first-person line that may be intended inner thought. All three now resolve to the approved Narrator.

Rescan after the change: 10 auto, 2 review, 3 blocked, 2 unresolved names (down from 3).

10. Open, for the operator

1. The Girl. The Girl and The Girl helping are the same person with two generated voices. Which ref survives — or regenerate one voice for both?
2. Officer. Same question, 24/4 against 5/2.
3. Guard 1. Merge The first guard in, or are they two different guards?
4. The Big One / The Biggest One. Merge cosmetically, or leave both?

Not duplicates, but worth knowing

Two voice reference files have no matching profile: narrator2_ref.wav and rieland_ref.wav. Rieland looks like a misspelling of the profile Rielan — a voice generated under a typo'd name and never wired to a speaker. Not a merge case; a prune-or-adopt decision. manifest.md already records orphan ref files as a known condition.