Narrator speaks the translation language; characters keep their own accent
Specification change. Two different rules, and they need different
mechanisms.
Narrator: accent = translation language. Its reference was generated by
VoiceDesign from a description that hardcoded '{accent} English', baking an
American-English timbre into the WAV; cloning that into German gives German
phonology from an American voice. Ten per-language references now exist
(narrator_EN.wav ... narrator_ZH.wav), each designed from the narrator's
timbre plus 'Speaking {language} as a native {language} speaker'.
Characters: keep the accent on the speaker page. Their accents were already
encoded in their existing refs - what destroyed them was the language
parameter forced onto every utterance, which made every voice adopt the
target language's prosody. That is why Deal Hunter came out French-native.
Phonology and accent are now separate controls: language carries phonology,
an explicit voice_description carries accent character. Deal Hunter gets
'with a strong Australian accent' in every language.
Measured A/B, French line from Deal Hunter: with and without the Australian
instruction the waveforms differ by 152% of signal amplitude and duration
moves 4.24s -> 4.08s, so the instruction is reaching the model. Whether it is
convincingly Australian is an ear test, not something measurable here.
Re-rendered all 120 clips under the new spec, 0 failures. The previous
render is kept in Lang_test_preSpec for A/B. 24 speakers carry a non-default
accent; the narrator is deliberately excluded from that set.
Suite: 721 passed, 8 pre-existing.
This commit is contained in:
83
specs/29-accent-and-voice-spec.md
Normal file
83
specs/29-accent-and-voice-spec.md
Normal file
@@ -0,0 +1,83 @@
|
||||
# Accent and voice specification for translated books
|
||||
|
||||
Status: specification, agreed 2026-10-13. Implemented by
|
||||
`scripts/generate_narrator_refs.py` + `scripts/render_lang_test.py`.
|
||||
|
||||
## The rule
|
||||
|
||||
Two rules, and they are different:
|
||||
|
||||
1. **The narrator speaks with the accent of the translation language.** An
|
||||
Italian book is narrated by an Italian-sounding narrator. A German book by a
|
||||
German one. Not an American reading Italian.
|
||||
|
||||
2. **Every other character keeps the accent defined on the speaker page.**
|
||||
Deal Hunter stays Australian. Duchess Victoria stays Canadian. Xu Ming stays
|
||||
Hong Kong. This holds in every target language.
|
||||
|
||||
The asymmetry is deliberate. The narrator is a narrator in *each* language and
|
||||
is heard far more than any single character, so a foreign accent on the
|
||||
narrating voice is the thing listeners notice. Characters are defined
|
||||
individually on the speaker page, and that definition is the contract.
|
||||
|
||||
## Why this needed two different mechanisms
|
||||
|
||||
**The narrator** needed a new reference voice per language. Its reference was
|
||||
generated by the VoiceDesign model from a description that hardcoded
|
||||
`"{accent} English"`, which baked an American-English timbre into the WAV.
|
||||
Cloning that into German gives German phonology from an American voice. Passing
|
||||
the target language fixes phonology but not identity.
|
||||
|
||||
So the narrator now has one reference per language, generated by VoiceDesign from
|
||||
the narrator's timbre plus "Speaking {language} as a native {language} speaker".
|
||||
`narrator_EN.wav` … `narrator_ZH.wav`, ten in all, in
|
||||
`<project>/.umbra/voices/`.
|
||||
|
||||
**Characters needed no new references.** Their accents are already encoded in
|
||||
their existing reference WAVs. What was destroying them was the `language`
|
||||
parameter: forced onto every utterance, it made every voice adopt the target
|
||||
language's prosody, which is why Deal Hunter came out French-native.
|
||||
|
||||
Phonology and accent are separate controls, and the fix uses them separately:
|
||||
|
||||
| | language parameter | accent carried by |
|
||||
|---|---|---|
|
||||
| Narrator | target language | its per-language reference |
|
||||
| Characters | target language | existing reference + explicit accent instruction |
|
||||
|
||||
The accent instruction is derived from the speaker page (`accent_for()` in the
|
||||
renderer) and passed as `voice_description` alongside the clone. Measured A/B on
|
||||
one French line from Deal Hunter: with and without her Australian instruction,
|
||||
waveforms differ by 152% of signal amplitude and duration moves 4.24s -> 4.08s.
|
||||
|
||||
## Deal Hunter's current accents
|
||||
|
||||
24 speakers have a non-default accent. The narrator is deliberately *not* in
|
||||
this list — its accent is the translation language.
|
||||
|
||||
australian Deal Hunter (strong) canadian 17 speakers incl.
|
||||
nigerian Adonai Maul, Dealer Kainda, Nanny,
|
||||
indian Alec, Interrogator Duchess Victoria
|
||||
kenyan Warbots hong_kong Trader Wakam, Xu Ming
|
||||
|
||||
Any character whose accent is `none` simply gets no instruction and speaks the
|
||||
target language plainly.
|
||||
|
||||
## What this does not fix
|
||||
|
||||
- **The ICL prompt path is still unusable.** One `prompt_slug` request takes the
|
||||
server from 5.4 GB to 19.1 GB and never returns. The render path uses
|
||||
`x_vector_only_mode=True`, which is stable. Cached prompts are effectively
|
||||
bypassed.
|
||||
- **Accent is verified by measurement, not by ear.** The instruction demonstrably
|
||||
changes the output. Whether "strong Australian" is *convincingly* Australian
|
||||
in French is a listening test for the reviewer, not something measurable here.
|
||||
- **One pass per language.** Languages are rendered sequentially so the system
|
||||
can reset between them; not all fifteen at once.
|
||||
|
||||
## Division of labour with the translation node
|
||||
|
||||
The translation node produces the text and the print paperback, including the
|
||||
full analysis needed for correct translation. This system turns that text into
|
||||
audio and knows nothing about translation. The renderer consumes
|
||||
`(index, speaker, text)` per utterance and emits WAV.
|
||||
Reference in New Issue
Block a user