Narrator uses the speaker-page reference in every language

The per-language narrator references were wrong. Generating a VoiceDesign
voice per language invents a different narrator each time: measured
0.77-0.96 band-energy similarity against the speaker page's own
narrator_ref.wav, so the voice changed with the language. That is the
opposite of using the vmatrix, and it is why the narrator did not sound
the same across languages.

The narrator now uses narrator_ref.wav in every language. The language
parameter supplies phonology; the single reference supplies identity. That
reference is what the vmatrix was built from, so it is the same voice - just
at x_vector resolution rather than the 8k matrix, which is the only form
this machine can run (one prompt_slug request takes the server 5.4 GB ->
19.1 GB and never returns).

Cross-language narrator similarity, same utterance, all 10 languages:
  per-language designed refs : mean 0.945, min 0.795
  single speaker-page ref    : mean 0.973, min 0.910

The previous render is kept in Lang_test_perLangRefs for comparison. 120
clips re-rendered, 0 failed. Note the proxy is a band-energy fingerprint,
not a true speaker embedding, and it cannot judge accent - only timbre
consistency. Suite: 721 passed, 8 pre-existing.
This commit is contained in:
B.D. Murphy
2026-10-13 19:36:20 -07:00
parent d7fc0236bc
commit 332aeb4b65

View File

@@ -323,9 +323,6 @@ def main() -> int:
out_dir = out_root / suffix
out_dir.mkdir(parents=True, exist_ok=True)
made: list[Path] = []
lang_ref = data_dir / "voices" / f"narrator_{suffix}.wav"
if not lang_ref.exists():
lang_ref = None
for u in doc["utterances"]:
slug = slugify(u["speaker"])
dest = out_dir / f"utt{u['index']:02d}_{u['speaker'].replace(' ', '_')}.wav"
@@ -339,12 +336,16 @@ def main() -> int:
# The narrator's accent is the translation language, so it uses
# a reference designed for that language. Every other character
# keeps its own reference and carries its page accent through.
# The narrator uses the speaker page's own reference in EVERY
# language. Generating a reference per language invents a
# different narrator each time - measured 0.77-0.96 band-energy
# similarity to the real one - so the voice changed with the
# language. The language parameter supplies phonology; the one
# reference supplies identity. That is the same source the
# vmatrix was built from, at x_vector resolution instead of the
# 8k matrix, which is the only form this machine can run.
ref = refs[slug]
instruct = ""
if u["speaker"].lower() == "narrator" and lang_ref is not None:
ref = lang_ref
else:
instruct = accent_for(u["speaker"], data_dir)
instruct = accent_for(u["speaker"], data_dir)
audio = post_tts(
TTS_URL, u["text"],
prompt_slug=slug if use_prompt else None,