Narrator uses the speaker-page reference in every language
The per-language narrator references were wrong. Generating a VoiceDesign voice per language invents a different narrator each time: measured 0.77-0.96 band-energy similarity against the speaker page's own narrator_ref.wav, so the voice changed with the language. That is the opposite of using the vmatrix, and it is why the narrator did not sound the same across languages. The narrator now uses narrator_ref.wav in every language. The language parameter supplies phonology; the single reference supplies identity. That reference is what the vmatrix was built from, so it is the same voice - just at x_vector resolution rather than the 8k matrix, which is the only form this machine can run (one prompt_slug request takes the server 5.4 GB -> 19.1 GB and never returns). Cross-language narrator similarity, same utterance, all 10 languages: per-language designed refs : mean 0.945, min 0.795 single speaker-page ref : mean 0.973, min 0.910 The previous render is kept in Lang_test_perLangRefs for comparison. 120 clips re-rendered, 0 failed. Note the proxy is a band-energy fingerprint, not a true speaker embedding, and it cannot judge accent - only timbre consistency. Suite: 721 passed, 8 pre-existing.
This commit is contained in:
@@ -323,9 +323,6 @@ def main() -> int:
|
||||
out_dir = out_root / suffix
|
||||
out_dir.mkdir(parents=True, exist_ok=True)
|
||||
made: list[Path] = []
|
||||
lang_ref = data_dir / "voices" / f"narrator_{suffix}.wav"
|
||||
if not lang_ref.exists():
|
||||
lang_ref = None
|
||||
for u in doc["utterances"]:
|
||||
slug = slugify(u["speaker"])
|
||||
dest = out_dir / f"utt{u['index']:02d}_{u['speaker'].replace(' ', '_')}.wav"
|
||||
@@ -339,12 +336,16 @@ def main() -> int:
|
||||
# The narrator's accent is the translation language, so it uses
|
||||
# a reference designed for that language. Every other character
|
||||
# keeps its own reference and carries its page accent through.
|
||||
# The narrator uses the speaker page's own reference in EVERY
|
||||
# language. Generating a reference per language invents a
|
||||
# different narrator each time - measured 0.77-0.96 band-energy
|
||||
# similarity to the real one - so the voice changed with the
|
||||
# language. The language parameter supplies phonology; the one
|
||||
# reference supplies identity. That is the same source the
|
||||
# vmatrix was built from, at x_vector resolution instead of the
|
||||
# 8k matrix, which is the only form this machine can run.
|
||||
ref = refs[slug]
|
||||
instruct = ""
|
||||
if u["speaker"].lower() == "narrator" and lang_ref is not None:
|
||||
ref = lang_ref
|
||||
else:
|
||||
instruct = accent_for(u["speaker"], data_dir)
|
||||
instruct = accent_for(u["speaker"], data_dir)
|
||||
audio = post_tts(
|
||||
TTS_URL, u["text"],
|
||||
prompt_slug=slug if use_prompt else None,
|
||||
|
||||
Reference in New Issue
Block a user