Let the TTS server be told what language to speak

Every generation branch hardcoded language="english", and the endpoint
accepted no language parameter at all. German, French and the rest were
synthesised with English phonology and prosody - which is why the German
samples sounded like an English narrator reading German.

- /v1/tts now accepts a language form field, passed to all four branches.
  Default stays "english" so the existing English pipeline is unchanged;
  the renderer sends the target language from the model's own
  codec_language_id keys (exactly ten: chinese english french german
  italian japanese korean portuguese russian spanish).
- A/B on one German line with the same cloned voice: language=english
  gives 4.56s, language=german 4.08s, waveforms differing by 151% of
  signal amplitude. Audibly different renditions, not a no-op.

Also documented the ICL memory finding in the server comments: the branch
that consumes a cached prompt leaves x_vector_only_mode at its default of
False, so it decodes the 8k ref_code matrix on every request. One such
request takes the footprint from 5.4GB to 19.1GB and does not return. The
ref_audio branch passes x_vector_only_mode=True and is stable - 120 clips,
0 failures, 13.1GB.

Note the language fix does NOT address that; it was tested and still
blows up with language=german. Separate problem.

qwen_tts_server.py stays at its 8 pre-existing ruff errors. Suite: 721
passed, 8 pre-existing.
This commit is contained in:
B.D. Murphy
2026-10-13 05:26:48 -07:00
parent 948034c365
commit f381c58160
2 changed files with 22 additions and 4 deletions

View File

@@ -403,6 +403,12 @@ async def set_attn(request: AttnRequest):
async def generate_tts(
text: str = Form(...),
sample_rate: int = Form(24000),
# Was hardcoded to "english" in every branch, so German and French text was
# synthesised with English phonology and prosody - an English-accented
# foreign voice. Defaults to "english" so the existing English pipeline is
# byte-for-byte unchanged; multilingual callers pass their own language.
# "Auto" is also accepted and lets the model detect it.
language: str = Form("english"),
voice_description: str | None = Form(None),
ref_audio: UploadFile | None = File(None),
ref_text: str | None = Form(None),
@@ -455,7 +461,7 @@ async def generate_tts(
wavs, sr = model.generate_voice_clone(
text=text,
voice_clone_prompt=items,
language="english",
language=language,
**kw,
)
return wavs, sr
@@ -480,7 +486,7 @@ async def generate_tts(
"x_vector_only_mode": [True],
"icl_mode": [False],
},
language="english",
language=language,
**gen_kwargs,
)
return wavs, sr
@@ -512,7 +518,7 @@ async def generate_tts(
ref_audio=(ref_audio_array, sample_rate),
ref_text=None,
x_vector_only_mode=True,
language="english",
language=language,
**kw,
)
return wavs, sr
@@ -524,7 +530,7 @@ async def generate_tts(
wavs, sr = model.generate_voice_design(
text=text,
instruct=voice_description,
language="english",
language=language,
**gen_kwargs,
)
return wavs, sr

View File

@@ -47,6 +47,13 @@ from umbraaudio.config import load_config # noqa: E402
TTS_URL = "http://127.0.0.1:8881"
# Qwen3-TTS language names, from the model's own codec_language_id keys.
LANG_CODES = {
"EN": "english", "ZH": "chinese", "JA": "japanese", "KO": "korean",
"DE": "german", "ES": "spanish", "FR": "french", "RU": "russian",
"IT": "italian", "PT": "portuguese",
}
# Chapter 1 of Deal Hunter is the voice-introduction sample.
CHAPTER_ID = 1
TRANSLATION_DIRNAME = "Lang_test"
@@ -103,6 +110,7 @@ def post_tts(
ref_audio: Path | None,
ref_text: str | None,
timeout: int,
language: str = "english",
) -> bytes:
"""POST multipart by hand; the boundary must match byte for byte."""
boundary = f"----umbra{uuid.uuid4().hex}"
@@ -123,6 +131,9 @@ def post_tts(
parts.append(b"\r\n")
field("text", text)
# Must be sent: the server used to hardcode language="english" in every
# branch, so German and French were synthesised with English phonology.
field("language", language)
if prompt_slug:
field("prompt_slug", prompt_slug)
if ref_audio:
@@ -286,6 +297,7 @@ def main() -> int:
ref_audio=None if use_prompt else refs[slug],
ref_text=None if use_prompt else REF_TEXT,
timeout=args.timeout,
language=LANG_CODES.get(suffix, "english"),
)
dest.write_bytes(audio)
made.append(dest)