Fix render_lang_test: /v1/tts returns raw WAV, and add a model-portable voice mode

Two bugs that would have failed on all 120 clips:

- The script parsed {"audio_base64": ...}. /v1/tts returns raw WAV bytes
  with media_type audio/wav. Now validated with a RIFF check so a bad
  response says so instead of raising a JSON error.
- It preferred the cached ICL prompt. Those are MODEL-SPECIFIC: narrator.pt
  was built by the 1.7B model and the 0.6B rejects its ref_code with
  'Expected size 1024 but got size 2048'. --voice-mode now defaults to ref,
  sending ref_audio + ref_text (the Rainbow Passage the refs were generated
  from) so the model builds its own prompt; --voice-mode prompt is available
  and documented as valid only for the model that built the cache.

Measured on a reboot with swap at 0: the 1.7B model still fails to generate
(footprint 23.4G on a 16G machine, zero POSTs completing). The 0.6B model
loads in 5s and renders 6-21s per clip.

Rendered 120 clips across 10 languages plus a stitched file each: no silent
clips, RMS -22 to -23 dB throughout. Duration ordering is itself a useful
signal - ZH 92s and EN 81s against DE 98s, FR/IT 102-107s.

Suite: 721 passed, 8 pre-existing.
This commit is contained in:
B.D. Murphy
2026-10-12 21:17:54 -07:00
parent 6b85e48b55
commit fe6337d3e5

View File

@@ -10,10 +10,16 @@ per-language file, which is what you actually listen to.
Qwen3-TTS auto-detects language from the text - ``/v1/tts`` takes no language
argument - so there is nothing to set per clip.
The speaker's cached ICL prompt is preferred over the raw reference WAV because
it gives the identical voice on every utterance. Where no prompt is cached the
script falls back to ``ref_audio`` + ``ref_text``, which needs the reference
transcript to be correct or the clone is worse, not better.
Voice is supplied two ways:
``--voice-mode ref`` (default) sends ``ref_audio`` + ``ref_text`` and lets the
model build its own prompt. ``--voice-mode prompt`` uses the cached ICL prompt,
which is more faithful but is **model-specific**: a prompt built by the 1.7B
model has a ref_code whose dimensions the 0.6B model rejects, with
"Expected size 1024 but got size 2048". Cached prompts are therefore only safe
when rendering with the model that built them.
/v1/tts returns raw WAV bytes, not a JSON envelope.
Usage:
python scripts/render_lang_test.py --project DealHunterShip_4.0e_whole-to-audio
@@ -41,6 +47,10 @@ TTS_URL = "http://127.0.0.1:8881"
CHAPTER_ID = 1
TRANSLATION_DIRNAME = "Lang_test"
# The reference WAVs were generated from this text, so it is the correct
# ref_text for the clone path. A wrong transcript degrades the clone.
from umbraaudio.voices.qwen_tts import VOICE_REFERENCE_TEXT as REF_TEXT # noqa: E402
def slugify(name: str) -> str:
return name.strip().lower().replace(" ", "_")
@@ -124,14 +134,16 @@ def post_tts(
headers={"Content-Type": f"multipart/form-data; boundary={boundary}"},
method="POST",
)
# Raw WAV, not JSON. An earlier version of this script parsed
# {"audio_base64": ...} and would have failed on every single clip.
with urllib.request.urlopen(req, timeout=timeout) as resp:
payload = json.loads(resp.read())
audio = payload.get("audio_base64")
if not audio:
raise RuntimeError(f"no audio in response: {str(payload)[:200]}")
import base64
return base64.b64decode(audio)
body = resp.read()
if not body.startswith(b"RIFF"):
raise RuntimeError(
f"expected WAV, got {body[:80]!r} "
f"(HTTP {getattr(resp, 'status', '?')})"
)
return body
def stitch(parts: list[Path], out: Path, gap_ms: int = 400) -> None:
@@ -162,6 +174,11 @@ def main() -> int:
ap.add_argument("--languages", help="comma-separated suffixes, default all found")
ap.add_argument("--timeout", type=int, default=900)
ap.add_argument("--dry-run", action="store_true")
ap.add_argument(
"--voice-mode", choices=("ref", "prompt"), default="ref",
help="ref: ref_audio+ref_text (model-portable). prompt: cached ICL "
"prompt (more faithful, but only valid for the model that built it)",
)
args = ap.parse_args()
config = load_config(args.project)
@@ -231,11 +248,12 @@ def main() -> int:
continue
t0 = time.time()
try:
use_prompt = args.voice_mode == "prompt" and slug in prompts
audio = post_tts(
TTS_URL, u["text"],
prompt_slug=slug if slug in prompts else None,
ref_audio=None if slug in prompts else refs[slug],
ref_text=None if slug in prompts else None,
prompt_slug=slug if use_prompt else None,
ref_audio=None if use_prompt else refs[slug],
ref_text=None if use_prompt else REF_TEXT,
timeout=args.timeout,
)
dest.write_bytes(audio)