Fix render_lang_test: /v1/tts returns raw WAV, and add a model-portable voice mode
Two bugs that would have failed on all 120 clips:
- The script parsed {"audio_base64": ...}. /v1/tts returns raw WAV bytes
with media_type audio/wav. Now validated with a RIFF check so a bad
response says so instead of raising a JSON error.
- It preferred the cached ICL prompt. Those are MODEL-SPECIFIC: narrator.pt
was built by the 1.7B model and the 0.6B rejects its ref_code with
'Expected size 1024 but got size 2048'. --voice-mode now defaults to ref,
sending ref_audio + ref_text (the Rainbow Passage the refs were generated
from) so the model builds its own prompt; --voice-mode prompt is available
and documented as valid only for the model that built the cache.
Measured on a reboot with swap at 0: the 1.7B model still fails to generate
(footprint 23.4G on a 16G machine, zero POSTs completing). The 0.6B model
loads in 5s and renders 6-21s per clip.
Rendered 120 clips across 10 languages plus a stitched file each: no silent
clips, RMS -22 to -23 dB throughout. Duration ordering is itself a useful
signal - ZH 92s and EN 81s against DE 98s, FR/IT 102-107s.
Suite: 721 passed, 8 pre-existing.
This commit is contained in:
@@ -10,10 +10,16 @@ per-language file, which is what you actually listen to.
|
||||
Qwen3-TTS auto-detects language from the text - ``/v1/tts`` takes no language
|
||||
argument - so there is nothing to set per clip.
|
||||
|
||||
The speaker's cached ICL prompt is preferred over the raw reference WAV because
|
||||
it gives the identical voice on every utterance. Where no prompt is cached the
|
||||
script falls back to ``ref_audio`` + ``ref_text``, which needs the reference
|
||||
transcript to be correct or the clone is worse, not better.
|
||||
Voice is supplied two ways:
|
||||
|
||||
``--voice-mode ref`` (default) sends ``ref_audio`` + ``ref_text`` and lets the
|
||||
model build its own prompt. ``--voice-mode prompt`` uses the cached ICL prompt,
|
||||
which is more faithful but is **model-specific**: a prompt built by the 1.7B
|
||||
model has a ref_code whose dimensions the 0.6B model rejects, with
|
||||
"Expected size 1024 but got size 2048". Cached prompts are therefore only safe
|
||||
when rendering with the model that built them.
|
||||
|
||||
/v1/tts returns raw WAV bytes, not a JSON envelope.
|
||||
|
||||
Usage:
|
||||
python scripts/render_lang_test.py --project DealHunterShip_4.0e_whole-to-audio
|
||||
@@ -41,6 +47,10 @@ TTS_URL = "http://127.0.0.1:8881"
|
||||
CHAPTER_ID = 1
|
||||
TRANSLATION_DIRNAME = "Lang_test"
|
||||
|
||||
# The reference WAVs were generated from this text, so it is the correct
|
||||
# ref_text for the clone path. A wrong transcript degrades the clone.
|
||||
from umbraaudio.voices.qwen_tts import VOICE_REFERENCE_TEXT as REF_TEXT # noqa: E402
|
||||
|
||||
|
||||
def slugify(name: str) -> str:
|
||||
return name.strip().lower().replace(" ", "_")
|
||||
@@ -124,14 +134,16 @@ def post_tts(
|
||||
headers={"Content-Type": f"multipart/form-data; boundary={boundary}"},
|
||||
method="POST",
|
||||
)
|
||||
# Raw WAV, not JSON. An earlier version of this script parsed
|
||||
# {"audio_base64": ...} and would have failed on every single clip.
|
||||
with urllib.request.urlopen(req, timeout=timeout) as resp:
|
||||
payload = json.loads(resp.read())
|
||||
audio = payload.get("audio_base64")
|
||||
if not audio:
|
||||
raise RuntimeError(f"no audio in response: {str(payload)[:200]}")
|
||||
import base64
|
||||
|
||||
return base64.b64decode(audio)
|
||||
body = resp.read()
|
||||
if not body.startswith(b"RIFF"):
|
||||
raise RuntimeError(
|
||||
f"expected WAV, got {body[:80]!r} "
|
||||
f"(HTTP {getattr(resp, 'status', '?')})"
|
||||
)
|
||||
return body
|
||||
|
||||
|
||||
def stitch(parts: list[Path], out: Path, gap_ms: int = 400) -> None:
|
||||
@@ -162,6 +174,11 @@ def main() -> int:
|
||||
ap.add_argument("--languages", help="comma-separated suffixes, default all found")
|
||||
ap.add_argument("--timeout", type=int, default=900)
|
||||
ap.add_argument("--dry-run", action="store_true")
|
||||
ap.add_argument(
|
||||
"--voice-mode", choices=("ref", "prompt"), default="ref",
|
||||
help="ref: ref_audio+ref_text (model-portable). prompt: cached ICL "
|
||||
"prompt (more faithful, but only valid for the model that built it)",
|
||||
)
|
||||
args = ap.parse_args()
|
||||
|
||||
config = load_config(args.project)
|
||||
@@ -231,11 +248,12 @@ def main() -> int:
|
||||
continue
|
||||
t0 = time.time()
|
||||
try:
|
||||
use_prompt = args.voice_mode == "prompt" and slug in prompts
|
||||
audio = post_tts(
|
||||
TTS_URL, u["text"],
|
||||
prompt_slug=slug if slug in prompts else None,
|
||||
ref_audio=None if slug in prompts else refs[slug],
|
||||
ref_text=None if slug in prompts else None,
|
||||
prompt_slug=slug if use_prompt else None,
|
||||
ref_audio=None if use_prompt else refs[slug],
|
||||
ref_text=None if use_prompt else REF_TEXT,
|
||||
timeout=args.timeout,
|
||||
)
|
||||
dest.write_bytes(audio)
|
||||
|
||||
Reference in New Issue
Block a user