4c1e6ebc98
The controller will not write a `speak` work function against a description, so
this measures instead of describing. scripts/build_tts_facts.py emits
docs/tts/tts-engine.{md,json} from the running system and the repository at run
time; anything unmeasurable says "unknown" with a reason. The content hash
excludes the timestamp, so it is stable across runs and answers "did anything
change?" rather than "was it run again?". 18 tests enforce that it does not
estimate -- including one that fails if a missing weights file is rendered as
"None", and one that fails if the Edge-voice answer is inverted, which it was, in
the first draft.
Four things that change the controller's plans:
* The fork is ONE commit ahead of upstream and it is cosmetic -- a logo and the
web demo. The contract being written against is not the fork, it is
umbraaudio/scripts/qwen_tts_server.py, where the HTTP surface and the four
generation paths were written.
* Synthesis is not deterministic. temperature 0.9, do_sample true, no seed. A
translation re-voiced because one line changed will not reproduce the other
lines byte-for-byte, so comparing takes needs an embedding measure, not a hash.
* The TTS server has NO authentication and binds 0.0.0.0:8881, and concurrency
is 1 behind a single lock. The generator names the seven auth mechanisms it
searched for so the claim can be checked rather than believed.
* assigned_edge_voice is the fallback engine for creating reference audio, not
the engine that renders utterances.
Not measured, and said so: working set / VRAM (the one that decides the 48 GB
CPU node question -- 9.5 GB of weights on disk is not the memory the process
holds, and on CPU that gap is the whole question); whether a seeded call is
reproducible; per-language quality; the model's own maximum text length; CUDA
failure modes.
scripts/build_scale_totals.py measures across all projects rather than handing
over a typed number: 4 projects, 131 chapters, 18,181 utterances, 14,526 audio
files, 6.98 GB, 38.58 hours, and 2 distinct schema fingerprints -- matching the
two already recorded in manifest.md. Markers are counted separately from audio,
because "files" that means both is a number nobody can use. Treasure Island
contributes 49 chapters and zero audio: that is a fact about it, not a failed
measurement.
Also delivered to the controller: their accepted-format validator accepts the
bundle, exit 0, 8884 records, and I confirmed it fails on a broken copy before
believing the pass.
292 lines
11 KiB
JSON
292 lines
11 KiB
JSON
{
|
|
"attn_implementation_now": {
|
|
"impl": "sdpa",
|
|
"ok": true
|
|
},
|
|
"content_hash": {
|
|
"algorithm": "sha256",
|
|
"of": "this document with generated_at and content_hash omitted, canonical JSON",
|
|
"value": "abe913526a4dc330e6cebd06e5dd3381df4053a7681ad971f8a829537db53017"
|
|
},
|
|
"determinism": {
|
|
"consequence": "A translation re-voiced because one line changed will NOT reproduce the other lines byte-for-byte. Comparing takes needs a perceptual or embedding measure, not a hash.",
|
|
"identical_by_default": false,
|
|
"seeded": [
|
|
"torch.manual_seed(seed)",
|
|
"torch.cuda.manual_seed_all(seed) if cuda"
|
|
],
|
|
"seeded_reproducible": "unknown",
|
|
"seeded_reproducible_why": "Not measured. torch.manual_seed is called when a seed is supplied, which seeds the global generator, but reproducibility across MPS kernels, across model reloads, and across attention implementations is a separate question that needs a deliberate experiment.",
|
|
"unseeded_sources": [
|
|
"sampling (temperature 0.9, do_sample True)"
|
|
],
|
|
"why": "temperature 0.9 with do_sample=True and no seed. Sampling is on, so the same request is expected to differ between calls."
|
|
},
|
|
"document_version": 1,
|
|
"generated_at": "2026-10-15T19:02:50Z",
|
|
"generated_by": "scripts/build_tts_facts.py",
|
|
"languages": {
|
|
"count": 10,
|
|
"declared_by_upstream": [
|
|
"Chinese",
|
|
"English",
|
|
"Japanese",
|
|
"Korean",
|
|
"German",
|
|
"French",
|
|
"Russian",
|
|
"Portuguese",
|
|
"Spanish",
|
|
"Italian"
|
|
],
|
|
"known_weak": {
|
|
"en": "English phonology and prosody are applied unless `language` is passed. This was a real bug here: the field was hardcoded to english in every branch, so German and French text was read with English prosody. Fixed by making it a parameter."
|
|
},
|
|
"quality_weaknesses": "unknown",
|
|
"quality_weaknesses_why": "No per-language evaluation has been run in this system. There is a quality gate, but it scores MOS/WER/artifact score over the book, not per language. Claiming a ranking would be an estimate.",
|
|
"selection": "the `language` form field; 'english' is the server default, 'auto' lets the model detect"
|
|
},
|
|
"models": {
|
|
"loaded_weights_bytes": 9520160568,
|
|
"models_present": [
|
|
{
|
|
"config_present": true,
|
|
"directory_bytes": 2516107403,
|
|
"model_id": "Qwen3-TTS-12Hz-0.6B-Base",
|
|
"weights_bytes": 1829344272,
|
|
"weights_file": "model.safetensors",
|
|
"weights_sha256": "180b3b10eb1c9f1b4db7806d5475bae3071c0243c299d49926bab1da3b6946f6",
|
|
"weights_state": "present"
|
|
},
|
|
{
|
|
"config_present": true,
|
|
"directory_bytes": 272829144,
|
|
"model_id": "Qwen3-TTS-12Hz-0.6B-CustomVoice",
|
|
"weights_bytes": null,
|
|
"weights_file": null,
|
|
"weights_sha256": null,
|
|
"weights_state": "absent"
|
|
},
|
|
{
|
|
"config_present": true,
|
|
"directory_bytes": 4544231050,
|
|
"model_id": "Qwen3-TTS-12Hz-1.7B-Base",
|
|
"weights_bytes": 3857413744,
|
|
"weights_file": "model.safetensors",
|
|
"weights_sha256": "38fc7fc51c5e776e840414b6fd443962e9411b9654888fd7913e4da643cb857c",
|
|
"weights_state": "present"
|
|
},
|
|
{
|
|
"config_present": true,
|
|
"directory_bytes": 406973095,
|
|
"model_id": "Qwen3-TTS-12Hz-1.7B-CustomVoice",
|
|
"weights_bytes": null,
|
|
"weights_file": null,
|
|
"weights_sha256": null,
|
|
"weights_state": "absent"
|
|
},
|
|
{
|
|
"config_present": true,
|
|
"directory_bytes": 4520165283,
|
|
"model_id": "Qwen3-TTS-12Hz-1.7B-VoiceDesign",
|
|
"weights_bytes": 3833402552,
|
|
"weights_file": "model.safetensors",
|
|
"weights_sha256": "391e8db219f292c515297cdceeb43e4eae67cdde35fa57e79a6a8a532fca0522",
|
|
"weights_state": "present"
|
|
}
|
|
],
|
|
"models_without_weights": [
|
|
"Qwen3-TTS-12Hz-0.6B-CustomVoice",
|
|
"Qwen3-TTS-12Hz-1.7B-CustomVoice"
|
|
],
|
|
"note": "weights_sha256 is of model.safetensors only, not the whole directory. A model directory also holds config and tokenizer files that change behaviour. A directory listed under models_without_weights has config but no weights at all - it cannot be loaded, and its absence is not an error.",
|
|
"total_directory_bytes": 12260305975
|
|
},
|
|
"output_contract": {
|
|
"consistency_note": "Renders are 24 kHz. The distribution masters are 44.1 kHz. They are not the same format, and a render produced by `speak` should be described as the first, not the second.",
|
|
"distribution_master": {
|
|
"bit_rate_bps": "192000",
|
|
"channels": 1,
|
|
"codec": "mp3",
|
|
"container": "mp3",
|
|
"duration_s": "2233.100000",
|
|
"measured_from": ".umbra/export/ar_package/Deal_Hunter_01_Salvage.mp3",
|
|
"sample_rate_hz": "44100"
|
|
},
|
|
"rendered_utterance": {
|
|
"bit_depth_bits": 16,
|
|
"bit_rate_bps": null,
|
|
"bit_rate_note": "PCM; follows from rate x depth x channels",
|
|
"channels": 1,
|
|
"codec": "pcm_s16le",
|
|
"container": "wav",
|
|
"duration_ms": 10560,
|
|
"measured_from": ".umbra/chapter_audio/ch1/utt_0000.wav",
|
|
"sample_rate_hz": 24000
|
|
}
|
|
},
|
|
"provenance": {
|
|
"commits_ahead_detail": [
|
|
"012e452 Add UmbraAudio Voice Studio branding to web demo (icon + favicon)"
|
|
],
|
|
"commits_ahead_of_upstream": 1,
|
|
"commits_behind_upstream": 0,
|
|
"diff_stat_vs_base": "assets/logo.png | Bin 0 -> 1102415 bytes\n qwen_tts/cli/demo.py | 65 ++++++++++++++++++++++++++++++++++++++++++++++++---\n 2 files changed, 62 insertions(+), 3 deletions(-)",
|
|
"license": "Apache License Version 2.0, January 2004",
|
|
"local_head": "012e45298d44e8fc6ec748ba8d194bc7d5cc6c94",
|
|
"note": "The fork delta touches only the web demo and a logo. It does not touch the synthesis path, so it does not alter the input or output contract. The contract that IS ours is umbraaudio's own server script, which is not part of this repository - see runtime_surface.",
|
|
"upstream_base_commit": "022e286b98fbec7e1e916cb940cdf532cd9f488e",
|
|
"upstream_repository": "https://github.com/QwenLM/Qwen3-TTS.git"
|
|
},
|
|
"runtime_surface": {
|
|
"authentication": {
|
|
"binds": "0.0.0.0:8881",
|
|
"mechanisms_found": [],
|
|
"mechanisms_searched": [
|
|
"api_token_env",
|
|
"auth_middleware",
|
|
"authorization_header_check",
|
|
"bearer_scheme",
|
|
"fastapi_dependency",
|
|
"oauth2_scheme",
|
|
"security_scheme"
|
|
],
|
|
"note": "No authentication mechanism is present, and it binds all interfaces. Any client - including a controller-side speak worker - reaches an open synthesis service that will happily spend minutes of CPU per request. This is a property of the deployment, not an oversight in the server code.",
|
|
"requires_authentication": false
|
|
},
|
|
"concurrency": {
|
|
"concurrent_generations": 1,
|
|
"model_lock": true,
|
|
"watchdog_restart_seconds": 1800,
|
|
"why": "a single asyncio.Lock serialises generation; the event loop stays responsive and work runs in threads"
|
|
},
|
|
"endpoints": {
|
|
"attn": {
|
|
"method": "POST",
|
|
"path": "/attn"
|
|
},
|
|
"generate": {
|
|
"method": "POST",
|
|
"path": "/v1/tts"
|
|
},
|
|
"health": {
|
|
"method": "GET",
|
|
"path": "/health"
|
|
},
|
|
"load": {
|
|
"method": "POST",
|
|
"path": "/load"
|
|
},
|
|
"unload": {
|
|
"method": "POST",
|
|
"path": "/unload"
|
|
},
|
|
"voice_prompt": {
|
|
"method": "POST",
|
|
"path": "/v1/voice-prompt"
|
|
}
|
|
},
|
|
"generation_paths": {
|
|
"clone": "ref_audio + ref_text -> prompt built per request.",
|
|
"design": "voice_description only -> VoiceDesign model.",
|
|
"embed": "speaker_embedding (.npy) -> coarse x-vector. Fastest.",
|
|
"icl": "prompt_slug -> cached vmatrix (.pt). Most faithful and most consistent."
|
|
},
|
|
"input_fields": {
|
|
"do_sample": {
|
|
"default": true
|
|
},
|
|
"language": {
|
|
"accepts": [
|
|
"english",
|
|
"auto",
|
|
"<other>"
|
|
],
|
|
"default": "english",
|
|
"note": "was hardcoded to english; a caller must pass it"
|
|
},
|
|
"prompt_slug": {
|
|
"default": null,
|
|
"note": "cached voice-clone prompt; wins over others"
|
|
},
|
|
"ref_audio": {
|
|
"default": null,
|
|
"type": "file"
|
|
},
|
|
"ref_text": {
|
|
"default": null
|
|
},
|
|
"sample_rate": {
|
|
"default": 24000
|
|
},
|
|
"seed": {
|
|
"default": null
|
|
},
|
|
"speaker_embedding": {
|
|
"default": null,
|
|
"note": "path to a cached .npy"
|
|
},
|
|
"temperature": {
|
|
"default": 0.9
|
|
},
|
|
"text": {
|
|
"required": true,
|
|
"type": "form"
|
|
},
|
|
"voice_description": {
|
|
"default": null,
|
|
"note": "VoiceDesign path, and also style direction when combined with a prompt"
|
|
}
|
|
},
|
|
"output": {
|
|
"max_text_length_enforced_by_server": null,
|
|
"max_text_length_note": "No length limit is enforced by this server. Any limit is the model's own and was not measured; treat as unknown rather than unbounded.",
|
|
"shape": "whole file, returned in-memory as an audio payload",
|
|
"streaming": false
|
|
}
|
|
},
|
|
"server_health_now": {
|
|
"base_loaded": true,
|
|
"current_model": "base",
|
|
"device": "mps",
|
|
"status": "ok",
|
|
"voicedesign_loaded": false
|
|
},
|
|
"source_system": {
|
|
"node_id": "umbraaudio-m4",
|
|
"system": "umbraaudio"
|
|
},
|
|
"voice_identity": {
|
|
"chain": [
|
|
"assigned_edge_voice -> an Edge-TTS voice id, e.g. en-CA-ClaraNeural:-100",
|
|
"voices/<name>_ref.wav -> the REFERENCE AUDIO, one per speaker",
|
|
"voice_prompts/<slug>.pt -> the cached 'vmatrix' extracted from that reference",
|
|
"utterance render -> Qwen conditions on the vmatrix (prompt_slug)"
|
|
],
|
|
"counts": {
|
|
"cached_voice_prompts": 56,
|
|
"ledger_entries": 9,
|
|
"prompts_without_a_reference_wav": 31,
|
|
"reference_wavs_on_disk": 93,
|
|
"refs_with_recorded_provenance": 8,
|
|
"refs_with_unknown_provenance": 85,
|
|
"speakers_with_an_edge_voice_assigned": 61
|
|
},
|
|
"orphan_prompts_note": "31 cached prompts have no matching reference file. A cached vmatrix outlives the audio it was built from, so a replaced reference does not automatically invalidate its prompt; the server takes force=1 to rebuild one deliberately.",
|
|
"tts_engine_used_semantics": {
|
|
"null": "UNKNOWN - no decision was recorded. Not 'edge'.",
|
|
"qwen": "this speaker's reference audio was generated by Qwen VoiceDesign"
|
|
},
|
|
"unknown_provenance_is_the_norm": "85 of 93 reference files have no recorded provenance. The ledger was added after those references were generated, so a null here means the system does not know, which is a different statement from 'it was Edge'.",
|
|
"what_the_edge_voice_actually_is": {
|
|
"answer": "the FALLBACK ENGINE for creating the reference audio",
|
|
"consequence": "tts_engine_used describes which engine produced the REFERENCE, not which engine rendered the utterance. Reading it as the latter is wrong.",
|
|
"not": [
|
|
"the engine that renders utterances",
|
|
"the clone source"
|
|
],
|
|
"why": "generate_voice_reference() tries Qwen VoiceDesign up to max_retries times; only on exhaustion does it call _generate_edge_fallback() and write the Edge-derived audio to the same ref path. The Edge id is also a historical voice assignment from when Edge-TTS was the voice source."
|
|
}
|
|
}
|
|
}
|