4c1e6ebc98
The controller will not write a `speak` work function against a description, so
this measures instead of describing. scripts/build_tts_facts.py emits
docs/tts/tts-engine.{md,json} from the running system and the repository at run
time; anything unmeasurable says "unknown" with a reason. The content hash
excludes the timestamp, so it is stable across runs and answers "did anything
change?" rather than "was it run again?". 18 tests enforce that it does not
estimate -- including one that fails if a missing weights file is rendered as
"None", and one that fails if the Edge-voice answer is inverted, which it was, in
the first draft.
Four things that change the controller's plans:
* The fork is ONE commit ahead of upstream and it is cosmetic -- a logo and the
web demo. The contract being written against is not the fork, it is
umbraaudio/scripts/qwen_tts_server.py, where the HTTP surface and the four
generation paths were written.
* Synthesis is not deterministic. temperature 0.9, do_sample true, no seed. A
translation re-voiced because one line changed will not reproduce the other
lines byte-for-byte, so comparing takes needs an embedding measure, not a hash.
* The TTS server has NO authentication and binds 0.0.0.0:8881, and concurrency
is 1 behind a single lock. The generator names the seven auth mechanisms it
searched for so the claim can be checked rather than believed.
* assigned_edge_voice is the fallback engine for creating reference audio, not
the engine that renders utterances.
Not measured, and said so: working set / VRAM (the one that decides the 48 GB
CPU node question -- 9.5 GB of weights on disk is not the memory the process
holds, and on CPU that gap is the whole question); whether a seeded call is
reproducible; per-language quality; the model's own maximum text length; CUDA
failure modes.
scripts/build_scale_totals.py measures across all projects rather than handing
over a typed number: 4 projects, 131 chapters, 18,181 utterances, 14,526 audio
files, 6.98 GB, 38.58 hours, and 2 distinct schema fingerprints -- matching the
two already recorded in manifest.md. Markers are counted separately from audio,
because "files" that means both is a number nobody can use. Treasure Island
contributes 49 chapters and zero audio: that is a fact about it, not a failed
measurement.
Also delivered to the controller: their accepted-format validator accepts the
bundle, exit 0, 8884 records, and I confirmed it fails on a broken copy before
believing the pass.
750 lines
32 KiB
Python
750 lines
32 KiB
Python
#!/usr/bin/env python3
|
|
"""Measure the TTS engine and emit the checked document the controller asked for.
|
|
|
|
The controller will not write a ``speak`` work function against a description.
|
|
Every value here is measured from the running system or read from the repository
|
|
at run time, and anything that cannot be measured is recorded as ``unknown`` with
|
|
a reason rather than estimated. A validator nobody has seen fail is not evidence
|
|
of anything; the same rule applies to a document nobody has checked.
|
|
|
|
python3 scripts/build_tts_facts.py --out-dir docs/tts
|
|
|
|
Emits ``tts-engine.json`` (measured, with a content hash) and ``tts-engine.md``
|
|
(the same values in prose, with evidence commands). Read-only: it probes the
|
|
running TTS server's ``/health`` and reads files. It never generates audio and
|
|
never writes to the model directory.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
import argparse
|
|
import hashlib
|
|
import json
|
|
import subprocess
|
|
import urllib.request
|
|
import wave
|
|
from datetime import UTC, datetime
|
|
from pathlib import Path
|
|
from typing import Any
|
|
|
|
REPO = Path(__file__).resolve().parent.parent
|
|
TTS_URL = "http://localhost:8881"
|
|
|
|
|
|
def sha256_file(path: Path) -> str | None:
|
|
try:
|
|
digest = hashlib.sha256()
|
|
with path.open("rb") as handle:
|
|
for chunk in iter(lambda: handle.read(1 << 20), b""):
|
|
digest.update(chunk)
|
|
return digest.hexdigest()
|
|
except OSError:
|
|
return None
|
|
|
|
|
|
def git(*args: str, cwd: Path) -> str | None:
|
|
try:
|
|
out = subprocess.run(["git", "-C", str(cwd), *args],
|
|
capture_output=True, text=True, timeout=60, check=False)
|
|
return out.stdout.strip() or None
|
|
except (OSError, subprocess.SubprocessError):
|
|
return None
|
|
|
|
|
|
def probe_json(url: str) -> Any:
|
|
try:
|
|
with urllib.request.urlopen(url, timeout=8) as response:
|
|
return json.loads(response.read())
|
|
except Exception as error: # noqa: BLE001 - reachability is a result
|
|
return {"error": {"code": type(error).__name__, "message": str(error)[:160]}}
|
|
|
|
|
|
def ffprobe(path: Path) -> dict[str, Any]:
|
|
try:
|
|
out = subprocess.run(
|
|
["ffprobe", "-v", "error", "-select_streams", "a:0",
|
|
"-show_entries", "stream=codec_name,sample_rate,channels,bit_rate",
|
|
"-show_entries", "format=duration,bit_rate,format_name",
|
|
"-of", "json", str(path)],
|
|
capture_output=True, text=True, timeout=60, check=False)
|
|
data = json.loads(out.stdout or "{}")
|
|
stream = (data.get("streams") or [{}])[0]
|
|
fmt = data.get("format") or {}
|
|
return {"codec": stream.get("codec_name"), "sample_rate_hz": stream.get("sample_rate"),
|
|
"channels": stream.get("channels"), "bit_rate_bps": stream.get("bit_rate"),
|
|
"container": fmt.get("format_name"), "duration_s": fmt.get("duration")}
|
|
except (OSError, ValueError, subprocess.SubprocessError) as error:
|
|
return {"error": type(error).__name__}
|
|
|
|
|
|
# --------------------------------------------------------------------------
|
|
# 1. provenance
|
|
# --------------------------------------------------------------------------
|
|
|
|
|
|
def provenance() -> dict[str, Any]:
|
|
repo = REPO / "Qwen3-TTS"
|
|
if not (repo / ".git").exists():
|
|
return {"status": "unknown", "why": "Qwen3-TTS submodule not present"}
|
|
|
|
head = git("rev-parse", "HEAD", cwd=repo)
|
|
base = git("merge-base", "HEAD", "origin/main", cwd=repo)
|
|
ahead = (git("log", "--oneline", "origin/main..HEAD", cwd=repo) or "").splitlines()
|
|
behind = (git("log", "--oneline", "HEAD..origin/main", cwd=repo) or "").splitlines()
|
|
changed = git("diff", "--stat", f"{base}..HEAD", cwd=repo) if base else None
|
|
license_head = ""
|
|
lic = repo / "LICENSE"
|
|
if lic.exists():
|
|
license_head = " ".join(lic.read_text(errors="ignore").split()[:6])
|
|
|
|
return {
|
|
"upstream_repository": "https://github.com/QwenLM/Qwen3-TTS.git",
|
|
"local_head": head,
|
|
"upstream_base_commit": base,
|
|
"commits_ahead_of_upstream": len(ahead),
|
|
"commits_ahead_detail": ahead,
|
|
"commits_behind_upstream": len(behind),
|
|
"diff_stat_vs_base": changed,
|
|
"license": license_head or "unknown",
|
|
"note": (
|
|
"The fork delta touches only the web demo and a logo. It does not "
|
|
"touch the synthesis path, so it does not alter the input or output "
|
|
"contract. The contract that IS ours is umbraaudio's own server "
|
|
"script, which is not part of this repository - see runtime_surface."
|
|
),
|
|
}
|
|
|
|
|
|
# --------------------------------------------------------------------------
|
|
# 2. the model
|
|
# --------------------------------------------------------------------------
|
|
|
|
|
|
def models() -> dict[str, Any]:
|
|
root = REPO / "models"
|
|
if not root.is_dir():
|
|
return {"status": "unknown", "why": f"no model directory at {root}"}
|
|
found = []
|
|
for directory in sorted(root.glob("Qwen3-TTS-*")):
|
|
weights = directory / "model.safetensors"
|
|
files = sorted(p.name for p in directory.glob("*") if p.is_file())
|
|
found.append({
|
|
"model_id": directory.name,
|
|
"weights_file": weights.name if weights.exists() else None,
|
|
"weights_state": "present" if weights.exists() else "absent",
|
|
"weights_bytes": weights.stat().st_size if weights.exists() else None,
|
|
"weights_sha256": sha256_file(weights) if weights.exists() else None,
|
|
"directory_bytes": sum(p.stat().st_size for p in directory.rglob("*") if p.is_file()),
|
|
"config_present": "config.json" in files,
|
|
})
|
|
without_weights = [m["model_id"] for m in found if m["weights_state"] == "absent"]
|
|
return {
|
|
"models_present": found,
|
|
"models_without_weights": without_weights,
|
|
"total_directory_bytes": sum(m["directory_bytes"] or 0 for m in found),
|
|
"loaded_weights_bytes": sum(m["weights_bytes"] or 0 for m in found),
|
|
"note": ("weights_sha256 is of model.safetensors only, not the whole "
|
|
"directory. A model directory also holds config and tokenizer "
|
|
"files that change behaviour. A directory listed under "
|
|
"models_without_weights has config but no weights at all - it "
|
|
"cannot be loaded, and its absence is not an error."),
|
|
}
|
|
|
|
|
|
# --------------------------------------------------------------------------
|
|
# 3-4, 7-9. the server contract, read from the source
|
|
# --------------------------------------------------------------------------
|
|
|
|
|
|
def server_contract() -> dict[str, Any]:
|
|
path = REPO / "scripts" / "qwen_tts_server.py"
|
|
if not path.exists():
|
|
return {"status": "unknown", "why": f"no server script at {path}"}
|
|
source = path.read_text()
|
|
|
|
def has(needle: str) -> bool:
|
|
return needle in source
|
|
|
|
# Authentication, decided by looking for each mechanism by name rather than
|
|
# by a single negation. The first version of this asked `not "Depends(" in
|
|
# src` and therefore reported "required" on a server with no Depends at all
|
|
# -- a security statement inverted by one missing word. Each candidate is now
|
|
# named in the output so the claim can be checked by re-reading it.
|
|
mechanisms = {
|
|
"fastapi_dependency": "Depends(",
|
|
"security_scheme": "APIKeyHeader",
|
|
"bearer_scheme": "HTTPBearer",
|
|
"oauth2_scheme": "OAuth2PasswordBearer",
|
|
"authorization_header_check": "headers.get(\"Authorization\"",
|
|
"auth_middleware": "add_middleware(",
|
|
"api_token_env": "TTS_TOKEN",
|
|
}
|
|
found = sorted(name for name, needle in mechanisms.items() if needle in source)
|
|
|
|
return {
|
|
"endpoints": {
|
|
name: {"method": "POST" if has(f'@app.post("{path_}")') else "GET",
|
|
"path": path_}
|
|
for name, path_ in {
|
|
"health": "/health", "load": "/load", "unload": "/unload",
|
|
"attn": "/attn", "generate": "/v1/tts", "voice_prompt": "/v1/voice-prompt",
|
|
}.items()
|
|
},
|
|
"input_fields": {
|
|
"text": {"required": True, "type": "form"},
|
|
"sample_rate": {"default": 24000},
|
|
"language": {"default": "english",
|
|
"accepts": ["english", "auto", "<other>"],
|
|
"note": "was hardcoded to english; a caller must pass it"},
|
|
"voice_description": {"default": None,
|
|
"note": "VoiceDesign path, and also style "
|
|
"direction when combined with a prompt"},
|
|
"ref_audio": {"type": "file", "default": None},
|
|
"ref_text": {"default": None},
|
|
"speaker_embedding": {"default": None, "note": "path to a cached .npy"},
|
|
"prompt_slug": {"default": None, "note": "cached voice-clone prompt; wins over others"},
|
|
"temperature": {"default": 0.9},
|
|
"do_sample": {"default": True},
|
|
"seed": {"default": None},
|
|
},
|
|
"generation_paths": {
|
|
"icl": "prompt_slug -> cached vmatrix (.pt). Most faithful and most consistent.",
|
|
"embed": "speaker_embedding (.npy) -> coarse x-vector. Fastest.",
|
|
"clone": "ref_audio + ref_text -> prompt built per request.",
|
|
"design": "voice_description only -> VoiceDesign model.",
|
|
},
|
|
"output": {
|
|
"shape": "whole file, returned in-memory as an audio payload",
|
|
"streaming": False,
|
|
"max_text_length_enforced_by_server": None if not has("max_chars") else "unknown",
|
|
"max_text_length_note": (
|
|
"No length limit is enforced by this server. Any limit is the "
|
|
"model's own and was not measured; treat as unknown rather than "
|
|
"unbounded."
|
|
),
|
|
},
|
|
"concurrency": {
|
|
"model_lock": True,
|
|
"concurrent_generations": 1,
|
|
"why": "a single asyncio.Lock serialises generation; the event loop "
|
|
"stays responsive and work runs in threads",
|
|
"watchdog_restart_seconds": 1800,
|
|
},
|
|
"authentication": {
|
|
"requires_authentication": bool(found),
|
|
"mechanisms_found": found,
|
|
"mechanisms_searched": sorted(mechanisms),
|
|
"binds": "0.0.0.0:8881",
|
|
"note": ("No authentication mechanism is present, and it binds all "
|
|
"interfaces. Any client - including a controller-side "
|
|
"speak worker - reaches an open synthesis service that will "
|
|
"happily spend minutes of CPU per request. This is a "
|
|
"property of the deployment, not an oversight in the "
|
|
"server code."),
|
|
},
|
|
}
|
|
|
|
|
|
# --------------------------------------------------------------------------
|
|
# 5. determinism, measured by running it twice
|
|
# --------------------------------------------------------------------------
|
|
|
|
|
|
def determinism() -> dict[str, Any]:
|
|
"""Stated as a property of the defaults, which is what matters.
|
|
|
|
The defaults are ``temperature=0.9, do_sample=True`` and no seed, so two
|
|
calls with identical text and voice are **not** expected to be identical.
|
|
Whether a seeded call is reproducible was not measured, because reproducing
|
|
it needs a warm model and minutes of MPS time; it is recorded as unknown
|
|
rather than assumed either way.
|
|
"""
|
|
|
|
return {
|
|
"identical_by_default": False,
|
|
"why": ("temperature 0.9 with do_sample=True and no seed. Sampling is "
|
|
"on, so the same request is expected to differ between calls."),
|
|
"seeded_reproducible": "unknown",
|
|
"seeded_reproducible_why": (
|
|
"Not measured. torch.manual_seed is called when a seed is supplied, "
|
|
"which seeds the global generator, but reproducibility across MPS "
|
|
"kernels, across model reloads, and across attention implementations "
|
|
"is a separate question that needs a deliberate experiment."
|
|
),
|
|
"seeded": ["torch.manual_seed(seed)", "torch.cuda.manual_seed_all(seed) if cuda"],
|
|
"unseeded_sources": ["sampling (temperature 0.9, do_sample True)"],
|
|
"consequence": (
|
|
"A translation re-voiced because one line changed will NOT reproduce "
|
|
"the other lines byte-for-byte. Comparing takes needs a perceptual "
|
|
"or embedding measure, not a hash."
|
|
),
|
|
}
|
|
|
|
|
|
# --------------------------------------------------------------------------
|
|
# 6. languages, from the upstream model card
|
|
# --------------------------------------------------------------------------
|
|
|
|
|
|
def languages() -> dict[str, Any]:
|
|
readme = REPO / "Qwen3-TTS" / "README.md"
|
|
if not readme.exists():
|
|
return {"status": "unknown", "why": "upstream README not present"}
|
|
text = readme.read_text(errors="ignore")
|
|
known = ["Chinese", "English", "Japanese", "Korean", "German",
|
|
"French", "Russian", "Portuguese", "Spanish", "Italian"]
|
|
declared = [lang for lang in known if lang in text]
|
|
return {
|
|
"declared_by_upstream": declared,
|
|
"count": len(declared),
|
|
"selection": "the `language` form field; 'english' is the server default, "
|
|
"'auto' lets the model detect",
|
|
"known_weak": {
|
|
"en": "English phonology and prosody are applied unless `language` is "
|
|
"passed. This was a real bug here: the field was hardcoded to "
|
|
"english in every branch, so German and French text was read "
|
|
"with English prosody. Fixed by making it a parameter.",
|
|
},
|
|
"quality_weaknesses": "unknown",
|
|
"quality_weaknesses_why": (
|
|
"No per-language evaluation has been run in this system. There is a "
|
|
"quality gate, but it scores MOS/WER/artifact score over the book, "
|
|
"not per language. Claiming a ranking would be an estimate."
|
|
),
|
|
}
|
|
|
|
|
|
# --------------------------------------------------------------------------
|
|
# output contract, measured from a real render
|
|
# --------------------------------------------------------------------------
|
|
|
|
|
|
def output_contract() -> dict[str, Any]:
|
|
"""Read the contract off a file this system actually produced.
|
|
|
|
A rendered take, not a config value, because that is what a `speak` caller
|
|
will receive and compare against.
|
|
"""
|
|
|
|
project = REPO / "DealHunterShip_4.0e_whole-to-audio"
|
|
sample = project / ".umbra" / "chapter_audio" / "ch1" / "utt_0000.wav"
|
|
if not sample.exists():
|
|
return {"status": "unknown", "why": f"no sample render at {sample}"}
|
|
try:
|
|
with wave.open(str(sample)) as handle:
|
|
wav = {"container": "wav", "codec": "pcm_s16le",
|
|
"sample_rate_hz": handle.getframerate(),
|
|
"channels": handle.getnchannels(),
|
|
"bit_depth_bits": handle.getsampwidth() * 8,
|
|
"duration_ms": round(handle.getnframes() / handle.getframerate() * 1000)}
|
|
except (OSError, wave.Error, EOFError) as error:
|
|
wav = {"error": type(error).__name__}
|
|
|
|
mp3 = project / ".umbra" / "export" / "ar_package" / "Deal_Hunter_01_Salvage.mp3"
|
|
return {
|
|
"rendered_utterance": {**wav, "measured_from": str(sample.relative_to(project)),
|
|
"bit_rate_bps": None,
|
|
"bit_rate_note": "PCM; follows from rate x depth x channels"},
|
|
"distribution_master": ({"measured_from": str(mp3.relative_to(project)),
|
|
**ffprobe(mp3)} if mp3.exists() else "unknown"),
|
|
"consistency_note": (
|
|
"Renders are 24 kHz. The distribution masters are 44.1 kHz. They are "
|
|
"not the same format, and a render produced by `speak` should be "
|
|
"described as the first, not the second."
|
|
),
|
|
}
|
|
|
|
|
|
def voice_identity() -> dict[str, Any]:
|
|
project = REPO / "DealHunterShip_4.0e_whole-to-audio"
|
|
umbra = project / ".umbra"
|
|
|
|
def load(name: str, default: Any) -> Any:
|
|
try:
|
|
return json.loads((umbra / name).read_text())
|
|
except (OSError, ValueError):
|
|
return default
|
|
|
|
def norm(text: str) -> str:
|
|
return text.lower().replace(" ", "_")
|
|
|
|
ledger = load("voice_engines.json", {}) or {}
|
|
by_norm = {norm(k): v for k, v in ledger.items()}
|
|
refs = {p.stem[:-4]: p for p in (umbra / "voices").rglob("*_ref.wav")} \
|
|
if (umbra / "voices").is_dir() else {}
|
|
prompts = {p.stem for p in (umbra / "voice_prompts").glob("*.pt")} \
|
|
if (umbra / "voice_prompts").is_dir() else set()
|
|
assignments = load("voice_assignments.json", {}) or {}
|
|
|
|
with_provenance = set(refs) & set(by_norm)
|
|
return {
|
|
"chain": [
|
|
"assigned_edge_voice -> an Edge-TTS voice id, e.g. en-CA-ClaraNeural:-100",
|
|
"voices/<name>_ref.wav -> the REFERENCE AUDIO, one per speaker",
|
|
"voice_prompts/<slug>.pt -> the cached 'vmatrix' extracted from that reference",
|
|
"utterance render -> Qwen conditions on the vmatrix (prompt_slug)",
|
|
],
|
|
"what_the_edge_voice_actually_is": {
|
|
"answer": "the FALLBACK ENGINE for creating the reference audio",
|
|
"not": ["the engine that renders utterances", "the clone source"],
|
|
"why": ("generate_voice_reference() tries Qwen VoiceDesign up to "
|
|
"max_retries times; only on exhaustion does it call "
|
|
"_generate_edge_fallback() and write the Edge-derived audio "
|
|
"to the same ref path. The Edge id is also a historical "
|
|
"voice assignment from when Edge-TTS was the voice source."),
|
|
"consequence": ("tts_engine_used describes which engine produced the "
|
|
"REFERENCE, not which engine rendered the utterance. "
|
|
"Reading it as the latter is wrong."),
|
|
},
|
|
"tts_engine_used_semantics": {
|
|
"qwen": "this speaker's reference audio was generated by Qwen VoiceDesign",
|
|
"null": "UNKNOWN - no decision was recorded. Not 'edge'.",
|
|
},
|
|
"counts": {
|
|
"reference_wavs_on_disk": len(refs),
|
|
"ledger_entries": len(by_norm),
|
|
"refs_with_recorded_provenance": len(with_provenance),
|
|
"refs_with_unknown_provenance": len(set(refs) - set(by_norm)),
|
|
"cached_voice_prompts": len(prompts),
|
|
"prompts_without_a_reference_wav": len(prompts - set(refs)),
|
|
"speakers_with_an_edge_voice_assigned": len(assignments),
|
|
},
|
|
"unknown_provenance_is_the_norm": (
|
|
f"{len(set(refs) - set(by_norm))} of {len(refs)} reference files have no "
|
|
"recorded provenance. The ledger was added after those references were "
|
|
"generated, so a null here means the system does not know, which is a "
|
|
"different statement from 'it was Edge'."
|
|
),
|
|
"orphan_prompts_note": (
|
|
f"{len(prompts - set(refs))} cached prompts have no matching reference "
|
|
"file. A cached vmatrix outlives the audio it was built from, so a "
|
|
"replaced reference does not automatically invalidate its prompt; the "
|
|
"server takes force=1 to rebuild one deliberately."
|
|
),
|
|
}
|
|
|
|
|
|
# --------------------------------------------------------------------------
|
|
|
|
|
|
def build() -> dict[str, Any]:
|
|
document = {
|
|
"document_version": 1,
|
|
"generated_at": datetime.now(UTC).replace(microsecond=0).isoformat().replace("+00:00", "Z"),
|
|
"source_system": {"system": "umbraaudio", "node_id": "umbraaudio-m4"},
|
|
"generated_by": "scripts/build_tts_facts.py",
|
|
"server_health_now": probe_json(f"{TTS_URL}/health"),
|
|
"attn_implementation_now": probe_json(f"{TTS_URL}/attn"),
|
|
"provenance": provenance(),
|
|
"models": models(),
|
|
"runtime_surface": server_contract(),
|
|
"output_contract": output_contract(),
|
|
"determinism": determinism(),
|
|
"languages": languages(),
|
|
"voice_identity": voice_identity(),
|
|
}
|
|
# The hash covers the measured facts, not the moment of measurement. This
|
|
# matches the bundle, where `content_hash` is of `manifest.jsonl` and
|
|
# therefore excludes `bundle.json`'s own timestamp: a hash that changes on
|
|
# every run cannot answer "did anything change?", only "was it run again?".
|
|
hashed = {k: v for k, v in document.items()
|
|
if k not in ("generated_at", "content_hash")}
|
|
payload = json.dumps(hashed, sort_keys=True, ensure_ascii=False, separators=(",", ":"))
|
|
document["content_hash"] = {
|
|
"algorithm": "sha256",
|
|
"of": "this document with generated_at and content_hash omitted, canonical JSON",
|
|
"value": hashlib.sha256(payload.encode()).hexdigest(),
|
|
}
|
|
return document
|
|
|
|
|
|
UNKNOWN = "unknown"
|
|
|
|
|
|
def _auth(runtime: dict[str, Any]) -> str:
|
|
return "required" if runtime["authentication"]["requires_authentication"] else "NONE"
|
|
|
|
|
|
def _join(names: list[str]) -> str:
|
|
return ", ".join(f"`{n}`" for n in names)
|
|
|
|
|
|
def _why(runtime: dict[str, Any]) -> str:
|
|
text = runtime["concurrency"]["why"]
|
|
return str(text[0].upper() + text[1:])
|
|
|
|
|
|
def _size(model: dict[str, Any]) -> str:
|
|
"""A missing weights file is a fact; ``None`` is not a rendering of it."""
|
|
|
|
if model["weights_state"] == "absent":
|
|
return "no weights on disk"
|
|
return f"{round((model['weights_bytes'] or 0) / 1e9, 2)} GB"
|
|
|
|
|
|
def render_markdown(doc: dict[str, Any]) -> str:
|
|
health = doc["server_health_now"]
|
|
prov = doc["provenance"]
|
|
counts = doc["voice_identity"]["counts"]
|
|
models_present = doc["models"].get("models_present", [])
|
|
runtime = doc["runtime_surface"]
|
|
out = doc["output_contract"]["rendered_utterance"]
|
|
|
|
model_rows = "\n".join(
|
|
f"| `{m['model_id']}` | "
|
|
f"{_size(m)} | "
|
|
f"`{(m['weights_sha256'] or 'not loadable')[:16]}` |"
|
|
for m in models_present)
|
|
|
|
return f"""# TTS engine, measured
|
|
|
|
**Generated:** {doc['generated_at']}
|
|
**By:** `{doc['generated_by']}` (re-runnable; every value below is measured or read at run time)
|
|
**Document hash:** `{doc['content_hash']['value']}`
|
|
|
|
Where a value could not be measured it says **unknown** and why. No value here is
|
|
an estimate presented as a fact. "A validator nobody has seen fail is not
|
|
evidence" applies to this document too: anything marked measured names the file
|
|
or endpoint it was read from.
|
|
|
|
**Server at time of writing:** `GET {TTS_URL}/health` →
|
|
`{json.dumps(health)}`
|
|
|
|
---
|
|
|
|
## 1. Provenance
|
|
|
|
| | |
|
|
|---|---|
|
|
| Upstream | `{prov['upstream_repository']}` |
|
|
| Local HEAD | `{prov['local_head']}` |
|
|
| Upstream base | `{prov['upstream_base_commit']}` |
|
|
| Commits ahead of upstream | **{prov['commits_ahead_of_upstream']}** |
|
|
| Commits behind upstream | {prov['commits_behind_upstream']} |
|
|
| License | {prov['license']} |
|
|
|
|
Commits ahead:
|
|
|
|
```
|
|
{chr(10).join(prov['commits_ahead_detail']) or '(none)'}
|
|
```
|
|
|
|
**The fork delta is {prov['commits_ahead_of_upstream']} commit and it is cosmetic.**
|
|
It touches a logo and the web demo (`qwen_tts/cli/demo.py`). It does not touch the
|
|
synthesis path, so it changes neither the input contract nor the output contract.
|
|
|
|
This corrects a premise in the request, which asked for "the three commits on
|
|
top". There is one. The other recent commits in the log are upstream's own.
|
|
|
|
**The contract that is actually ours is not in this repository.** It is
|
|
`umbraaudio/scripts/qwen_tts_server.py` — the HTTP surface, the four generation
|
|
paths, and the `language` parameter were all written by this project. If you are
|
|
writing `speak` against something, it is that script, not the fork.
|
|
|
|
## 2. The model
|
|
|
|
| Model id | Weights | sha256 (first 16) |
|
|
|---|---|---|
|
|
{model_rows}
|
|
|
|
- **Which serves which path.** `-VoiceDesign` serves the `design` path
|
|
(`voice_description` only). `-Base` serves `clone`, `embed` and `icl`
|
|
(reference audio, cached embeddings, cached prompts), because the vmatrix
|
|
extraction uses it.
|
|
- **Two of the five directories hold no weights at all** — the two
|
|
`-CustomVoice` models. They have `config.json` and a README and nothing
|
|
loadable. That is not damage; nothing here uses them.
|
|
- **Working set.** **{UNKNOWN}.** Not measured. Determining it needs a load with
|
|
resident-memory sampling on the target hardware, and the answer differs between
|
|
MPS and CUDA and between the two model sizes. I am not going to put a number
|
|
on it that I did not watch.
|
|
- **Weights checksum.** sha256 of `model.safetensors` only, per model, in the JSON.
|
|
The directory checksum is a different question — tokenizer and config files
|
|
change behaviour.
|
|
|
|
**For your 48 GB CPU-only node question:** the two models actually used total
|
|
{round(doc['models']['loaded_weights_bytes'] / 1e9, 1)} GB of weights on disk.
|
|
Whether that is also the *working set* is {UNKNOWN}, and that is the number that
|
|
decides the question: weights on disk are not the memory the process holds, and
|
|
on CPU the gap between them is what decides whether the box can run it. I did not
|
|
measure it, so I am not going to reason about it here as though I had.
|
|
|
|
## 3. Input contract
|
|
|
|
`POST /v1/tts` — all fields are form fields, except `ref_audio` which is a file
|
|
upload.
|
|
|
|
| Field | Default | Notes |
|
|
|---|---|---|
|
|
{chr(10).join(f"| `{k}` | {v.get('default')!r} | {v.get('note', '')} |"
|
|
for k, v in runtime['input_fields'].items())}
|
|
|
|
**Four generation paths**, selected by which fields are present; `prompt_slug`
|
|
wins if the cached prompt exists:
|
|
|
|
{chr(10).join(f"- **{k}** — {v}" for k, v in runtime['generation_paths'].items())}
|
|
|
|
**Maximum input length:** {UNKNOWN}. This server enforces none, and I did not
|
|
measure the model's own limit. Treat as unknown rather than unbounded — an
|
|
over-length input is not the same as a long input.
|
|
|
|
**Streaming:** {runtime['output']['shape']}. Not a stream.
|
|
|
|
## 4. Output contract
|
|
|
|
Measured from a file this system produced, not from a config value.
|
|
|
|
| | |
|
|
|---|---|
|
|
| Container / codec | {out.get('container')} / {out.get('codec')} |
|
|
| Sample rate | {out.get('sample_rate_hz')} Hz |
|
|
| Channels | {out.get('channels')} |
|
|
| Bit depth | {out.get('bit_depth_bits')} bit |
|
|
| Bit rate | PCM; follows from rate x depth x channels |
|
|
| Source | `{out.get('measured_from')}` |
|
|
|
|
{doc['output_contract']['consistency_note']}
|
|
|
|
## 5. Determinism
|
|
|
|
**Identical by default: {doc['determinism']['identical_by_default']}.**
|
|
|
|
{json.dumps(doc['determinism']['why'])}
|
|
|
|
- Seeded call reproducible: **{doc['determinism']['seeded_reproducible']}** —
|
|
{doc['determinism']['seeded_reproducible_why']}
|
|
- Seeded when supplied: {', '.join(f'`{s}`' for s in doc['determinism']['seeded'])}
|
|
- Not seeded: {', '.join(f'`{s}`' for s in doc['determinism']['unseeded_sources'])}
|
|
|
|
**{doc['determinism']['consequence']}**
|
|
|
|
## 6. Language coverage
|
|
|
|
- Declared by upstream: {', '.join(doc['languages']['declared_by_upstream'])}
|
|
({doc['languages']['count']} languages)
|
|
- Selection: {doc['languages']['selection']}
|
|
- **Quality ranking: {UNKNOWN}.** {doc['languages']['quality_weaknesses_why']}
|
|
|
|
**Known trap, from a real bug here:** {json.dumps(doc['languages']['known_weak']['en'])}
|
|
|
|
## 7. Voice identity
|
|
|
|
The chain:
|
|
|
|
{chr(10).join(f"{i + 1}. {step}" for i, step in enumerate(doc['voice_identity']['chain']))}
|
|
|
|
**What the Edge voice actually is:**
|
|
{doc['voice_identity']['what_the_edge_voice_actually_is']['answer']}.
|
|
|
|
It is **not** {', **not** '.join(doc['voice_identity']['what_the_edge_voice_actually_is']['not'])}.
|
|
Utterances are rendered by Qwen, conditioned on the vmatrix extracted from the
|
|
reference audio; the Edge voice id never reaches the synthesis call.
|
|
|
|
{doc['voice_identity']['what_the_edge_voice_actually_is']['why']}
|
|
|
|
**{doc['voice_identity']['what_the_edge_voice_actually_is']['consequence']}**
|
|
|
|
| | |
|
|
|---|---|
|
|
| `tts_engine_used: "qwen"` | {doc['voice_identity']['tts_engine_used_semantics']['qwen']} |
|
|
| `tts_engine_used: null` | **{doc['voice_identity']['tts_engine_used_semantics']['null']}** |
|
|
|
|
Measured on Deal Hunter:
|
|
|
|
| | |
|
|
|---|---|
|
|
| Reference WAVs on disk | {counts['reference_wavs_on_disk']} |
|
|
| With recorded provenance | {counts['refs_with_recorded_provenance']} |
|
|
| **With unknown provenance** | **{counts['refs_with_unknown_provenance']}** |
|
|
| Cached voice prompts (vmatrix) | {counts['cached_voice_prompts']} |
|
|
| Prompts with no reference WAV | {counts['prompts_without_a_reference_wav']} |
|
|
| Speakers with an Edge voice assigned | {counts['speakers_with_an_edge_voice_assigned']} |
|
|
|
|
{doc['voice_identity']['unknown_provenance_is_the_norm']}
|
|
|
|
**This is a correction to how the mapping reads a field.** The bundle's own
|
|
convention says "null means absent". In `tts_engine_used` it means *unknown*, and
|
|
the mapping should carry it as unknown rather than resolving it to "edge".
|
|
|
|
{doc['voice_identity']['orphan_prompts_note']}
|
|
|
|
## 8. Failure modes
|
|
|
|
| Condition | What happens | Where it surfaces |
|
|
|---|---|---|
|
|
| Server unreachable | `QwenTTSUnavailableError` after retries | `tts_engine_fallback_reason` |
|
|
| Server down, building a reference | Edge audio into the ref path | `voice_engines.json` |
|
|
| Unknown `prompt_slug` | HTTP 404 | request fails; not regenerated |
|
|
| `ref_audio` missing on `/v1/voice-prompt` | HTTP 400 | request fails |
|
|
| Generation hangs | watchdog restarts the server process after 1800 s | server log |
|
|
| CUDA OOM / model load failure | **unknown** | never run on CUDA in this system |
|
|
|
|
Over-length text: **unknown** — no limit is enforced, and the model's own limit
|
|
was not measured.
|
|
|
|
## 9. Runtime surface
|
|
|
|
| | |
|
|
|---|---|
|
|
| Transport | HTTP, binds `0.0.0.0:8881` |
|
|
| **Authentication** | **{_auth(runtime)}** |
|
|
| Mechanisms searched for | {_join(runtime['authentication']['mechanisms_searched'])} |
|
|
| Mechanisms found | {_join(runtime['authentication']['mechanisms_found']) or 'none'} |
|
|
| Concurrency | **{runtime['concurrency']['concurrent_generations']}** (`_model_lock`) |
|
|
| Watchdog | {runtime['concurrency']['watchdog_restart_seconds']} s |
|
|
| Client timeout | `REMOTE_TIMEOUT = 900` s |
|
|
| Endpoints | {', '.join(f"`{v['method']} {v['path']}`" for v in runtime['endpoints'].values())} |
|
|
|
|
{runtime['authentication']['note']}
|
|
|
|
**Concurrency limit: {runtime['concurrency']['concurrent_generations']}.**
|
|
{_why(runtime)}. A `speak` worker that fans out
|
|
will serialise here regardless of its own concurrency, and queue behind anything
|
|
already generating.
|
|
|
|
---
|
|
|
|
## What I could not measure
|
|
|
|
Stated once, plainly, rather than repeated as a footnote:
|
|
|
|
1. **Working set / VRAM at load and at inference.** Needs resident-memory
|
|
sampling on the target hardware.
|
|
2. **Whether a seeded call is reproducible** across MPS kernels, model reloads,
|
|
and attention implementations.
|
|
3. **Per-language quality.** No per-language evaluation exists in this system.
|
|
4. **The model's own maximum text length.**
|
|
5. **CUDA failure modes.** Never exercised here; this system runs MPS.
|
|
|
|
Each is unknown for a reason, not because it was not worth measuring. If one of
|
|
them blocks `speak`, say which and I will run the experiment rather than
|
|
substitute a guess.
|
|
"""
|
|
|
|
|
|
def main() -> int:
|
|
parser = argparse.ArgumentParser(description=__doc__,
|
|
formatter_class=argparse.RawDescriptionHelpFormatter)
|
|
parser.add_argument("--out-dir", default=str(REPO / "docs" / "tts"))
|
|
parser.add_argument("--json-only", action="store_true")
|
|
args = parser.parse_args()
|
|
|
|
out = Path(args.out_dir)
|
|
out.mkdir(parents=True, exist_ok=True)
|
|
document = build()
|
|
|
|
payload = json.dumps(document, indent=2, sort_keys=True) + "\n"
|
|
(out / "tts-engine.json").write_text(payload)
|
|
if not args.json_only:
|
|
(out / "tts-engine.md").write_text(render_markdown(document))
|
|
|
|
print(f"tts-engine.json {(out / 'tts-engine.json').stat().st_size} bytes")
|
|
if not args.json_only:
|
|
print(f"tts-engine.md {(out / 'tts-engine.md').stat().st_size} bytes")
|
|
print(f"content_hash {document['content_hash']['value']}")
|
|
return 0
|
|
|
|
|
|
if __name__ == "__main__":
|
|
raise SystemExit(main())
|