2cafbabe82
This was the one open prerequisite for any target-language audio lane: the survey in spec 26 says the model covers Chinese, but that is a model-card claim, and our whole voice catalogue is English references. Nothing had been measured about what happens when an English reference is asked to speak Chinese. scripts/probe_cross_language_tts.py now also takes an optional second English reference and makes the identity question differential rather than a transcription question - a transcript has no timbre. Two English references reading the same text should stay as far apart when they read Chinese as when they read English; if they do, the reference is carrying identity into the Chinese output rather than being ignored. The measurement is made with ECAPA-TDNN cosine similarity. The result, on the real Deal Hunter narrator and chief-of-the-guard references, is in logs/cross_language_tts_probe/probe-report.json: - the English reference speaks Chinese. Whisper hears zh on every Chinese-text clip and en on every English-text clip, recovering 12 of the source's 22 distinct characters. - the text's script decides the language, not the `language` parameter. Chinese text with language=english still comes back as Chinese, so a caller that trusted the parameter to select a language would get the wrong phonology without an error. - the reference is carried: different voices are 0.383 apart reading English and 0.369 apart reading Chinese. What the measurement does not establish is stated in the spec rather than glossed: ECAPA is an English VoxCeleb model, so its Chinese embeddings are out of domain and the same-speaker/different-speaker separation collapses in Chinese. It can show that two clips differ; it cannot certify that two clips are the same speaker across a language boundary. How much of the identity survives is still a listening question, and the clips are saved for it. Spec 26 gains the measurement and a sixth open question - whether the identity survives for all 61 speakers, not just the two measured. Spec 27 gains the parameterisation record (UmbraTrans 3f05489) and a measurement of what the Chinese edition actually contains: 4,305 delivery rows, of which the 90 not-voiced rows are 44 normal collapses and 46 drift-dropped paragraphs covering 33 paragraphs and 356 source words, about 2.4 minutes of speech. Two of the dropped paragraphs are upstream translation defects the aligner cannot fix - ch-15-p0100 and ch-25-p0089 carry text that belongs to a different sentence - so fixing them means re-translating. Suite: 913 passed, 8 failed (the same pre-existing pronunciation and voice_profile_manager accent failures), 6 skipped.
UmbraAudio Specifications
Index of all specification documents for the UmbraAudio audiobook generation system.
Specifications
| File | Topic | Description |
|---|---|---|
| 00-project-overview.md | Architecture | Pipeline overview, tech stack, data layout |
| 01-parsing.md | Pass 1a | EPUB, PDF, TXT, MD manuscript parsing |
| 02-analysis.md | Pass 1b | BookNLP, character profiling, pronunciation |
| 03-voice-design.md | Pass 2 | Per-character voice reference generation |
| 04-mapping.md | Pass 3 | Text segmentation, speaker attribution, manifests |
| 05-generation.md | Pass 5 | Audio generation, caching, retry, CUDA management |
| 06-normalization-watermarking.md | Post-gen | Audio normalization, ACX compliance, Perth watermark |
| 07-assembly-export.md | Export | M4B, MP3, WAV export via FFmpeg |
| 08-database.md | Database | SQLite schema, entity relationships, CRUD API |
| 09-web-review-ui.md | Web UI | FastAPI + HTMX review interface |
| 10-cli-orchestrator.md | CLI | Click commands, full pipeline orchestration |
| 11-configuration-dependencies.md | Config | Pydantic settings, dependencies, build system |
| 12-quality.md | Quality | MOS, WER, consistency, ACX compliance |
| 18-UmbraTrans.md | Multilingual | Meaning-preserving translation, target voices, and multilingual audiobook production |
| 19-enterprise-control-plane.md | Enterprise | Multi-tenant projects, access control, artifact lineage, and commercial voice library |
| 20-enterprise-database-integration.md | Enterprise DB | Shared-PostgreSQL contract, blockers, cutover phases P1-P4 |
| 21-umbra-system-nodes-tasks-and-changes.md | System | Worker nodes, task assignment, cross-product change management, UI topology |
| 22-releases-and-the-publish-flag.md | Release | The publish flag, pinned artifact sets, post-publication change, and the Deal Hunter lesson |
| 23-speaker-duplicate-detection.md | Speakers | Duplicate-speaker scan, approval-safe merging, and the crowd veto |
| 24-analysis-quality-plan.md | Analysis | Why the ch6 speakers were missed, and a ranked plan to improve attribution |
Quick Reference
Pipeline Passes
1a: Parse → chapters table
1b: Analyze → characters, pronunciation_guide
2: Voices → voice_ref WAVs
3: Map → manifests/*.json
4: Test → sample audio (optional)
5: Generate → audio/*.wav, audio_normalized/*.wav
Export → export/*.m4b, *.mp3
Key Entry Points
| Component | File | Class/Function |
|---|---|---|
| CLI | cli.py |
main Click group |
| Pipeline | orchestrator.py |
run_pipeline() |
| TTS | voices/chatterbox.py |
ChatterboxEngine |
| Segmenter | mapping/segmenter.py |
Segmenter |
| Generator | generation/chapter.py |
ChapterGenerator |
| Normalizer | assembly/normalize.py |
AudioNormalizer |
| Web UI | web/__init__.py |
create_app() |
| Database | database.py |
Database |