f381c58160
Every generation branch hardcoded language="english", and the endpoint accepted no language parameter at all. German, French and the rest were synthesised with English phonology and prosody - which is why the German samples sounded like an English narrator reading German. - /v1/tts now accepts a language form field, passed to all four branches. Default stays "english" so the existing English pipeline is unchanged; the renderer sends the target language from the model's own codec_language_id keys (exactly ten: chinese english french german italian japanese korean portuguese russian spanish). - A/B on one German line with the same cloned voice: language=english gives 4.56s, language=german 4.08s, waveforms differing by 151% of signal amplitude. Audibly different renditions, not a no-op. Also documented the ICL memory finding in the server comments: the branch that consumes a cached prompt leaves x_vector_only_mode at its default of False, so it decodes the 8k ref_code matrix on every request. One such request takes the footprint from 5.4GB to 19.1GB and does not return. The ref_audio branch passes x_vector_only_mode=True and is stable - 120 clips, 0 failures, 13.1GB. Note the language fix does NOT address that; it was tested and still blows up with language=german. Separate problem. qwen_tts_server.py stays at its 8 pre-existing ruff errors. Suite: 721 passed, 8 pre-existing.
UmbraAudio
Quality-first multi-pass audiobook generation from English manuscripts.
Features
- 5-pass pipeline: Analysis → Voice Design → Mapping → Test → Generate
- Multi-engine TTS: Chatterbox (primary), Qwen3-TTS (secondary)
- LLM-enhanced: BookNLP for NLP, LLM for profiling/emotion/pronunciation
- Web review: FastAPI + HTMX at checkpoints for human approval
Quick Start
python -m umbraaudio.cli init <manuscript.epub> --name 'My Book'
python -m umbraaudio.cli analyze
python -m umbraaudio.cli voices
python -m umbraaudio.cli review
Requirements
- Python 3.11+
- NVIDIA GPU with CUDA support (RTX 3060 or better recommended)
- ffmpeg
Development
pip install -e ".[dev]"
ruff check umbraaudio/
pytest tests/ -v
Commit cadence: commit at least once every Sunday. Never leave a week (let alone a month) of work uncommitted — a single accidental
git checkouton 2026-09-20 wiped out a month of changes toumbraaudio/web/routes.py. SeeAGENTS.md→ Commit Cadence.
scripts/commit_check.sh enforces the nudge: it exits non-zero once the tree has been dirty
longer than a week. The LaunchAgent com.umbraaudio.commitcheck runs it every Sunday at
18:00 and posts a macOS notification when there is uncommitted work.
scripts/commit_check.sh # warn if dirty > 7 days
scripts/commit_check.sh --days 3 # tighter threshold
scripts/commit_check.sh --any # warn on any uncommitted change
Rebuilding from source
The repo tracks source only; large runtime data is reconstructed as needed.
-
Models: weights are not committed (see
.gitignore). Download them:bash scripts/download_models.sh # ~9 GB, targets ./models -
Qwen3-TTS is a git submodule pinned to upstream commit
022e286:git submodule update --init --recursiveThe
tts_server/web-demo backend includes a local UI customization toqwen_tts/cli/demo.py. It is preserved as a patch — reapply it after checking out the submodule:cd Qwen3-TTS && git apply ../scripts/qwen3_tts_demo_umbra_patch.patch -
Environment: copy/keep
.env(contains secrets) and setUMBRA_TTS__*model paths undermodels/.
Book projects (e.g. DealHunterShip_...) are committed separately under their
own project directories and are not part of this program repository.