332aeb4b65
The per-language narrator references were wrong. Generating a VoiceDesign voice per language invents a different narrator each time: measured 0.77-0.96 band-energy similarity against the speaker page's own narrator_ref.wav, so the voice changed with the language. That is the opposite of using the vmatrix, and it is why the narrator did not sound the same across languages. The narrator now uses narrator_ref.wav in every language. The language parameter supplies phonology; the single reference supplies identity. That reference is what the vmatrix was built from, so it is the same voice - just at x_vector resolution rather than the 8k matrix, which is the only form this machine can run (one prompt_slug request takes the server 5.4 GB -> 19.1 GB and never returns). Cross-language narrator similarity, same utterance, all 10 languages: per-language designed refs : mean 0.945, min 0.795 single speaker-page ref : mean 0.973, min 0.910 The previous render is kept in Lang_test_perLangRefs for comparison. 120 clips re-rendered, 0 failed. Note the proxy is a band-energy fingerprint, not a true speaker embedding, and it cannot judge accent - only timbre consistency. Suite: 721 passed, 8 pre-existing.
UmbraAudio
Quality-first multi-pass audiobook generation from English manuscripts.
Features
- 5-pass pipeline: Analysis → Voice Design → Mapping → Test → Generate
- Multi-engine TTS: Chatterbox (primary), Qwen3-TTS (secondary)
- LLM-enhanced: BookNLP for NLP, LLM for profiling/emotion/pronunciation
- Web review: FastAPI + HTMX at checkpoints for human approval
Quick Start
python -m umbraaudio.cli init <manuscript.epub> --name 'My Book'
python -m umbraaudio.cli analyze
python -m umbraaudio.cli voices
python -m umbraaudio.cli review
Requirements
- Python 3.11+
- NVIDIA GPU with CUDA support (RTX 3060 or better recommended)
- ffmpeg
Development
pip install -e ".[dev]"
ruff check umbraaudio/
pytest tests/ -v
Commit cadence: commit at least once every Sunday. Never leave a week (let alone a month) of work uncommitted — a single accidental
git checkouton 2026-09-20 wiped out a month of changes toumbraaudio/web/routes.py. SeeAGENTS.md→ Commit Cadence.
scripts/commit_check.sh enforces the nudge: it exits non-zero once the tree has been dirty
longer than a week. The LaunchAgent com.umbraaudio.commitcheck runs it every Sunday at
18:00 and posts a macOS notification when there is uncommitted work.
scripts/commit_check.sh # warn if dirty > 7 days
scripts/commit_check.sh --days 3 # tighter threshold
scripts/commit_check.sh --any # warn on any uncommitted change
Rebuilding from source
The repo tracks source only; large runtime data is reconstructed as needed.
-
Models: weights are not committed (see
.gitignore). Download them:bash scripts/download_models.sh # ~9 GB, targets ./models -
Qwen3-TTS is a git submodule pinned to upstream commit
022e286:git submodule update --init --recursiveThe
tts_server/web-demo backend includes a local UI customization toqwen_tts/cli/demo.py. It is preserved as a patch — reapply it after checking out the submodule:cd Qwen3-TTS && git apply ../scripts/qwen3_tts_demo_umbra_patch.patch -
Environment: copy/keep
.env(contains secrets) and setUMBRA_TTS__*model paths undermodels/.
Book projects (e.g. DealHunterShip_...) are committed separately under their
own project directories and are not part of this program repository.