umbragit/UmbraAudio
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Weekly commit: attribution QA, dramatic variations, GPU TTS, AR export

Browse Source 
Squashes ~3 weeks of uncommitted work (2026-09-03 → 09-21). The repo is the
only durable copy; a single accidental `git checkout` on routes.py nearly lost
a month of work (recovered from an editor snapshot, not git). See AGENTS.md →
"Commit Cadence" — commit every Sunday.

Attribution & QA
- New umbraaudio/analysis/attribution_qa.py: mechanical check_attribution /
  fix_high_confidence / verify_conversation_flow (speech-tag mismatch, narrator
  dialogue, role-tag guessing) plus scripts/qa_attribution.py CLI.
- llm_attributor prompt rules 3e-3h (unnamed roles, pronoun coreference,
  repeated lines) with wider scene/position context.
- Per-utterance QA box in chapter audio detail UI: hover issues, Recheck
  (auto-fix loop) and Accept (persisted qa_accepted* + text/speaker signature,
  auto re-surfaces when either changes). New POST /api/utterance/{id}/recheck
  and /api/utterance/{id}/qa-accept. _ensure_utterance_columns migration fixes
  legacy DBs that lacked the columns.

Generation
- Dramatic utterance variations: 26 emotions + pace + custom instruct,
  composed into every TTS voice mode (clone/icl/ref/embedding) and all
  generation sites; markers record the instruct so edits re-trigger.
- Staleness markers are JSON {speaker,text}; legacy plain-text markers always
  stale.
- Scene-break segmentation (generation/scene_break.py) + resegment merge.
- AR package exporter (assembly/ar_export.py) with tests.
- Voice ref resolution: rglob refs, canonical alias / first-word fallback.
- Edge-TTS fallback visibility: durable voice_engines.json ledger + UI badge.

GPU / TTS server
- qwen_tts_server: instruct + clone/icl support (voice_description →
  instruct_ids prepended to the talker input).
- qwen_tts.py remote path sends voice_description alongside voice_ref/slug.

Config / ops
- .env: UMBRA_TTS__QWEN_TTS_URL=http://localhost:8881.
- .gitignore: book projects, tmp_out, logs, one-off samples A/B demo.
- AGENTS.md + README.md: Commit Cadence (commit at least weekly, on Sunday).

New tests: test_qa_acceptance, test_variations, test_ar_export,
test_scene_break, test_resegment_merge, test_vocals_project. Suite: 327 pass
(6 pre-existing voice-profile accent-wording failures, unrelated).
This commit is contained in:
B.D. Murphy
2026-09-21 06:44:22 -07:00
parent 85b1137a2e
commit df36dad8aa
62 changed files with 7145 additions and 564 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
.env
View File

@@ -13,6 +13,7 @@ UMBRA_TTS__MAX_RETRIES=3
UMBRA_TTS__RETRY_DELAY=2.0
UMBRA_TTS__VOICEDESIGN_MODEL=/Volumes/1TB_APL_Storage/UmbraAudio/models/Qwen3-TTS-12Hz-1.7B-VoiceDesign
UMBRA_TTS__BASE_MODEL=/Volumes/1TB_APL_Storage/UmbraAudio/models/Qwen3-TTS-12Hz-1.7B-Base
UMBRA_TTS__QWEN_TTS_URL=http://localhost:8881
# --- LLM ---
UMBRA_LLM__PROVIDER=auto

10
.gitignore vendored
View File

@@ -34,6 +34,16 @@ umbraaudio_package.zip
# Book projects are committed separately (not part of this program repo)
/DealHunterShip_4.0e_whole-to-audio/
/Last Dwarf/
/Treasure Island/
/Vocals/
# App runtime data (per-project book data; projects committed separately)
/.umbra/
# Scratch / temp artifacts
/log.log
/tmp_out/
# One-off voice A/B demo (large binary WAVs, not wired into the app)
/umbraaudio/web/static/samples/

198
AGENTS.md
View File

@@ -1,5 +1,80 @@
# UmbraAudio — Development Conventions
## Commit Cadence (do not skip)
- **Commit at least once every Sunday — never let a week of work sit uncommitted.** On
2026-09-20 a single accidental `git checkout -- umbraaudio/web/routes.py` destroyed ~2,300
lines of a month's uncommitted work; the file only survived because OpenCode keeps its own
editor snapshots, *not* because of git. Do not rely on that again.
- End-of-Sunday ritual: `git status` → stage the **code** (source, tests, specs, scripts,
config) → `git commit`. Book projects, `tmp_out/`, logs and the one-off
`umbraaudio/web/static/samples/` A/B demo are intentionally ignored (see `.gitignore`);
never `git add -A` without checking, and never leave a dirty `umbraaudio/web/routes.py`
for days.
## System & Deployment Notes
- **macOS 26.6.2, home relocated**: user `worstauthor`'s home dir is
`/Volumes/Dock/worstauthor` (dscl + `$HOME`); `/Users/worstauthor` still exists as a
legacy/boot-volume path and is fine for scratch scripts. launchd scans the relocated home's
`Library/LaunchAgents`.
- **Web UI access**: `umbraudio.bdmurphy.me` is served by a **Caddy reverse proxy on ANOTHER
machine on the LAN**, forwarding to this Mac — so the web server MUST bind `0.0.0.0` (it does,
port **8420**). LAN IPs: `192.168.123.12` (en0) and `192.168.123.154` (en1). macOS App Firewall
is **disabled** (inbound 8420 open). Any new firewall/user grant must keep `0.0.0.0:8420` open.
- **launchd PATH is bare** (`/usr/bin:/bin:/usr/sbin:/sbin`): anything depending on Homebrew tools
(notably **ffmpeg** -> m4b/MP3 export) fails with `[Errno 2] No such file or directory: 'ffmpeg'`
when run under the web server. `umbraweb_login.sh` now exports
`PATH="/opt/homebrew/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin"` before exec'ing python —
keep that export if editing the launcher. A server started before 2026-09-09 needs a restart to
pick it up.
- **macOS 26 external-volume restriction (critical)**: processes NOT launched from an approved
context (everything spawned by launchd/cron — shells like `/bin/zsh`/`cat`, system `python3`)
get `EPERM: Operation not permitted` on any read/write of `/Volumes/1TB_APL_Storage`. Only the
**venv python binary** (`.venv/bin/python`) is volume-approved, and only in its own process —
verified: under launchd it can `import umbraaudio`, `open()`/write project files, but a shell
cannot even open a *script* on that volume (`/bin/zsh: can't open input file`), and launchd
dies with `EX_CONFIG 78` if a job's StandardOut/ErrorPath points at the volume.
Consequence: **auto-start must keep ALL project I/O inside `python -c '...'` — never let a
shell open an external-volume script, never point launchd stdio at the volume.**
- **Login auto-start (not boot)**: LaunchAgent `com.umbraaudio.web` (`RunAtLoad` + `KeepAlive`,
crash → auto-restart) runs `/bin/zsh /Users/worstauthor/.umbraaudio/umbraweb_login.sh`, which
`cd`s into the project and `exec .venv/bin/python -c "<inline uvicorn>"`, with python `dup2`-
ing stdout/stderr into `logs/web_server.log`. Plist lives at `/Volumes/Dock/worstauthor/
Library/LaunchAgents/com.umbraaudio.web.plist` (login-scanned) AND `/Users/worstauthor/
Library/LaunchAgents/com.umbraaudio.web.plist` (manual `launchctl load`). Starts when
`worstauthor` logs in. For headless/pre-login (boot) start: a root LaunchDaemon
(`sudo launchctl bootstrap system`) using the same python-inline pattern is the proven route.
- **Rejected/unused**: boot-time LaunchAgent/LaunchDaemon with shell-opened external scripts or
external-volume stdio (TCC-blocked, EX_CONFIG/127); cron (same TCC denial); a `sudoers.d`
NOPASSWD-to-`worstauthor` bridge was trialed and removed — unnecessary once everything stays in
the venv python. Test plists/probes were cleaned up.
- **Qwen TTS server (port 8881) is MANUAL start only** — the login LaunchAgent starts only the
web server. After any reboot the Qwen server is down and speaker "re-voice" silently degrades
to Edge-TTS (web log: `Qwen3-TTS voice reference failed, falling back to Edge-TTS: Cannot
connect to Qwen TTS server at http://localhost:8881`), which cannot do raspy/texture and may
pick an arbitrary voice (e.g. a male for a female-low request). Restore with
`nohup .venv/bin/python scripts/qwen_tts_server.py` (matches the prover-2821 pattern; LLM and
Qwen must not be co-resident on the 16 GB Mac — wait until any LLM attribution batch is done).
- **GPU Qwen3-TTS box (192.168.123.14) + Local Network TCC (critical)**: macOS 15+'s **Local
Network privacy** blocks launchd-spawned user processes (incl. the web server) from ANY LAN
IP — they get `EHOSTUNREACH` ("No route to host") while GUI/terminal processes reach the LAN
fine. Loopback is NOT blocked. Workaround (deployed): a **root LaunchDaemon relay**
(`/Library/LaunchDaemons/com.umba.qwen-relay.plist` + `qwen_relay.py`,
`sudo launchctl bootstrap system ...`) forwards `127.0.0.1:18881 -> 192.168.123.14:8881`;
the web app targets `UMBRA_TTS__QWEN_TTS_URL=http://127.0.0.1:18881`. Root system daemons DO
bypass Local Network TCC (verified LAN_OK 200). Box runs `scripts/qwen_tts_server.py` with
`QWEN_DEVICE=cuda` on an RTX 3060 (12 GiB), models at `/home/bm/umba/models/…`, prompt cache at
`/home/bm/umba/voice_prompts`, deps in `/home/bm/umba/venv` (torch 2.11.0+cu128,
accelerate==1.12.0, python-multipart + onnxruntime REQUIRED), started manually via
`nohup ./gpu_server_env.sh`. Box GPU shared with an LM Studio LLM (~6.6 GiB); when TTS runs, the
LLM must be unloaded first (`curl -X POST http://192.168.123.14:1234/api/v1/models/unload -d
'{"instance_id":"qwen/qwen3.5-9b"}'`) or model swaps OOM and the connection drops
("Server disconnected without sending a response." → Edge fallback). **Relay bug history**: a
`socket.create_connection(..., timeout=10)` left a 10s recv timeout on upstream — any TTS call
>10s was killed mid-response while `/health` and `/load` survived; fixed with `up.settimeout(None)`
after connect. Test plists/probes (`com.umba.netcheck`, `netpost`, `rootnetcheck`) were bootstrapped
into launchd and cleaned up; after any of these env changes, restart web via
`launchctl kickstart -k "gui/$(id -u)/com.umbraaudio.web"`.
## Project Overview
Quality-first multi-pass audiobook generation from English manuscripts.
Each pass has quality gates. Quality is the primary concern; generation speed is secondary.
@@ -7,7 +82,7 @@ Each pass has quality gates. Quality is the primary concern; generation speed is
## Architecture
- **5-pass pipeline**: Analysis → Voice Design → Mapping → Test → Generate
- **Multi-engine TTS**: Edge-TTS (primary for voice variety), Chatterbox (secondary for cloning)
- **LLM-enhanced**: gemma-4-12b-it-qat for character discovery and profiling
- **LLM-enhanced**: google/gemma-4-12b-qat for character discovery and profiling
- **Web review**: FastAPI + HTMX at checkpoints for human approval
## Voice System
@@ -65,6 +140,84 @@ Each pass has quality gates. Quality is the primary concern; generation speed is
## Current Session Summary
### Session: Xylia hiss fixed (post-TTS white-noise) + GPU box live
- Xylia's "hissing at the end" was NOT the TTS: her `vocal_traits` pile in `speakers.json`
(LLM-discovery dump incl. "raspy"/"gravelly") triggered `TRAIT_EFFECTS["raspy"]` =
**`noise_inject` (uniform white noise @0.04) + pitch_jitter applied POST-TTS** to every
utterance — most audible as hiss when speech stops. Measured E>6 kHz: ref 18%, her clip 33.8%,
raw clone w/o effects 19.4%, after fix 16.6% (tail rms 0.001 = calm stop). Fixed by cleaning
Xylia's `vocal_traits` to `["deep","measured","ancient","highly articulate"]` (+ kept
`distortion_effects: []`) — no effect matched; the Qwen ref/ICL already carries the gravel.
She was the ONLY speaker whose traits matched effects (audit over all speakers). Existing
Xylia utterances across ch24–46 still carry the old noise layer until regenerated; ch1 sample
utt_0012 regenerated clean (16.6% HF). Backup: `/tmp/speakers.json.bak`.
### Session: Dramatic utterance variations (emotion/pace/custom instruct)
**Goal**: The per-utterance Emotion/Pace/Custom-instruct controls in the chapter UI didn't
"change from neutral". Root cause: **variations were dropped, not weak** — every voice with a
ref clones via `generate_voice_clone`, which never received the instruct, and `pace` was read
nowhere.
**Root cause chain**: `_remote_generate` (`qwen_tts.py`) only sent `voice_description` in the
no-ref branch → any speaker with a `_ref.wav` (narrator + all 50 characters) ignored emotion/
instruct; `generate_voice_clone` (emb = `_clone`) and ICL (`_icl`) in `qwen_tts_server.py` had no
instruct slot; `pace` never consumed. Model already supports instruct+clone simultaneously via
`instruct_ids` prepended to the talker input (`modeling_qwen3_tts.py:2076`).
**Fixes**:
- `scripts/qwen_tts_server.py`: `_icl` + `_clone` accept `voice_description` → build `instruct_ids`
via `model._tokenize_texts([model._build_instruct_text(s)])[0]` and pass through `**kwargs`
(flows via `_merge_generate_kwargs` → `model.generate(instruct_ids=...)`).
- `qwen_tts.py _remote_generate`: sends `voice_description` alongside `voice_ref`/`prompt_slug`
(server keeps speaker identity, instruct bends delivery).
- `generation/chapter.py`: **26 emotions** (`EMOTION_OPTIONS`: original 18 + `tense`/`urgent`/
`horror` + drama tiers `furious`/`joyful`/`terrified`/`griefstricken`/`shocked`), all presets
rewritten as short directorial directives (≤100 char, ruff E501); `PACE_OPTIONS` slow/moderate/
fast + `PACE_INSTRUCT`; new `compose_utterance_instruct(instruct, emotion, pace)` — neutral/
moderate contribute NOTHING so no-variation clones stay natural; `_build_utterance_instruct`
composes base profile + custom + emotion + pace.
- All three generation sites use the composer and pass instruct through **every** voice mode:
`chapter_builder.py` (embedding/prompt_slug/ref branches), `scripts/generate_all_audio.py`,
`routes.py` single-utt `/api/utterance/{id}/generate`. `write_marker` now records the instruct
(staleness regenerates on variation changes).
- UI `chapter_audio_detail.html`: emotion dropdown now lists all 26 options.
- **Verification**: A/B on narrator clone (`/tmp/var_{neutral,angry,furious_fast,slow}.wav`) —
angry hotter/louder (mean −20.6 vs −23.5 dB, peak −3.6 vs −6.5), furious+fast ~8% shorter +
darker, slow distinct; audible delivery change while voice identity holds. New
`tests/test_variations.py` (14 tests: composer, sync checks, remote payload carries
`voice_description` with ref). Full suite green; ruff clean; mypy only pre-existing env noise.
- **Full regen — COMPLETE**: cleared all 4101 `*.wav.txt` markers (+ `ch*_full.wav/mp3` and
`ch*_build_status.json`) because newer JSON markers lack `instruct` and would have SKIPPED regen.
Ran `nohup .venv/bin/python scripts/generate_all_audio.py --project DealHunterShip_4.0e_whole-to-audio`.
**Result (09-09 04:18 local): `=== COMPLETE: 4304 generated, 0 failed, 678.6 min ===` (11.3 h).**
All 32 chapters re-stitched fresh; markers now carry `instruct` (variation edits re-trigger regen).
- **`Vocals` project (expression showcase) rebuilt**: Rainbow Passage × all 26 emotions × 2 voices
(Narrator utt0–17, Deal Hunter utt18–35). Cleared its 36 markers and ran
`ChapterBuilder(config, chapter_id=1).run()` in a fresh process (the :8420 web server still held
pre-wiring imports). 36/36 generated, quality passed, 6m12s. Every marker now records the composed
instruct; delivery measurably distinct (angry +4.9 dB mean vs neutral, romantic 8.7 s vs 6.2 s,
whisper soft peak, shock longer/wilder). User confirmed "usable".
- **Transient TTS stall episode (was NOT content)**: regen and a probe both stalled ~20 min at ch4
utt_0002, blocking `_model_lock`. Suspected `©`/`℗` — **falsified**: direct `/v1/tts` with raw
text (27.7 s), exact batch params + `ref_text` + phonetics (18.7 s) both fine. It's the known
random MPS/orphan-generation class (per-token `_local_scalar_dense_mps`→`MPSStream::synchronize`
crawl). Recovery per docs: kill -9 qwen server + restart (`scripts/qwen_tts_server.py`); batch
resumed marker-safe at utt_0002 and completed. If a stall recurs: restart the server, don't
rebuild the batch.
- **AR package rebuilt + verified (auto-watcher)**: `logs/ar_pre_rebuild_snapshot.json` captured the
stale package (built_at 2026-09-08T02:29:59Z + 32 MP3 md5s); on regen completion the one-shot
watcher `/tmp/auto_rebuild_ar.py` ran
`ARPackageExporter(data_dir).build(discover_chapters(data_dir), load_meta(data_dir), force=True)`.
**Verified** (`logs/ar_rebuild_report.json`): `built_at` 2026-09-09T11:21:11Z, 32 tracks,
0 warnings, all 32 tracks changed vs snapshot, `loudness_ok`/`peak_ok` all true, ch4 on-demand
MP3 spot check RMS −20.5 / peak −3.0. Dashboard shows the new `built {{ ar_manifest.built_at }}`
automatically.
- **Operational notes**: web `POST /api/audiobook/export-mp3` only guards web-side generation, NOT
external batch processes — do not click AR export while a `generate_all_audio.py` batch runs (ch12+
would be silently dropped while their `ch*_full.wav` are missing). Watcher-side, completion gate =
batch PID gone + `=== COMPLETE: … 0 failed ===` in `logs/regen_variations.log` + all 32
`ch{id}_full.wav` mtime ≥ regen start; on failure it aborts without rebuilding.
### Goal
Fix speaker-attribution errors so the user can review/accept voices with confidence. Reported
failures in chapter "Salvage" (ch6): utt35 "We have strict orders…" (after "she asks the lead guard…
@@ -198,3 +351,46 @@ future tooling: a per-utterance Whisper-based QA transcription pass over `ch*_fu
### Previous Session (chapter-5 audio fix)
Complete — ch5 audio (markers, staleness, pronunciation, curly punctuation) verified. History below
this section documents the model-coordinator session and speaker-attribution completion.
### Session: Edge-TTS fallback visibility + GPU Qwen3 box probe
**Goal**: the user had no way to know when a voice silently fell back to Edge-TTS, wants Qwen3 used
almost always, and asked whether a GPU box's Qwen3-TTS (192.168.123.14:1234) could serve generation.
**Done (this session)**:
- **Fallback is no longer silent.** `VoiceDesigner.generate_voice_reference` (`voice_designer.py`)
now retries Qwen up to `TTSConfig.max_retries` (default 3, backoff `retry_delay`, tests used 1-2)
and on exhausted attempts logs ERROR "FALLING BACK to Edge-TTS" **and records it durably** in a new
per-project ledger `{project}/.umbra/voice_engines.json`: `{speaker: {engine: qwen|edge, reason,
at, ref_mtime}}`. Ledger written atomically (tmp+replace) next to the `voices/` dir.
- **Web UI indication** (`routes.py` `_load_speakers` + templates): each speaker row shows a red
"⚠ Edge-TTS fallback" badge (title = reason) when the ledger says `edge` AND `ref_mtime` matches the
current ref wav (so stale entries don't flag), else a green "Qwen3-TTS ✓" after a Qwen generation.
`/speakers/partial` also runs a live health check on `config.tts.qwen_tts_url` (via `QwenTTSEngine.
check_health`) and `_speakers_list.html` renders an amber banner *before* generating when Qwen is
unreachable ("will fall back to Edge-TTS") or a red N-voices banner when fallbacks exist.
- Verified: real Qwen gen records engine=qwen; forced bogus-URL run retries 2×, falls back, records
engine=edge with connection reason + matching ref_mtime; both project /speakers pages render (200).
ruff clean on changed regions (routes.py has 116 pre-existing errors, none from these edits);
mypy only pre-existing stubs noise (soundfile/parselmouth/numpy pyi).
- Note: existing refs predate the feature (no ledger) → no badge until next regen. Regenerate any
speaker flagged "edge" after restarting Qwen to get the real Qwen3 voice.
**GPU box (192.168.123.14:1234) probe — findings**:
- It is an **LM Studio server**. `/v1/chat/completions` is a real endpoint; every audio path
(`/v1/audio/speech`, `/tts`, `/v1/tts`, `/audio/speech`) returns LM Studio's catch-all
`{"error":"Unexpected endpoint or method. (POST ...)"}` → **this LM Studio build cannot synthesize
audio**. Catalog only has the **`qwen3-tts-tokenizer` GGUF (Q8_0, 291 MB)** downloaded — the 12 Hz
codec/tokenizer component, NOT a usable full TTS model.
- Our app does NOT speak LM Studio's protocol anyway: `QwenTTSEngine` (`qwen_tts.py`) talks the
custom FastAPI `/v1/tts` + `/v1/voice-prompt` protocol of `scripts/qwen_tts_server.py` (transformers
`Qwen3-TTS-12Hz-1.7B-VoiceDesign`, `QWEN_DEVICE` env selects cpu/mps/cuda, default mps).
- **To use the GPU for real**: deploy `scripts/qwen_tts_server.py` on the GPU box (venv with CUDA
torch + `qwen_tts` pip package + the HF Qwen3-TTS-12Hz-1.7B-VoiceDesign model — local copies already
exist under `models/`; if using LM Studio instead, a TTS-capable LM Studio version + full model set
would still need a client rewrite, so it's NOT worth it). Then point the Mac at it via
`TTSConfig.qwen_tts_url` (env `UMBRA_TTS__QWEN_TTS_URL=http://192.168.123.14:8881`). SSH from this
Mac to the box is denied (key+password) — needs user-provided access or manual provisioning.
- Xylia (Last Dwarf): regenerated lower — median F0 **141.6 → 111.9 Hz** via a pinned
`voice_description` in `speakers.json` ("lowest baritone register possible, much deeper than any
man… light Irish accent"). Pin overrides auto-built description verbatim (`voice_profile_manager.py`
~619) and is preserved on future regenerations.

3
Issues.txt Normal file
View File

@@ -0,0 +1,3 @@
Issues
Split of book - identify chapters, paragraphs and utterances. Split utterances by ".?!" the only three valid end of sentence punctuation.

4
README.md
View File

@@ -32,6 +32,10 @@ ruff check umbraaudio/
pytest tests/ -v
```
> **Commit cadence:** commit at least once every **Sunday**. Never leave a week (let alone a
> month) of work uncommitted — a single accidental `git checkout` on 2026-09-20 wiped out a
> month of changes to `umbraaudio/web/routes.py`. See `AGENTS.md` → *Commit Cadence*.
## Rebuilding from source
The repo tracks source only; large runtime data is reconstructed as needed.

220
scripts/create_vocals_project.py Normal file
View File

@@ -0,0 +1,220 @@
#!/usr/bin/env python3
"""Create the standalone "Vocals" example-vocals project (no upload needed).
The project has a single chapter that speaks the Rainbow Passage once per
emotion variation with the Narrator voice, then repeats the exact same set
with the Deal Hunter voice. Speakers, voice references, and voice profiles are
copied from an existing project's library so both voices match the audiobook.
Usage:
python scripts/create_vocals_project.py [--force] [--generate]
- Without options: build (or leave in place) the Vocals project structure.
- ``--force``: wipe and rebuild the project from scratch.
- ``--generate``: synthesize every variation, stitch the chapter, and export an
MP3 to the project export dir (requires the Qwen TTS server on port 8881).
"""
from __future__ import annotations
import argparse
import json
import shutil
import sqlite3
import subprocess
import sys
from pathlib import Path
from typing import Any
sys.path.insert(0, str(Path(__file__).parent.parent))
from umbraaudio.config import load_config # noqa: E402
from umbraaudio.database import Database # noqa: E402
DEFAULT_SOURCE_PROJECT = "DealHunterShip_4.0e_whole-to-audio"
PROJECT_NAME = "Vocals"
CHAPTER_TITLE = "Rainbow Passage Variations"
RAINBOW_SENTENCES = [
"When the sunlight strikes raindrops in the air, they act as a prism and form a rainbow.", # noqa: E501
"The rainbow is a division of white light into many beautiful colors.",
"These take the shape of a long round arch, with its path high above, "
"and its two ends apparently beyond the horizon.",
"There is, according to legend, a boiling pot of gold at one end.",
"People look, but no one ever finds it.",
"When a man looks for something beyond his reach, "
"his friends say he is looking for the pot of gold at the end of the rainbow.", # noqa: E501
]
# Mirrors the emotion options in the per-utterance Variation editor so every
# variation the narrator/he ro can perform is demonstrated once.
VARIATIONS = [
"neutral", "angry", "sad", "happy", "fearful", "surprised", "whisper",
"pleading", "sarcastic", "authoritative", "thoughtful", "excited",
"anxious", "tired", "romantic", "disgusted", "amused", "nostalgic",
]
SPEAKERS = ["Narrator", "Deal Hunter"]
VOICE_REF_FILES = ["narrator_ref.wav", "deal_hunter_ref.wav"]
def plan_utterances() -> list[dict[str, str]]:
"""One utterance per (speaker, variation) pair, cycling rainbow sentences."""
plan: list[dict[str, str]] = []
for speaker in SPEAKERS:
for i, variation in enumerate(VARIATIONS):
plan.append(
{
"speaker_name": speaker,
"text": RAINBOW_SENTENCES[i % len(RAINBOW_SENTENCES)],
"emotion": variation,
"pace": "moderate",
}
)
return plan
def _copy_library_speakers(source_dir: Path) -> list[dict[str, Any]]:
"""Return the Narrator + Deal Hunter entries from the source speakers.json."""
source_file = source_dir / "speakers.json"
data = json.loads(source_file.read_text())
wanted = {s.lower() for s in SPEAKERS}
return [s for s in data.get("speakers", []) if s.get("primary_name", "").lower() in wanted]
def _copy_voice_references(source_dir: Path, dest_voices: Path) -> list[Path]:
copied: list[Path] = []
for name in VOICE_REF_FILES:
src = source_dir / "voices" / name
if src.exists():
dst = dest_voices / name
shutil.copy2(src, dst)
copied.append(dst)
return copied
def setup(force: bool) -> Path:
source_cfg = load_config(DEFAULT_SOURCE_PROJECT)
cfg = load_config(PROJECT_NAME)
project_dir = cfg.project_dir
if project_dir.exists() and not force:
print(f"Project '{PROJECT_NAME}' already exists at {project_dir} (use --force to rebuild)")
return project_dir
if project_dir.exists():
shutil.rmtree(project_dir)
cfg.ensure_dirs()
speakers = _copy_library_speakers(source_cfg.data_dir)
if len(speakers) != len(SPEAKERS):
found = [s["primary_name"] for s in speakers]
raise SystemExit(f"Source library missing speakers: found {found}")
db = Database(cfg.db_path)
import asyncio
async def _bootstrap() -> int:
await db.connect()
project_id = await db.create_project(
name=PROJECT_NAME, source_path="", source_format="json"
)
await db.update_project_status(project_id, "parsed")
full_passage = " ".join(RAINBOW_SENTENCES)
chapter_id = await db.insert_chapter(
project_id=project_id,
index=0,
title=CHAPTER_TITLE,
source_text=full_passage,
word_count=len(full_passage.split()),
)
await db.close()
return chapter_id
chapter_id = asyncio.run(_bootstrap())
conn = sqlite3.connect(str(cfg.db_path))
for i, utt in enumerate(plan_utterances()):
conn.execute(
"INSERT INTO utterances "
"(chapter_id, utterance_index, text, type, emotion, pace, speaker_name, status) "
"VALUES (?, ?, ?, 'narration', ?, ?, ?, 'pending')",
(chapter_id, i, utt["text"], utt["emotion"], utt["pace"], utt["speaker_name"]),
)
conn.commit()
conn.close()
speakers_file = cfg.data_dir / "speakers.json"
payload = {
"speakers": speakers,
"book_title": PROJECT_NAME,
"total_chapters": 1,
"total_words": sum(len(u["text"].split()) for u in plan_utterances()),
}
speakers_file.write_text(json.dumps(payload, indent=2))
copied = _copy_voice_references(source_cfg.data_dir, cfg.voices_dir)
print(f"Voice references copied: {[p.name for p in copied]}")
profiles_src = source_cfg.data_dir / "voice_profiles.json"
if profiles_src.exists():
profiles = json.loads(profiles_src.read_text())
wanted = {s.lower() for s in SPEAKERS}
kept = {k: v for k, v in profiles.items() if k.lower() in wanted}
if kept:
(cfg.data_dir / "voice_profiles.json").write_text(json.dumps(kept, indent=2))
print(f"Created '{PROJECT_NAME}' project at {project_dir}")
plan = plan_utterances()
print(f"Chapter {chapter_id}: {CHAPTER_TITLE} ({len(plan)} utterances, "
f"{len(plan) // len(SPEAKERS)} variations per speaker)")
return project_dir
def export_chapter_mp3() -> None:
"""Stitch (if needed) and converts the single chapter to a 192k MP3."""
cfg = load_config(PROJECT_NAME)
db = sqlite3.connect(str(cfg.db_path))
chapter_id = db.execute("SELECT id FROM chapters ORDER BY id LIMIT 1").fetchone()[0]
db.close()
chapter_dir = cfg.data_dir / "chapter_audio"
full_wav = chapter_dir / f"ch{chapter_id}_full.wav"
if not full_wav.exists():
hint = f"stitch ch{chapter_id} or run with --generate first"
raise SystemExit(f"No stitched audio at {full_wav} - {hint}")
cfg.export_dir.mkdir(parents=True, exist_ok=True)
out = cfg.export_dir / f"{PROJECT_NAME}_rainbow_variations.mp3"
result = subprocess.run(
[
"ffmpeg", "-y", "-i", str(full_wav),
"-ac", "1", "-ar", "44100",
"-c:a", "libmp3lame", "-b:a", "192k",
str(out),
],
capture_output=True, text=True,
)
if result.returncode != 0:
raise SystemExit(f"MP3 export failed: {result.stderr[-300:]}")
print(f"Exported {out} ({out.stat().st_size / 1048576:.1f} MB)")
def generate() -> None:
from scripts.generate_all_audio import main as gen_main
gen_main(project=PROJECT_NAME, emotion_fx=True)
export_chapter_mp3()
def main() -> None:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--force", action="store_true", help="rebuild the project from scratch")
parser.add_argument("--generate", action="store_true", help="synthesize audio and export MP3")
args = parser.parse_args()
setup(force=args.force)
if args.generate:
generate()
if __name__ == "__main__":
main()

66
scripts/generate_all_audio.py
View File

@@ -16,13 +16,25 @@ sys.path.insert(0, str(Path(__file__).parent.parent))
from umbraaudio.config import load_config
from umbraaudio.generation.audio_effects import (
EMOTION_FX_VERSION,
apply_effects_to_file,
apply_effects_to_file_by_name,
apply_emotion_effects_to_file,
load_speaker_distortion_effects,
load_speaker_traits,
)
from umbraaudio.generation.chapter import compose_utterance_instruct
from umbraaudio.generation.scene_break import (
is_scene_break,
needs_silence,
write_scene_break_audio,
)
from umbraaudio.generation.staleness import is_current, write_marker
from umbraaudio.pronunciation import apply_phonetics, load_phonetic_names
from umbraaudio.pronunciation import (
load_phonetic_names,
load_proper_names,
prepare_tts_text,
)
from umbraaudio.voices.lmstudio_lifecycle import get_vram_usage_mb
from umbraaudio.voices.qwen_tts import VOICE_REFERENCE_TEXT, QwenTTSEngine
@@ -109,11 +121,13 @@ def generate_chapter(
db_path: Path,
narrator_ref: Path | None = None,
phonetic: dict[str, str] | None = None,
proper_names: frozenset[str] | None = None,
speaker_traits: dict[str, list[str]] | None = None,
distortion_effects: dict[str, list[str]] | None = None,
utterance_callback=None,
progress_callback=None,
progress_context=None,
emotion_fx: bool = False,
) -> tuple[int, int]:
db = sqlite3.connect(str(db_path))
db.row_factory = sqlite3.Row
@@ -125,8 +139,8 @@ def generate_chapter(
title = chapter["title"] or f"Chapter {chapter['chapter_index'] + 1}"
utterances = db.execute(
"SELECT id, speaker_name, text, utterance_index FROM utterances "
"WHERE chapter_id = ? ORDER BY utterance_index",
"SELECT id, speaker_name, text, utterance_index, emotion, pace, instruct "
"FROM utterances WHERE chapter_id = ? ORDER BY utterance_index",
(chapter_id,),
).fetchall()
db.close()
@@ -150,8 +164,13 @@ def generate_chapter(
utterance_callback()
continue
if text.strip() == "<<<>>>":
skipped += 1
if is_scene_break(text):
output_path = chapter_dir / f"utt_{idx:04d}.wav"
if needs_silence(output_path):
write_scene_break_audio(output_path)
generated += 1
else:
skipped += 1
if utterance_callback:
utterance_callback()
continue
@@ -160,9 +179,20 @@ def generate_chapter(
ref_path = resolve_reference(speaker, voice_refs, alias_map, narrator_ref)
spoken_text = apply_phonetics(text, phonetic or {})
spoken_text = prepare_tts_text(text, phonetic or {}, proper_names)
if is_current(spoken_text, output_path, speaker=speaker, voice_ref=ref_path):
instruct = compose_utterance_instruct(
(utt["instruct"] or "").strip() or None,
utt["emotion"],
utt["pace"],
)
fx_version = EMOTION_FX_VERSION if (emotion_fx and utt["emotion"]) else None
if is_current(
spoken_text, output_path, speaker=speaker, voice_ref=ref_path,
instruct=instruct, fx_version=fx_version,
):
skipped += 1
if utterance_callback:
utterance_callback()
@@ -176,12 +206,13 @@ def generate_chapter(
output_path=output_path,
voice_ref=ref_path,
ref_text=VOICE_REFERENCE_TEXT,
instruct=instruct,
)
else:
engine.generate(
text=spoken_text,
output_path=output_path,
instruct="A calm, neutral voice speaking clearly.",
instruct=instruct or "A calm, neutral voice speaking clearly.",
)
if output_path.exists():
@@ -192,7 +223,14 @@ def generate_chapter(
traits = speaker_traits.get(speaker, [])
if traits:
apply_effects_to_file(output_path, traits)
write_marker(spoken_text, output_path, speaker=speaker, voice_ref=ref_path)
if fx_version is not None:
apply_emotion_effects_to_file(output_path, utt["emotion"])
write_marker(
spoken_text, output_path, speaker=speaker,
voice_ref=ref_path,
voice_mode="embedding" if ref_path else None,
instruct=instruct, fx_version=fx_version,
)
generated += 1
utt_success = True
else:
@@ -222,7 +260,12 @@ def generate_chapter(
return generated, failed
def main(status_callback=None, gen_manager=None, project: str | None = None):
def main(
status_callback=None,
gen_manager=None,
project: str | None = None,
emotion_fx: bool = False,
):
cfg = load_config(project)
data_dir = cfg.data_dir
@@ -238,6 +281,7 @@ def main(status_callback=None, gen_manager=None, project: str | None = None):
speakers = load_speakers(data_dir)
alias_map = get_speaker_name_map(speakers)
phonetic = load_phonetic_names(data_dir)
proper_names = load_proper_names(data_dir)
speaker_traits = load_speaker_traits(data_dir / "speakers.json")
distortion_effects = load_speaker_distortion_effects(
data_dir / "speakers.json"
@@ -356,8 +400,10 @@ def main(status_callback=None, gen_manager=None, project: str | None = None):
cfg.db_path,
narrator_ref,
phonetic=phonetic,
proper_names=proper_names,
speaker_traits=speaker_traits,
distortion_effects=distortion_effects,
emotion_fx=emotion_fx,
utterance_callback=_utterance_done,
progress_callback=_progress_tick,
progress_context=progress_ctx,

4
scripts/lmstudio_control.py
View File

@@ -23,9 +23,9 @@ from umbraaudio.voices.lmstudio_lifecycle import (
)
PREFERRED_MODELS = [
"gemma-4-12b-it-qat",
"google/gemma-4-12b-qat",
"google/gemma-4-e4b",
"qwen3.5-9b",
"qwen/qwen3.5-9b",
]

22
scripts/qwen_tts_server.py
View File

@@ -422,6 +422,10 @@ async def generate_tts(
prompt_slug takes priority: if the cached prompt exists it is used, giving
the speaker the identical reference voice on every utterance.
voice_description also combines with a clone/ICL prompt: when ref_audio or
prompt_slug is given, the instruction text is prepended as style direction
(emotion, pace, delivery) while the cloned speaker identity is preserved.
"""
gen_kwargs: dict = {"temperature": temperature, "do_sample": do_sample}
@@ -441,11 +445,18 @@ async def generate_tts(
def _icl() -> tuple[np.ndarray, int]:
model = _load_model("base")
kw = dict(gen_kwargs)
if voice_description:
kw["instruct_ids"] = [
model._tokenize_texts(
[model._build_instruct_text(voice_description)]
)[0]
]
wavs, sr = model.generate_voice_clone(
text=text,
voice_clone_prompt=items,
language="english",
**gen_kwargs,
**kw,
)
return wavs, sr
@@ -489,13 +500,20 @@ async def generate_tts(
ref_audio_array, orig_sr=ref_sr, target_sr=sample_rate
)
kw = dict(gen_kwargs)
if voice_description:
kw["instruct_ids"] = [
model._tokenize_texts(
[model._build_instruct_text(voice_description)]
)[0]
]
wavs, sr = model.generate_voice_clone(
text=text,
ref_audio=(ref_audio_array, sample_rate),
ref_text=None,
x_vector_only_mode=True,
language="english",
**gen_kwargs,
**kw,
)
return wavs, sr

9
scripts/reattribute_all.py
View File

@@ -114,8 +114,13 @@ def main():
for u in attributed:
db.execute(
"UPDATE utterances SET speaker_name=? WHERE id=?",
(u.get("speaker_name", "Narrator"), u["id"]),
"UPDATE utterances SET speaker_name=?, attribution_confidence=?, attribution_reasoning=? WHERE id=?",
(
u.get("speaker_name", "Narrator"),
u.get("attribution_confidence", u.get("confidence", 0.5)),
u.get("attribution_reasoning", "") or "",
u["id"],
),
)
db.commit()

2
scripts/resegment_all.py
View File

@@ -25,7 +25,7 @@ def main():
parser.add_argument(
"--chapters", type=str, default=None, help="Comma-separated chapter IDs (e.g. '5,6,7')"
)
parser.add_argument("--model", type=str, default="gemma-4-12b-it-qat", help="LLM model name")
parser.add_argument("--model", type=str, default="google/gemma-4-12b-qat", help="LLM model name")
parser.add_argument(
"--mechanical-only", action="store_true", help="Skip LLM attribution, use mechanical only"
)

413
scripts/resegment_merge.py Normal file
View File

@@ -0,0 +1,413 @@
#!/usr/bin/env python3
"""Non-destructive chapter re-segmentation.
Re-runs the *current* Segmenter over selected chapters and merges the result
into the existing utterance table instead of wiping it. The goal is to apply
segmentation fixes without destroying data the user has already reviewed,
edited, or added:
* rows whose normalized text still matches a new segment keep their row id,
``speaker_name`` (including user-checked speakers), emotion/pace/instruct,
status, and ``user_edited`` flag;
* rows the user added/edited (``user_edited == 1`` or text not derivable from
the new segmentation) are always preserved even if the normalized text no
longer appears;
* narration rows replaced by corrected segmentation (leading-comma artifacts,
merged chapter titles, stale tense) are rewritten to the new text but stay
``narrator``;
* scene breaks stay ``scene_break`` and bind by order (old scenes are never
duplicated or dropped).
"""
from __future__ import annotations
import argparse
import contextlib
import json
import re
import sqlite3
import sys
import unicodedata
from difflib import SequenceMatcher
from pathlib import Path
from typing import Any
sys.path.insert(0, str(Path(__file__).parent.parent))
from umbraaudio.config import load_config
from umbraaudio.mapping.segmenter import Segmenter
def _norm(text: str) -> str:
"""Fold text for comparison: NFC, strip quotes, lowercase, collapse space."""
text = unicodedata.normalize("NFC", text or "")
text = text.strip(" \u201c\u201d\u2018\u2019\"'.,;:!?-\t\n")
for c in ("\u201c", "\u201d", "\u2018", "\u2019", '"', "'"):
text = text.replace(c, "")
return re.sub(r"\s+", " ", text.lower()).strip()
def _similar(a: str, b: str) -> float:
return SequenceMatcher(None, a, b).ratio()
def _row_fields(r: Any, rewritten: bool) -> dict:
return {
"text": r["text"],
"type": r["type"],
"speaker_name": r["speaker_name"],
"emotion": r["emotion"],
"pace": r["pace"],
"instruct": r["instruct"],
"status": r["status"],
"user_edited": r["user_edited"],
"attribution_confidence": r["attribution_confidence"],
"attribution_reasoning": r["attribution_reasoning"],
"context_before": r["context_before"],
"context_after": r["context_after"],
"rewritten": rewritten,
}
class _TargetPool:
"""Multiset of new segments keyed by normalized text."""
def __init__(self, new_segs: list[dict]) -> None:
self._groups: dict[str, list[dict]] = {}
for s in new_segs:
self._groups.setdefault(_norm(s.get("text", "")), []).append(s)
def exact(self, n: str) -> dict | None:
bucket = self._groups.get(n)
if not bucket:
return None
s = bucket.pop(0)
if not bucket:
del self._groups[n]
return s
def fuzzy_dialogue(self, n: str, threshold: float) -> dict | None:
best_key, best_sim, best = None, 0.0, None
for norm_key, cands in self._groups.items():
if "<<" in norm_key:
continue
sim = _similar(n, norm_key)
if sim > best_sim:
best_sim, best_key, best = sim, norm_key, cands[0]
if best is None or best_sim < threshold:
return None
bucket = self._groups.pop(best_key)
if len(bucket) > 1:
self._groups[best_key] = bucket[1:]
return best
def narration_rewrite(self, n: str) -> dict | None:
"""Match narration that got merged/split (title lines, comma fixes)."""
for norm_key, _cands in list(self._groups.items()):
if "<<" in norm_key:
continue
if n in norm_key or norm_key in n:
bucket = self._groups.pop(norm_key)
if len(bucket) > 1:
self._groups[norm_key] = bucket[1:]
return bucket[0]
return None
def _merge_chapter(
seg: Segmenter,
db_path: Path,
ch: Any,
character_names: list[str],
threshold: float,
dry_run: bool = False,
) -> tuple[int, int, int, list[int]]:
"""Rewrite one chapter's utterances in place. Returns (kept,new,rewritten,dropped)."""
ch_id = ch["id"]
title = ch["title"] or f"Chapter {ch['chapter_index']}"
source = ch["source_text"] or ""
new_segs = seg.segment(source, ch_id, character_names, chapter_title=title)
db = sqlite3.connect(str(db_path))
db.row_factory = sqlite3.Row
old_rows = db.execute(
"SELECT id, utterance_index, text, type, speaker_name, emotion, pace, "
"instruct, status, user_edited, attribution_confidence, "
"attribution_reasoning, context_before, context_after "
"FROM utterances WHERE chapter_id=? ORDER BY utterance_index",
(ch_id,),
).fetchall()
old_by_id: dict[int, Any] = {r["id"]: r for r in old_rows}
pool = _TargetPool(new_segs)
binds: dict[int, dict] = {}
for r in old_rows:
if r["type"] == "dialogue":
s = pool.exact(_norm(r["text"])) or pool.fuzzy_dialogue(_norm(r["text"]), threshold)
if s is not None:
binds[r["id"]] = s
for r in old_rows:
if r["type"] != "narration":
continue
s = pool.exact(_norm(r["text"])) or pool.narration_rewrite(_norm(r["text"]))
if s is not None:
binds[r["id"]] = s
# Scene breaks bind by order.
old_scenes = [r for r in old_rows if r["type"] == "scene_break"]
new_scenes = [s for s in new_segs if s.get("type") == "scene_break"]
scene_bind: dict[int, dict | None] = {
r["id"]: (new_scenes[i] if i < len(new_scenes) else None)
for i, r in enumerate(old_scenes)
}
assembly: list[dict] = []
consumed_old: set[int] = set()
def emit_row(rid: int, s: dict | None, scene: bool = False) -> None:
if rid in consumed_old:
return
r = old_by_id[rid]
fields = _row_fields(r, rewritten=False)
fields["old_index"] = r["utterance_index"]
if scene:
fields["text"] = "<<<>>>"
fields["type"] = "scene_break"
fields["speaker_name"] = "scene_break"
elif s is not None:
n_txt, n_tgt = _norm(r["text"]), _norm(s.get("text", ""))
fields["rewritten"] = bool(n_txt and n_tgt and n_txt != n_tgt)
if s.get("type") == "narration" and not r["user_edited"]:
fields["speaker_name"] = "narrator"
fields["text"] = s.get("text", r["text"])
fields["type"] = r["type"]
fields["new"] = False
fields["old_id"] = rid
assembly.append(fields)
consumed_old.add(rid)
for s in new_segs:
if s.get("type") == "scene_break":
rid = next((oid for oid, tgt in scene_bind.items() if tgt is s), None)
if rid is not None:
emit_row(rid, None, scene=True)
else:
assembly.append(
{
"text": "<<<>>>",
"type": "scene_break",
"speaker_name": "scene_break",
"new": True,
"old_id": None,
"rewritten": False,
}
)
continue
rid = next(
(oid for oid, tgt in binds.items() if tgt is s and oid not in consumed_old),
None,
)
if rid is not None:
emit_row(rid, s)
else:
assembly.append(
{
"text": s.get("text", ""),
"type": s.get("type", "narration"),
"speaker_name": s.get("character_name") or "narrator",
"new": True,
"old_id": None,
"rewritten": False,
}
)
for rid in sorted(scene_bind):
if scene_bind[rid] is None:
emit_row(rid, None, scene=True)
for rid in sorted(old_by_id):
if rid in consumed_old:
continue
r = old_by_id[rid]
n = _norm(r["text"])
derivable = any(
_similar(n, _norm(s.get("text", ""))) >= threshold for s in new_segs
)
if r["user_edited"] or not n or not derivable:
fields = _row_fields(r, rewritten=False)
fields["new"] = False
fields["old_id"] = rid
fields["old_index"] = r["utterance_index"]
assembly.append(fields)
consumed_old.add(rid)
dropped = [rid for rid in old_by_id if rid not in consumed_old]
kept = sum(1 for a in assembly if a["old_id"] is not None)
newly = sum(1 for a in assembly if a["new"])
rewritten = sum(1 for a in assembly if a["old_id"] is not None and a["rewritten"])
if dry_run:
db.close()
return kept, newly, rewritten, dropped
if dropped:
ph = ",".join("?" * len(dropped))
db.execute(f"DELETE FROM utterances WHERE id IN ({ph})", dropped)
for new_idx, a in enumerate(assembly):
if a["old_id"] is not None:
db.execute(
"UPDATE utterances SET utterance_index=?, text=?, speaker_name=?, "
"emotion=?, pace=?, instruct=?, status=? WHERE id=?",
(
new_idx,
a["text"],
a["speaker_name"],
a["emotion"],
a["pace"],
a["instruct"],
a["status"],
a["old_id"],
),
)
else:
db.execute(
"INSERT INTO utterances (chapter_id, utterance_index, text, type, "
"speaker_name, user_edited) VALUES (?, ?, ?, ?, ?, 0)",
(ch_id, new_idx, a["text"], a["type"], a["speaker_name"]),
)
db.commit()
db.close()
_remove_dropped_audio(db_path, ch_id, dropped, old_by_id)
_rename_chapter_audio(db_path, ch_id, assembly)
return kept, newly, rewritten, dropped
def _rename_chapter_audio(db_path: Path, chapter_id: int, assembly: list[dict]) -> None:
"""Rename utt_*.wav (+ sidecar .wav.txt) so audio follows the new indices."""
audio_dir = db_path.parent / "chapter_audio" / f"ch{chapter_id}"
if not audio_dir.exists():
return
index_map: dict[int, int] = {}
for new_idx, a in enumerate(assembly):
if a["old_id"] is not None and a.get("old_index") is not None:
index_map.setdefault(a["old_index"], new_idx)
if not index_map:
return
staged: list[tuple[Path, int]] = []
for file in sorted(audio_dir.glob("utt_*.wav")):
try:
old = int(file.stem.split("_")[1])
except (IndexError, ValueError):
continue
new = index_map.get(old)
if new is not None and new != old:
staged.append((file, new))
if not staged:
return
tmp_names: list[tuple[Path, int, Path]] = []
for file, new in staged:
tmp = audio_dir / f"._move_{file.stem}.wav.tmp"
file.rename(tmp)
tmp_names.append((tmp, new, file))
for tmp, new, orig in tmp_names:
tmp.rename(audio_dir / f"utt_{new:04d}.wav")
txt = audio_dir / f"{orig.stem}.wav.txt"
if txt.exists():
txt.rename(audio_dir / f"utt_{new:04d}.wav.txt")
def _remove_dropped_audio(
db_path: Path, chapter_id: int, dropped: list[int], old_by_id: dict
) -> None:
"""Delete wav/txt files whose utterances no longer exist to avoid stale audio."""
if not dropped:
return
audio_dir = db_path.parent / "chapter_audio" / f"ch{chapter_id}"
if not audio_dir.exists():
return
for rid in dropped:
r = old_by_id.get(rid)
if r is None:
continue
idx = r["utterance_index"]
for suffix in (".wav", ".wav.txt", ".mp3"):
f = audio_dir / f"utt_{idx:04d}{suffix}"
with contextlib.suppress(OSError):
f.unlink()
def main() -> None:
parser = argparse.ArgumentParser()
parser.add_argument("--project", type=str, default=None)
parser.add_argument(
"--chapters",
type=str,
default=None,
help="Comma-separated chapter_index list, e.g. '6,7,8'",
)
parser.add_argument("--dry-run", action="store_true", help="Report changes without writing")
parser.add_argument(
"--match-threshold",
type=float,
default=0.82,
help="Similarity ratio required to bind a row (default 0.82)",
)
args = parser.parse_args()
cfgs = load_config(args.project)
db_path = cfgs.db_path
if not db_path.exists():
print(f"Database not found: {db_path}")
sys.exit(1)
speakers_path = cfgs.data_dir / "speakers.json"
character_names: list[str] = []
alias_map: dict[str, str] = {}
if speakers_path.exists():
with open(speakers_path) as f:
sdata = json.load(f)
for s in sdata.get("speakers", []):
pn = s.get("primary_name", "")
if pn:
character_names.append(pn)
for a in s.get("aliases", []):
alias_map[a] = pn
db = sqlite3.connect(str(db_path))
db.row_factory = sqlite3.Row
wanted = {int(x) for x in args.chapters.split(",") if x.strip()} if args.chapters else None
chapters = db.execute(
"SELECT id, title, chapter_index, source_text FROM chapters ORDER BY chapter_index"
).fetchall()
if wanted is not None:
chapters = [c for c in chapters if c["chapter_index"] in wanted]
db.close()
print(f"Segmenter re-run over {len(chapters)} chapters")
seg = Segmenter(alias_map=alias_map)
total_kept = total_new = total_dropped = total_rewritten = 0
for ch in chapters:
kept, newly, rewritten, dropped = _merge_chapter(
seg, db_path, ch, character_names, args.match_threshold, dry_run=args.dry_run
)
title = ch["title"] or ""
print(
f"ch{ch['chapter_index']:>2} {title[:26]:<26} "
f"kept={kept} new={newly} rewritten={rewritten} dropped={len(dropped)}"
)
total_kept += kept
total_new += newly
total_dropped += len(dropped)
total_rewritten += rewritten
print(
f"\nTotals: kept={total_kept} new={total_new} "
f"rewritten={total_rewritten} dropped={total_dropped}"
)
if __name__ == "__main__":
main()

6
scripts/start_web_server.sh Executable file
View File

@@ -0,0 +1,6 @@
#!/bin/zsh
PROJ=/Volumes/1TB_APL_Storage/UmbraAudio
exec >> "$PROJ/logs/web_server.log" 2>&1
echo "[$(date '+%F %T')] web server starting (wrapper pid $$)"
cd "$PROJ"
exec "$PROJ/.venv/bin/python" -c "from pathlib import Path; from umbraaudio.web import create_app; from umbraaudio.config import load_config; import uvicorn; c = load_config(Path('/Volumes/1TB_APL_Storage/UmbraAudio')); uvicorn.run(create_app(c), host='0.0.0.0', port=8420, log_level='info')"

249
scripts/test_speaker_scottish.py Normal file
View File

@@ -0,0 +1,249 @@
#!/usr/bin/env python3
"""Test a Scottish voice for a given speaker once project discovery completes.
Waits for ``analysis_status.json`` to reach ``complete`` (or ``error``), then:
1. loads speakers.json and locates the requested speaker (primary/alias, case-insensitive),
2. starts the Qwen3-TTS server if it isn't already up,
3. generates a voice reference via VoiceDesign using the speaker's demographics +
a mid-low, adult, medium-Scottish description,
4. writes it next to the other project voices, drops a copy into the shared
web samples dir for listening, appends it to the samples index page, and
pins the description so the real "Generate All Voices" run uses this accent.
Usage:
scripts/test_speaker_scottish.py --speaker Hallr --project "Last Dwarf"
"""
from __future__ import annotations
import argparse
import io
import json
import logging
import os
import subprocess
import sys
import time
from pathlib import Path
ROOT = Path(__file__).resolve().parent.parent
sys.path.insert(0, str(ROOT))
import httpx
import soundfile as sf
from umbraaudio.config import load_config
TTS_URL = os.getenv("UMBRA_TTS_URL", "http://localhost:8881")
SAMPLES_DIR = ROOT / "umbraaudio" / "web" / "static" / "samples"
MAX_WAIT_SECONDS = 6 * 3600
POLL_INTERVAL = 20
VOICE_REFERENCE_TEXT = (
"When the sunlight strikes raindrops in the air, they act as a prism and form a rainbow. "
"The rainbow is a division of white light into many beautiful colors. These take the shape "
"of a long round arch, with its path high above, and its two ends apparently beyond the "
"horizon. There is, according to legend, a boiling pot of gold at one end. People look, "
"but no one ever finds it. When a man looks for something beyond his reach, his friends "
"say he is looking for the pot of gold at the end of the rainbow."
)
def log(msg: str) -> None:
print(msg, flush=True)
def build_description(speaker: dict) -> str:
gender = ((speaker.get("demographics") or {}).get("gender") or "").strip().lower()
if "female" in gender:
return (
"An adult female voice with a medium Scottish accent, mid-low pitch, "
"warm and resonant, clear and intelligible."
)
if "male" in gender or "man" in gender:
return (
"An adult male voice with a medium Scottish accent, mid-low pitch, "
"warm and slightly gravelly, clear and intelligible."
)
return (
"An adult voice with a medium Scottish accent, mid-low pitch, "
"warm and slightly gravelly, clear and intelligible."
)
def wait_for_analysis(status_path: Path) -> dict:
started = time.monotonic()
while time.monotonic() - started < MAX_WAIT_SECONDS:
try:
with open(status_path) as f:
st = json.load(f)
except (OSError, json.JSONDecodeError):
st = {"status": "unknown"}
status = st.get("status")
if status == "complete":
log(f"Discovery complete: {st.get('detail')}")
return st
if status == "error":
raise RuntimeError(f"Discovery failed: {st.get('error') or st.get('detail')}")
log(f"[poll] discovering — {st.get('detail')} ({st.get('progress')}%)")
time.sleep(POLL_INTERVAL)
raise TimeoutError("Discovery did not complete within the wait window")
def find_speaker(speakers_path: Path, target: str) -> tuple[dict, str]:
with open(speakers_path) as f:
data = json.load(f)
speakers = data.get("speakers", [])
tl = target.lower()
for s in speakers:
if s.get("primary_name", "").lower() == tl:
return s, s["primary_name"]
for a in s.get("aliases", []):
if a.lower() == tl:
return s, s["primary_name"]
raise KeyError(
f"Speaker '{target}' not found. Known: {[s.get('primary_name') for s in speakers]}"
)
def ensure_tts_server(cfg) -> None:
try:
r = httpx.get(f"{TTS_URL}/", timeout=3)
if r.status_code < 500:
log("TTS server already running")
return
except Exception:
pass
log("Starting Qwen3-TTS server...")
env = dict(os.environ)
env["QWEN_PROMPT_DIR"] = str(cfg.data_dir / "voice_prompts")
log_fh = open(ROOT / "logs" / "qwen_tts_server.log", "a")
subprocess.Popen(
[sys.executable, "-u", str(ROOT / "scripts" / "qwen_tts_server.py")],
cwd=str(ROOT),
env=env,
stdout=log_fh,
stderr=subprocess.STDOUT,
)
for _ in range(120):
try:
r = httpx.get(f"{TTS_URL}/", timeout=3)
if r.status_code < 500:
log("TTS server ready")
return
except Exception:
pass
time.sleep(5)
raise RuntimeError("TTS server did not become ready")
def generate_ref(description: str, output_path: Path) -> float:
r = httpx.post(
f"{TTS_URL}/v1/tts",
data={
"text": VOICE_REFERENCE_TEXT,
"voice_description": description,
"sample_rate": 24000,
},
timeout=900.0,
)
r.raise_for_status()
wav, sr = sf.read(io.BytesIO(r.content), dtype="float32")
sf.write(str(output_path), wav, sr)
info = sf.info(str(output_path))
return info.duration
def pin_description(speakers_path: Path, primary_name: str, description: str) -> None:
with open(speakers_path) as f:
data = json.load(f)
for s in data.get("speakers", []):
if s.get("primary_name") == primary_name:
s["voice_description"] = description
break
with open(speakers_path, "w") as f:
json.dump(data, f, indent=2)
profiles_path = speakers_path.parent / "voice_profiles.json"
profiles = {}
if profiles_path.exists():
with open(profiles_path) as f:
profiles = json.load(f)
profiles[primary_name] = {
"voice_description": description,
"source": "hallr_scottish_test",
}
with open(profiles_path, "w") as f:
json.dump(profiles, f, indent=2)
def append_to_samples_index(display: str, filename: str, desc: str, tag: str) -> None:
idx = SAMPLES_DIR / "index.html"
section = (
f'\n<h2 style="border-color:#8be9fd">★ {display}</h2>\n'
f'<div class="row"><label>{desc}</label>'
f'<audio controls preload="none" src="{filename}"></audio>'
f'<span class="tag">{tag}</span></div>\n'
)
with open(idx) as f:
html = f.read()
if filename in html:
return
with open(idx, "a") as f:
f.write(section)
def main() -> None:
ap = argparse.ArgumentParser()
ap.add_argument("--project", default="Last Dwarf")
ap.add_argument("--speaker", default="Hallr")
args = ap.parse_args()
cfg = load_config(args.project)
status_path = cfg.data_dir / "analysis_status.json"
speakers_path = cfg.data_dir / "speakers.json"
result = wait_for_analysis(status_path)
if not speakers_path.exists():
raise RuntimeError("Discovery report 'complete' but speakers.json is missing")
speaker, primary_name = find_speaker(speakers_path, args.speaker)
log(f"Found '{primary_name}': {json.dumps(speaker, indent=2)[:800]}")
description = build_description(speaker)
log(f"Voice description:\n {description}")
ensure_tts_server(cfg)
voices_dir = cfg.data_dir / "voices"
voices_dir.mkdir(parents=True, exist_ok=True)
ref_name = primary_name.lower().replace(" ", "_") + "_ref.wav"
output_path = voices_dir / ref_name
duration = generate_ref(description, output_path)
log(f"Wrote {output_path} ({duration:.1f}s)")
pin_description(speakers_path, primary_name, description)
log("Pinned voice_description on the speaker + voice_profiles.json")
SAMPLES_DIR.mkdir(parents=True, exist_ok=True)
sample_name = f"{output_path.stem}_scottish.wav"
sample_path = SAMPLES_DIR / sample_name
import shutil
shutil.copy2(output_path, sample_path)
append_to_samples_index(
f"{args.project} — {primary_name} (Scottish test)",
sample_name,
"mid-low pitch, adult, medium Scottish accent (waits on VoiceDesign)",
f"{duration:.0f}s",
)
log(f"Sample: http://<webui>/static/samples/{sample_name} (see index.html)")
if __name__ == "__main__":
try:
main()
except Exception as e:
log(f"HALLR SCOTTISH TEST FAILED: {e}")
raise

4
specs/00-project-overview.md
View File

@@ -114,7 +114,7 @@ EPUB/PDF/TXT/MD
| Database | SQLite (aiosqlite) | 0.19+ |
| LLM | OpenAI-compatible (LM Studio) | 1.10+ |
| TTS | Qwen3-TTS 1.7B (local GPU) | 0.1+ |
| LLM Model | gemma-4-12b-it-qat (remote LM Studio, context_length=8192) | remote |
| LLM Model | google/gemma-4-12b-qat (remote LM Studio, context_length=8192) | remote |
| Web | FastAPI + HTMX + Jinja2 | 0.104+ |
| Parsing | ebooklib, beautifulsoup4, PyMuPDF | 0.18+ / 4.0+ / 1.23+ |
@@ -122,7 +122,7 @@ EPUB/PDF/TXT/MD
- **Qwen3-TTS 1.7B Base**: ~3.9GB VRAM (voice cloning)
- **Qwen3-TTS 1.7B VoiceDesign**: ~3.9GB VRAM (voice reference generation)
- **LM Studio (gemma-4-12b-it-qat)**: remote server; load with `context_length: 8192` (speaker attribution)
- **LM Studio (google/gemma-4-12b-qat)**: remote server; load with `context_length: 8192` (speaker attribution)
- **Note**: Only one Qwen3-TTS model can be loaded at a time (auto-swaps). LM Studio must be unloaded before TTS generation if VRAM is tight.
## Module Structure

2
specs/02-analysis.md
View File

@@ -89,7 +89,7 @@ class CharacterProfiler:
```
**LLM Provider Selection (auto mode):**
1. Try LM Studio at `config.llm.lmstudio_base_url/models` (default `http://192.168.123.14:1234/v1`, overridable via `LLMConfig.lmstudio_base_url`); the model is `config.llm.lmstudio_model` (default `gemma-4-12b-it-qat` — must be loaded with `context_length: 8192`), and the configured model is preferred over whatever the server has loaded
1. Try LM Studio at `config.llm.lmstudio_base_url/models` (default `http://192.168.123.14:1234/v1`, overridable via `LLMConfig.lmstudio_base_url`); the model is `config.llm.lmstudio_model` (default `google/gemma-4-12b-qat` — must be loaded with `context_length: 8192`), and the configured model is preferred over whatever the server has loaded
2. Fall back to OpenAI (requires `OPENAI_API_KEY`)
3. Raise `RuntimeError` if neither available

6
specs/04-mapping.md
View File

@@ -130,7 +130,7 @@ class LLMAttributor:
- Scene breaks (`<<<>>>`) reset the carried speaker
**Performance:**
- ~5 seconds per batch of 40 utterances with `gemma-4-12b-it-qat` (~34 tok/s on the remote LM Studio server)
- ~5 seconds per batch of 40 utterances with `google/gemma-4-12b-qat` (~34 tok/s on the remote LM Studio server)
- The model must be loaded with `context_length: 8192`; the default 262k-token context requests ~44.9 GB of memory and blocks loading
- Processes all dialogue utterances; deterministic fallback recovers any that the LLM misses (target 100% dialogue attribution)
@@ -226,7 +226,7 @@ INSERT INTO utterances (
```bash
# Run on all chapters with LLM attribution
python scripts/resegment_all.py --model gemma-4-12b-it-qat
python scripts/resegment_all.py --model google/gemma-4-12b-qat
# Run on specific chapters
python scripts/resegment_all.py --chapters 5,6,7
@@ -241,7 +241,7 @@ Re-runs LLM speaker attribution on existing utterances **without re-segmenting**
```bash
# All chapters
python scripts/reattribute_all.py --model gemma-4-12b-it-qat
python scripts/reattribute_all.py --model google/gemma-4-12b-qat
# Specific chapters (--sleep adds a delay between chapters to pace the server)
python scripts/reattribute_all.py --chapters 5,6,7 --sleep 0.2

14
specs/14-two-phase-segmentation.md
View File

@@ -212,7 +212,7 @@ Return ONLY the JSON array, no other text.
```
**Performance:**
- ~5 seconds per batch of 40 utterances with `gemma-4-12b-it-qat` (~34 tok/s on the remote LM Studio server)
- ~5 seconds per batch of 40 utterances with `google/gemma-4-12b-qat` (~34 tok/s on the remote LM Studio server)
- The model must be loaded with `context_length: 8192` (default 262k-token context requests ~44.9 GB of memory and blocks loading)
- ~20-30 minutes for all 35 chapters; every dialogue utterance is processed (not just unattributed ones)
@@ -343,10 +343,10 @@ Additional checks:
```bash
# Run on all chapters with LLM attribution
python scripts/resegment_all.py --model gemma-4-12b-it-qat
python scripts/resegment_all.py --model google/gemma-4-12b-qat
# Run on specific chapters
python scripts/resegment_all.py --chapters 5,6,7 --model gemma-4-12b-it-qat
python scripts/resegment_all.py --chapters 5,6,7 --model google/gemma-4-12b-qat
# Mechanical-only (no LLM)
python scripts/resegment_all.py --mechanical-only
@@ -358,10 +358,10 @@ Re-runs LLM speaker attribution on existing utterances **without re-segmenting**
```bash
# All chapters (--sleep paces requests to the remote server)
python scripts/reattribute_all.py --model gemma-4-12b-it-qat --sleep 0.2
python scripts/reattribute_all.py --model google/gemma-4-12b-qat --sleep 0.2
# Specific chapters
python scripts/reattribute_all.py --chapters 5,6,7 --model gemma-4-12b-it-qat
python scripts/reattribute_all.py --chapters 5,6,7 --model google/gemma-4-12b-qat
```
**Note:** before running attribution, ensure the remote LM Studio server has the model loaded with `context_length: 8192`:
@@ -369,7 +369,7 @@ python scripts/reattribute_all.py --chapters 5,6,7 --model gemma-4-12b-it-qat
```bash
curl -X POST http://192.168.123.14:1234/api/v1/models/load \
-H 'Content-Type: application/json' \
-d '{"model": "gemma-4-12b-it-qat", "context_length": 8192, "flash_attention": true, "echo_load_config": true}'
-d '{"model": "google/gemma-4-12b-qat", "context_length": 8192, "flash_attention": true, "echo_load_config": true}'
```
## Key Design Decisions
@@ -382,4 +382,4 @@ curl -X POST http://192.168.123.14:1234/api/v1/models/load \
6. **Deterministic fallback**: Speech-tag regex + speaker carry-forward guarantee un-attributed dialogue still gets a speaker
7. **Batch processing**: 40 utterances per LLM call for efficiency
8. **Scene break preservation**: `<<<>>>` markers maintained as special utterances; they reset the carried speaker
9. **gemma-4-12b-it-qat model**: ~34 tok/s on the remote LM Studio server, must be loaded with `context_length: 8192` (~44.9 GB context otherwise blocks loading)
9. **google/gemma-4-12b-qat model**: ~34 tok/s on the remote LM Studio server, must be loaded with `context_length: 8192` (~44.9 GB context otherwise blocks loading)

130
specs/16-audiobook-production.md Normal file
View File

@@ -0,0 +1,130 @@
# Spec: Audiobook Production for Author's Republic (MP3 Package)
> **Status: In development.** This spec defines how the Deal Hunter audiobook is
> packaged for submission to Author's Republic (AR). It maps every AR audio
> requirement to a concrete artifact in the build so we can verify compliance
> before upload. The existing M4B build is **unchanged**; the AR submission is a
> **parallel** export produced from the same chapter audio.
>
> **Reviewed:** 2026-09-05 against
> <https://authorsrepublic.zendesk.com/hc/en-us/articles/17274538468244--Audio-Requirements-List>
---
## 1. Goal
Produce a single self-contained MP3 package that satisfies AR's 18 audio-file
requirements, plus a dashboard **"Build Final Audiobook (MP3)"** step that
builds it, shows per-track statistics, and offers a download for every track.
## 2. AR requirements → how Deal Hunter meets them
| # | AR requirement | How we meet it |
|---|----------------|----------------|
| 1 | Consistent overall sound & formatting | Every track: same mono channel layout, same 44.1 kHz sample rate, same 192 kbps CBR MP3, same 2 s leading/trailing silence, same loudness target (≈ −20 dB RMS). |
| 2 | Mono **or** stereo for all files | Force mono (`-ac 1`) on every track. |
| 3 | Same bit rate for all files | All tracks libmp3lame **192 kbps CBR** (no VBR). |
| 4 | Opening **and** closing credits in separate tracks | Dedicated `01_Opening.mp3` (title/author/narrator per cover+metadata) and `…_Closing.mp3` (ends with «The End.» + full credits). |
| 5 | Sample track 1–5 min | `02_Sample.mp3` = Chapter 1 promo (≈ 1 min 35 s; within 1–5 min). It was written as the audiobook intro and introduces the book, Umbra Audio, and several voices. |
| 6 | One chapter/section per file | One content track per manuscript chapter. |
| 7 | ≤ 119 min per track | Longest chapter today ≈ 38 min (Salvage); no split needed, but the builder **flags** any track > 119 min. |
| 8 | Chapter announcement at start of each file | Narrator-voice announcement «Chapter N, _Title_.» synthesized and prepended to every story chapter track (Salvage already self-announces «…Chapter one, Salvage.» so it is not duplicated). |
| 9 | 1–5 s silence at start **and** end of each track | 2 s silence applied to both ends of every track during encoding. |
| 10 | No plosives/pops/mouse clicks | Existing audio pipeline (utterance-level generation + LCM/resampled TTS) has no live recording artifacts. |
| 11 | −23 to −18 dB RMS | Chapters are normalized to a −20 dB RMS target; builder measures and reports RMS per track (`ffmpeg volumedetect`). |
| 12 | Peak ≤ −3 dB | Global peak limit −3 dB applied by the normalizer; builder reports peak per track. |
| 13 | Noise floor ≤ −60 dB RMS | Digital silence between utterances is ≥ −90 dB; TTS output is clean. |
| 14 | 192 kbps or higher | 192 kbps CBR on all tracks. |
| 15 | MP3 file format | `.mp3` for every track. |
| 16 | Constant Bit Rate (CBR) | `-b:a 192k` with libmp3lame = CBR. |
| 17 | 44.1 kHz | `-ar 44100` on every track (source WAVs are 24 kHz → resampled). |
| 18 | ≤ 170 MB per file | Longest chapter output is well under; total package ≈ 400 MB across ~34 files. |
### Opening / closing / sample / chapter-track rules (from AR's intros guidance)
**Opening track** (`…_01_Opening_Credits.mp3`, < 3 min)
- Must include **title, author, narrator**, matching cover art + metadata exactly.
- Text (from `m4b_meta.json`, kept in sync):
> This is Deal Hunter. Written by B. D. Murphy, and read for you by Umbra Audio.
- Narrator voice, ≈ 25–45 s including silence.
**Sample track** (`…_02_Sample.mp3`, 1–5 min)
- Chapter 1 ("Welcome to the audio book Deal Hunter…") — the purpose-built intro.
**Chapter tracks** (`…_NN_Title.mp3`)
- One per content chapter, in manuscript order, correctly sorted (zero-padded sequence in the file name).
- Spoken announcement prepended for story chapters that do not already self-announce.
**Closing track** (`…_NN_Closing_Credits.mp3`, < 3 min)
- Base = final track of the book, Chapter 33 in the UI («Thank You for listening to Deal Hunter…» / About-the-author).
- Appended narrator line provides the required book-ending reference + full credits:
> The End. You have been listening to Deal Hunter. Written by B. D. Murphy, and read for you by Umbra Audio. Published by WorstAuthor LLC. Artwork by WorstAuthor LLC.
## 3. Track manifest (as of this session)
Sequence numbers are assigned at build time (opening first, then sample, then chapters in book order, closing last). Base name derives from `m4b_meta.json` `output_name` → `Deal_Hunter`.
| Seq | File | Role | Source |
|-----|------|------|--------|
| 01 | `Deal_Hunter_01_Opening_Credits.mp3` | opening (synth) | narrator TTS from template |
| 02 | `Deal_Hunter_02_Sample.mp3` | sample | `ch1_full.wav` (promo intro) |
| 03 | `Deal_Hunter_03_Copyright.mp3` | chapter | `ch4_full.wav` (Copyright) |
| 04 | `Deal_Hunter_04_Authors_Note.mp3` | chapter | `ch5_full.wav` (Author's Note) |
| 05 | `Deal_Hunter_05_Salvage.mp3` | chapter | `ch6_full.wav` (self-announces) |
| 06…| `Deal_Hunter_06_Family_Reunion.mp3` … | chapter | `ch7..ch32_full.wav` + synth announcement |
| NN | `Deal_Hunter_NN_Closing_Credits.mp3` | closing | `ch35_full.wav` + appended credits |
Notes:
- Empty title-page chapters (`Title`, `DEAL HUNTER` echo) are skipped (no audio).
- Chapter-track sequence `NN` is zero-padded so retailer file-sort = book order.
- Announcement numbering follows the book's story order (Salvage = chapter one),
e.g. Family Reunion → «Chapter 2, Family Reunion.»
## 4. Build flow (dashboard → `/api/audiobook/export-mp3`)
```
m4b_meta.json (title/author/narrator/publisher) ─┐
project.db (chapter order + titles + first ütterance) │
chapter_audio/chN_full.wav (content) ───────────┴─▶ ARPackageExporter
voices/narrator_ref.wav (voice for new narration) ─▶ 1. synth opening/announcements/closing-tail
2. assemble per-track concatenation
3. encode 44.1 kHz / mono / 192k CBR MP3
with 2 s leading+trailing silence
4. measure duration/size/RMS/peak per track
5. write export/ar_package/ + ar_package.json
```
- Work/narration clips cache to `export/ar_work/` (announcements regenerated only when text changes).
- Background thread mirrors the existing M4B export UX (status JSON + polling).
- Output lands under `export/` so the existing Step-7 download list picks it up automatically.
## 5. New/changed code
| File | Change |
|------|--------|
| `specs/16-audiobook-production.md` | This spec |
| `umbraaudio/assembly/ar_export.py` | New `ARPackageExporter` (build + stats + manifest) |
| `umbraaudio/web/routes.py` | `POST /api/audiobook/export-mp3`, `GET /api/audiobook/ar-status`, `GET /api/audiobook/ar-files` |
| `umbraaudio/web/templates/index.html` | "Build Final Audiobook (MP3)" card + stats table + download links |
| `tests/test_ar_export.py` | Unit tests with mocked TTS + ffmpeg |
## 6. Quality gates before upload
- All tracks: 44.1 kHz, mono, 192 kbps CBR MP3 (ffprobe report in manifest).
- RMS in [−23, −18] dB, peak ≤ −3 dB (volumedetect). Manual spot-check on final files.
- Every chapter track has a spoken announcement; opening/closing ≤ 3 min; sample 1–5 min.
- No track > 119 min; no file > 170 MB.
- Naming sorted so retailer display order == book order.
## 7. Open decisions / risks
- **Qwen TTS server dependency**: opening/closing/announcement narration needs the
TTS server (port 8881) up with the narrator vmatrix cached (`prompt_slug="narrator"`).
If it is down the build fails fast with `QwenTTSUnavailableError`. (~26 short clips.)
- **"About the author" becomes part of the closing track.** If instead it should stay
as its own numbered chapter track, move its role to `chapter` in `_ABOUT_AUTHOR_ROLE`.
- **Loudness**: if final per-track RMS drifts outside [−23, −18] dB the encode should
add `-af loudnorm`; today chapters are normalized to −20 dB so this is expected-pass.
- **Narrator metadata mismatch**: opening/closing text is templated from `m4b_meta.json`;
any change to title/author/narrator there must be re-exported (or spoken credits won't
match the metadata/cover as AR requires).

318
tests/test_ar_export.py Normal file
View File

@@ -0,0 +1,318 @@
from __future__ import annotations
import json
import shutil
import sqlite3
import subprocess
from typing import TYPE_CHECKING
import numpy as np
import pytest
import soundfile as sf
if TYPE_CHECKING:
from pathlib import Path
from umbraaudio.assembly.ar_export import (
ARChapter,
ARPackageExporter,
discover_chapters,
export_chapter_mp3,
)
DUR = 24000
META = {
"title": "Deal Hunter",
"author": "B.D. Murphy",
"narrator": "Umbra Audio",
"publisher": "WorstAuthor LLC",
"output_name": "Deal_Hunter.m4b",
}
def _fake_chapter_wav(path: Path, seconds: float = 1.0, amp: float = 0.1) -> Path:
sf.write(str(path), (amp * np.random.randn(int(DUR * seconds))).astype(np.float32), DUR)
return path
def _sample_book(data_dir: Path) -> list[ARChapter]:
audio = data_dir / "chapter_audio"
audio.mkdir(parents=True, exist_ok=True)
return [
ARChapter(1, 0, "Sample Track", "Welcome to the audio book Deal Hunter.",
_fake_chapter_wav(audio / "ch1_full.wav"), role="sample"),
ARChapter(4, 3, "Copyright", "Copyright c 2023 by B.D. Murphy",
_fake_chapter_wav(audio / "ch4_full.wav"), role="chapter"),
ARChapter(5, 4, "Author's Note", "Author's Note",
_fake_chapter_wav(audio / "ch5_full.wav"), role="chapter"),
ARChapter(6, 5, "Salvage", "Deal Hunter. Chapter one, Salvage.",
_fake_chapter_wav(audio / "ch6_full.wav"), role="chapter"),
ARChapter(7, 6, "Family Reunion", "The princess is back home in the palace on Albert-4.",
_fake_chapter_wav(audio / "ch7_full.wav"), role="chapter"),
ARChapter(35, 34, "Closing Track", "Thank You for listening to Deal Hunter.",
_fake_chapter_wav(audio / "ch35_full.wav"), role="closing"),
]
def _build_book(data_dir: Path, book: list[ARChapter] | None = None, **kwargs) -> dict:
exporter = ARPackageExporter(data_dir)
return exporter.build(book or _sample_book(data_dir), dict(META), **kwargs)
def test_full_build_manifest(temp_dir: Path) -> None:
manifest = _build_book(temp_dir / ".umbra")
assert manifest["track_count"] == 6
assert manifest["format"] == {
"codec": "mp3", "sample_rate": 44100, "channels": 1,
"bit_rate_kbps": 192, "cbr": True, "silence_s": 2.0,
}
roles = [t["role"] for t in manifest["tracks"]]
assert roles == ["sample", "closing", "chapter", "chapter", "chapter", "chapter"]
assert [t["number"] for t in manifest["tracks"]] == ["", "ii", "", "", "1", "2"]
assert [t["name"] for t in manifest["tracks"]] == [
"Sample Track", "Closing Track", "Copyright", "Author's Note",
"Salvage", "Family Reunion",
]
assert [t["title"] for t in manifest["tracks"]] == [
"Sample Track", "ii Closing Track", "Copyright", "Author's Note",
"1 Salvage", "2 Family Reunion",
]
files = sorted((temp_dir / ".umbra" / "export" / "ar_package").glob("*.mp3"))
names = [f.name for f in files]
assert len(names) == 6
assert "Deal_Hunter_Sample_Track.mp3" in names
assert "Deal_Hunter_Copyright.mp3" in names
assert "Deal_Hunter_Authors_Note.mp3" in names
assert "Deal_Hunter_01_Salvage.mp3" in names
assert "Deal_Hunter_02_Family_Reunion.mp3" in names
assert "Deal_Hunter_ii_Closing_Track.mp3" in names
def test_tracks_are_straight_chapter_audio_no_synthesis(temp_dir: Path) -> None:
manifest = _build_book(temp_dir / ".umbra")
for t in manifest["tracks"]:
assert t["source"].endswith("_full.wav")
assert t["source"] != ""
assert all("announcement" not in t for t in manifest["tracks"])
def test_chapters_without_audio_are_skipped(temp_dir: Path) -> None:
book = _sample_book(temp_dir / ".umbra")
book[3] = ARChapter(6, 5, "Salvage", "Deal Hunter. Chapter one, Salvage.",
None, role="chapter")
manifest = _build_book(temp_dir / ".umbra", book)
assert manifest["track_count"] == 5
assert all(t["title"] != "Salvage" for t in manifest["tracks"])
def test_loudness_warning_for_quiet_chapter(temp_dir: Path) -> None:
data_dir = temp_dir / ".umbra"
book = _sample_book(data_dir)
quiet = book[3].audio_path
assert quiet is not None
_fake_chapter_wav(quiet, amp=0.0001)
book[3] = ARChapter(6, 5, "Salvage", "Deal Hunter. Chapter one, Salvage.",
quiet, role="chapter")
manifest = _build_book(data_dir, book)
assert any("RMS" in w for w in manifest["warnings"])
def test_loudness_normalization_brings_quiet_book_to_spec(temp_dir: Path) -> None:
data_dir = temp_dir / ".umbra"
audio = data_dir / "chapter_audio"
audio.mkdir(parents=True, exist_ok=True)
book = [
ARChapter(1, 0, "Sample Track", "Welcome to the audio book Deal Hunter.",
_fake_chapter_wav(audio / "ch1_full.wav", amp=0.02),
role="sample"),
ARChapter(4, 3, "Copyright", "Copyright c 2023 by B.D. Murphy",
_fake_chapter_wav(audio / "ch4_full.wav", amp=0.02),
role="chapter"),
ARChapter(35, 34, "Closing Track", "Thank You for listening to Deal Hunter.",
_fake_chapter_wav(audio / "ch35_full.wav", amp=0.02),
role="closing"),
]
manifest = _build_book(data_dir, book)
assert manifest["warnings"] == [], manifest["warnings"]
for t in manifest["tracks"]:
assert -22.5 <= t["rms_db"] <= -18.5, t
assert t["peak_db"] <= -2.9, t
assert t["loudness_ok"] and t["peak_ok"], t
def test_peak_limiter_caps_clipping_source(temp_dir: Path) -> None:
data_dir = temp_dir / ".umbra"
audio = data_dir / "chapter_audio"
audio.mkdir(parents=True, exist_ok=True)
hot = audio / "ch1_full.wav"
sf.write(str(hot), (4.0 * np.random.randn(DUR)).astype(np.float32), DUR)
book = _sample_book(data_dir)
book[0] = ARChapter(1, 0, "Sample Track", "Welcome to the audio book Deal Hunter.",
hot, role="sample")
manifest = _build_book(data_dir, book)
sample = next(t for t in manifest["tracks"] if t["role"] == "sample")
assert sample["peak_db"] <= -2.9
assert sample["peak_ok"]
def test_rebuild_is_idempotent(temp_dir: Path) -> None:
data_dir = temp_dir / ".umbra"
exporter = ARPackageExporter(data_dir)
first = exporter.build(_sample_book(data_dir), dict(META))
second = exporter.build(_sample_book(data_dir), dict(META))
assert isinstance(second, dict)
assert second["track_count"] == first["track_count"]
assert [t["title"] for t in second["tracks"]] == [t["title"] for t in first["tracks"]]
def test_manifest_written_and_total_duration(temp_dir: Path) -> None:
manifest_path = temp_dir / ".umbra" / "export" / "ar_package.json"
manifest = _build_book(temp_dir / ".umbra")
assert manifest_path.exists()
on_disk = json.loads(manifest_path.read_text())
assert on_disk["track_count"] == manifest["track_count"]
expected_duration = sum(t["duration_s"] for t in manifest["tracks"])
assert abs(manifest["total_duration_s"] - expected_duration) < 0.01
@pytest.mark.skipif(shutil.which("ffmpeg") is None or shutil.which("ffprobe") is None,
reason="ffmpeg/ffprobe required")
def test_encoded_files_are_44100_mono_192k_cbr(temp_dir: Path) -> None:
_build_book(temp_dir / ".umbra")
mp3 = next((temp_dir / ".umbra" / "export" / "ar_package").glob("*.mp3"))
probe = subprocess.run(
[
"ffprobe", "-v", "error",
"-show_entries", "stream=codec_name,sample_rate,channels,bit_rate",
"-show_entries", "format=duration",
"-of", "json",
str(mp3),
],
capture_output=True, text=True,
)
assert probe.returncode == 0, probe.stderr
data = json.loads(probe.stdout)
stream = data["streams"][0]
assert stream["codec_name"] == "mp3"
assert stream["sample_rate"] == "44100"
assert int(stream["channels"]) == 1
assert stream["bit_rate"] == "192000"
@pytest.mark.skipif(shutil.which("ffprobe") is None, reason="ffprobe required")
def test_chapter_mp3_export_is_ar_compliant(temp_dir: Path) -> None:
data_dir = temp_dir / ".umbra"
audio = data_dir / "chapter_audio"
audio.mkdir(parents=True, exist_ok=True)
wav = _fake_chapter_wav(audio / "ch7_full.wav", seconds=1.0, amp=0.1)
out = audio / "ch7_full.mp3"
stats = export_chapter_mp3(data_dir, wav, out)
assert stats["duration_s"] == 5.0
assert stats["size_bytes"] == out.stat().st_size
assert stats["loudness_ok"] is True
assert stats["peak_ok"] is True
probe = subprocess.run(
[
"ffprobe", "-v", "error",
"-show_entries", "stream=codec_name,sample_rate,channels,bit_rate",
"-show_entries", "format=duration",
"-of", "json",
str(out),
],
capture_output=True, text=True,
)
assert probe.returncode == 0, probe.stderr
data = json.loads(probe.stdout)
stream = data["streams"][0]
assert stream["codec_name"] == "mp3"
assert stream["sample_rate"] == "44100"
assert int(stream["channels"]) == 1
assert stream["bit_rate"] == "192000"
assert abs(float(data["format"]["duration"]) - 5.0) < 0.3
@pytest.mark.skipif(shutil.which("ffprobe") is None, reason="ffprobe required")
def test_track_duration_matches_file_length(temp_dir: Path) -> None:
manifest = _build_book(temp_dir / ".umbra")
pkg = temp_dir / ".umbra" / "export" / "ar_package"
for t in manifest["tracks"]:
probe = subprocess.run(
["ffprobe", "-v", "error", "-show_entries", "format=duration",
"-of", "json", str(pkg / t["file"])],
capture_output=True, text=True,
)
assert probe.returncode == 0
actual = float(json.loads(probe.stdout)["format"]["duration"])
assert abs(actual - t["duration_s"]) < 0.3
def _make_project_db(data_dir: Path) -> None:
data_dir.mkdir(parents=True, exist_ok=True)
db = sqlite3.connect(data_dir / "project.db")
db.executescript(
"""
CREATE TABLE chapters (id INTEGER PRIMARY KEY, chapter_index INTEGER,
title TEXT, source_text TEXT);
CREATE TABLE utterances (
id INTEGER PRIMARY KEY,
chapter_id INTEGER,
utterance_index INTEGER,
type TEXT,
speaker_name TEXT,
text TEXT);
"""
)
db.executemany(
"INSERT INTO chapters (id, chapter_index, title, source_text) VALUES (?,?,?,?)",
[
(1, 0, "Sample Track", ""),
(2, 1, "Title Page", ""),
(3, 2, "Opening Track", ""),
(4, 3, "Copyright", ""),
(6, 5, "Salvage", ""),
(35, 34, "Closing Track", ""),
],
)
db.executemany(
"INSERT INTO utterances "
"(chapter_id, utterance_index, type, speaker_name, text) VALUES (?,?,?,?,?)",
[
(1, 0, "narrator", "Narrator", "Welcome to the audio book Deal Hunter."),
(4, 0, "narrator", "Narrator", "Copyright c 2023"),
(6, 0, "narrator", "Narrator", "Deal Hunter. Chapter one, Salvage."),
(35, 0, "narrator", "Narrator", "Thank You for listening to Deal Hunter."),
],
)
db.commit()
db.close()
audio = data_dir / "chapter_audio"
audio.mkdir()
_fake_chapter_wav(audio / "ch1_full.wav")
_fake_chapter_wav(audio / "ch4_full.wav")
_fake_chapter_wav(audio / "ch6_full.wav")
_fake_chapter_wav(audio / "ch35_full.wav")
def test_discover_chapters_assigns_roles(temp_dir: Path) -> None:
data_dir = temp_dir / ".umbra"
_make_project_db(data_dir)
chapters = discover_chapters(data_dir)
roles = {c.role for c in chapters}
assert roles == {"sample", "chapter", "closing"}
sample = next(c for c in chapters if c.role == "sample")
closing = next(c for c in chapters if c.role == "closing")
assert sample.chapter_id == 1
assert sample.title == "Sample Track"
assert closing.chapter_id == 35
assert closing.title == "Closing Track"
assert all(c.title not in ("Title Page", "Opening Track") for c in chapters)
assert [c.title for c in chapters] == [
"Sample Track", "Copyright", "Salvage", "Closing Track",
]
if __name__ == "__main__":
pytest.main([__file__, "-v"])

81
tests/test_generation/test_audio_effects.py
View File

@@ -8,6 +8,8 @@ import numpy as np
import soundfile as sf
from umbraaudio.generation.audio_effects import (
EFFECT_FN,
EMOTION_EFFECTS,
TRAIT_EFFECTS,
_bandpass,
_bitcrusher,
@@ -20,6 +22,8 @@ from umbraaudio.generation.audio_effects import (
_tremolo,
apply_effects,
apply_effects_to_file,
apply_emotion_effects,
apply_emotion_effects_to_file,
get_effects_for_traits,
load_speaker_traits,
)
@@ -257,3 +261,80 @@ class TestLoadSpeakerTraits:
}
path.write_text(json.dumps(data))
assert load_speaker_traits(path) == {}
def _rms(audio: np.ndarray) -> float:
return float(np.sqrt(np.mean(np.square(audio))))
def _speechlike() -> np.ndarray:
"""A broadband harmonic signal with a syllable-rate amplitude envelope."""
t = np.linspace(0, DURATION, SAMPLES, dtype=np.float32)
am = 0.5 + 0.5 * np.sin(2 * np.pi * 3.5 * t)
sig = am * (
np.sin(2 * np.pi * 140 * t)
+ 0.5 * np.sin(2 * np.pi * 280 * t)
+ 0.25 * np.sin(2 * np.pi * 420 * t)
)
return (0.6 * sig).astype(np.float32)
class TestEmotionEffects:
def test_neutral_is_untouched_baseline(self) -> None:
out = apply_emotion_effects(_sine(), SR, "neutral")
np.testing.assert_allclose(out, _sine(), atol=1e-6)
def test_unknown_emotion_is_untouched(self) -> None:
out = apply_emotion_effects(_sine(), SR, "confused")
np.testing.assert_allclose(out, _sine(), atol=1e-6)
def test_angry_is_louder_and_faster(self) -> None:
base = _speechlike()
out = apply_emotion_effects(base, SR, "angry")
assert _rms(out) > _rms(base)
assert len(out) < len(base)
def test_whisper_is_softer(self) -> None:
base = _speechlike()
out = apply_emotion_effects(base, SR, "whisper")
assert _rms(out) < _rms(base) * 0.5
def test_sad_is_quieter_and_slower(self) -> None:
base = _speechlike()
out = apply_emotion_effects(base, SR, "sad")
assert _rms(out) < _rms(base)
assert len(out) > len(base)
def test_angry_and_sad_differ_in_delivery(self) -> None:
base = _speechlike()
angry = apply_emotion_effects(base, SR, "angry")
sad = apply_emotion_effects(base, SR, "sad")
assert len(angry) < len(sad)
def test_all_chain_effects_exist(self) -> None:
for emotion, chain in EMOTION_EFFECTS.items():
for effect_name, _ in chain:
assert effect_name in EFFECT_FN, f"{emotion}: {effect_name}"
def test_all_chain_effects_finite(self) -> None:
base = _speechlike()
for emotion in EMOTION_EFFECTS:
out = apply_emotion_effects(base, SR, emotion)
assert np.all(np.isfinite(out))
assert np.max(np.abs(out)) <= 1.0001
def test_apply_emotion_effects_to_file(self, tmp_path: Path) -> None:
wav = tmp_path / "e.wav"
sf.write(str(wav), _speechlike(), SR)
applied = apply_emotion_effects_to_file(wav, "angry")
assert applied is True
result, _ = sf.read(str(wav), dtype="float32")
assert len(result) != SAMPLES or not np.allclose(result, _speechlike(), atol=0.01)
def test_apply_emotion_effects_to_file_neutral_is_false(self, tmp_path: Path) -> None:
wav = tmp_path / "n.wav"
audio = _sine()
sf.write(str(wav), audio, SR)
assert apply_emotion_effects_to_file(wav, "neutral") is False
result, _ = sf.read(str(wav), dtype="float32")
np.testing.assert_allclose(result, audio, atol=1e-4)

63
tests/test_generation/test_staleness.py
View File

@@ -113,3 +113,66 @@ def test_voice_fingerprint_stale_when_ref_deleted(tmp_path: Path) -> None:
write_marker("Gone ref.", wav, speaker="Nanny", voice_ref=ref)
ref.unlink()
assert is_current("Gone ref.", wav, speaker="Nanny", voice_ref=ref) is False
def test_changed_instruct_is_stale(tmp_path: Path) -> None:
wav = tmp_path / "utt_0015.wav"
wav.write_bytes(b"RIFF....")
write_marker("Same words.", wav, speaker="Narrator", instruct="speak softly")
assert is_current("Same words.", wav, speaker="Narrator", instruct="speak loudly") is False
def test_matching_instruct_is_current(tmp_path: Path) -> None:
wav = tmp_path / "utt_0016.wav"
wav.write_bytes(b"RIFF....")
write_marker("Same words.", wav, speaker="Narrator", instruct="speak softly")
assert is_current("Same words.", wav, speaker="Narrator", instruct="speak softly") is True
def test_marker_without_instruct_matches_no_instruct_call(tmp_path: Path) -> None:
wav = tmp_path / "utt_0017.wav"
wav.write_bytes(b"RIFF....")
write_marker("Plain line.", wav, speaker="Narrator")
assert is_current("Plain line.", wav, speaker="Narrator") is True
def test_marker_without_instruct_stale_when_instruct_expected(tmp_path: Path) -> None:
wav = tmp_path / "utt_0018.wav"
wav.write_bytes(b"RIFF....")
write_marker("Plain line.", wav, speaker="Narrator")
assert is_current("Plain line.", wav, speaker="Narrator", instruct="speak softly") is False
def test_instruct_recorded_in_marker(tmp_path: Path) -> None:
wav = tmp_path / "utt_0019.wav"
write_marker("Some text", wav, speaker="Narrator", instruct="speak softly")
marker = marker_path(wav).read_text(encoding="utf-8")
assert marker == '{"speaker": "Narrator", "text": "Some text", "instruct": "speak softly"}'
def test_matching_fx_version_is_current(tmp_path: Path) -> None:
wav = tmp_path / "utt_0020.wav"
wav.write_bytes(b"RIFF....")
write_marker("Same words.", wav, speaker="Narrator", fx_version=1)
assert is_current("Same words.", wav, speaker="Narrator", fx_version=1) is True
def test_fx_version_mismatch_is_stale(tmp_path: Path) -> None:
wav = tmp_path / "utt_0021.wav"
wav.write_bytes(b"RIFF....")
write_marker("Same words.", wav, speaker="Narrator", fx_version=1)
assert is_current("Same words.", wav, speaker="Narrator", fx_version=2) is False
def test_marker_without_fx_stale_when_fx_expected(tmp_path: Path) -> None:
wav = tmp_path / "utt_0022.wav"
wav.write_bytes(b"RIFF....")
write_marker("Same words.", wav, speaker="Narrator")
assert is_current("Same words.", wav, speaker="Narrator", fx_version=1) is False
def test_fx_marked_audio_stale_for_plain_calls(tmp_path: Path) -> None:
wav = tmp_path / "utt_0023.wav"
wav.write_bytes(b"RIFF....")
write_marker("Same words.", wav, speaker="Narrator", fx_version=1)
assert is_current("Same words.", wav, speaker="Narrator") is False

118
tests/test_pronunciation.py
View File

@@ -178,6 +178,124 @@ class TestApplyPhonetics:
assert pr.apply_phonetics("don’t won’t", {}) == "don't won't"
def _write_speakers(temp_dir: Path) -> None:
_write_json(
temp_dir / "speakers.json",
{
"speakers": [
{"primary_name": "Kaleb", "aliases": ["The young man"]},
{"primary_name": "Hallr", "aliases": []},
{"primary_name": "Sheyla", "aliases": ["Healer", "The woman"]},
{"primary_name": "Captain Ulgary", "aliases": []},
{"primary_name": "Guard 1", "aliases": []},
{"primary_name": "Narrator", "aliases": []},
{"primary_name": "scum leader", "aliases": []},
]
},
)
class TestLoadProperNames:
def test_primary_names_and_alias_first_words(self, temp_dir: Path):
_write_speakers(temp_dir)
names = pr.load_proper_names(temp_dir)
assert "kaleb" in names
assert "hallr" in names
assert "sheyla" in names
# role/title/unknown words are excluded
assert "captain" not in names
assert "healer" not in names
assert "guard" not in names
assert "narrator" not in names
assert "scum" not in names
assert "the" not in names
def test_user_confirmed_phonetic_names_included(self, temp_dir: Path):
_write_json(
temp_dir / pr.PHONETIC_NAMES_FILE,
{
"Feliu": {
"phonetic": "FEH-lee-oo",
"source": "user",
"confirmed": True,
"common": False,
},
"Meriel": {
"phonetic": "MEH-ree-el",
"source": "behindthename",
"confirmed": False,
"common": False,
},
},
)
names = pr.load_proper_names(temp_dir)
assert "feliu" in names
assert "meriel" not in names
def test_missing_files_empty(self, temp_dir: Path):
assert pr.load_proper_names(temp_dir) == frozenset()
class TestPrefixProperNamePause:
def test_narration_start(self):
names = frozenset({"kaleb", "hallr", "sheyla"})
assert pr.prefix_proper_name_pause("Kaleb stood by the fire.", names) == (
", Kaleb stood by the fire."
)
def test_dialogue_after_open_quote(self):
names = frozenset({"kaleb"})
assert pr.prefix_proper_name_pause("“Kaleb, come here.”", names) == (
"“, Kaleb, come here.”"
)
def test_already_prefixed_untouched(self):
names = frozenset({"hallr"})
assert pr.prefix_proper_name_pause(", Hallr, with his axe, stood.", names) == (
", Hallr, with his axe, stood."
)
assert pr.prefix_proper_name_pause("“, Hallr, what do you mean?”", names) == (
"“, Hallr, what do you mean?”"
)
def test_non_proper_name_untouched(self):
names = frozenset({"kaleb"})
assert pr.prefix_proper_name_pause("The boy ran home.", names) == "The boy ran home."
assert pr.prefix_proper_name_pause("“What is Kaleb doing?”", names) == (
"“What is Kaleb doing?”"
)
def test_possessive_matches(self):
names = frozenset({"xylia"})
assert pr.prefix_proper_name_pause("Xylia's wing healed.", names) == (
", Xylia's wing healed."
)
def test_empty_inputs(self):
assert pr.prefix_proper_name_pause("", frozenset({"kaleb"})) == ""
assert pr.prefix_proper_name_pause("Kaleb nods.", frozenset()) == "Kaleb nods."
class TestPrepareTtsText:
def test_pause_then_phonetics(self):
phonetic = {"Kaleb": "KAYLEB", "Hallr": "HALR"}
names = frozenset({"kaleb", "hallr"})
assert pr.prepare_tts_text("Kaleb replies,", phonetic, names) == (
", KAYLEB replies,"
)
def test_no_proper_names_skips_pause(self):
assert pr.prepare_tts_text("The boy ran.", {"Weston": "WES-ton"}, frozenset()) == (
"The boy ran."
)
def test_matches_apply_phonetics_without_pause(self):
phonetic = {"Kainda": "KAEIN-duh"}
assert pr.prepare_tts_text("Kainda steps forward.", phonetic) == (
pr.apply_phonetics("Kainda steps forward.", phonetic)
)
class TestNormalizeTtsText:
def test_ascii_identity(self):
assert pr.normalize_tts_text("Plain ASCII text.") == "Plain ASCII text."

217
tests/test_qa_acceptance.py Normal file
View File

@@ -0,0 +1,217 @@
"""Per-utterance QA box: issue detection, recheck/fix loop, and accept state.
These cover the helpers behind ``/api/utterance/{id}/recheck`` and
``/api/utterance/{id}/qa-accept`` without spinning up the web app.
"""
from __future__ import annotations
import json
import sqlite3
import types
from typing import TYPE_CHECKING
if TYPE_CHECKING:
from pathlib import Path
from umbraaudio.web.routes import (
SCHEMA_SQL,
_ensure_utterance_columns,
_load_speaker_alias_map,
_qa_accept_utterance,
_qa_accepted_for,
_qa_accepted_sig,
_qa_issues_for_chapter,
_recheck_utterance_qa,
)
def _make_db(temp_dir: Path) -> tuple[sqlite3.Connection, types.SimpleNamespace]:
db = sqlite3.connect(str(temp_dir / "project.db"))
db.row_factory = sqlite3.Row
db.executescript(SCHEMA_SQL)
db.execute(
"INSERT INTO project (id, name, source_path, source_format) VALUES (1, 'p', '', 'txt')"
)
db.execute(
"INSERT INTO chapters (id, project_id, chapter_index, title, source_text) "
"VALUES (1, 1, 1, 'Slate', '')"
)
db.commit()
return db, types.SimpleNamespace(data_dir=temp_dir)
def _write_speakers(temp_dir: Path) -> None:
(temp_dir / "speakers.json").write_text(
json.dumps(
{
"speakers": [
{"primary_name": "Slate", "aliases": []},
{"primary_name": "Xylia", "aliases": []},
]
}
)
)
def _insert_utt(
db: sqlite3.Connection, utt_id: int, idx: int, text: str, utt_type: str, speaker: str | None
) -> None:
db.execute(
"INSERT INTO utterances (id, chapter_id, utterance_index, text, type, speaker_name) "
"VALUES (?, 1, ?, ?, ?, ?)",
(utt_id, idx, text, utt_type, speaker),
)
db.commit()
def test_qa_accepted_sig_responds_to_speaker_and_text() -> None:
sig = _qa_accepted_sig("Slate", "Move now.")
assert sig == _qa_accepted_sig("Slate", "Move now.")
assert sig != _qa_accepted_sig("Xylia", "Move now.")
assert sig != _qa_accepted_sig("Slate", "Move now")
def test_qa_accepted_for_staleness() -> None:
sig = _qa_accepted_sig("Slate", "Move now.")
assert _qa_accepted_for(1, sig, "Slate", "Move now.") is True
# Speaker or text changed -> acceptance must not hide a fresh issue.
assert _qa_accepted_for(1, sig, "Xylia", "Move now.") is False
assert _qa_accepted_for(1, sig, "Slate", "Different") is False
assert _qa_accepted_for(0, sig, "Slate", "Move now.") is False
def test_load_speaker_alias_map_includes_aliases_and_first_words(temp_dir: Path) -> None:
_write_speakers(temp_dir)
db, _ = _make_db(temp_dir)
db.close()
(temp_dir / "speakers.json").write_text(
json.dumps(
{
"speakers": [
{
"primary_name": "Captain Slate Ironbeard",
"aliases": ["Slate", "The Captain"],
},
{"primary_name": "Xylia", "aliases": []},
]
}
)
)
alias_map = _load_speaker_alias_map(types.SimpleNamespace(data_dir=temp_dir))
assert alias_map["slate"] == "Captain Slate Ironbeard"
assert alias_map["the captain"] == "Captain Slate Ironbeard"
assert alias_map["captain"] == "Captain Slate Ironbeard"
assert alias_map["xylia"] == "Xylia"
assert alias_map.get("another") is None
def test_qa_issues_for_chapter_flags_mismatch_and_narrator_dialogue(temp_dir: Path) -> None:
_write_speakers(temp_dir)
db, stub = _make_db(temp_dir)
_insert_utt(db, 1, 0, 'Slate says, "We have to hurry."', "narration", None)
_insert_utt(db, 2, 1, "Fine, let's go.", "dialogue", "Xylia")
_insert_utt(db, 3, 2, "Great, we're ready.", "dialogue", None)
issues = _qa_issues_for_chapter(db, 1, _load_speaker_alias_map(stub))
db.close()
assert {i["type"] for i in issues[1]} == {"speech_tag_mismatch"}
tag = issues[1][0]
assert tag["expected"] == "Slate"
assert tag["confidence"] == 0.9
assert {i["type"] for i in issues[2]} == {"narrator_dialogue"}
assert issues[2][0]["confidence"] == 0.95
assert 0 not in issues
def test_recheck_auto_fixes_resolved_mismatch(temp_dir: Path) -> None:
_write_speakers(temp_dir)
db, stub = _make_db(temp_dir)
_insert_utt(db, 1, 0, 'Slate says, "We have to hurry."', "narration", None)
_insert_utt(db, 2, 1, "Fine, let's go.", "dialogue", "Xylia")
payload = _recheck_utterance_qa(db, stub, 2)
assert payload["clean"] is True
assert payload["issues"] == []
assert payload["fixed"] == [{"utt_id": 2, "utt_index": 1, "from": "Xylia", "to": "Slate"}]
row = db.execute("SELECT speaker_name FROM utterances WHERE id = 2").fetchone()
assert row["speaker_name"] == "Slate"
db.close()
def test_recheck_leaves_unresolved_role_tag_flagged(temp_dir: Path) -> None:
_write_speakers(temp_dir)
db, stub = _make_db(temp_dir)
_insert_utt(db, 1, 0, 'Another says, "Someone must go."', "narration", None)
_insert_utt(db, 2, 1, "I will not stay.", "dialogue", "Slate")
payload = _recheck_utterance_qa(db, stub, 2)
assert payload["clean"] is False
assert payload["fixed"] == []
assert payload["issues"][0]["type"] == "speech_tag_mismatch"
assert payload["issues"][0]["confidence"] == 0.7
row = db.execute("SELECT speaker_name FROM utterances WHERE id = 2").fetchone()
assert row["speaker_name"] == "Slate"
db.close()
def test_recheck_unknown_utterance_returns_error(temp_dir: Path) -> None:
db, stub = _make_db(temp_dir)
payload = _recheck_utterance_qa(db, stub, 999)
db.close()
assert payload["error"] == "utterance not found"
def test_accept_records_across_speaker_change(temp_dir: Path) -> None:
_write_speakers(temp_dir)
db, stub = _make_db(temp_dir)
_insert_utt(db, 1, 0, 'Slate says, "We have to hurry."', "narration", None)
_insert_utt(db, 2, 1, "Fine, let's go.", "dialogue", "Xylia")
payload = _qa_accept_utterance(db, 2)
assert payload == {"utterance_id": 2, "accepted": True}
row = db.execute("SELECT qa_accepted, qa_accepted_sig FROM utterances WHERE id = 2").fetchone()
assert row["qa_accepted"] == 1
assert _qa_accepted_for(row["qa_accepted"], row["qa_accepted_sig"], "Xylia", "Fine, let's go.")
# The recheck loop re-assigns the speaker; the old acceptance must lapse.
db.execute("UPDATE utterances SET speaker_name = 'Slate' WHERE id = 2")
db.commit()
stale = db.execute(
"SELECT qa_accepted, qa_accepted_sig FROM utterances WHERE id = 2"
).fetchone()
assert (
_qa_accepted_for(stale["qa_accepted"], stale["qa_accepted_sig"], "Slate", "Fine, let's go.")
is False
)
db.close()
def test_accept_unknown_utterance_returns_error(temp_dir: Path) -> None:
db, _ = _make_db(temp_dir)
payload = _qa_accept_utterance(db, 999)
db.close()
assert payload["error"] == "utterance not found"
def test_ensure_utterance_columns_migrates_legacy_db(temp_dir: Path) -> None:
"""A project DB predating the qa_accepted/user_edited columns must upgrade.
Raw-connection routes (chapter audio, speaker/text/emotion edits, add
utterance) read and write these columns, so a missing migration 500s the
whole page.
"""
db = sqlite3.connect(str(temp_dir / "project.db"))
db.execute("CREATE TABLE utterances (id INTEGER PRIMARY KEY, chapter_id INTEGER, text TEXT)")
db.commit()
_ensure_utterance_columns(db)
cols = {r[1] for r in db.execute("PRAGMA table_info(utterances)")}
assert {"qa_accepted", "qa_accepted_at", "qa_accepted_sig", "user_edited"} <= cols
# Idempotent: a second call must not raise (columns already present).
_ensure_utterance_columns(db)
db.close()

207
tests/test_resegment_merge.py Normal file
View File

@@ -0,0 +1,207 @@
#!/usr/bin/env python3
"""Regression tests for resegment_merge.py."""
from __future__ import annotations
import sqlite3
from typing import TYPE_CHECKING
from scripts.resegment_merge import _merge_chapter, _norm, _TargetPool
if TYPE_CHECKING:
from pathlib import Path
# --------------------------------------------------------------------------- #
# helpers
# --------------------------------------------------------------------------- #
def _make_db(tmp_path: Path, chapter_rows: list[dict], title: str, source: str) -> Path:
"""Create a minimal project.db with a single chapter + utterance rows."""
db_path = tmp_path / "project.db"
db = sqlite3.connect(str(db_path))
db.execute("""
CREATE TABLE chapters (
id INTEGER PRIMARY KEY,
title TEXT,
chapter_index INTEGER NOT NULL,
source_text TEXT,
status TEXT
)
""")
db.execute("""
CREATE TABLE utterances (
id INTEGER PRIMARY KEY,
chapter_id INTEGER NOT NULL,
utterance_index INTEGER NOT NULL,
text TEXT NOT NULL,
type TEXT NOT NULL DEFAULT 'narration',
context_before TEXT,
context_after TEXT,
emotion TEXT,
pace TEXT,
emphasis_words TEXT,
pause_before_ms INTEGER,
pause_after_ms INTEGER,
instruct TEXT,
speaker_name TEXT,
attribution_confidence REAL DEFAULT 1.0,
attribution_reasoning TEXT,
emotion_confidence REAL DEFAULT 1.0,
user_edited INTEGER NOT NULL DEFAULT 0,
status TEXT NOT NULL DEFAULT 'pending'
)
""")
db.execute("INSERT INTO chapters (id, title, chapter_index, source_text) VALUES (1,?,?,?)",
(title, 1, source))
for r in chapter_rows:
db.execute(
"INSERT INTO utterances (id, chapter_id, utterance_index, text, type, "
"speaker_name, user_edited, emotion, pace, instruct, status) "
"VALUES (?, 1, ?, ?, ?, ?, ?, ?, ?, ?, 'pending')",
(r["id"], r["index"], r["text"], r["type"],
r.get("speaker", "narrator"), r.get("user_edited", 0),
r.get("emotion"), r.get("pace"), r.get("instruct")),
)
db.commit()
db.close()
return db_path
def _read_rows(db_path: Path, chapter_id: int = 1) -> list[dict]:
db = sqlite3.connect(str(db_path))
db.row_factory = sqlite3.Row
rows = db.execute(
"SELECT id, utterance_index, text, type, speaker_name, user_edited "
"FROM utterances WHERE chapter_id=? ORDER BY utterance_index",
(chapter_id,),
).fetchall()
db.close()
return [dict(r) for r in rows]
def _make_seg():
from umbraaudio.mapping.segmenter import Segmenter
return Segmenter()
# --------------------------------------------------------------------------- #
# tests
# --------------------------------------------------------------------------- #
class TestNorm:
def test_strips_quotes_and_commas(self):
assert _norm("\u201cHello,\u201d he said.") == "hello, he said"
def test_case_insensitive(self):
assert _norm("Hallr says,") == _norm("hallr says,")
class TestTargetPool:
def test_exact_pop(self):
pool = _TargetPool([
{"text": "Hallr shrugs.", "type": "narration"},
{"text": "Hallr shrugs.", "type": "narration"},
])
s = pool.exact("hallr shrugs")
assert s is not None
assert s["text"] == "Hallr shrugs."
s2 = pool.exact("hallr shrugs")
assert s2 is not None
assert pool.exact("hallr shrugs") is None
def test_narration_rewrite_containment(self):
pool = _TargetPool([
{"text": "Ready to Fight\nHallr was up early.", "type": "narration"},
])
s = pool.narration_rewrite("hallr was up early")
assert s is not None
assert "Hallr" in s["text"]
class TestMergeChapter:
def test_dialogue_preserved(self, tmp_path):
source = 'Hallr says, \u201cI will fight.\u201d\nFeliu nods.\n\u201cSounds good.\u201d'
db = _make_db(tmp_path, [
{"id": 1, "index": 0, "text": "\u201cI will fight.\u201d",
"type": "dialogue", "speaker": "Hallr"},
{"id": 2, "index": 1, "text": "Feliu nods.", "type": "narration"},
{"id": 3, "index": 2, "text": "\u201cSounds good.\u201d",
"type": "dialogue", "speaker": "Feliu"},
], "Test", source)
seg = _make_seg()
kept, new, rewritten, dropped = _merge_chapter(
seg, db, _read_db_chapter(db, 1), ["Hallr", "Feliu"], 0.82, dry_run=True
)
rows = _read_rows(db, 1)
# dialogue ids and speakers unchanged
assert rows[0]["id"] == 1 and rows[0]["speaker_name"] == "Hallr"
assert rows[2]["id"] == 3 and rows[2]["speaker_name"] == "Feliu"
def test_leading_comma_rewrite(self, tmp_path):
source = "Hallr shrugs.\nHe moves on."
db = _make_db(tmp_path, [
{"id": 1, "index": 0, "text": ", Hallr shrugs.", "type": "narration"},
{"id": 2, "index": 1, "text": "He moves on.", "type": "narration"},
], "Test", source)
seg = _make_seg()
_merge_chapter(
seg, db, _read_db_chapter(db, 1), [], 0.82, dry_run=False
)
rows = _read_rows(db, 1)
assert rows[0]["text"] == "Hallr shrugs."
assert rows[0]["speaker_name"] == "narrator"
def test_user_edited_row_not_dropped(self, tmp_path):
source = "Hallr was tired."
db = _make_db(tmp_path, [
{"id": 1, "index": 0, "text": "Hallr was tired.", "type": "narration"},
{"id": 2, "index": 1, "text": "I hope you like the story.",
"type": "narration", "user_edited": 1},
], "Test", source)
seg = _make_seg()
kept, new, rewritten, dropped = _merge_chapter(
seg, db, _read_db_chapter(db, 1), [], 0.82, dry_run=True
)
assert dropped == []
rows = _read_rows(db, 1)
assert len(rows) == 2
def test_scene_break_preserved_and_added(self, tmp_path):
source = "Scene one.\n<<<>>>\nScene two."
db = _make_db(tmp_path, [
{"id": 1, "index": 0, "text": "Scene one.", "type": "narration"},
{"id": 2, "index": 1, "text": "<<<>>>", "type": "scene_break"},
{"id": 3, "index": 2, "text": "Scene two.", "type": "narration"},
], "Test", source)
seg = _make_seg()
_merge_chapter(
seg, db, _read_db_chapter(db, 1), [], 0.82, dry_run=False
)
rows = _read_rows(db, 1)
scene_rows = [r for r in rows if r["type"] == "scene_break"]
assert len(scene_rows) == 1
assert scene_rows[0]["speaker_name"] == "scene_break"
def test_indices_renumbered(self, tmp_path):
source = "One.\n<<<>>>\nTwo.\nThree."
db = _make_db(tmp_path, [
{"id": 1, "index": 0, "text": "One.", "type": "narration"},
{"id": 3, "index": 2, "text": "Three.", "type": "narration"},
{"id": 2, "index": 1, "text": "Two.", "type": "narration"},
], "Test", source)
seg = _make_seg()
_merge_chapter(
seg, db, _read_db_chapter(db, 1), [], 0.82, dry_run=False
)
rows = _read_rows(db, 1)
indices = [r["utterance_index"] for r in rows]
assert indices == list(range(len(rows)))
def _read_db_chapter(db_path: Path, chapter_id: int) -> dict:
db = sqlite3.connect(str(db_path))
db.row_factory = sqlite3.Row
r = db.execute("SELECT id, title, chapter_index, source_text FROM chapters WHERE id=?",
(chapter_id,)).fetchone()
db.close()
return dict(r)

65
tests/test_scene_break.py Normal file
View File

@@ -0,0 +1,65 @@
from __future__ import annotations
import numpy as np
import soundfile as sf
from umbraaudio.assembly import AudioStitcher
from umbraaudio.generation.scene_break import (
SCENE_BREAK_SILENCE_MS,
is_scene_break,
needs_silence,
write_scene_break_audio,
)
def test_is_scene_break_by_text() -> None:
assert is_scene_break("<<<>>>")
assert is_scene_break(" <<<>>> ")
def test_is_scene_break_by_type() -> None:
assert is_scene_break("", "scene_break")
assert not is_scene_break("Hello", "dialogue")
assert is_scene_break("<<<>>>", "narration")
def test_write_scene_break_audio(tmp_path) -> None:
out = tmp_path / "utt_0000.wav"
assert needs_silence(out)
write_scene_break_audio(out, sample_rate=24000)
assert out.exists()
data, sr = sf.read(str(out))
assert sr == 24000
expected = int(24000 * SCENE_BREAK_SILENCE_MS / 1000)
assert abs(len(data) - expected) < 5
assert np.allclose(data, 0.0)
assert not needs_silence(out)
def test_zero_byte_file_is_stale(tmp_path) -> None:
out = tmp_path / "utt_0000.wav"
out.write_bytes(b"")
assert needs_silence(out)
def test_stitch_includes_scene_break_silence(tmp_path) -> None:
tone = np.full(2400, 0.1, dtype=np.float32)
before = tmp_path / "utt_0000.wav"
scene = tmp_path / "utt_0001.wav"
after = tmp_path / "utt_0002.wav"
sf.write(str(before), tone, 24000)
sf.write(str(after), tone, 24000)
write_scene_break_audio(scene, sample_rate=24000)
output = tmp_path / "ch1_full.wav"
AudioStitcher().stitch_utterances([before, scene, after], output)
data, sr = sf.read(str(output))
assert sr == 24000
gap = int(24000 * 500 / 1000)
scene_samples = int(24000 * SCENE_BREAK_SILENCE_MS / 1000)
expected = 2400 + gap + scene_samples + gap + 2400
assert abs(len(data) - expected) < 10
assert np.allclose(data[2400 + gap : 2400 + gap + scene_samples], 0.0)

37
tests/test_segmenter.py
View File

@@ -83,6 +83,43 @@ def test_author_note_splits_into_sentences() -> None:
assert texts[3].startswith("Deal Hunter uses the generic terms")
def test_author_note_numbered_list_splits_on_sentence_end() -> None:
seg = Segmenter()
text = (
"Author’s Note\n\n"
"This novel started as an exploration of several questions.\n\n"
"1) A dragon buys nothing, yet it hoards gold. Why does the dragon"
" consider gold plates, cups, and ornaments a treasure?\n"
"2) How does a dwarven city inside the mountains feed everyone with no farms?\n"
"3) Why is the city located inside that specific mountain?\n"
"4) Dragons and dwarves live in the mountains. Hmm. Is there more?"
)
utts = seg.segment(text, 5, chapter_title="Author's Note")
texts = [u["text"] for u in utts]
question = "Why does the dragon consider gold plates, cups, and ornaments a treasure?"
item2 = "2) How does a dwarven city inside the mountains feed everyone with no farms?"
item3 = "3) Why is the city located inside that specific mountain?"
assert any(t == question for t in texts)
assert any(t == item2 for t in texts)
assert any(t == item3 for t in texts)
assert any(t == "4) Dragons and dwarves live in the mountains." for t in texts)
assert any(t == "Is there more?" for t in texts)
assert all(u["type"] == "narration" for u in utts)
def test_sentence_end_does_not_split_on_money_or_number() -> None:
seg = Segmenter()
text = (
"The price is $2. 50 more than the last model. Section 3. 4 of the manual"
" explains it all."
)
utts = seg.segment(text, 1, character_names=["Nanny"])
narration = [u for u in utts if u["type"] == "narration"]
joined = " ".join(u["text"] for u in narration)
assert "$2. 50 more than the last model" in joined
assert "Section 3. 4 of the manual" in joined
def test_author_note_all_narration() -> None:
seg = Segmenter()
text = (

156
tests/test_variations.py Normal file
View File

@@ -0,0 +1,156 @@
from __future__ import annotations
import io
import httpx
import numpy as np
import soundfile as sf
from umbraaudio.generation.chapter import (
EMOTION_INSTRUCT_MAP,
EMOTION_OPTIONS,
NARRATOR_COMPRESSION_INSTRUCT,
PACE_INSTRUCT,
PACE_OPTIONS,
compose_utterance_instruct,
)
from umbraaudio.voices.qwen_tts import QwenTTSEngine
class _FakeResponse:
def __init__(self, content: bytes) -> None:
self.status_code = 200
self.content = content
@property
def headers(self) -> dict[str, str]:
return {"content-type": "audio/wav"}
def json(self) -> dict[str, object]:
return {}
def _tiny_wav_bytes() -> bytes:
sr = 24000
t = np.linspace(0, 0.1, int(sr * 0.1), dtype=np.float32)
data = (0.1 * np.sin(2 * np.pi * 220 * t)).astype(np.float32)
buf = io.BytesIO()
sf.write(buf, data, sr, format="WAV")
return buf.getvalue()
class TestComposeUtteranceInstruct:
def test_no_variation_returns_none(self) -> None:
assert compose_utterance_instruct() is None
assert compose_utterance_instruct(emotion="neutral", pace="moderate") is None
assert compose_utterance_instruct("", "neutral", "moderate") is None
def test_custom_instruct_only(self) -> None:
assert compose_utterance_instruct("Growl this line.") == "Growl this line."
def test_emotion_adds_prescriptive_text(self) -> None:
result = compose_utterance_instruct(emotion="angry")
assert "anger" in result
assert result == EMOTION_INSTRUCT_MAP["angry"]
def test_neutral_emotion_contributes_nothing(self) -> None:
result = compose_utterance_instruct("Say it plain.", emotion="neutral")
assert result == "Say it plain."
def test_pace_drives_tempo(self) -> None:
fast = compose_utterance_instruct(pace="fast")
assert fast and "quick" in fast
slow = compose_utterance_instruct(pace="slow")
assert slow and "slow" in slow
assert compose_utterance_instruct(pace="moderate") is None
def test_unknown_emotion_is_skipped(self) -> None:
result = compose_utterance_instruct("Keep this.", emotion="not-a-real-emotion")
assert result == "Keep this."
def test_combines_custom_emotion_pace(self) -> None:
result = compose_utterance_instruct(
"Hold the line.", emotion="furious", pace="fast"
)
assert result.startswith("Hold the line.")
assert "rage" in result
assert "quick" in result
assert len(result.split(". ")) == 1 + 2 + 1
def test_every_emotion_option_has_a_directive_except_neutral(self) -> None:
for option in EMOTION_OPTIONS:
assert option in EMOTION_INSTRUCT_MAP, f"missing {option} in map"
if option == "neutral":
assert EMOTION_INSTRUCT_MAP[option] == ""
else:
assert EMOTION_INSTRUCT_MAP[option].strip(), (
f"{option} has empty directive"
)
def test_map_keys_stay_in_sync_with_options(self) -> None:
assert set(EMOTION_INSTRUCT_MAP) == set(EMOTION_OPTIONS)
def test_pace_options_stay_in_sync_with_map(self) -> None:
assert set(PACE_INSTRUCT) == set(PACE_OPTIONS)
assert PACE_INSTRUCT["moderate"] == ""
assert PACE_INSTRUCT["slow"] and PACE_INSTRUCT["fast"]
class TestNarratorProfile:
def test_narrator_compression_targets_calm_even_delivery(self) -> None:
assert "Do not dramatize" in NARRATOR_COMPRESSION_INSTRUCT
assert "steadily" in NARRATOR_COMPRESSION_INSTRUCT
class TestRemoteInstructComposition:
def test_instruct_sent_alongside_voice_ref(self, tmp_path, monkeypatch) -> None:
ref = tmp_path / "narrator_ref.wav"
ref.write_bytes(_tiny_wav_bytes())
captured: dict[str, object] = {}
def fake_post(
url: str,
data: dict[str, object],
files: dict[str, object],
timeout: float,
) -> _FakeResponse:
captured["data"] = data
captured["files"] = files
return _FakeResponse(_tiny_wav_bytes())
monkeypatch.setattr(httpx, "post", fake_post)
engine = QwenTTSEngine(qwen_tts_url="http://tts.test:8881")
out = tmp_path / "out.wav"
engine.generate(
text="Hello.",
output_path=out,
voice_ref=ref,
instruct=EMOTION_INSTRUCT_MAP["angry"],
)
data: dict[str, object] = captured["data"]
assert data["voice_description"] == EMOTION_INSTRUCT_MAP["angry"]
assert captured["files"], "ref audio must still be attached"
assert out.exists()
def test_no_instruct_means_no_voice_description(self, tmp_path, monkeypatch) -> None:
ref = tmp_path / "kainda_ref.wav"
ref.write_bytes(_tiny_wav_bytes())
captured: dict[str, object] = {}
def fake_post(
url: str,
data: dict[str, object],
files: dict[str, object],
timeout: float,
) -> _FakeResponse:
captured["data"] = data
return _FakeResponse(_tiny_wav_bytes())
monkeypatch.setattr(httpx, "post", fake_post)
engine = QwenTTSEngine(qwen_tts_url="http://tts.test:8881")
engine.generate(text="Hello.", output_path=tmp_path / "out.wav", voice_ref=ref)
assert "voice_description" not in captured["data"]

38
tests/test_vocals_project.py Normal file
View File

@@ -0,0 +1,38 @@
from __future__ import annotations
import sys
from pathlib import Path
sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
from scripts.create_vocals_project import ( # noqa: E402
RAINBOW_SENTENCES,
SPEAKERS,
VARIATIONS,
plan_utterances,
)
def test_plan_utterances_covers_both_speakers_and_all_variations() -> None:
plan = plan_utterances()
assert len(plan) == len(SPEAKERS) * len(VARIATIONS)
assert [u["speaker_name"] for u in plan] == (
[SPEAKERS[0]] * len(VARIATIONS) + [SPEAKERS[1]] * len(VARIATIONS)
)
assert [u["emotion"] for u in plan[: len(VARIATIONS)]] == VARIATIONS
assert [u["emotion"] for u in plan[len(VARIATIONS):]] == VARIATIONS
def test_plan_utterances_use_rainbow_sentences() -> None:
plan = plan_utterances()
used = {u["text"] for u in plan}
assert set(RAINBOW_SENTENCES) == used
for u in plan:
assert u["text"] in RAINBOW_SENTENCES
assert u["pace"] == "moderate"
def test_rainbow_sentences_are_phonetically_distinct() -> None:
assert len(RAINBOW_SENTENCES) >= 6
long_enough = all(len(s.split()) >= 6 for s in RAINBOW_SENTENCES)
assert long_enough

27
umbraaudio/analysis/character_discovery.py
View File

@@ -6,6 +6,7 @@ import re
import time
from dataclasses import dataclass, field
from pathlib import Path
from collections.abc import Callable
from typing import TYPE_CHECKING, Any
if TYPE_CHECKING:
@@ -228,17 +229,22 @@ class CharacterDiscoverer:
m["id"] for m in models if m.get("id") and m["id"] not in embedding_models
]
if chat_models:
preferred = ["google/gemma-4-e4b", "qwen3.5-9b", "google/gemma-4-26b-a4b"]
preferred = [
self.config.discovery_model or "google/gemma-4-e4b",
"google/gemma-4-e4b",
"qwen3.5-9b",
]
self._active_model = next(
(m for m in preferred if m in chat_models),
chat_models[0],
)
logger.info("Using LM Studio model: %s", self._active_model)
logger.info("Discovery using LM Studio model: %s", self._active_model)
from openai import OpenAI
self._client = OpenAI(
base_url=self.config.lmstudio_base_url,
api_key="lm-studio",
timeout=600,
)
return self._client
except Exception:
@@ -260,6 +266,7 @@ class CharacterDiscoverer:
max_chars: int = 40000,
known_speakers: str = "",
pass_context: str = "",
on_retry: Callable[[int], None] | None = None,
) -> dict:
if len(text) > max_chars:
logger.info("Truncating chapter text from %d to %d chars", len(text), max_chars)
@@ -293,6 +300,7 @@ class CharacterDiscoverer:
messages=[{"role": "user", "content": prompt}],
temperature=0.2,
max_tokens=8192,
timeout=600,
)
break
except Exception as e:
@@ -300,8 +308,13 @@ class CharacterDiscoverer:
logger.warning(
"LLM call failed (attempt %d/%d): %s", attempt + 1, max_attempts, err
)
if on_retry is not None:
try:
on_retry(attempt + 1)
except Exception:
pass
if attempt < max_attempts - 1:
time.sleep(20 * (attempt + 1))
time.sleep(min(60, 5 * (attempt + 1)))
else:
logger.error("LLM call failed after %d attempts: %s", max_attempts, err)
raise
@@ -649,6 +662,8 @@ class CharacterDiscoverer:
if name_a_lower == name_b_lower:
self._absorb_into_richer(merged, name_a, name_b)
changed = True
if name_a not in merged:
break
continue
b_aliases_lower = [a.lower() for a in speaker_b.aliases]
@@ -663,6 +678,8 @@ class CharacterDiscoverer:
):
self._absorb_into_richer(merged, name_a, name_b)
changed = True
if name_a not in merged:
break
elif shared_specific:
logger.info(
"Dedup: merging '%s' and '%s' (shared specific alias)",
@@ -670,6 +687,8 @@ class CharacterDiscoverer:
)
self._absorb_into_richer(merged, name_a, name_b)
changed = True
if name_a not in merged:
break
return merged
@@ -693,6 +712,8 @@ class CharacterDiscoverer:
def _absorb_into_richer(
self, merged: dict[str, Speaker], name_a: str, name_b: str
) -> None:
if name_a not in merged or name_b not in merged:
return
speaker_a = merged[name_a]
speaker_b = merged[name_b]

14
umbraaudio/analysis/llm_attributor.py
View File

@@ -56,26 +56,26 @@ You are attributing dialogue for: {scene_label}
1. Each utterance is already correctly split — do NOT merge or split them
2. EVERY utterance MUST be attributed to a speaker. Return EXACTLY one entry per utterance index. Do not skip any.
3. ALL these utterances are dialogue (in quotes) — they are spoken by a CHARACTER, almost never "Narrator". Only use "Narrator" if the line is truly un-attributable.
3b. NARRATION-OVERRIDES-CONTENT: The 1-2 narration utterances IMMEDIATELY BEFORE a dialogue line are the STRONGEST signal for speaker identity. Always check these FIRST before looking at dialogue content. When narration says a character IS speaking — using ANY pattern ("Kainda says", "Kainda nods and says", "Kainda checks the screen and says", "Kainda hits a mute button and says", "Kainda is looking at X and says", "Kainda continues talking with Hunter", "Kainda pauses and says", "Kainda is telling the princess") — the following quoted lines are THAT character's dialogue, until another narration tag indicates a different speaker. This rule OVERRIDES any guess based on the dialogue content.
3b. NARRATION-OVERRIDES-CONTENT: The 1-2 narration utterances IMMEDIATELY BEFORE a dialogue line are the STRONGEST signal for speaker identity. Always check these FIRST before looking at dialogue content. When narration says a character IS speaking — using ANY pattern ("X says", "X nods and says", "X checks the screen and says", "X hits a mute button and says", "X is looking at Y and says", "X continues talking with Y", "X pauses and says", "X is telling Y") where X is the speaker's name or role — the following quoted lines are THAT character's dialogue, until another narration tag indicates a different speaker. This rule OVERRIDES any guess based on the dialogue content.
3c. A multi-line speech block from one character continues until dialogue tags, action, or the responder's lines indicate a change.
3d. A single quoted dialogue utterance may contain MULTIPLE sentences (e.g. one paragraph of speech) — every sentence within it belongs to the SAME speaker. Do not split the speaker across sentences within one utterance.
3e. Attribute to Deal Hunter (or AI-1) ONLY when narration explicitly identifies a computer/AI/communication system as the speaker (e.g. "the ship says", "the computer says", "Hunter says", "her ear device says", "the comm crackles, '...'"). A bare "a voice is heard" / "the voice calls again" / "someone says" over a radio or in the distance is a HUMAN speaker unless the text says otherwise — use gender cues in the narration ("it sounds female") to pick the human.
3e. Attribute to a computer/AI/communication-system character ONLY when narration explicitly identifies a computer/AI/communication system as the speaker (e.g. "the ship says", "the computer says", "'[name]' says", "her ear device says", "the comm crackles, '...'") AND such a character exists in the Character Profiles. A bare "a voice is heard" / "the voice calls again" / "someone says" over a radio or in the distance is a HUMAN speaker unless the text says otherwise — use gender cues in the narration ("it sounds female") to pick the human.
3f. When narration says "She says," or "He says," resolve the pronoun to the most recently mentioned character of the appropriate gender in the recent context above. Never resolve a pronoun to a character who is absent from the scene, and never resolve to a named character when the pronoun's antecedent is an unnamed role ("she asks the lead guard... he replies" → the guard speaks).
3g. UNNAMED ROLES: when a speech tag names a role or title WITHOUT a proper name ("the lead guard", "the doctor", "a pilot", "a pirate", "a Warbot", "a guard"), the speaker is that ROLE. If the Character Profiles list contains a matching role speaker (e.g. "Chief of the Guard", "The Doctor", "Warbots", "Pilot (Freighter)"), use its EXACT primary_name. Otherwise use a descriptive Title Case role name (e.g. "Lead Guard"). NEVER attribute such a line to an unrelated named character merely because that character is present in the scene. Guessing a wrong named character is worse than using a role name.
3h. REPEATED LINES: an identical quoted line that appears again in the same scene ("The voice calls again, 'Help me, please.'") is spoken by the SAME speaker as the first occurrence. When narration says "again", "another voice", or restates a previous line, reuse the earlier speaker.
3i. RETROACTIVE NAMING: If an unnamed role ("the robot", "a guard") is later given a name ("Niñera", "Support Robot 74532") in the same scene, ALL prior dialogue by that role should be attributed to that name.
3j. ADDRESS PATTERN: When dialogue addresses someone by name or title ("Hunter, kill the connection" / "Yes, Captain" / "Captain, we need to jump"), the SPEAKER is NOT the person being addressed. The speaker is whoever was identified as speaking in the preceding narration. If narration says "Kainda says" and the dialogue reads "Hunter, kill the connection", the speaker is Kainda (she is talking TO Hunter). If narration says "Deal Hunter says" and the dialogue reads "Captain, we need to jump", the speaker is Deal Hunter (it is talking TO the Captain).
3k. SELF-ADDRESS IMPOSSIBLE: A character never addresses themselves by name or title in their own dialogue. If a line says "Yes, Captain" and the Captain is Kainda, then Kainda is NOT the speaker — someone else is responding to her. If a line says "Hunter, do X" and Hunter is Deal Hunter, then Deal Hunter is NOT the speaker.
3j. ADDRESS PATTERN: When dialogue addresses someone by name or title ("X, kill the connection" / "Yes, Captain" / "Captain, we need to jump"), the SPEAKER is NOT the person being addressed. The speaker is whoever was identified as speaking in the preceding narration. If narration says "X says" and the dialogue reads "Y, kill the connection", the speaker is X (talking TO Y). If narration says "the ship's AI says" and the dialogue reads "Captain, we need to jump", the speaker is the AI (talking TO the Captain).
3k. SELF-ADDRESS IMPOSSIBLE: A character never addresses themselves by name or title in their own dialogue. If a line says "Yes, Captain" and the Captain is X, then X is NOT the speaker — someone else is responding to her. If a line says "Y, do Z" and Y is X, then X is NOT the speaker.
4. Use dialogue tags (X says/replies/asks) to identify speakers
5. Use context: who is present, who was speaking before, who is addressed
6. Resolve pronouns: "she says" refers to the most recently mentioned female character
7. If a character has aliases, use their EXACT full primary_name shown in the profiles (e.g. "Kainda Walters Foster Murphy", NOT "Kainda" or "Princess Kainda")
7. If a character has aliases, use their EXACT full primary_name shown in the profiles (e.g. the full name as listed, NOT a short form or nickname)
8. The "current_speaker" field shows who the mechanical segmenter guessed — use it as a hint but correct if wrong
9. In a conversation, speakers usually alternate. If one line is Character A, the next line is likely a different character (often the one addressed).
10. CRITICAL — only return the EXACT utterance indices provided in the input. Do NOT invent or hallucinate extra indices.
11. CRITICAL — every returned speaker must be the ACTUAL character speaking. Assigning the wrong speaker ruins the audiobook. When genuinely uncertain, prefer an unnamed role speaker over a random named character.
12. AI-1 is a specific AI character voice. Attribute dialogue to AI-1 when the text indicates an AI or computer is speaking (e.g., "ear device", "ship AI", "computer voice"). Do NOT default undetermined utterances to AI-1 if another character is more likely.
13. KEYWORD NON-MATCHING: When a character's name appears in the TEXT of a dialogue line (e.g., "The ship is the Deal Hunter", "I know Captain Weston", "We are near the Deal Hunter"), it does NOT mean that character is speaking. The speaker is determined by the preceding narration tag, NOT by character names mentioned inside the dialogue. A character can be TALKED ABOUT without being the speaker. Only attribute to a character when a narration tag or dialogue tag explicitly identifies them as the speaker.
12. If the Character Profiles include an AI/computer character (e.g. a ship AI), attribute dialogue to it only when the text indicates an AI or computer is speaking (e.g., "ear device", "ship AI", "computer voice"). Do NOT default undetermined utterances to an AI character if another character is more likely.
13. KEYWORD NON-MATCHING: When a character's name appears in the TEXT of a dialogue line (e.g., "The ship is the Kestrel", "I know Captain Smith", "We are near the Kestrel"), it does NOT mean that character is speaking. The speaker is determined by the preceding narration tag, NOT by character names mentioned inside the dialogue. A character can be TALKED ABOUT without being the speaker. Only attribute to a character when a narration tag or dialogue tag explicitly identifies them as the speaker.
## Scene Attribution Rules (CRITICAL)
- Chapters can contain multiple scenes separated by scene breaks (<<<>>>)

550
umbraaudio/assembly/ar_export.py Normal file
View File

@@ -0,0 +1,550 @@
from __future__ import annotations
import json
import logging
import re
import subprocess
from dataclasses import dataclass, field
from datetime import UTC, datetime
from pathlib import Path
from typing import TYPE_CHECKING, Any
import numpy as np
import soundfile as sf
if TYPE_CHECKING:
from collections.abc import Callable
logger = logging.getLogger(__name__)
SAMPLE_RATE = 24000
TARGET_SAMPLE_RATE = 44100
BIT_RATE = "192k"
SILENCE_MS = 2000
MAX_TRACK_SEC = 119 * 60
MAX_FILE_BYTES = 170 * 1024 * 1024
RMS_LO_DB = -23.0
RMS_HI_DB = -18.0
PEAK_MAX_DB = -3.0
NORM_TARGET_RMS_DB = -20.5
NORM_MAX_GAIN_DB = 12.0
PEAK_LIMIT_LINEAR = float(10 ** (PEAK_MAX_DB / 20))
SAMPLE_MARKER_TEXT = "Welcome to the audio book"
STORY_START_RE = re.compile(r"\bchapter\s+(one|1)\b", re.IGNORECASE)
@dataclass
class ARChapter:
"""One manuscript chapter and how it participates in the AR package."""
chapter_id: int
chapter_index: int
title: str
first_utterance: str = ""
audio_path: Path | None = None
role: str = "chapter"
number: str = ""
@property
def display_title(self) -> str:
return (self.title or f"Chapter {self.chapter_id}").strip()
@dataclass
class ARTrack:
seq: int
role: str
title: str
filename: str
source: str
duration_s: float
size_bytes: int
number: str = ""
rms_db: float | None = None
peak_db: float | None = None
warning: str = ""
loudness_ok: bool = True
peak_ok: bool = True
segments: list[tuple[str, Any]] = field(default_factory=list)
@property
def display_title(self) -> str:
prefix = self.number
return f"{prefix} {self.title}".strip() if prefix else self.title
def as_dict(self) -> dict[str, Any]:
return {
"seq": self.seq,
"role": self.role,
"number": self.number,
"title": self.display_title,
"name": self.title,
"file": self.filename,
"source": self.source,
"duration_s": round(self.duration_s, 2),
"size_bytes": self.size_bytes,
"rms_db": self.rms_db,
"peak_db": self.peak_db,
"warning": self.warning,
"loudness_ok": self.loudness_ok,
"peak_ok": self.peak_ok,
}
class ARPackageExporter:
"""Build the Author's Republic MP3 submission package.
Every track maps 1:1 to a manuscript chapter that the author edits:
the sample chapter ("Sample Track"), each story chapter, and the final
chapter ("Closing Track"). No audio is synthesized — all content comes
from the project's ``chapter_audio`` files. Every file is forced to mono,
44.1 kHz, 192 kbit CBR MP3 with 2 seconds of silence at both ends. Each
track is loudness-normalized to a −20.5 dB RMS target with a hard −3 dBFS
peak ceiling so the package meets AR's loudness gates; a JSON manifest of
per-track statistics (duration, size, loudness, peak, AR compliance flags)
is written beside the files.
"""
def __init__(self, data_dir: Path, ffmpeg: str = "ffmpeg", ffprobe: str = "ffprobe") -> None:
self.data_dir = Path(data_dir)
self.ffmpeg = ffmpeg
self.ffprobe = ffprobe
self.package_dir = self.export_dir / "ar_package"
self.work_dir = self.export_dir / "ar_work"
@property
def export_dir(self) -> Path:
return self.data_dir / "export"
@property
def chapter_audio_dir(self) -> Path:
return self.data_dir / "chapter_audio"
# ------------------------------------------------------------------
# Public build entry point
# ------------------------------------------------------------------
def build(
self,
chapters: list[ARChapter],
meta: dict[str, str],
force: bool = False,
progress_cb: Callable[[int, str], None] | None = None,
) -> dict[str, Any]:
"""Assemble the full AR package and return the manifest.
``chapters`` must be in final book order with roles already assigned.
``meta`` is the publishing metadata (title, author, narrator,
publisher, output_name) used for file naming.
``progress_cb`` (if given) is called with ``(percent, detail)``.
"""
self.package_dir.mkdir(parents=True, exist_ok=True)
self.work_dir.mkdir(parents=True, exist_ok=True)
def _progress(percent: int, detail: str) -> None:
if progress_cb is not None:
progress_cb(percent, detail)
ordered = [c for c in chapters if c.audio_path is not None]
self._assign_numbers(ordered)
roles = {c.role: c for c in ordered}
base = self._base_name(meta)
tracks: list[ARTrack] = []
seq = 0
for role in ("sample", "opening", "closing"):
special = roles.get(role)
if special is not None:
track = self._make_track(
seq=seq + 1, role=role, number=special.number,
title=special.display_title,
segments=[("wav", special.audio_path)],
source=str(special.audio_path.name),
)
tracks.append(track)
seq += 1
for ch in ordered:
if ch.role != "chapter":
continue
track = self._make_track(
seq=seq + 1, role="chapter", number=ch.number,
title=ch.display_title,
segments=[("wav", ch.audio_path)],
source=str(ch.audio_path.name),
)
tracks.append(track)
seq += 1
filenames = self._filename_plan(base, tracks)
for i, (track, fname) in enumerate(zip(tracks, filenames, strict=True)):
track.filename = fname
out = self.package_dir / fname
self._encode_track(track, out)
_progress(
10 + int((i + 1) / len(tracks) * 90),
f"Encoding track {i + 1}/{len(tracks)} ({fname})...",
)
manifest = self._manifest(meta, base, tracks)
manifest_path = self.export_dir / "ar_package.json"
manifest_path.write_text(json.dumps(manifest, indent=2))
logger.info("AR package built: %s (%d tracks)", self.package_dir, len(tracks))
_progress(100, f"AR package built: {len(tracks)} tracks")
return manifest
# ------------------------------------------------------------------
# Chapter numbering
# ------------------------------------------------------------------
def _assign_numbers(self, chapters: list[ARChapter]) -> None:
"""Set per-chapter display numbers.
- sample / unnumbered front matter -> no number
- opening -> ``i``
- closing -> ``ii``
- story chapters -> 01, 02, ... in manuscript order, starting at the
first chapter that self-announces as ``Chapter one``/``Chapter 1``.
"""
story = 0
started = False
for ch in chapters:
if ch.role == "sample":
ch.number = ""
elif ch.role == "opening":
ch.number = "i"
elif ch.role == "closing":
ch.number = "ii"
elif ch.role == "chapter":
if not started and STORY_START_RE.search(ch.first_utterance or ""):
started = True
if started:
story += 1
ch.number = str(story)
else:
ch.number = ""
# ------------------------------------------------------------------
# Text / naming
# ------------------------------------------------------------------
def _base_name(self, meta: dict[str, str]) -> str:
raw = meta.get("output_name") or meta.get("title") or "audiobook"
stem = Path(raw).stem
stem = re.sub(r"[^\w\s.-]", "", stem).strip()
stem = re.sub(r"\s+", "_", stem)
return stem or "audiobook"
def _filename_plan(self, base: str, tracks: list[ARTrack]) -> list[str]:
plan = []
for track in tracks:
title = re.sub(r"[^\w\s-]", "", track.title).strip()
title = re.sub(r"\s+", "_", title) or "Track"
number = track.number
prefix = number.zfill(2) if number.isdigit() else number
part = f"{prefix}_" if prefix else ""
plan.append(f"{base}_{part}{title}.mp3")
return plan
# ------------------------------------------------------------------
# Track assembly, encoding, measurement
# ------------------------------------------------------------------
def _make_track(
self,
seq: int,
role: str,
title: str,
segments: list[tuple[str, Any]],
source: str = "",
number: str = "",
) -> ARTrack:
combined = self._concatenate(segments)
rms, peak = self._measure_loudness(combined)
duration_s = len(combined) / SAMPLE_RATE + 2 * SILENCE_MS / 1000
warning = (
f"track exceeds 119 minutes ({duration_s / 60:.1f} min)"
if duration_s > MAX_TRACK_SEC
else ""
)
return ARTrack(
seq=seq,
role=role,
number=number,
title=title,
filename="",
source=source,
duration_s=duration_s,
size_bytes=0,
rms_db=rms,
peak_db=peak,
warning=warning,
segments=segments,
)
def _encode_track(self, track: ARTrack, out: Path) -> None:
content = self._concatenate(track.segments)
content = self._normalize(content, track.rms_db)
rms, peak = self._measure_loudness(content)
if rms is not None:
track.rms_db, track.peak_db = rms, peak
track.warning = self._loudness_warning(rms, peak, track.warning)
track.loudness_ok = RMS_LO_DB <= rms <= RMS_HI_DB
track.peak_ok = peak is not None and peak <= PEAK_MAX_DB
combined = self._pad_silence(content)
wav = self.work_dir / f"combined_{track.seq:02d}.wav"
sf.write(str(wav), combined, SAMPLE_RATE)
self._encode_wav(wav, out)
track.size_bytes = out.stat().st_size
wav.unlink(missing_ok=True)
def encode_chapter_mp3(self, wav_path: Path, out: Path) -> dict[str, Any]:
"""Encode one chapter WAV into an AR-compliant MP3.
Mirrors :meth:`_encode_track` exactly (mono, 44.1 kHz, 192 kbit CBR
MP3, loudness-normalized to the −20.5 dB RMS target with the ±12 dB
gain limit and −3 dBFS peak ceiling, 2 s silence at both ends) so a
per-chapter ``Build Chapter`` result and the assembled AR package
produce identical audio. Returns per-file stats (duration, size,
loudness, peaks, AR compliance flags) for the caller.
"""
self.work_dir.mkdir(parents=True, exist_ok=True)
content, sr = sf.read(str(wav_path), dtype="float32")
if content.ndim > 1:
content = content.mean(axis=1)
if sr != SAMPLE_RATE:
content = self._resample_chunk(wav_path, sr)
content = np.asarray(content, dtype=np.float32)
src_rms, _ = self._measure_loudness(content)
content = self._normalize(content, src_rms)
rms, peak = self._measure_loudness(content)
content = self._pad_silence(content)
duration_s = round(len(content) / SAMPLE_RATE, 2)
warning = self._loudness_warning(rms, peak)
bits = [warning] if warning else []
if duration_s > MAX_TRACK_SEC:
bits.append(f"{duration_s:.0f}s exceeds {MAX_TRACK_SEC // 60} min")
work = self.work_dir / f"chapter_{wav_path.stem}.wav"
sf.write(str(work), content, SAMPLE_RATE)
self._encode_wav(work, out)
work.unlink(missing_ok=True)
size_bytes = out.stat().st_size
if size_bytes > MAX_FILE_BYTES:
bits.append(f"{size_bytes / (1024 * 1024):.0f} MB exceeds 170 MB")
return {
"duration_s": duration_s,
"size_bytes": size_bytes,
"rms_db": rms,
"peak_db": peak,
"warning": "; ".join(bits),
"loudness_ok": rms is not None and RMS_LO_DB <= rms <= RMS_HI_DB,
"peak_ok": peak is not None and peak <= PEAK_MAX_DB,
}
def _normalize(self, audio: np.ndarray, source_rms: float | None) -> np.ndarray:
audio = np.asarray(audio, dtype=np.float32)
if source_rms is None:
return audio
gain_db = min(max(NORM_TARGET_RMS_DB - source_rms, -NORM_MAX_GAIN_DB), NORM_MAX_GAIN_DB)
scaled = audio * float(10 ** (gain_db / 20))
np.clip(scaled, -PEAK_LIMIT_LINEAR, PEAK_LIMIT_LINEAR, out=scaled)
return scaled
def _loudness_warning(
self, rms: float | None, peak: float | None, base: str = ""
) -> str:
bits = [base] if base else []
if rms is not None and not (RMS_LO_DB <= rms <= RMS_HI_DB):
bits.append(f"RMS {rms:.1f} dB outside [{RMS_LO_DB}, {RMS_HI_DB}] dB")
if peak is not None and peak > PEAK_MAX_DB:
bits.append(f"peak {peak:.1f} dB above {PEAK_MAX_DB} dB")
return "; ".join(bits)
def _concatenate(self, segments: list[tuple[str, Any]]) -> np.ndarray:
chunks: list[np.ndarray] = []
for kind, value in segments:
if kind == "gap":
ms = int(value) # type: ignore[arg-type]
chunks.append(np.zeros(int(SAMPLE_RATE * ms / 1000), dtype=np.float32))
elif kind == "wav":
audio, sr = sf.read(str(value), dtype="float32")
if audio.ndim > 1:
audio = audio.mean(axis=1)
if sr != SAMPLE_RATE:
chunks.append(self._resample_chunk(value, sr))
else:
chunks.append(audio.astype(np.float32))
if not chunks:
return np.zeros(1, dtype=np.float32)
return np.concatenate(chunks)
def _resample_chunk(self, path: Path, sr: int) -> np.ndarray:
tmp = self.work_dir / f"resample_{path.stem}_{sr}.wav"
subprocess.run(
[self.ffmpeg, "-y", "-i", str(path), "-ar", str(SAMPLE_RATE), str(tmp)],
capture_output=True,
check=True,
)
audio, _ = sf.read(str(tmp), dtype="float32")
tmp.unlink(missing_ok=True)
return audio.astype(np.float32)
def _pad_silence(self, audio: np.ndarray) -> np.ndarray:
pad = np.zeros(int(SAMPLE_RATE * SILENCE_MS / 1000), dtype=np.float32)
return np.concatenate([pad, audio, pad])
def _measure_loudness(self, audio: np.ndarray) -> tuple[float | None, float | None]:
tmp = self.work_dir / "loudness_probe.wav"
sf.write(str(tmp), audio, SAMPLE_RATE)
try:
result = subprocess.run(
[self.ffmpeg, "-i", str(tmp), "-af", "volumedetect", "-f", "null", "-"],
capture_output=True, text=True, check=True,
)
except subprocess.CalledProcessError:
tmp.unlink(missing_ok=True)
return None, None
tmp.unlink(missing_ok=True)
stderr = result.stderr or ""
rms = peak = None
m = re.search(r"mean_volume:\s*([-\d.]+) dB", stderr)
if m:
rms = float(m.group(1))
m = re.search(r"max_volume:\s*([-\d.]+) dB", stderr)
if m:
peak = float(m.group(1))
return rms, peak
def _encode_wav(self, wav_path: Path, out: Path) -> None:
result = subprocess.run(
[
self.ffmpeg, "-y",
"-i", str(wav_path),
"-ar", str(TARGET_SAMPLE_RATE),
"-ac", "1",
"-c:a", "libmp3lame",
"-b:a", BIT_RATE,
str(out),
],
capture_output=True, text=True,
)
if result.returncode != 0:
raise RuntimeError(f"MP3 encoding failed: {result.stderr[-500:]}")
# ------------------------------------------------------------------
# Manifest
# ------------------------------------------------------------------
def _manifest(self, meta: dict[str, str], base: str, tracks: list[ARTrack]) -> dict[str, Any]:
return {
"book": {
"title": meta.get("title", ""),
"author": meta.get("author", ""),
"narrator": meta.get("narrator", ""),
"publisher": meta.get("publisher", ""),
"output_name": meta.get("output_name", ""),
},
"base_name": base,
"built_at": datetime.now(UTC).isoformat(timespec="seconds"),
"format": {
"codec": "mp3",
"sample_rate": TARGET_SAMPLE_RATE,
"channels": 1,
"bit_rate_kbps": 192,
"cbr": True,
"silence_s": SILENCE_MS / 1000,
},
"track_count": len(tracks),
"total_duration_s": round(sum(t.duration_s for t in tracks), 2),
"tracks": [t.as_dict() for t in tracks],
"warnings": [t.warning for t in tracks if t.warning],
}
def discover_chapters(data_dir: Path) -> list[ARChapter]:
"""Read the project DB + ``chapter_audio`` and assign AR package roles.
Role rules (transparent, per-project):
- **sample**: chapter whose first utterance opens with the book's promo
intro (``Welcome to the audio book``);
- **opening**: chapter titled ``Opening Track`` (intro, numbered ``i``);
- **closing**: the last chapter that has audio (final "Closing Track");
- **skip**: chapters with no utterances or no audio file — no track kept;
- **chapter**: everything else, in manuscript order. Display numbers are
assigned at build time by :meth:`ARPackageExporter._assign_numbers`.
"""
import sqlite3
db = sqlite3.connect(str(data_dir / "project.db"))
db.row_factory = sqlite3.Row
totals: dict[int, int] = {}
for r in db.execute("SELECT chapter_id, COUNT(*) AS n FROM utterances GROUP BY chapter_id"):
totals[r["chapter_id"]] = r["n"]
first: dict[int, str] = {}
for r in db.execute(
"SELECT chapter_id, utterance_index, text FROM utterances "
"ORDER BY chapter_id, utterance_index"
):
if r["chapter_id"] not in first:
first[r["chapter_id"]] = (r["text"] or "").strip()
rows = db.execute(
"SELECT id, chapter_index, title FROM chapters ORDER BY chapter_index"
).fetchall()
db.close()
audio_dir = data_dir / "chapter_audio"
chapters: list[ARChapter] = []
for r in rows:
cid = r["id"]
if totals.get(cid, 0) == 0:
continue
title = (r["title"] or "").strip() or f"Chapter {r['chapter_index']}"
first_text = first.get(cid, "")
audio_path = audio_dir / f"ch{cid}_full.wav"
role = "chapter"
if first_text.lower().startswith(SAMPLE_MARKER_TEXT.lower()):
role = "sample"
elif title.lower().startswith("opening"):
role = "opening"
chapters.append(
ARChapter(
chapter_id=cid,
chapter_index=r["chapter_index"],
title=title,
first_utterance=first_text,
audio_path=audio_path if audio_path.exists() else None,
role=role,
)
)
last_with_audio = next(
(c for c in reversed(chapters) if c.audio_path is not None), None
)
if last_with_audio is not None:
last_with_audio.role = "closing"
return chapters
def export_chapter_mp3(data_dir: Path, wav_path: Path, out: Path) -> dict[str, Any]:
"""Encode ``wav_path`` into an AR-compliant MP3 at ``out``.
Convenience wrapper used by the per-chapter ``Build Chapter`` pipeline and
the web per-chapter MP3 download so both bake the exact AR normalization
(44.1 kHz mono, 192 kbit CBR, −20.5 dB RMS target, −3 dBFS peak ceiling,
2 s edge silence) into every delivered file.
"""
return ARPackageExporter(data_dir).encode_chapter_mp3(wav_path, out)

267
umbraaudio/audiobook_meta.py
View File

@@ -9,6 +9,7 @@ from __future__ import annotations
import json
import logging
import re
from pathlib import Path
from typing import Any
@@ -88,3 +89,269 @@ def completeness(meta: dict[str, str]) -> dict[str, Any]:
"required_total": sum(1 for f in META_FIELDS if f["required"]),
"required_filled": len(filled),
}
# --- EPUB publishing-metadata extraction ------------------------------------
# When a book is uploaded as an EPUB we can lift its package (OPF) metadata and
# cover directly, instead of asking the author to retype everything. Epubs do
# NOT carry narrator info, so that field (and any others the author already
# filled in) is left untouched.
def find_project_epub(data_dir: Path, source_path: str = "") -> Path | None:
"""Locate the project's stored EPUB upload, if any.
Prefers ``project.source_path`` (pristine copy) then falls back to the most
recent ``*.epub`` in ``<data_dir>/uploads``.
"""
candidates: list[Path] = []
if source_path and Path(source_path).suffix.lower() == ".epub":
candidates.append(Path(source_path))
upload_dir = data_dir / "uploads"
if upload_dir.is_dir():
candidates.extend(
sorted(
upload_dir.glob("*.epub"),
key=lambda p: p.stat().st_mtime if p.exists() else 0,
reverse=True,
)
)
for c in candidates:
try:
if c.is_file():
return c
except OSError:
continue
return None
def _collect_epub_metadata(book: Any) -> dict[str, list[tuple[str, dict[str, Any]]]]:
"""Normalize ``book.metadata`` ({uri: {tag: [(value, attrs), ...]}}) into a
flat {tag_tail: [(value, attrs), ...]} mapping, lowercasing tag tails and
collecting OPF ``<meta name=... content=...>`` pairs under the key ``_meta``.
"""
import ebooklib as _ebl
out: dict[str, list[tuple[str, dict[str, Any]]]] = {}
metas: dict[str, list[str]] = {}
for _uri, tag_map in book.metadata.items():
for tag, entries in (tag_map or {}).items():
tail = str(tag).split(":")[-1].strip().lower()
if not tail:
continue
if tail == "meta":
for e in entries or []:
attrs = (
e[1]
if isinstance(e, (list, tuple)) and len(e) > 1 and isinstance(e[1], dict)
else (dict(e) if isinstance(e, dict) else {})
)
name = str(attrs.get("name") or "").strip()
content = str(attrs.get("content") or "").strip()
if name and content:
metas.setdefault(name.replace("calibre:", "").lower(), []).append(content)
continue
bucket = out.setdefault(tail, [])
for e in entries or []:
if isinstance(e, dict):
v = e.get("content") or e.get("#text")
attrs = dict(e)
elif isinstance(e, (list, tuple)) and len(e) >= 1:
v = e[0]
attrs = e[1] if len(e) > 1 and isinstance(e[1], dict) else {}
else:
v = e
attrs = {}
if v is None:
continue
text = str(v).strip()
if text:
bucket.append((text, dict(attrs) if isinstance(attrs, dict) else {}))
if metas:
out["_meta"] = [(k, v) for k, v in metas.items()]
return out
def extract_epub_metadata(epub_path: Path) -> dict[str, Any]:
"""Read an EPUB's package metadata + cover.
Returns a dict keyed by metadata tag (``title``, ``creator``, ``publisher``,
``description``, ``subject``, ``identifier``, ``date``, ``language``,
``rights``, ...) mapping to lists of ``(value, attrs)`` tuples, plus a
``_meta`` key for OPF ``<meta>`` pairs (e.g. series info) and, when a cover
is present, ``_cover`` = ``(bytes, extension)``.
"""
from ebooklib import epub as _epub
book = _epub.read_epub(str(epub_path))
data = _collect_epub_metadata(book)
cover = _extract_epub_cover(book)
if cover is not None:
data["_cover"] = cover
data["_title_attr"] = [(str(getattr(book, "title", "") or "").strip(), {})]
return data
def _extract_epub_cover(book: Any) -> tuple[bytes, str] | None:
"""Return ``(bytes, ext)`` for the best candidate cover image, or None."""
import ebooklib as _ebl
item = None
for e in book.get_metadata("OPF", "cover") or []:
attrs = e[1] if isinstance(e, (list, tuple)) and len(e) > 1 and isinstance(e[1], dict) else {}
cover_id = str(attrs.get("content") or "").strip()
if cover_id:
try:
item = book.get_item_with_id(cover_id)
except Exception:
item = None
if item is not None:
break
if item is None:
for it in book.get_items_of_type(_ebl.ITEM_COVER):
item = it
break
if item is None:
for it in book.get_items():
if it.get_type() in (_ebl.ITEM_IMAGE, _ebl.ITEM_DOCUMENT) and "cover" in it.get_name().lower():
item = it
break
if item is None:
return None
content = item.get_content()
if not content:
return None
ext = Path(item.get_name()).suffix.lower() or ".jpg"
if ext not in (".jpg", ".jpeg", ".png", ".webp", ".gif"):
ext = ".jpg"
return content, ext
def _first(data: dict[str, list[tuple[str, Any]]], *tags: str) -> str:
for tag in tags:
vals = data.get(tag, [])
for v, _a in vals:
s = str(v).strip()
if s:
return s
return ""
def _join(data: dict[str, list[tuple[str, Any]]], tag: str) -> str:
return ", ".join(
dict.fromkeys(str(v).strip() for v, _a in data.get(tag, []) if str(v).strip())
)
def _meta_val(data: dict[str, list[tuple[str, Any]]], name: str) -> str:
for n, vals in data.get("_meta", []):
if n.replace("calibre:", "").lower() == name.lower() and vals:
return str(vals[0])
return ""
def epub_meta_to_fields(data: dict[str, Any]) -> dict[str, str]:
"""Map normalized EPUB metadata onto META_FIELDS keys (empty values only)."""
fields: dict[str, str] = {}
title = _first(data, "title", "_title_attr")
fields["title"] = title
fields["subtitle"] = _first(data, "subtitle")
fields["author"] = _join(data, "creator")
fields["narrator"] = ""
fields["synopsis"] = _first(data, "description", "abstract", "long_description")
fields["language"] = _first(data, "language")
fields["publisher"] = _first(data, "publisher")
rights = _first(data, "rights", "license")
fields["copyright_holder"] = rights if rights and not rights.lower().startswith("public") else ""
date_val = _first(data, "date", "created", "issued")
m = re.match(r"\s*(\d{4})-(\d{1,2})-(\d{1,2})", date_val)
if m:
fields["release_date"] = f"{m.group(1)}-{int(m.group(2)):02d}-{int(m.group(3)):02d}"
ym = re.match(r"\s*(\d{4})", date_val)
if ym:
fields["copyright_year"] = ym.group(1)
fields["series"] = _first(data, "series") or _meta_val(data, "series")
series_num = _first(data, "series_pos", "series_index") or _meta_val(data, "series_index") or _meta_val(data, "series_pos")
if series_num:
fields["series_number"] = series_num
fields["genre"] = _join(data, "subject")
isbn = ""
for v, attrs in data.get("identifier", []):
scheme = " ".join(str(x) for x in attrs.get("scheme", [])) if isinstance(attrs, dict) else ""
if "isbn" in str(v).lower() or "isbn" in scheme.lower():
isbn = str(v).strip()
break
if not isbn:
for v, attrs in data.get("identifier", []):
scheme = " ".join(str(x) for x in attrs.get("scheme", [])) if isinstance(attrs, dict) else ""
text = str(v).strip()
if not text:
continue
if "uuid" in text.lower() or "uuid" in scheme.lower():
continue
isbn = text
break
if not isbn and data.get("identifier"):
isbn = str(data["identifier"][0][0]).strip()
fields["isbn"] = isbn
return fields
def merge_epub_metadata(data_dir: Path, epub_path: Path | None = None) -> dict[str, Any]:
"""Populate empty m4b-meta fields + cover from the project's EPUB.
Never overwrites values the user has already entered. Returns a summary with
``filled`` (field -> value) and the stored cover path (relative to the
project dir, matching how the cover-upload endpoint persists it).
"""
cover = None
if epub_path is None:
epub_path = find_project_epub(data_dir)
if epub_path is None:
return {"ok": False, "error": "No uploaded EPUB found for this project."}
data = extract_epub_metadata(epub_path)
fields = epub_meta_to_fields(data)
meta = load_meta(data_dir)
filled: dict[str, str] = {}
for key, value in fields.items():
if key in meta and value and not str(meta.get(key, "")).strip():
meta[key] = value
filled[key] = value
if not str(meta.get("output_name", "")).strip() and (title := fields.get("title")):
safe = re.sub(r"[^\w\s.-]", "", title).strip()
meta["output_name"] = (safe or "audiobook") + ".m4b"
filled["output_name"] = meta["output_name"]
cover = data.get("_cover")
stored_cover_path = ""
if cover and not str(meta.get("cover_path") or "").strip():
ext = cover[1] if isinstance(cover, tuple) and cover[1] else ".jpg"
assets = data_dir / "m4b_assets"
assets.mkdir(parents=True, exist_ok=True)
dest = assets / f"cover{ext}"
dest.write_bytes(cover[0])
rel = dest.relative_to(data_dir.parent).as_posix()
meta["cover_path"] = rel
filled["cover_path"] = rel
stored_cover_path = str(dest)
if filled:
save_meta(data_dir, meta)
return {
"ok": True,
"epub": str(epub_path),
"filled": filled,
"cover": stored_cover_path,
"source": data_dir,
}

2
umbraaudio/cli.py
View File

@@ -149,7 +149,7 @@ def discover(ctx: click.Context, output: str | None, max_chars: int) -> None:
table.add_row(
speaker.primary_name,
names_str,
speaker.gender,
speaker.demographics.gender,
speaker_type,
speaker.description[:60] + "..."
if len(speaker.description) > 60

1
umbraaudio/config.py
View File

@@ -89,6 +89,7 @@ class LLMConfig(BaseSettings):
lmstudio_base_url: str = "http://localhost:1234/v1"
lmstudio_model: str = "google/gemma-4-e4b"
openai_model: str = "gpt-4o"
discovery_model: str = "google/gemma-4-e4b"
temperature: float = 0.3
max_tokens: int = 4096
qwen_tts_url: str = "http://localhost:8881"

8
umbraaudio/database.py
View File

@@ -95,6 +95,7 @@ CREATE TABLE IF NOT EXISTS utterances (
attribution_confidence REAL DEFAULT 1.0,
attribution_reasoning TEXT,
emotion_confidence REAL DEFAULT 1.0,
user_edited INTEGER NOT NULL DEFAULT 0,
status TEXT NOT NULL DEFAULT 'pending'
);
@@ -181,6 +182,13 @@ class Database:
if col not in existing_cols:
await self._conn.execute(f"ALTER TABLE characters ADD COLUMN {col} {typedef}")
cursor = await self._conn.execute("PRAGMA table_info(utterances)")
utt_cols = {row[1] for row in await cursor.fetchall()}
if "user_edited" not in utt_cols:
await self._conn.execute(
"ALTER TABLE utterances ADD COLUMN user_edited INTEGER NOT NULL DEFAULT 0"
)
cursor = await self._conn.execute(
"SELECT name FROM sqlite_master WHERE type='table' AND name='voice_review_history'"
)

352
umbraaudio/generation/audio_effects.py
View File

@@ -92,6 +92,60 @@ def _lowpass(audio: np.ndarray, sr: int, cutoff_hz: float = 2000.0, order: int =
return lfilter(b, a, audio).astype(np.float32)
def _highpass(audio: np.ndarray, sr: int, cutoff_hz: float = 600.0, order: int = 2) -> np.ndarray:
"""High-pass to strip human chest/body resonance (syrinx: no ~100-600Hz body)."""
nyq = sr / 2.0
c = min(cutoff_hz / nyq, 0.999)
b, a = butter(order, c, btype="high")
return lfilter(b, a, audio).astype(np.float32)
def _tube_sat(audio: np.ndarray, sr: int, drive_db: float = 8.0, mix: float = 1.0) -> np.ndarray:
"""Soft-knee (tanh) saturation for harsh squawky harmonics without digital clipping."""
drive = 10.0 ** (drive_db / 20.0)
dry = audio.astype(np.float64)
wet = np.tanh(drive * dry) / np.tanh(drive)
out = mix * wet + (1.0 - mix) * dry
return out.astype(np.float32)
def _chorus(
audio: np.ndarray, sr: int, rate_hz: float = 1.8, depth: float = 0.25,
centre_delay_ms: float = 7.0, feedback: float = 0.15, mix: float = 0.35,
) -> np.ndarray:
"""Dual-modulated-tap chorus, emulating a syrinx's two independent membranes.
Two short delays sawing against each other produce the subtle
thickening/beating of a bird's dual sound source.
"""
n = len(audio)
if n < 2:
return audio.astype(np.float32)
t = np.arange(n, dtype=np.float64) / sr
base = centre_delay_ms / 1000.0 * sr
depth_samp = depth * base
d1 = base + depth_samp * np.sin(2.0 * np.pi * rate_hz * t)
d2 = base * 0.6 + depth_samp * np.sin(2.0 * np.pi * (rate_hz * 1.31) * t + 0.7)
idx = np.arange(n, dtype=np.float64)
read1 = np.clip(idx - d1, 0, n - 1)
read2 = np.clip(idx - d2, 0, n - 1)
tap1 = np.interp(read1, idx, audio).astype(np.float32)
tap2 = np.interp(read2, idx, audio).astype(np.float32)
wet = (0.7 * tap1 + 0.3 * tap2).astype(np.float64)
out = (1.0 - mix) * audio + mix * wet
if feedback > 0.0:
take = min(int(sr * 0.05), n)
if take < n:
fb = np.concatenate([np.zeros(take), out[: n - take]])
out = out + feedback * fb
peak = float(np.max(np.abs(out)))
if peak > 1.0:
out = out / peak
return out.astype(np.float32)
def _noise_inject(audio: np.ndarray, sr: int, amount: float = 0.03) -> np.ndarray:
"""Add white noise for raspy/grainy quality."""
rng = np.random.default_rng(42)
@@ -117,6 +171,75 @@ def _ring_modulate(audio: np.ndarray, sr: int, freq: float = 40.0) -> np.ndarray
return (audio * np.sin(2.0 * np.pi * freq * t)).astype(np.float32)
def _pitch_shift(
audio: np.ndarray, sr: int, semitones: float = 6.0
) -> np.ndarray:
"""Shift pitch (and formants) upward by resampling.
Raising pitch via resampling also raises the spectral envelope, shrinking
the apparent vocal tract (the classic "chipmunk" effect). Positive
values make voices sound smaller, faster and higher — the core of a
cartoon parrot/squawk voice. Negative values do the reverse.
"""
factor = 2.0 ** (semitones / 12.0)
n_out = max(1, int(len(audio) / factor))
idx = np.linspace(0.0, len(audio) - 1.0, n_out)
return np.interp(idx, np.arange(len(audio), dtype=np.float64), audio).astype(
np.float32
)
def _pitch_shift_pv(
audio: np.ndarray, sr: int, semitones: float = 7.0
) -> np.ndarray:
"""Duration-preserving pitch shift via the phase vocoder.
Unlike resampling (``_pitch_shift``), the speech rate stays the same
while pitch rises — so a parrot keeps its natural hopping delivery but
sounds higher. Falls back to resampling if ``librosa`` is unavailable.
"""
try:
import librosa # noqa: PLC0415
except ImportError: # pragma: no cover
return _pitch_shift(audio, sr, semitones)
shifted = librosa.effects.pitch_shift(y=audio, sr=sr, n_steps=semitones)
return shifted.astype(np.float32)
def _vibrato(
audio: np.ndarray, sr: int, rate: float = 20.0, depth: float = 0.15
) -> np.ndarray:
"""Rapid pitch wobble (warbling squawk) via time-domain resampling."""
t = np.linspace(0, len(audio) / sr, len(audio), dtype=np.float32)
mod = 1.0 + depth * np.sin(2.0 * np.pi * rate * t)
indices = np.clip(np.round(np.cumsum(mod)).astype(int), 0, len(audio) - 1)
return audio[indices].astype(np.float32)
def _time_stretch(audio: np.ndarray, sr: int, rate: float = 1.0) -> np.ndarray:
"""Pitch-preserving tempo change for emotion delivery.
librosa semantics: ``rate > 1`` speeds the signal up (shorter), ``rate < 1``
slows it down (longer). Falls back to resampling (which also shifts pitch)
when librosa is unavailable.
"""
try:
import librosa # noqa: PLC0415
except ImportError: # pragma: no cover
n_out = max(1, int(len(audio) * rate))
idx = np.linspace(0.0, len(audio) - 1.0, n_out)
return np.interp(idx, np.arange(len(audio), dtype=np.float64), audio).astype(
np.float32
)
return librosa.effects.time_stretch(y=audio, rate=rate).astype(np.float32)
def _gain_db(audio: np.ndarray, _sr: int, db: float = 0.0) -> np.ndarray:
"""Linear gain stage in decibels (positive = louder)."""
factor = 10.0 ** (db / 20.0)
return (audio * factor).astype(np.float32)
TRAIT_EFFECTS: TraitEffectMap = {
"broken": [
("bitcrusher", {"bits": 6, "sample_rate_reduction": 2}),
@@ -168,6 +291,20 @@ TRAIT_EFFECTS: TraitEffectMap = {
("ring_modulate", {"freq": 50.0}),
("bandpass", {"low_hz": 800.0, "high_hz": 3500.0}),
],
"parrot": [
("pitch_shift_pv", {"semitones": 7.0}),
("bandpass", {"low_hz": 700.0, "high_hz": 4800.0}),
("tremolo", {"rate": 16.0, "depth": 0.32}),
("vibrato", {"rate": 24.0, "depth": 0.15}),
("hard_clip", {"threshold": 0.4}),
],
"squawky": [
("pitch_shift_pv", {"semitones": 7.0}),
("bandpass", {"low_hz": 700.0, "high_hz": 4800.0}),
("tremolo", {"rate": 16.0, "depth": 0.32}),
("vibrato", {"rate": 24.0, "depth": 0.15}),
("hard_clip", {"threshold": 0.4}),
],
}
EFFECT_FN = {
@@ -177,9 +314,159 @@ EFFECT_FN = {
"random_gating": _random_gating,
"bandpass": _bandpass,
"lowpass": _lowpass,
"highpass": _highpass,
"tube_sat": _tube_sat,
"chorus": _chorus,
"noise_inject": _noise_inject,
"pitch_jitter": _pitch_jitter,
"pitch_shift": _pitch_shift,
"pitch_shift_pv": _pitch_shift_pv,
"vibrato": _vibrato,
"ring_modulate": _ring_modulate,
"time_stretch": _time_stretch,
"gain_db": _gain_db,
}
# Named chains that expand to one or more effects with tuned parameters.
# ``squawk`` / ``parrot`` is the recipe for a high, warbling cartoon-parrot
# voice: duration-preserving pitch/formant lift, mid-high bandpass, fast
# flutter, vibrato and a touch of grit. No noise injection (that just adds
# hiss on an already bright band).
PRESET_EFFECTS: dict[str, list[tuple[str, dict[str, Any]]]] = {
"squawk": [
("pitch_shift_pv", {"semitones": 7.0}),
("bandpass", {"low_hz": 700.0, "high_hz": 4800.0}),
("tremolo", {"rate": 16.0, "depth": 0.32}),
("vibrato", {"rate": 24.0, "depth": 0.15}),
("hard_clip", {"threshold": 0.4}),
],
"parrot": [
("pitch_shift_pv", {"semitones": 7.0}),
("bandpass", {"low_hz": 700.0, "high_hz": 4800.0}),
("tremolo", {"rate": 16.0, "depth": 0.32}),
("vibrato", {"rate": 24.0, "depth": 0.15}),
("hard_clip", {"threshold": 0.4}),
],
# ``polly`` follows the avian-syrinx blueprint: strip human chest
# resonance, lift pitch/formants, fast throat flutter (~30 Hz), soft
# overdrive for harsh squawk harmonics, dual-membrane micro-chorus,
# then roll off hypersonic brittleness.
"polly": [
("highpass", {"cutoff_hz": 600.0}),
("pitch_shift_pv", {"semitones": 6.0}),
("tremolo", {"rate": 30.0, "depth": 0.30}),
("tube_sat", {"drive_db": 8.0, "mix": 1.0}),
("chorus", {"rate_hz": 1.8, "depth": 0.25, "centre_delay_ms": 7.0, "feedback": 0.15, "mix": 0.35}),
("lowpass", {"cutoff_hz": 6800.0}),
],
}
# Bump this when the emotion DSP recipe changes so existing fx'd audio is
# regenerated instead of reused.
EMOTION_FX_VERSION = 1
# Per-emotion exaggeration chains. The TTS voice clone carries the speaker's
# timbre but not emotion delivery, so each emotion is stamped onto the cloned
# audio deterministically (tempo, pitch, brightness, energy). ``neutral`` is
# intentionally empty so it stays the untouched baseline. Note librosa
# ``time_stretch`` semantics: rate > 1 = faster, rate < 1 = slower.
EMOTION_EFFECTS: dict[str, list[tuple[str, dict[str, Any]]]] = {
"neutral": [],
"angry": [
("time_stretch", {"rate": 1.08}),
("pitch_shift_pv", {"semitones": 1.5}),
("tube_sat", {"drive_db": 6.0, "mix": 0.5}),
("gain_db", {"db": 2.5}),
],
"sad": [
("time_stretch", {"rate": 0.89}),
("pitch_shift_pv", {"semitones": -1.5}),
("lowpass", {"cutoff_hz": 5000.0}),
("gain_db", {"db": -2.0}),
],
"happy": [
("time_stretch", {"rate": 1.06}),
("pitch_shift_pv", {"semitones": 1.0}),
("gain_db", {"db": 2.0}),
],
"fearful": [
("time_stretch", {"rate": 0.94}),
("pitch_shift_pv", {"semitones": 2.5}),
("tremolo", {"rate": 5.0, "depth": 0.25}),
("gain_db", {"db": 1.5}),
],
"surprised": [
("time_stretch", {"rate": 1.02}),
("pitch_shift_pv", {"semitones": 3.0}),
("gain_db", {"db": 1.5}),
],
"whisper": [
("lowpass", {"cutoff_hz": 4500.0}),
("gain_db", {"db": -7.0}),
("noise_inject", {"amount": 0.012}),
],
"pleading": [
("time_stretch", {"rate": 0.93}),
("lowpass", {"cutoff_hz": 5500.0}),
("gain_db", {"db": -1.5}),
],
"sarcastic": [
("time_stretch", {"rate": 1.04}),
("bandpass", {"low_hz": 400.0, "high_hz": 3200.0}),
("gain_db", {"db": 0.5}),
],
"authoritative": [
("time_stretch", {"rate": 1.04}),
("pitch_shift_pv", {"semitones": -1.0}),
("lowpass", {"cutoff_hz": 3500.0}),
("gain_db", {"db": 2.5}),
],
"thoughtful": [
("time_stretch", {"rate": 0.94}),
("gain_db", {"db": -1.0}),
],
"excited": [
("time_stretch", {"rate": 1.10}),
("pitch_shift_pv", {"semitones": 1.5}),
("tube_sat", {"drive_db": 4.0, "mix": 0.4}),
("gain_db", {"db": 3.0}),
],
"anxious": [
("time_stretch", {"rate": 1.05}),
("pitch_shift_pv", {"semitones": 1.0}),
("tremolo", {"rate": 6.0, "depth": 0.18}),
("gain_db", {"db": 0.5}),
],
"tired": [
("time_stretch", {"rate": 0.88}),
("pitch_shift_pv", {"semitones": -2.0}),
("lowpass", {"cutoff_hz": 4000.0}),
("gain_db", {"db": -3.0}),
],
"romantic": [
("time_stretch", {"rate": 0.95}),
("pitch_shift_pv", {"semitones": -0.5}),
("lowpass", {"cutoff_hz": 5200.0}),
("gain_db", {"db": -1.0}),
],
"disgusted": [
("time_stretch", {"rate": 0.92}),
("pitch_shift_pv", {"semitones": -1.0}),
("bandpass", {"low_hz": 350.0, "high_hz": 3000.0}),
],
"amused": [
("time_stretch", {"rate": 1.05}),
("pitch_shift_pv", {"semitones": 1.0}),
("tremolo", {"rate": 3.5, "depth": 0.12}),
("gain_db", {"db": 1.0}),
],
"nostalgic": [
("time_stretch", {"rate": 0.93}),
("pitch_shift_pv", {"semitones": -1.0}),
("lowpass", {"cutoff_hz": 4500.0}),
("gain_db", {"db": -1.0}),
],
}
@@ -228,13 +515,19 @@ def apply_effects(audio: np.ndarray, sr: int, vocal_traits: list[str]) -> np.nda
def get_effects_by_name(effect_names: list[str]) -> list[tuple[str, dict[str, Any]]]:
"""Build an effect chain from explicit effect names (e.g. from UI selection).
Each name must be a key in ``EFFECT_FN``. Unknown names are skipped.
Each name must be a preset in ``PRESET_EFFECTS`` or a key in
``EFFECT_FN``. Unknown names are skipped.
"""
chain: list[tuple[str, dict[str, Any]]] = []
seen: set[str] = set()
for name in effect_names:
key = name.strip().lower()
if key in EFFECT_FN and key not in seen:
if key in PRESET_EFFECTS:
for eff, params in PRESET_EFFECTS[key]:
if eff not in seen:
seen.add(eff)
chain.append((eff, dict(params)))
elif key in EFFECT_FN and key not in seen:
seen.add(key)
chain.append((key, {}))
return chain
@@ -245,16 +538,16 @@ def apply_effects_by_name(
) -> np.ndarray:
"""Apply DSP effects by explicit effect names (not trait-derived).
Uses default parameters for each effect.
Uses tuned parameters for presets and defaults for individual effects.
"""
chain = get_effects_by_name(effect_names)
if not chain:
return audio
out = audio.copy()
for effect_name, _params in chain:
for effect_name, params in chain:
fn = EFFECT_FN[effect_name]
out = fn(out, sr)
out = fn(out, sr, **params)
peak = float(np.max(np.abs(out)))
if peak > 1.0:
out = out / peak
@@ -358,3 +651,52 @@ def load_speaker_distortion_effects(speakers_path: Path) -> dict[str, list[str]]
if name and effects:
result[name] = list(effects)
return result
def apply_emotion_effects(audio: np.ndarray, sr: int, emotion: str | None) -> np.ndarray:
"""Apply the emotion exaggeration chain for *emotion* to *audio*.
Unknown or empty emotions (e.g. ``neutral``) return the audio unchanged so
they remain the untouched baseline for comparison.
"""
chain = EMOTION_EFFECTS.get((emotion or "").strip().lower(), [])
if not chain:
return audio
out = audio.copy()
for effect_name, params in chain:
fn = EFFECT_FN.get(effect_name)
if fn is None:
continue
out = fn(out, sr, **params)
if out.size == 0:
return audio
peak = float(np.max(np.abs(out)))
if peak > 1.0:
out = out / peak
if float(np.max(np.abs(out))) < 1e-6:
logger.warning("Emotion effect %s silenced output", effect_name)
return audio
return out
def apply_emotion_effects_to_file(
wav_path: Path,
emotion: str | None,
output_path: Path | None = None,
) -> bool:
"""Apply the emotion exaggeration chain to a WAV file in-place (or to
``output_path``). Returns True when an effect chain was applied."""
if not EMOTION_EFFECTS.get((emotion or "").strip().lower()):
return False
import soundfile as sf
audio, sr = sf.read(str(wav_path), dtype="float32")
if audio.ndim > 1:
audio = audio.mean(axis=1)
processed = apply_emotion_effects(audio, sr, emotion)
dest = output_path or wav_path
sf.write(str(dest), processed, sr, subtype="PCM_16")
return True

215
umbraaudio/generation/chapter.py
View File

@@ -3,38 +3,189 @@ from __future__ import annotations
import hashlib
import logging
import time
import typing
from pathlib import Path
from typing import Any
import numpy as np
import torch
if typing.TYPE_CHECKING:
import numpy as np
logger = logging.getLogger(__name__)
EMOTION_OPTIONS: list[str] = [
"neutral",
"angry",
"sad",
"happy",
"fearful",
"surprised",
"whisper",
"pleading",
"sarcastic",
"authoritative",
"thoughtful",
"excited",
"anxious",
"tired",
"romantic",
"disgusted",
"amused",
"nostalgic",
"tense",
"urgent",
"horror",
"furious",
"joyful",
"terrified",
"griefstricken",
"shocked",
]
PACE_OPTIONS: list[str] = ["slow", "moderate", "fast"]
EMOTION_INSTRUCT_MAP: dict[str, str] = {
"angry": "Speak with anger and frustration. Sharp, aggressive tone.",
"sad": "Speak with sadness and melancholy. Slow, heavy delivery.",
"happy": "Speak with joy and enthusiasm. Bright, lively tone.",
"fearful": "Speak with fear and anxiety. Trembling, uncertain delivery.",
"surprised": "Speak with surprise and wonder. Higher pitch, faster pace.",
"whisper": "Speak in a soft whisper. Quiet, intimate delivery.",
"pleading": "Speak with desperate pleading. Urgent, emotional.",
"sarcastic": "Speak with sarcasm and irony. Exaggerated, knowing tone.",
"authoritative": "Speak with authority and command. Firm, unwavering tone.",
"thoughtful": "Speak thoughtfully, as if thinking aloud. Pensive, slow.",
"excited": "Speak with excitement and energy. Fast, animated delivery.",
"anxious": "Speak with nervous anxiety. Quick, uncertain delivery, slightly breathless.",
"tired": "Speak with exhaustion. Slow, heavy, drained delivery.",
"disgusted": "Speak with disgust and revulsion. Sharp, repulsed tone.",
"amused": "Speak with amusement and humor. Light, chuckling delivery.",
"nostalgic": "Speak with wistful nostalgia. Soft, reflective tone.",
"romantic": "Speak with warmth and tenderness. Soft, intimate delivery.",
"neutral": "Speak in a neutral, balanced tone. Even delivery.",
"tense": "Speak with controlled tension. Tight, measured delivery, alert tone.",
"urgent": "Speak with urgency. Fast, clipped delivery, high energy.",
"horror": "Speak with horror and dread. Hushed, terrified delivery.",
"angry": (
"Speak in hot, rising anger. Voice tight, teeth clenched, "
"clipped aggressive delivery."
),
"sad": (
"Speak in heavy, aching sorrow. Voice cracks and dips; "
"draw words out on a defeated breath."
),
"happy": (
"Speak with bright, unguarded joy. Smile in the voice; "
"light, bouncy, almost-laughing rhythm."
),
"fearful": (
"Speak with sharp, trembling fear. Quick shallow breaths, "
"an unsteady high voice that cuts words short."
),
"surprised": (
"Speak with genuine shock. Pitch jumps, words tumble out "
"in disbelief, then a stunned silence."
),
"whisper": (
"Speak in a close, hushed whisper. Barely audible, intimate "
"and secret, tense with withheld breath."
),
"pleading": (
"Speak with desperate, vulnerable pleading. Straining voice, "
"swaying pitch, reaching drawn-out vowels."
),
"sarcastic": (
"Speak with dragged-out, knowing sarcasm. Heavy irony, "
"mocking emphasis, an amused cruel edge."
),
"authoritative": (
"Speak with commanding authority. Deep, slow, immovable; "
"every line lands without argument."
),
"thoughtful": (
"Speak slowly, drifting into thought. Words hang and search "
"mid-air; quiet revelations between pauses."
),
"excited": (
"Speak bursting with excitement. Rapid-fire, bright, rising; "
"barely pause for breath, energy spills over."
),
"anxious": (
"Speak with nervous, breathless anxiety. Quick flits between "
"words, swallowed endings, quiet panic beneath."
),
"tired": (
"Speak drained and exhausted. Heavy sagging words, slow and "
"quiet, barely any energy to finish a line."
),
"disgusted": (
"Speak curling with disgust. Bitten-off endings, a grimace "
"in every syllable, contempt in each word."
),
"amused": (
"Speak with barely suppressed amusement. The smile leaks "
"into the voice; light, teasing, warm laughter."
),
"nostalgic": (
"Speak with wistful, aching nostalgia. Warm and faraway, "
"lingering on memories, gentle sadness beneath."
),
"romantic": (
"Speak with tender intimate warmth. Low velvet delivery, "
"long caressed vowels, soft and close to the ear."
),
"tense": (
"Speak coiled with silent tension. Every syllable measured "
"and charged, shallow held breath, about to snap."
),
"urgent": (
"Speak at a racing, clipped emergency clip. Breath-driven, "
"decisive, tiny electric pauses, unignorable."
),
"horror": (
"Speak with dawning horror. Hushed, shaking dread, "
"flattening to a whisper as the terror settles in."
),
"furious": (
"Speak in barely controlled rage. A low growl: fast, violent, "
"almost shouting, one slip from destruction."
),
"joyful": (
"Speak with uncontainable exuberance. Laugh mid-sentence, "
"breathless and bright, delight tumbling words."
),
"terrified": (
"Speak in absolute terror. Strangled shaking whisper, gasping, "
"words cut short in panic, barely formed."
),
"griefstricken": (
"Speak devastated, newly grieving. Voice trembling and breaking, "
"long defeated pauses, tears barely held."
),
"shocked": (
"Speak stunned into stillness. Hollow and disbelieving; "
"each word lands like a blow, world just stopped."
),
"neutral": "",
}
PACE_INSTRUCT: dict[str, str] = {
"slow": (
"Deliver at a slow, deliberate tempo; drag the words out "
"and let the pauses linger."
),
"moderate": "",
"fast": (
"Deliver at a quick, clipped tempo; rapid syllables, "
"minimal pauses, constant momentum."
),
}
def compose_utterance_instruct(
instruct: str | None = None,
emotion: str | None = None,
pace: str | None = None,
) -> str | None:
"""Compose the TTS instruct from custom text + emotion + pace.
Only non-default choices contribute: custom instruct always, emotion
anything but neutral, pace anything but moderate. With no variation set the
result is None so a voice clone keeps its natural delivery.
"""
parts: list[str] = []
if instruct:
parts.append(instruct.strip())
if emotion and emotion not in ("neutral", ""):
emotion_instr = EMOTION_INSTRUCT_MAP.get(emotion)
if emotion_instr:
parts.append(emotion_instr)
if pace and pace not in ("moderate", ""):
pace_instr = PACE_INSTRUCT.get(pace)
if pace_instr:
parts.append(pace_instr)
return " ".join(parts) if parts else None
NARRATOR_COMPRESSION_INSTRUCT = (
"Read steadily and evenly. Low emphasis. Maintain calm, even pace. "
"Do not dramatize. Smooth, measured delivery for descriptive prose."
@@ -66,22 +217,14 @@ class ChapterGenerator:
if is_narrator:
return NARRATOR_COMPRESSION_INSTRUCT
emotion = utterance.get("emotion") or utterance.get("emotional_baseline", "neutral")
instruct = EMOTION_INSTRUCT_MAP.get(emotion)
speaker_profile = utterance.get("_speaker_profile")
base_instruct = self._build_speaker_base_instruct(speaker_profile)
utterance_instruct = utterance.get("instruct")
parts = []
if base_instruct:
parts.append(base_instruct)
if utterance_instruct:
parts.append(utterance_instruct)
if instruct:
parts.append(instruct)
return " ".join(parts) if parts else None
varied = compose_utterance_instruct(
utterance.get("instruct"),
utterance.get("emotion"),
utterance.get("pace"),
)
return " ".join(p for p in (base_instruct, varied) if p) or None
ENERGY_INSTRUCT = {
"whispered": "Speak in a near-whisper, quiet and subdued.",

119
umbraaudio/generation/chapter_builder.py
View File

@@ -18,7 +18,6 @@ import json
import logging
import os
import sqlite3
import subprocess
import threading
import time
from dataclasses import dataclass, field
@@ -358,9 +357,14 @@ class ChapterBuilder:
self._set_phase("validating", "Checking utterance staleness...", 8)
from umbraaudio.generation.staleness import is_current
from umbraaudio.pronunciation import apply_phonetics, load_phonetic_names
from umbraaudio.pronunciation import (
load_phonetic_names,
load_proper_names,
prepare_tts_text,
)
phonetic = load_phonetic_names(self.data_dir)
proper_names = load_proper_names(self.data_dir)
speakers = _load_speakers(self.data_dir)
alias_map = _build_alias_map(speakers)
voices_dir = self.data_dir / "voices"
@@ -382,19 +386,25 @@ class ChapterBuilder:
audio_dir = self.data_dir / "chapter_audio" / f"ch{self.chapter_id}"
audio_dir.mkdir(parents=True, exist_ok=True)
from umbraaudio.generation.scene_break import is_scene_break, needs_silence
missing: list[dict] = []
for u in utterances:
idx = u["utterance_index"]
text = u["text"] or ""
utt_type = u["type"] or "narration"
if utt_type == "scene_break" or text.strip() == "<<<<<<>>>>>":
continue
if not text.strip():
continue
fname = f"utt_{idx:04d}.wav"
spoken_text = apply_phonetics(text, phonetic)
if is_scene_break(text, utt_type):
if needs_silence(audio_dir / fname):
missing.append(dict(u))
continue
spoken_text = prepare_tts_text(text, phonetic, proper_names)
spk = (u["speaker_name"] or "narrator").strip()
vref = _resolve_voice_ref(spk, alias_map, voices_dir)
@@ -425,9 +435,17 @@ class ChapterBuilder:
load_speaker_distortion_effects,
load_speaker_traits,
)
from umbraaudio.generation.chapter import EMOTION_INSTRUCT_MAP
from umbraaudio.generation.chapter import compose_utterance_instruct
from umbraaudio.generation.scene_break import (
is_scene_break,
write_scene_break_audio,
)
from umbraaudio.generation.staleness import write_marker
from umbraaudio.pronunciation import apply_phonetics, load_phonetic_names
from umbraaudio.pronunciation import (
load_phonetic_names,
load_proper_names,
prepare_tts_text,
)
from umbraaudio.voices.qwen_tts import (
VOICE_REFERENCE_TEXT,
QwenTTSEngine,
@@ -437,6 +455,7 @@ class ChapterBuilder:
self._set_phase("generating", f"Generating {len(missing)} utterances...", 15)
phonetic = load_phonetic_names(self.data_dir)
proper_names = load_proper_names(self.data_dir)
speakers = _load_speakers(self.data_dir)
alias_map = _build_alias_map(speakers)
voices_dir = self.data_dir / "voices"
@@ -456,6 +475,8 @@ class ChapterBuilder:
# use_embedding_mode.
prompt_map: dict[str, str] = {}
for u in missing:
if is_scene_break(u.get("text"), u.get("type")):
continue
spk = (u.get("speaker_name") or "narrator").strip()
if spk in prompt_map:
continue
@@ -485,16 +506,22 @@ class ChapterBuilder:
pct,
)
if not text.strip() or text.strip() == "<<<<<<>>>>>":
if not text.strip():
continue
spoken_text = apply_phonetics(text, phonetic)
out = audio_dir / f"utt_{idx:04d}.wav"
if is_scene_break(text, u["type"]):
write_scene_break_audio(out)
self.result.generated += 1
continue
spoken_text = prepare_tts_text(text, phonetic, proper_names)
voice_ref = _resolve_voice_ref(speaker, alias_map, voices_dir)
tts_instruct = instruct
if not tts_instruct and emotion in EMOTION_INSTRUCT_MAP:
tts_instruct = EMOTION_INSTRUCT_MAP[emotion]
tts_instruct = compose_utterance_instruct(
instruct, emotion, u["pace"]
)
seed = (self.chapter_id * 10000 + idx) if self.use_seed else None
success = False
@@ -526,6 +553,7 @@ class ChapterBuilder:
output_path=out,
voice_ref=voice_ref,
ref_text=VOICE_REFERENCE_TEXT,
instruct=tts_instruct,
temperature=self.temperature,
seed=seed,
)
@@ -535,6 +563,7 @@ class ChapterBuilder:
text=spoken_text,
output_path=out,
prompt_slug=prompt_slug,
instruct=tts_instruct,
temperature=self.temperature,
seed=seed,
)
@@ -545,6 +574,7 @@ class ChapterBuilder:
output_path=out,
voice_ref=voice_ref,
ref_text=VOICE_REFERENCE_TEXT,
instruct=tts_instruct,
temperature=self.temperature,
seed=seed,
)
@@ -570,8 +600,9 @@ class ChapterBuilder:
write_marker(
spoken_text, out,
speaker=speaker, voice_ref=voice_ref,
voice_mode=voice_mode,
voice_mode=voice_mode, instruct=tts_instruct,
)
self.result.generated += 1
success = True
break
@@ -619,7 +650,32 @@ class ChapterBuilder:
from umbraaudio.assembly import AudioStitcher
audio_dir = self.data_dir / "chapter_audio" / f"ch{self.chapter_id}"
audio_files = sorted(audio_dir.glob("utt_*.wav"))
# Only stitch audio that belongs to a real, non-blank utterance.
# Leftover wavs for blank/stale utterances (text cleared or empty)
# must not be spliced into the chapter — silence/artifacts.
def _idx(p: Path) -> int:
try:
return int(p.stem.split("_")[1])
except (IndexError, ValueError):
return -1
valid_idx: set[int] = set()
db = sqlite3.connect(str(self.data_dir / "project.db"))
db.row_factory = sqlite3.Row
for u in db.execute(
"SELECT utterance_index, text, type FROM utterances WHERE chapter_id = ?",
(self.chapter_id,),
):
t = (u["text"] or "").strip()
if t:
valid_idx.add(u["utterance_index"])
db.close()
audio_files = sorted(
(p for p in audio_dir.glob("utt_*.wav") if _idx(p) in valid_idx),
key=_idx,
)
if not audio_files:
self._set_phase("error", "No audio files to stitch", 0)
@@ -638,7 +694,7 @@ class ChapterBuilder:
# ------------------------------------------------------------------
def _export_mp3(self) -> None:
self._set_phase("exporting", "Converting to MP3...", 88)
self._set_phase("exporting", "Converting to AR-compliant MP3...", 88)
wav_path = (
self.data_dir / "chapter_audio" / f"ch{self.chapter_id}_full.wav"
@@ -649,30 +705,27 @@ class ChapterBuilder:
mp3_path = wav_path.with_suffix(".mp3")
try:
result = subprocess.run(
[
"ffmpeg", "-y",
"-i", str(wav_path),
"-codec:a", "libmp3lame",
"-b:a", "192k",
str(mp3_path),
],
capture_output=True,
text=True,
timeout=300,
)
if result.returncode == 0 and mp3_path.exists():
from umbraaudio.assembly.ar_export import export_chapter_mp3
stats = export_chapter_mp3(self.data_dir, wav_path, mp3_path)
if mp3_path.exists():
self.result.chapter_mp3 = mp3_path.name
self._set_phase("exporting", f"MP3 exported: {mp3_path.name}", 92)
detail = f"AR MP3 exported: {mp3_path.name} ({stats['duration_s']}s"
if stats.get("rms_db") is not None:
detail += f", RMS {stats['rms_db']:.1f} dB"
detail += ")"
if stats.get("warning"):
detail += f" ⚠ {stats['warning']}"
self._set_phase("exporting", detail, 92)
else:
logger.warning("ffmpeg MP3 conversion failed: %s", result.stderr[:300])
logger.warning("ffmpeg MP3 conversion failed")
self._set_phase("exporting", "MP3 conversion failed (non-fatal)", 92)
except FileNotFoundError:
logger.warning("ffmpeg not found, skipping MP3 export")
self._set_phase("exporting", "ffmpeg not available, skipping MP3", 92)
except subprocess.TimeoutExpired:
logger.warning("ffmpeg MP3 conversion timed out")
self._set_phase("exporting", "MP3 conversion timed out (non-fatal)", 92)
except Exception as e:
logger.warning("ffmpeg MP3 conversion failed: %s", e)
self._set_phase("exporting", f"MP3 conversion failed: {e}", 92)
# ------------------------------------------------------------------
# Phase 6: Verify

72
umbraaudio/generation/scene_break.py Normal file
View File

@@ -0,0 +1,72 @@
"""Scene-break (``<<<>>>``) audio handling.
A scene break is not spoken text; it marks a change of location, time, or
perspective. Historically these rows produced no audio at all, which left
behind 0-byte ``utt_XXXX.wav`` artifacts that broke the chapter stitcher
(``soundfile`` cannot open an empty file) and gave the utterance regen button
nothing to show. Every scene break now gets a fixed-length silence clip so
the final chapter audio carries the intended pause and the stitcher always has
a valid file to read.
"""
from __future__ import annotations
from typing import TYPE_CHECKING
import numpy as np
import soundfile as sf
if TYPE_CHECKING:
from pathlib import Path
from umbraaudio.generation.staleness import is_current, write_marker
SCENE_BREAK_TEXT = "<<<>>>"
SCENE_BREAK_SPEAKER = "scene_break"
SCENE_BREAK_SILENCE_MS = 2000
DEFAULT_SAMPLE_RATE = 24000
_WAV_HEADER_BYTES = 44
def is_scene_break(text: str | None, utt_type: str | None = None) -> bool:
"""True when an utterance is a scene break (by type or by its text)."""
if (utt_type or "").strip() == SCENE_BREAK_SPEAKER:
return True
return (text or "").strip() == SCENE_BREAK_TEXT
def _is_valid_wav(path: Path) -> bool:
try:
return path.exists() and path.stat().st_size > _WAV_HEADER_BYTES
except OSError:
return False
def needs_silence(path: Path) -> bool:
"""True when *path* is missing, empty/corrupt, or lacks a matching marker.
A 0-byte file (a historical artifact) must trigger a rewrite, not be
treated as "already generated".
"""
if not _is_valid_wav(path):
return True
return not is_current(SCENE_BREAK_TEXT, path, speaker=SCENE_BREAK_SPEAKER)
def write_scene_break_audio(
output_path: Path,
duration_ms: int = SCENE_BREAK_SILENCE_MS,
sample_rate: int = DEFAULT_SAMPLE_RATE,
) -> Path:
"""Write a silence wav (plus staleness marker) for a scene break."""
samples = int(sample_rate * duration_ms / 1000)
output_path.parent.mkdir(parents=True, exist_ok=True)
if output_path.exists():
output_path.unlink()
sf.write(str(output_path), np.zeros(samples, dtype=np.float32), sample_rate)
write_marker(
SCENE_BREAK_TEXT,
output_path,
speaker=SCENE_BREAK_SPEAKER,
)
return output_path

33
umbraaudio/generation/staleness.py
View File

@@ -6,7 +6,7 @@ check silently reuses audio that no longer matches the utterance text. Each
wav gets a sidecar marker (utt_XXXX.wav.txt) recording the exact spoken text
AND the assigned speaker; a file is only considered current when both match.
Markers are JSON ``{"speaker": ..., "text": ..., "voice_fp": ...}``.
Markers are JSON ``{"speaker": ..., "text": ..., "voice_fp": ..., "instruct": ...}``.
Legacy plain-text markers carry no speaker information, so they are always
treated as stale. The ``voice_fp`` field is a truncated SHA-256 of the
voice-reference wav used during generation; when a speaker's ref changes
@@ -42,11 +42,15 @@ def is_current(
voice_ref: Path | None = None,
voice_mode: str | None = None,
accepted_voice_modes: set[str] | None = None,
instruct: str | None = None,
fx_version: int | None = None,
) -> bool:
"""True when the wav exists, its marker is JSON, and text (+ speaker if
given) match exactly. If *voice_ref* is provided the marker's
``voice_fp`` must also match the current ref's fingerprint; when no
ref is given the fingerprint check is skipped (backward compatible).
If *instruct* is provided the marker's ``instruct`` must match too
(a changed emotion/variation therefore invalidates old audio).
Mode handling: if *voice_mode* is given (e.g. ``"icl"``) the marker must
record that exact mode; otherwise if *accepted_voice_modes* is given the
@@ -72,6 +76,8 @@ def is_current(
return False
if speaker is not None and payload.get("speaker") != speaker:
return False
if instruct is not None and payload.get("instruct") != instruct:
return False
if voice_ref is not None:
if not voice_ref.exists():
return False
@@ -81,10 +87,15 @@ def is_current(
recorded = payload.get("voice_mode")
if voice_mode is not None and recorded != voice_mode:
return False
if voice_mode is None and accepted_voice_modes is not None:
if recorded not in accepted_voice_modes:
return False
return True
if (
voice_mode is None
and accepted_voice_modes is not None
and recorded not in accepted_voice_modes
):
return False
if fx_version is not None and payload.get("fx_version") != fx_version:
return False
return not (fx_version is None and payload.get("fx_version") is not None)
def write_marker(
@@ -93,14 +104,24 @@ def write_marker(
speaker: str | None = None,
voice_ref: Path | None = None,
voice_mode: str | None = None,
instruct: str | None = None,
fx_version: int | None = None,
) -> None:
"""Record *text*, *speaker*, and (optionally) the voice-reference
fingerprint as the spoken content of *wav_path*."""
fingerprint as the spoken content of *wav_path*. When *instruct* is given
(the exact TTS instruction used) it is recorded too, so a later run with a
different emotion/instruct regenerates instead of reusing the file.
*fx_version* records which post-processing emotion recipe (if any) was
stamped onto the audio; a bump forces regeneration of fx'd files."""
payload: dict[str, str] = {"speaker": speaker or "", "text": text}
if voice_ref is not None and voice_ref.exists():
payload["voice_fp"] = _voice_fingerprint(voice_ref)
if voice_mode is not None:
payload["voice_mode"] = voice_mode
if instruct is not None:
payload["instruct"] = instruct
if fx_version is not None:
payload["fx_version"] = fx_version
marker_path(wav_path).write_text(
json.dumps(payload, ensure_ascii=False), encoding="utf-8"
)

16
umbraaudio/mapping/segmenter.py
View File

@@ -118,8 +118,12 @@ ABBREVIATIONS = (
)
ABBREV_PATTERN = re.compile(r"\b([A-Za-z]\.)+$")
# A sentence boundary is .!? followed by whitespace and the start of the next
# sentence: an uppercase letter, an opening quote, or a numbered-list marker.
# Digits are only accepted when followed by a list separator (")", ".", ":") so
# prose like "$2. 50" or "section 3. 4" is never split but "treasure? 2) How" is.
SENTENCE_END_RE = re.compile(
r'(?<=[.!?])\s+(?=[A-Z"\u201c\u2018])|(?<=[.!?])(?=["\u201d\u2019])|(?<=[.!?])$'
r'(?<=[.!?])\s+(?=[A-Z"\u201c\u2018]|\d+[).:])|(?<=[.!?])(?=["\u201d\u2019])|(?<=[.!?])$'
)
@@ -165,12 +169,14 @@ class Segmenter:
def _split_narration_sentences(self, text: str) -> list[str]:
"""Split narration-only text into sentence segments.
A newline designates a paragraph break (the manuscript convention).
Splits only at sentence-ending punctuation (`.`, `!`, `?`), never at
commas or inline quotation marks, so quoted terms mid-sentence (e.g.
"Android") stay inside their sentence.
"Android") stay inside their sentence. A standalone title line or a
numbered-list item becomes its own paragraph/utterance.
"""
sentences: list[str] = []
for para in text.split("\n\n"):
for para in text.split("\n"):
para = para.strip()
if not para:
continue
@@ -537,6 +543,10 @@ class Segmenter:
utt["character_name"] = "narrator"
continue
if utt["type"] == "scene_break":
utt["character_name"] = None
continue
if idx in tagged_speakers:
utt["character_name"] = tagged_speakers[idx]
continue

5
umbraaudio/pipeline_manager.py
View File

@@ -95,8 +95,11 @@ class PipelineProgress:
raw_state = "failed"
if raw_state in ("starting", "queued", "generating", "segmenting", "attributing", "saving"):
raw_state = "running"
step = data.get("step", "upload")
if step == "segmenting":
step = "segment"
return cls(
step=PipelineStep(data.get("step", "upload")),
step=PipelineStep(step),
state=PipelineState(raw_state),
detail=data.get("detail", ""),
progress=data.get("progress", 0),

117
umbraaudio/pronunciation.py
View File

@@ -187,6 +187,123 @@ def apply_phonetics(text: str, phonetic: dict[str, str]) -> str:
return text
# First-word tokens that are NOT proper names even when capitalized (role labels,
# titles, narrator variants) — an utterance starting with one of these must not
# gain a ", " pause.
_PAUSE_EXCLUDED_TOKENS = frozenset({
"a", "an", "and", "apprentice", "at", "blacksmith", "border", "boy",
"captain", "child", "children", "dr", "elder", "from", "general", "girl",
"guard", "guards", "handler", "he", "healer", "helper", "her", "his", "i",
"in", "it", "king", "lady", "lieutenant", "lord", "madam", "man", "men",
"mr", "mrs", "ms", "mute", "my", "narrator", "narratorold", "of", "officer",
"officers", "old", "older", "oldest", "on", "onlooker", "or", "orignarrator",
"our", "prince", "princess", "queen", "rider", "royal", "scavenger",
"scum", "sergeant", "she", "sir", "skeleton", "skin", "skinny", "smith",
"soldier", "soldiers", "stranger", "sword", "tallest", "the", "they", "to",
"troll", "unidentified", "unnamed", "vendor", "voice", "walking", "we",
"witch", "wolf", "wolves", "woman", "women", "you", "young", "younger",
"youngest", "your",
})
_OPENING_QUOTES = frozenset({'"', "\u201c", "\u2018", "'"})
def _name_token(name: str) -> str | None:
"""First-word, casefolded token of a proper-name candidate, or None when
the candidate carries a digit, a parenthetical (role/scene marker such as
``Guard 1`` / ``Aide (Royal)``), or a non-name first word."""
if not name:
return None
if any(ch.isdigit() for ch in name) or "(" in name or ")" in name:
return None
first = name.split()[0].casefold() if name.split() else ""
if not first or first in _PAUSE_EXCLUDED_TOKENS:
return None
return first
def load_proper_names(data_dir: Path) -> frozenset[str]:
"""Casefolded first-word tokens of proper names that may begin an utterance.
Sources: every speaker's primary name and alias (first word only), plus
user-confirmed phonetic records (``source == "user"``, e.g. the confirmed
names Feliu / Hallr / Kaleb / Merial / Sheyla / Xylia). Role labels,
narrator variants, and common words are excluded so narration such as
``"The boy ran"`` never gains a spurious pause. Used by
:func:`prepare_tts_text` to insert the leading ``', '`` pause.
"""
tokens: set[str] = set()
speakers_path = data_dir / "speakers.json"
if speakers_path.exists():
try:
with open(speakers_path) as f:
data = json.load(f)
except (OSError, ValueError):
data = {}
for s in data.get("speakers", []):
if not isinstance(s, dict):
continue
for cand in [s.get("primary_name", ""), *s.get("aliases", [])]:
token = _name_token(cand)
if token:
tokens.add(token)
for name, record in load_records(data_dir).items():
if record.get("source") != "user":
continue
token = _name_token(name)
if token:
tokens.add(token)
return frozenset(tokens)
def prefix_proper_name_pause(text: str, proper_names: frozenset[str]) -> str:
"""Insert ``', '`` before a proper name that opens an utterance.
Only the very first word of the utterance is examined (after an optional
leading quote), so names mentioned later in the line are untouched. Lines
that already carry the pause (the hand-edited ``, Hallr ...`` rows) pass
through unchanged. Possessives (``Kaleb's``) are allowed.
"""
if not text or not proper_names:
return text
start = 0
n = len(text)
while start < n and text[start].isspace():
start += 1
if start < n and text[start] in _OPENING_QUOTES:
start += 1
while start < n and text[start].isspace():
start += 1
if start >= n:
return text
if text.startswith(", ", start) or text[start] == ",":
return text
m = re.match(r"[A-Za-z]+", text[start:])
if not m:
return text
if m.group(0).casefold() not in proper_names:
return text
return text[:start] + ", " + text[start:]
def prepare_tts_text(
text: str,
phonetic: dict[str, str],
proper_names: frozenset[str] | None = None,
) -> str:
"""Full TTS text for an utterance: insert the leading proper-name pause
(when enabled) before applying phonetics. Every generation path funnels
through here so markers, staleness checks, and TTS input agree."""
if proper_names:
text = prefix_proper_name_pause(text, proper_names)
return apply_phonetics(text, phonetic)
def ipa_to_simple(ipa: str, word: str = "") -> str:
"""Convert IPA to an uppercase ASCII TTS-readable respelling.

5
umbraaudio/voices/qwen_tts.py
View File

@@ -332,7 +332,10 @@ class QwenTTSEngine:
payload["prompt_slug"] = prompt_slug
elif voice_ref and Path(voice_ref).exists():
files = {"ref_audio": ("ref.wav", Path(voice_ref).read_bytes(), "audio/wav")}
elif instruct:
# instruct composes with voice clone / ICL: the server prepends it as
# style direction (emotion/pace/delivery) while keeping the speaker.
if instruct:
payload["voice_description"] = instruct
try:

4
umbraaudio/voices/unified_model_manager.py
View File

@@ -131,7 +131,7 @@ class UnifiedModelManager:
self,
qwen_tts_url: str,
lm_studio_url: str = "http://localhost:1234",
lm_model: str = "gemma-4-12b-it-qat",
lm_model: str = "google/gemma-4-12b-qat",
) -> None:
self.qwen_manager = ModelManager(qwen_tts_url)
@@ -194,6 +194,6 @@ class UnifiedModelManager:
async def get_unified_model_manager(
qwen_tts_url: str,
lm_studio_url: str = "http://localhost:1234",
lm_model: str = "gemma-4-12b-it-qat",
lm_model: str = "google/gemma-4-12b-qat",
) -> UnifiedModelManager:
return UnifiedModelManager(qwen_tts_url, lm_studio_url, lm_model)

108
umbraaudio/voices/voice_designer.py
View File

@@ -1,8 +1,11 @@
from __future__ import annotations
import hashlib
import json
import logging
import time
from dataclasses import dataclass, field
from datetime import UTC, datetime
from pathlib import Path
from typing import Any
@@ -10,7 +13,7 @@ import numpy as np
from umbraaudio.analysis import CharacterProfile
from umbraaudio.config import TTSConfig
from umbraaudio.voices.model_manager import PipelineModelManager, get_pipeline_model_manager
from umbraaudio.voices.model_manager import ModelType, PipelineModelManager, get_pipeline_model_manager
logger = logging.getLogger(__name__)
@@ -248,27 +251,96 @@ class VoiceDesigner:
desc = character.voice_description or self._build_qwen_description(character)
if self._model_manager:
self._model_manager.for_voice_design()
self._model_manager.mm.ensure_model(ModelType.VOICEDESIGN)
try:
qwen = self._get_qwen()
qwen.generate_voice_reference(
text=text,
output_path=output_path,
voice_description=desc,
)
logger.info(
"Generated Qwen3-TTS voice reference for %s -> %s: %s",
character.name,
resolved_name,
output_path,
)
except Exception as e:
logger.warning("Qwen3-TTS voice reference failed, falling back to Edge-TTS: %s", e)
self._generate_edge_fallback(character, text, output_path)
max_retries = max(1, int(getattr(self.config, "max_retries", 1)))
retry_delay = float(getattr(self.config, "retry_delay", 2.0))
last_error: Exception | None = None
for i in range(max_retries):
try:
qwen = self._get_qwen()
qwen.generate_voice_reference(
text=text,
output_path=output_path,
voice_description=desc,
)
logger.info(
"Generated Qwen3-TTS voice reference for %s -> %s: %s",
character.name,
resolved_name,
output_path,
)
self._record_voice_engine(output_dir, resolved_name, "qwen", "ok")
return output_path
except Exception as e:
last_error = e
logger.warning(
"Qwen3-TTS voice reference failed for %s (attempt %d/%d): %s",
character.name,
i + 1,
max_retries,
e,
)
if i < max_retries - 1:
time.sleep(retry_delay * (i + 1))
# Qwen exhausted its attempts: fall back to Edge-TTS, but say so loudly
# and durably so the caller knows the ref is Edge-derived, not Qwen.
logger.error(
"Qwen3-TTS unavailable for %s after %d attempts; FALLING BACK to Edge-TTS: %s",
character.name,
max_retries,
last_error,
)
self._generate_edge_fallback(character, text, output_path)
self._record_voice_engine(
output_dir,
resolved_name,
"edge",
str(last_error)[:300] if last_error else "unknown qwen error",
)
return output_path
@staticmethod
def _voice_engines_path(output_dir: Path) -> Path:
"""Per-project engine ledger lives next to the voices dir (`.umbra/`)."""
return output_dir.parent / "voice_engines.json"
def _record_voice_engine(
self, output_dir: Path, name: str, engine: str, reason: str
) -> None:
"""Persist which engine produced the current ref for this speaker.
The web UI reads this ledger to show a per-speaker badge and a
project-wide fallback banner, so an Edge-TTS fallback is never silent.
"""
path = self._voice_engines_path(output_dir)
ledger: dict[str, Any] = {}
if path.exists():
try:
with open(path) as f:
ledger = json.load(f)
except (json.JSONDecodeError, OSError):
ledger = {}
try:
ref_mtime = (output_dir / f"{name.lower().replace(' ', '_')}_ref.wav").stat().st_mtime
except OSError:
ref_mtime = 0
ledger[name] = {
"engine": engine,
"reason": reason,
"at": datetime.now(UTC).isoformat(),
"ref_mtime": ref_mtime,
}
tmp = path.with_suffix(".json.tmp")
try:
with open(tmp, "w") as f:
json.dump(ledger, f, indent=2)
tmp.replace(path)
except OSError as e:
logger.warning("Could not write voice engine ledger %s: %s", path, e)
def _get_edge_engine(self):
if self._edge_engine is None:
from umbraaudio.voices.edge_tts import EdgeTTSEngine

121
umbraaudio/voices/voice_profile_manager.py
View File

@@ -354,30 +354,67 @@ def build_profile_from_speaker(speaker: dict[str, Any]) -> CharacterProfile:
explicit_pitch: str = demographics.get("pitch", "")
if is_narrator:
narrator_accent = (accent or "american").lower()
if narrator_accent == "none":
narrator_accent = "american"
accent_article = "an" if narrator_accent[0] in "aeiou" else "a"
# Narrators are driven by their OWN settings too (first-person books
# like Treasure Island have a young, high, accented narrator), not by
# the old hardcoded "late-night radio host" baritone. An explicit
# ``voice_description`` pin always wins.
narrator_pin = speaker.get("voice_description")
if narrator_pin:
profile_desc = narrator_pin.strip().rstrip(".") + "."
else:
narrator_parts: list[str] = []
narr_strength = speaker.get("accent_strength", "medium")
narr_strength_phrase = {
"light": "subtle",
"medium": "clear",
"strong": "strong, broad, unmistakable",
}.get(narr_strength, "clear")
if accent and accent != "none":
narr_base = f"{accent[0].upper() + accent[1:]} English"
narr_article = "an" if narr_base[:1].lower() in "aeiou" else "a"
narr_lead = (
f"A voice speaking with {narr_article} {narr_strength_phrase} "
f"{narr_base} accent"
)
if narr_base == "British English":
narr_lead += " in Received Pronunciation"
narrator_parts.append(narr_lead + ".")
narrator_parts.append(
{
"male": "A male voice",
"female": "A female voice",
}.get(gender.lower(), "A voice")
)
if age in AGE_DESCRIPTORS:
narrator_parts.append(AGE_DESCRIPTORS[age])
explicit_lower = explicit_pitch.lower()
if explicit_lower in ("high", "very_high", "ultra_high"):
narrator_parts.append("at high pitch")
elif explicit_lower not in ("mid", "mid_low", "low", "very_low") and age in ("Child", "Young"):
narrator_parts.append("at high pitch")
narrator_parts.append(
"speaking evenly as the narrator of the story, clearly and "
"observantly, recounting events in the first person with "
"calm composure, occasional urgency and a wondering curiosity"
)
profile_desc = " ".join(narrator_parts) + "."
return CharacterProfile(
name=name,
aliases=aliases,
gender="Male",
gender=gender if gender.lower() in ("male", "female") else "Male",
is_narrator=True,
pitch_register="very_low",
pace_baseline="slow",
energy="commanding",
timbre="resonant",
texture="smooth",
formality="formal",
emotional_baseline="neutral_calm",
distinguishing_trait="late-night radio host style",
accent_marker=narrator_accent,
voice_description=(
"A deep, resonant male voice with a calm, authoritative tone. "
"Rich baritone timbre, slow and deliberate pacing, "
f"spoken with {accent_article} {narrator_accent} accent, "
"like a late-night radio host. Very controlled and even."
pitch_register=(
explicit_pitch if explicit_pitch in VALID_PITCH_REGISTERS else "high"
),
pace_baseline="moderate",
energy="measured",
timbre="clear",
texture="smooth",
formality="conversational",
emotional_baseline="neutral_calm",
distinguishing_trait="storytelling narrator delivery",
accent_marker=accent or "american",
voice_description=profile_desc,
)
demo_desc = DEMOGRAPHIC_TYPE_TO_DESC.get(demo_type, demo_type.lower())
@@ -468,6 +505,32 @@ def build_profile_from_speaker(speaker: dict[str, Any]) -> CharacterProfile:
# Phase 5: Build rich voice description
desc_parts: list[str] = []
# The spoken accent leads the description so the voice-design model treats it as
# the dominant conditioning signal rather than a buried clause. British/other
# accents are stated as the primary marker and reinforced by strength.
accent_strength = speaker.get("accent_strength", "medium")
accent_article = "a"
accent_base = f"{accent[0].upper() + accent[1:]} English"
strength_phrase = {
"light": "subtle",
"medium": "clear",
"strong": "strong, broad, unmistakable",
}.get(accent_strength, "clear")
if accent and accent != "none":
gender_lead = {
"male": "A male voice",
"female": "A female voice",
}.get(inferred_gender, "A voice")
accent_lead = (
f"{gender_lead} speaking with {accent_article} {strength_phrase} "
f"{accent_base} accent"
)
if accent_base == "British English":
accent_lead += " in Received Pronunciation"
accent_lead += "."
desc_parts.append(accent_lead)
if inferred_gender == "male":
desc_parts.append("A male voice")
elif inferred_gender == "female":
@@ -490,15 +553,6 @@ def build_profile_from_speaker(speaker: dict[str, Any]) -> CharacterProfile:
if timbre in timbre_desc_map:
desc_parts.append(f"with a {timbre_desc_map[timbre]} timbre")
if accent and accent != "none":
accent_strength = speaker.get("accent_strength", "medium")
accent_article = "an" if accent[0] in "aeiou" else "a"
strength_prefix = {
"light": f"{accent_article} subtle",
"strong": f"{accent_article} strong, broad",
}.get(accent_strength, accent_article)
desc_parts.append(f"with {strength_prefix} {accent} accent")
if age in AGE_DESCRIPTORS:
desc_parts.append(AGE_DESCRIPTORS[age])
@@ -555,7 +609,16 @@ def build_profile_from_speaker(speaker: dict[str, Any]) -> CharacterProfile:
if description:
desc_parts.append(description[:200].rstrip(".") + ".")
voice_description = " ".join(desc_parts) + "."
voice_description = " ".join(part.rstrip(".") for part in desc_parts) + "."
# Pinned voice_description full override: if the speaker carries an explicit
# voice_description field, use it VERBATIM and skip the assembled text. This is
# the escape hatch for characters whose voice the trait-driven assembly gets
# wrong (e.g. Captain Flint's parrot), so any regeneration path (UI button,
# generate-all, chapter build) produces the same, intended voice.
pinned_description = speaker.get("voice_description")
if pinned_description:
voice_description = pinned_description.strip().rstrip(".") + "."
# Phase 6: Distinguishing trait
trait_parts: list[str] = []

15
umbraaudio/web/__init__.py
View File

@@ -3,7 +3,8 @@ from __future__ import annotations
from pathlib import Path
from typing import TYPE_CHECKING
from fastapi import FastAPI
from fastapi import FastAPI, Request
from fastapi.responses import JSONResponse, RedirectResponse
from fastapi.staticfiles import StaticFiles
if TYPE_CHECKING:
@@ -27,10 +28,20 @@ def create_app(config: AppConfig) -> FastAPI:
from umbraaudio.config import list_projects
from umbraaudio.pipeline_manager import PipelineManager
from umbraaudio.voices.generation_manager import GenerationManager
from umbraaudio.web.routes import router
from umbraaudio.web.routes import NoActiveProjectError, router
app.include_router(router)
@app.exception_handler(NoActiveProjectError)
async def _no_active_project_handler(request: Request, exc: NoActiveProjectError):
"""HTML pages go to the project picker; API callers get a 400 error."""
accept = (request.headers.get("accept") or "").lower()
if "text/html" in accept:
return RedirectResponse(url="/projects", status_code=303)
return JSONResponse(
{"detail": str(exc), "redirect": "/projects"}, status_code=400
)
app.state.config = config
app.state.projects = list_projects()
if config.data_dir and config.data_dir.exists() and config.data_dir.name != ".umbra":

2297
umbraaudio/web/routes.py
View File

File diff suppressed because it is too large Load Diff

20
umbraaudio/web/templates/_library_speakers.html
View File

@@ -1,7 +1,7 @@
<div class="card">
<div style="display:flex; justify-content:space-between; align-items:center; margin-bottom:1rem">
<h3 style="margin:0">Library Speakers ({{ speakers|length }})</h3>
<span style="color:#64748b; font-size:0.85rem">Add speakers from the project Speakers page using the "Lib" button.</span>
<span style="color:#64748b; font-size:0.85rem">Global voices (library + registry). Use "Import ➜" to add one to the current project.</span>
</div>
{% if not speakers %}
<p style="color:#64748b">No speakers in the library yet.</p>
@@ -51,8 +51,12 @@
{% endif %}
</td>
<td>
<button class="btn btn-generate" style="font-size:0.7rem; padding:0.2rem 0.5rem"
onclick="importToProject({{ s.id }}, '{{ s.source_kind }}', '{{ s.primary_name|e }}')">Import ➜</button>
{% if s.source_kind == 'lib' %}
<button class="btn btn-reject" style="font-size:0.7rem; padding:0.2rem 0.5rem"
onclick="removeLibSpeaker({{ s.id }}, '{{ s.primary_name|e }}')">✕</button>
{% endif %}
</td>
</tr>
{% endfor %}
@@ -61,6 +65,20 @@
{% endif %}
</div>
<script>
function importToProject(id, kind, name) {
if (!confirm('Import "' + name + '" into the current project?')) return;
var url = kind === 'registry' ? '/api/registry/' + id + '/import' : '/api/library/speakers/' + id + '/import';
fetch(url, {method: 'POST'})
.then(function (r) { return r.json(); })
.then(function (d) {
var msg = d.status === 'added'
? 'Imported "' + name + '"' + (d.ref_copied ? ' with its voice reference.' : ' (no reference audio found).')
: d.status === 'already' ? '"' + name + '" already exists in this project.'
: 'Import failed: ' + (d.reason || 'unknown error');
alert(msg);
htmx.ajax('GET', '/library/speakers', {target: '#lib-tab', swap: 'innerHTML'});
});
}
function removeLibSpeaker(id, name) {
if (!confirm('Remove "' + name + '" from the library?')) return;
fetch('/api/library/speakers/' + id, {method: 'DELETE'})

53
umbraaudio/web/templates/_model_management.html
View File

@@ -194,27 +194,42 @@ async function unloadLmModel() {
async function refreshModels() {
const btn = event.target;
btn.disabled = true;
btn.textContent = 'Refreshing...';
if (btn) { btn.disabled = true; btn.textContent = 'Refreshing...'; }
const select = document.getElementById('lm-model-select');
const current = select ? select.value : null;
try {
const resp = await fetch('/api/models/lm/available');
const data = await resp.json();
const select = document.getElementById('lm-model-select');
const current = select.value;
select.innerHTML = '';
data.models.forEach(m => {
const opt = document.createElement('option');
opt.value = m.key;
opt.textContent = `${m.display_name || m.key} (${m.params}, ${m.size_gb}GB)${m.loaded ? ' ✓' : ''}`;
if (m.key === current) opt.selected = true;
select.appendChild(opt);
});
if (select) {
select.innerHTML = '';
if (Array.isArray(data.models) && data.models.length) {
data.models.forEach(m => {
const opt = document.createElement('option');
opt.value = m.key;
opt.textContent = `${m.display_name || m.key} (${m.params || '?'}, ${m.size_gb || '?'}GB)${m.loaded ? ' ✓' : ''}`;
if (m.key === current) opt.selected = true;
select.appendChild(opt);
});
} else {
const opt = document.createElement('option');
opt.value = '';
opt.disabled = true;
opt.textContent = data.error || 'No models available';
select.appendChild(opt);
}
}
refreshModelStatus();
} catch (e) {
alert('Error: ' + e.message);
console.error('Failed to refresh models:', e);
if (select && select.options.length === 0) {
const opt = document.createElement('option');
opt.value = '';
opt.disabled = true;
opt.textContent = 'Could not load models';
select.appendChild(opt);
}
} finally {
btn.disabled = false;
btn.textContent = 'Refresh';
if (btn) { btn.disabled = false; btn.textContent = 'Refresh'; }
}
}
@@ -222,14 +237,14 @@ async function refreshModelStatus() {
try {
const resp = await fetch('/api/models/status');
const data = await resp.json();
// Update Qwen
const qwen = data.qwen_tts;
const qwen = data.qwen_tts || {};
updateModelBadge('VoiceDesign', qwen.voicedesign_loaded);
updateModelBadge('Base Clone', qwen.base_loaded);
// Update LM Studio
const lm = data.lm_studio;
const lm = data.lm_studio || {};
const lmStatus = document.querySelector('#model-management-section .model-status:last-of-type');
if (lmStatus) {
const indicator = lmStatus.querySelector('.status-indicator');

19
umbraaudio/web/templates/_registry_list.html
View File

@@ -62,9 +62,26 @@
<span style="color:#64748b">No ref</span>
{% endif %}
</td>
<td>
<button class="btn btn-generate" style="font-size:0.7rem; padding:0.2rem 0.5rem"
onclick="importRegistryToProject({{ char.id }}, '{{ char.character_name|e }}')">Import ➜</button>
</td>
</tr>
{% endfor %}
</tbody>
</table>
{% endif %}
</div>
</div>
<script>
function importRegistryToProject(id, name) {
if (!confirm('Import "' + name + '" into the current project?')) return;
fetch('/api/registry/' + id + '/import', {method: 'POST'})
.then(r => r.json()).then(d => {
alert(d.status === 'added'
? 'Imported "' + name + '"' + (d.ref_copied ? ' with its voice reference.' : ' (no reference audio found).')
: d.status === 'already' ? '"' + name + '" already exists in this project.'
: 'Import failed: ' + (d.reason || 'unknown error'));
htmx.ajax('GET', '/registry/partial', {target: '#registry-list', swap: 'innerHTML'});
});
}
</script>

20
umbraaudio/web/templates/_speaker_row.html
View File

@@ -49,17 +49,24 @@
{% set ref_file = ref_name ~ '_ref.wav' %}
<div style="display:flex; gap:0.3rem; align-items:center; flex-wrap:wrap">
{% if speaker.has_voice_ref %}
<button class="btn btn-generate" style="font-size:0.75rem; padding:0.2rem 0.5rem"
onclick="playSpeakerSample('{{ ref_file }}')">
<button class="btn btn-generate" id="play-btn-{{ speaker_idx }}" style="font-size:0.75rem; padding:0.2rem 0.5rem"
onclick="toggleSpeakerPlay({{ speaker_idx }}, '{{ ref_file }}')">
▶ Play
</button>
<audio id="audio-{{ speaker_idx }}" style="height:28px; width:140px" preload="none">
<audio id="audio-{{ speaker_idx }}" style="display:none" preload="none">
<source src="/voice-test-audio/{{ ref_file }}" type="audio/wav">
</audio>
{% else %}
<span style="font-size:0.75rem; color:#64748b; font-style:italic">No sample</span>
{% endif %}
<div style="width:100%; margin-top:0.25rem">
{% if speaker.engine_fallback %}
<span class="badge badge-red" style="font-size:0.7rem"
title="Generated with Edge-TTS, not Qwen3-TTS. Reason: {{ speaker.voice_engine_reason }}">⚠ Edge-TTS fallback</span>
<span style="font-size:0.65rem; color:#f87171; display:block">not the Qwen3 voice — regenerate</span>
{% elif speaker.voice_engine == 'qwen' and speaker.has_voice_ref %}
<span class="badge" style="font-size:0.7rem; background:#065f46; color:#6ee7b7">Qwen3-TTS ✓</span>
{% endif %}
{% if speaker.voice_status == 'approved' %}
<span class="badge badge-green" style="font-size:0.7rem">Approved ✓</span>
<span style="font-size:0.65rem; color:#64748b; display:block">saved to registry</span>
@@ -183,7 +190,7 @@
<label style="color:#94a3b8; font-size:0.75rem; display:block; margin-bottom:0.2rem">Post-gen effects (applied after TTS)</label>
<select name="distortion_effects" multiple size="3"
style="width:100%; padding:0.3rem; background:#1a1a2e; border:1px solid #333; border-radius:3px; color:#e0e0e0; font-size:0.8rem">
{% set available_effects = ['bitcrusher','hard_clip','tremolo','random_gating','bandpass','lowpass','noise_inject','pitch_jitter','ring_modulate'] %}
{% set available_effects = ['squawk','polly','bitcrusher','hard_clip','tremolo','random_gating','bandpass','lowpass','highpass','tube_sat','chorus','noise_inject','pitch_jitter','pitch_shift','pitch_shift_pv','vibrato','ring_modulate'] %}
{% set current_effects = speaker.get('distortion_effects', []) %}
{% for eff in available_effects %}
<option value="{{ eff }}" {% if eff in current_effects %}selected{% endif %}>{{ eff }}</option>
@@ -207,6 +214,11 @@
style="width:100%; padding:0.4rem; background:#1a1a2e; border:1px solid #333; border-radius:3px; color:#e0e0e0; font-size:0.85rem; margin-bottom:0.5rem">
<input type="text" name="description" value="{{ speaker.description }}"
style="width:100%; padding:0.4rem; background:#1a1a2e; border:1px solid #333; border-radius:3px; color:#e0e0e0; font-size:0.85rem; margin-bottom:0.5rem">
<div style="margin-bottom:0.5rem">
<label style="color:#94a3b8; font-size:0.75rem">Pinned voice instruct (overrides auto-built description — leave empty to auto-build)</label>
<textarea name="voice_description" rows="3"
style="width:100%; padding:0.4rem; background:#1a1a2e; border:1px solid #333; border-radius:3px; color:#e0e0e0; font-size:0.85rem; margin-top:0.2rem">{{ speaker.voice_description }}</textarea>
</div>
<input type="text" name="sample_dialogue" value="{{ speaker.sample_dialogue }}"
style="width:100%; padding:0.4rem; background:#1a1a2e; border:1px solid #333; border-radius:3px; color:#e0e0e0; font-size:0.85rem; margin-bottom:0.5rem">
<div style="display:grid; grid-template-columns:1fr 1fr; gap:0.5rem; margin-bottom:0.5rem">

13
umbraaudio/web/templates/_speakers_list.html
View File

@@ -17,6 +17,19 @@
</div>
</div>
{% set fallback_count = speakers|selectattr('engine_fallback')|list|length %}
{% if not qwen_up %}
<div style="padding:0.75rem 1rem; margin-bottom:1rem; background:#1a1a2e; border:1px solid #b45309; border-left:4px solid #f59e0b; border-radius:6px">
<strong style="color:#f59e0b">⚠ Qwen3-TTS server unreachable</strong>
<span style="color:#94a3b8; font-size:0.85rem"> — {{ qwen_url or 'no qwen URL configured' }}. Voice generation will <strong style="color:#f59e0b">fall back to Edge-TTS</strong>, which loses raspy/texture and may pick an arbitrary voice. Restart the Qwen server first for real Qwen3 voices.</span>
</div>
{% elif fallback_count %}
<div style="padding:0.75rem 1rem; margin-bottom:1rem; background:#1a1a2e; border:1px solid #b91c1c; border-left:4px solid #ef4444; border-radius:6px">
<strong style="color:#ef4444">⚠ {{ fallback_count }} voice{{ 's' if fallback_count != 1 }} were generated with Edge-TTS fallback</strong>
<span style="color:#94a3b8; font-size:0.85rem"> — those rows show a red badge. Regenerate them now with a running Qwen3-TTS server to get the intended voices.</span>
</div>
{% endif %}
<div id="add-speaker-form" style="display:none; margin-bottom:1.5rem; padding:1rem; background:#0f3460; border-radius:8px">
<form hx-post="/api/speakers"
hx-target="#speakers-list"

2
umbraaudio/web/templates/_utterance_list.html
View File

@@ -25,7 +25,7 @@
{% set ch_id = chapter_id %}
{% endif %}
<audio controls preload="none">
<source src="/audio/{{ ch_id }}/{{ fname }}" type="audio/wav">
<source src="/audio/{{ active_project(request).slug }}/{{ ch_id }}/{{ fname }}" type="audio/wav">
</audio>
{% if u.duration_ms %}
<small style="color: #64748b;"> {{ "%.1f"|format(u.duration_ms / 1000) }}s</small>

2
umbraaudio/web/templates/_voice_analysis_report.html
View File

@@ -28,7 +28,7 @@
<div class="label">Failing</div>
</div>
<div class="stat">
<div class="num" style="color:{% if report.summary.near_duplicate_count > 0 %}#ef4444{% else %}#7c3aed{% endif %}">{{ report.summary.near_duplicate_count }}</div>
<div class="num" style="color:{% if (report.summary.near_duplicate_count|default(0)) > 0 %}#ef4444{% else %}#7c3aed{% endif %}">{{ report.summary.near_duplicate_count|default(0) }}</div>
<div class="label">Near-duplicate pairs</div>
</div>
<div class="stat">

22
umbraaudio/web/templates/base.html
View File

@@ -56,12 +56,33 @@
.card-completed { border-left: 3px solid #10b981; }
.progress-bar { width: 100%; height: 6px; background: #374151; border-radius: 3px; overflow: hidden; }
.progress-fill { height: 100%; background: #7c3aed; border-radius: 3px; transition: width 0.3s ease; }
.qa-box { display: inline-flex; align-items: center; gap: 0.3rem; background: #b91c1c; border: 2px solid #fff; border-radius: 6px; padding: 0.15rem 0.3rem; }
.qa-badge-wrap { position: relative; display: inline-flex; }
.qa-badge { background: #fff; color: #b91c1c; font-weight: 900; width: 1.5rem; height: 1.5rem; border-radius: 4px; display: inline-flex; align-items: center; justify-content: center; font-size: 0.95rem; cursor: help; }
.qa-tooltip { display: none; position: absolute; bottom: calc(100% + 8px); left: 0; z-index: 60; width: 340px; max-height: 260px; overflow-y: auto; background: #0b1120; color: #f8fafc; border: 2px solid #f87171; border-radius: 8px; padding: 0.6rem 0.75rem; font-size: 0.75rem; line-height: 1.45; box-shadow: 0 8px 20px rgba(0,0,0,0.65); }
.qa-badge-wrap:hover .qa-tooltip, .qa-badge-wrap:focus-within .qa-tooltip { display: block; }
.qa-tl-head { font-weight: 700; color: #fca5a5; margin-bottom: 0.35rem; }
.qa-tl-item { margin-bottom: 0.5rem; }
.qa-tl-item:last-child { margin-bottom: 0; }
.qa-tl-item b { color: #fca5a5; }
.qa-tl-ev { color: #cbd5e1; }
.qa-btn { margin-right: 0; font-size: 0.65rem; padding: 0.15rem 0.4rem; border: none; border-radius: 4px; cursor: pointer; font-weight: 700; white-space: nowrap; }
.qa-recheck { background: #f59e0b; color: #1a1a2e; }
.qa-recheck:hover { background: #fbbf24; }
.qa-accept { background: #10b981; color: #fff; }
.qa-accept:hover { background: #34d399; }
.qa-btn:disabled { opacity: 0.6; cursor: default; }
.qa-accepted-badge { display: inline-flex; align-items: center; gap: 0.2rem; background: #065f46; color: #6ee7b7; font-size: 0.65rem; font-weight: 700; border-radius: 4px; padding: 0.15rem 0.4rem; cursor: help; }
</style>
</head>
<body>
{% set ap = active_project(request) %}
<h1 style="display:flex; align-items:center; gap:0.75rem;">
<img src="/static/logo.png" alt="UmbraAudio logo" style="height:56px; width:56px; border-radius:12px;">
UmbraAudio Review
<span class="badge badge-blue" style="margin-left:auto; font-size:0.9rem; padding:0.4rem 0.9rem; display:inline-flex; align-items:center; gap:0.5rem;" title="Project: {{ ap.slug }}">
Active project: <strong style="color:#e0e7ff">{{ ap.name }}</strong>
</span>
</h1>
<nav class="nav">
<a href="/">Dashboard</a>
@@ -69,7 +90,6 @@
<a href="/proper-names">Proper Names</a>
<a href="/chapters-audio">Chapters</a>
<a href="/quality">Quality</a>
<a href="/registry">Voice Registry</a>
<a href="/library">Lib</a>
<a href="/m4b-meta">m4b Meta</a>
<a href="/projects" style="margin-left: auto; border-color: #a78bfa;">Projects</a>

236
umbraaudio/web/templates/chapter_audio_detail.html
View File

@@ -3,7 +3,15 @@
<div class="card">
<div style="display:flex; justify-content:space-between; align-items:center; margin-bottom:1rem">
<div>
<h2 style="margin:0">{{ chapter_title }}</h2>
<div style="display:flex; gap:0.5rem; align-items:center; flex-wrap:wrap">
<input type="text" id="chapter-title-input" value="{{ chapter_title }}"
onkeydown="if(event.key === 'Enter'){event.preventDefault(); saveChapterTitle();}"
style="font-size:1.5rem; font-weight:bold; background:#1a1a2e; border:1px solid #333; border-radius:4px; color:#e0e0e0; padding:0.25rem 0.5rem; min-width:320px">
<button class="btn btn-approve" onclick="saveChapterTitle()">Save Title</button>
<button class="btn" style="background:#374151; color:#9ca3af" onclick="useFirstUtteranceTitle()"
title="Copy the first utterance's text into the title field">Use 1st Utterance</button>
<span id="chapter-title-status" style="color:#64748b; font-size:0.8rem"></span>
</div>
<p style="color:#94a3b8; margin:0.25rem 0">{{ total_utterances }} utterances</p>
</div>
<a href="/chapters-audio" class="btn" style="background:#374151; color:#9ca3af; text-decoration:none">
@@ -47,6 +55,13 @@
</div>
</div>
{% if blank_count %}
<div class="card" style="margin-bottom:1rem; border-left:3px solid #dc2626">
<h3 style="color:#dc2626; margin:0">⚠ {{ blank_count }} utterance{% if blank_count != 1 %}s{% endif %} with no text — flagged for review</h3>
<p style="color:#94a3b8; font-size:0.85rem; margin:0.25rem 0 0">Blank utterances are skipped at build time and their stray audio is no longer stitched into the chapter. Restore the text or delete the row (✕) to clear the flag.</p>
</div>
{% endif %}
{% if chapter_id|string == '1' %}
<div class="card" style="margin-bottom:1rem; border-left:3px solid #f59e0b">
<h3 style="color:#f59e0b; margin-bottom:0.5rem">Personalization</h3>
@@ -75,9 +90,13 @@
</thead>
<tbody>
{% for audio in audio_files %}
<tr id="utterance-{{ audio.index }}">
<td>
<tr id="utterance-{{ audio.index }}"{% if audio.is_blank %} style="background:rgba(220,38,38,0.08); outline:1px solid #dc2626"{% endif %}>
<td style="text-align:center; vertical-align:top">
<button class="btn" title="Move up" style="font-size:0.65rem; padding:0.05rem 0.35rem; display:block; margin:0 auto 0.15rem"
onclick="moveUtterance({{ audio.utt_id }}, 1)">↑</button>
<small style="color:#64748b">{{ audio.index }}</small>
<button class="btn" title="Move down" style="font-size:0.65rem; padding:0.05rem 0.35rem; display:block; margin:0.15rem auto 0"
onclick="moveUtterance({{ audio.utt_id }}, 0)">↓</button>
</td>
<td>
<select id="speaker-{{ audio.index }}" onchange="updateSpeaker({{ audio.utt_id }}, {{ audio.index }})"
@@ -92,7 +111,7 @@
<div style="display:flex; gap:0.3rem; align-items:center">
<select id="emotion-{{ audio.index }}"
style="padding:0.2rem; background:#1a1a2e; border:1px solid #333; border-radius:3px; color:#e0e0e0; font-size:0.75rem; flex:1">
{% for e in ["neutral","angry","sad","happy","fearful","surprised","whisper","pleading","sarcastic","authoritative","thoughtful","excited","anxious","tired","romantic","disgusted","amused","nostalgic"] %}
{% for e in ["neutral","angry","sad","happy","fearful","surprised","whisper","pleading","sarcastic","authoritative","thoughtful","excited","anxious","tired","romantic","disgusted","amused","nostalgic","tense","urgent","horror","furious","joyful","terrified","griefstricken","shocked"] %}
<option value="{{ e }}" {% if e == audio.emotion %}selected{% endif %}>{{ e }}</option>
{% endfor %}
</select>
@@ -107,15 +126,46 @@
value="{{ audio.instruct }}"
placeholder="Custom instruction (optional)..."
style="padding:0.2rem; background:#1a1a2e; border:1px solid #333; border-radius:3px; color:#e0e0e0; font-size:0.75rem; width:100%">
<button id="save-btn-{{ audio.index }}" class="btn btn-approve" style="font-size:0.65rem; padding:0.1rem 0.4rem; width:fit-content"
onclick="saveVariation({{ audio.utt_id }}, {{ audio.index }})">
Save Variation
</button>
<div style="display:flex; gap:0.3rem; align-items:center; flex-wrap:wrap">
<button id="save-btn-{{ audio.index }}" class="btn btn-approve" style="font-size:0.65rem; padding:0.1rem 0.4rem; width:fit-content"
onclick="saveVariation({{ audio.utt_id }}, {{ audio.index }})">
Save Variation
</button>
{% if audio.qa_issues %}
<div id="qa-box-{{ audio.index }}" class="qa-box"
style="{% if audio.qa_accepted %}display:none{% endif %}">
<div class="qa-badge-wrap" tabindex="0" aria-label="QA issues">
<span class="qa-badge">⚠</span>
<div class="qa-tooltip">
<div class="qa-tl-head">QA flag — fix then Recheck, or Accept as-is:</div>
{% for iss in audio.qa_issues %}
<div class="qa-tl-item">
<b>[{{ iss.type }}]</b>
{% if iss.expected %} expected <i>{{ iss.expected }}</i>{% endif %}
(conf {{ "%.2f"|format(iss.confidence) }})<br>
{% if iss.evidence %}<span class="qa-tl-ev">{{ iss.evidence }}</span><br>{% endif %}
{% if iss.text %}<i>“{{ iss.text }}”</i>{% endif %}
</div>
{% endfor %}
</div>
</div>
<button id="qa-recheck-{{ audio.index }}" class="btn qa-btn qa-recheck"
onclick="recheckUtterance({{ audio.utt_id }}, {{ audio.index }})">↻ Recheck</button>
<button id="qa-accept-{{ audio.index }}" class="btn qa-btn qa-accept"
onclick="acceptUtterance({{ audio.utt_id }}, {{ audio.index }})">✓ Accept</button>
</div>
<span id="qa-accepted-{{ audio.index }}" class="qa-accepted-badge"
style="{% if not audio.qa_accepted %}display:none{% endif %}"
title="Accepted as-is — the box reappears if the speaker/text changes or a recheck finds issues">✓ Accepted</span>
{% endif %}
<span id="qa-status-{{ audio.index }}" style="color:#64748b; font-size:0.7rem"></span>
</div>
</div>
</td>
<td>
{% if audio.is_blank %}<span class="badge badge-red" style="color:#ef4444">⚠ no text</span>{% endif %}
<div id="text-display-{{ audio.index }}" onclick="editText({{ audio.index }})" style="cursor:pointer; padding:0.2rem; border-radius:3px" title="Click to edit">
<small style="color:#94a3b8">{{ audio.text }}</small>
<small style="color:#94a3b8">{{ audio.text or '(click to add text)' }}</small>
</div>
<div id="text-edit-{{ audio.index }}" style="display:none">
<textarea id="text-input-{{ audio.index }}" rows="2"
@@ -133,7 +183,7 @@
▶
</button>
<audio id="audio-{{ audio.index }}" style="height:32px; width:150px; margin-left:0.3rem">
<source src="/chapter-audio-file/{{ chapter_dir }}/{{ audio.filename }}" type="audio/wav">
<source src="/chapter-audio-file/{{ active_project(request).slug }}/{{ chapter_dir }}/{{ audio.filename }}" type="audio/wav">
</audio>
{% else %}
<span style="color:#64748b; font-size:0.75rem" id="audio-status-{{ audio.index }}">No audio</span>
@@ -170,6 +220,11 @@
<input type="text" id="new-utterance-text" placeholder="Enter utterance text..."
style="width:100%; padding:0.4rem; background:#1a1a2e; border:1px solid #333; border-radius:4px; color:#e0e0e0; font-size:0.85rem">
</div>
<div>
<label style="color:#94a3b8; font-size:0.8rem; display:block; margin-bottom:0.25rem">Insert before #</label>
<input type="number" id="new-utterance-index" min="0" placeholder="end"
style="width:100%; padding:0.4rem; background:#1a1a2e; border:1px solid #333; border-radius:4px; color:#e0e0e0; font-size:0.85rem">
</div>
<button class="btn btn-generate" onclick="addUtterance()" style="white-space:nowrap">Add</button>
</div>
<p style="color:#64748b; font-size:0.75rem; margin-top:0.5rem" id="add-utterance-status"></p>
@@ -180,6 +235,40 @@ let currentAudio = null;
let playAllStopped = false;
let playAllIndex = -1;
var FIRST_UTT_TEXT = {{ (audio_files[0].full_text if audio_files else '')|tojson }};
function saveChapterTitle() {
const input = document.getElementById('chapter-title-input');
const status = document.getElementById('chapter-title-status');
const title = input.value.trim();
status.textContent = 'Saving...';
status.style.color = '#94a3b8';
fetch('/api/chapter/{{ chapter_id }}/title', {
method: 'PUT',
headers: {'Content-Type': 'application/json'},
body: JSON.stringify({title: title})
}).then(r => r.json()).then(data => {
if (data.ok) {
status.textContent = '✓ Saved';
status.style.color = '#10b981';
setTimeout(() => { status.textContent = ''; }, 2000);
} else {
status.textContent = 'Failed: ' + (data.error || 'unknown');
status.style.color = '#ef4444';
}
}).catch(() => {
status.textContent = 'Failed: request error';
status.style.color = '#ef4444';
});
}
function useFirstUtteranceTitle() {
if (!FIRST_UTT_TEXT) { return; }
const input = document.getElementById('chapter-title-input');
input.value = FIRST_UTT_TEXT.length > 80 ? FIRST_UTT_TEXT.slice(0, 80) : FIRST_UTT_TEXT;
saveChapterTitle();
}
function updateSpeaker(uttId, index) {
const select = document.getElementById('speaker-' + index);
const newSpeaker = select.value;
@@ -229,6 +318,114 @@ function saveVariation(uttId, index) {
});
}
function escHtml(s) {
const d = document.createElement('div');
d.textContent = String(s == null ? '' : s);
return d.innerHTML;
}
function qaTooltipHtml(issues) {
let html = '<div class="qa-tl-head">QA flag — fix then Recheck, or Accept as-is:</div>';
for (const iss of issues) {
html += '<div class="qa-tl-item"><b>[' + escHtml(iss.type) + ']</b>';
if (iss.expected) html += ' expected <i>' + escHtml(iss.expected) + '</i>';
html += ' (conf ' + (iss.confidence != null ? Number(iss.confidence).toFixed(2) : '?') + ')<br>';
if (iss.evidence) html += '<span class="qa-tl-ev">' + escHtml(iss.evidence) + '</span><br>';
if (iss.text) html += '<i>“' + escHtml(iss.text) + '”</i>';
html += '</div>';
}
return html;
}
function recheckUtterance(uttId, index) {
const btn = document.getElementById('qa-recheck-' + index);
const status = document.getElementById('qa-status-' + index);
const box = document.getElementById('qa-box-' + index);
if (btn) { btn.disabled = true; btn.textContent = '…'; }
if (status) { status.textContent = 'Rechecking…'; status.style.color = '#f59e0b'; }
// Flush any open text edit so the check sees the current wording.
const textEdit = document.getElementById('text-edit-' + index);
const textInput = document.getElementById('text-input-' + index);
const flush = (textEdit && textEdit.style.display !== 'none' && textInput)
? fetch('/api/utterance/' + uttId + '/text', {
method: 'PUT',
headers: {'Content-Type': 'application/json'},
body: JSON.stringify({text: textInput.value})
}).then(r => r.json()).then(d => { if (!d.ok) { throw new Error('save text failed'); } })
: Promise.resolve();
flush.then(() => fetch('/api/utterance/' + uttId + '/recheck', { method: 'POST' }))
.then(r => r.json()).then(data => {
if (!data.ok) { throw new Error(data.error || 'recheck failed'); }
// Apply any auto-fixed speaker so the dropdown matches the DB.
const fix = (data.fixed || []).find(f => f.utt_id === uttId);
if (fix) {
const sel = document.getElementById('speaker-' + index);
if (sel) {
const esc = fix.to.replace(/"/g, '\\"');
let opt = sel.querySelector('option[value="' + esc + '"]');
if (!opt) {
opt = document.createElement('option');
opt.value = fix.to;
opt.textContent = fix.to;
sel.appendChild(opt);
}
sel.value = fix.to;
}
}
if (box) {
if (data.issues && data.issues.length > 0) {
const tip = box.querySelector('.qa-tooltip');
if (tip) { tip.innerHTML = qaTooltipHtml(data.issues); }
box.style.display = 'inline-flex';
} else {
box.style.display = 'none';
const accBadge = document.getElementById('qa-accepted-' + index);
if (accBadge) { accBadge.style.display = 'inline-flex'; }
}
}
if (status) {
if (data.clean) {
status.textContent = '✓ No issues';
status.style.color = '#10b981';
} else if (fix) {
status.textContent = '✓ Auto-fixed to ' + fix.to + ' — still flagged';
status.style.color = '#f59e0b';
} else if (data.fixed && data.fixed.length > 0) {
status.textContent = '✓ Auto-fixed ' + data.fixed.length + ' in chapter — still flagged';
status.style.color = '#f59e0b';
} else {
status.textContent = '⚠ Still flagged';
status.style.color = '#ef4444';
}
}
if (btn) { btn.disabled = false; btn.textContent = '↻ Recheck'; }
}).catch((err) => {
if (status) { status.textContent = '✗ ' + (err.message || 'failed'); status.style.color = '#ef4444'; }
if (btn) { btn.disabled = false; btn.textContent = '↻ Recheck'; }
alert('Recheck failed: ' + (err.message || 'unknown'));
});
}
function acceptUtterance(uttId, index) {
const btn = document.getElementById('qa-accept-' + index);
if (btn) { btn.disabled = true; btn.textContent = '…'; }
fetch('/api/utterance/' + uttId + '/qa-accept', { method: 'POST' })
.then(r => r.json()).then(data => {
if (!data.ok) { throw new Error(data.error || 'accept failed'); }
const box = document.getElementById('qa-box-' + index);
const accBadge = document.getElementById('qa-accepted-' + index);
const status = document.getElementById('qa-status-' + index);
if (box) { box.style.display = 'none'; }
if (accBadge) { accBadge.style.display = 'inline-flex'; }
if (status) { status.textContent = '✓ Accepted'; status.style.color = '#10b981'; }
}).catch((err) => {
if (btn) { btn.disabled = false; btn.textContent = '✓ Accept'; }
alert('Accept failed: ' + (err.message || 'unknown'));
});
}
function showGenPopup(message) {
let el = document.getElementById('gen-popup');
if (!el) {
@@ -310,7 +507,7 @@ function pollGeneration(uttId, index, filename) {
let elapsed = 0;
const checkInterval = setInterval(() => {
elapsed += 2000;
fetch('/chapter-audio-file/{{ chapter_dir }}/' + filename, { method: 'HEAD' })
fetch('/chapter-audio-file/{{ active_project(request).slug }}/{{ chapter_dir }}/' + filename, { method: 'HEAD' })
.then(r => {
if (r.ok) {
clearInterval(checkInterval);
@@ -370,10 +567,14 @@ function addUtterance() {
const status = document.getElementById('add-utterance-status');
status.textContent = 'Adding...';
status.style.color = '#94a3b8';
const idxInput = document.getElementById('new-utterance-index');
const payload = {text: text, speaker_name: speaker};
const idxVal = idxInput.value.trim();
if (idxVal !== '') { payload.index = parseInt(idxVal, 10); }
fetch('/api/chapter/{{ chapter_id }}/utterance', {
method: 'POST',
headers: {'Content-Type': 'application/json'},
body: JSON.stringify({text: text, speaker_name: speaker})
body: JSON.stringify(payload)
}).then(r => r.json()).then(data => {
if (data.ok) {
status.textContent = 'Added! Reloading...';
@@ -386,6 +587,17 @@ function addUtterance() {
});
}
function moveUtterance(uttId, up) {
fetch('/api/utterance/' + uttId + '/move', {
method: 'PUT',
headers: {'Content-Type': 'application/json'},
body: JSON.stringify({direction: up === 1 ? 'up' : 'down'})
}).then(r => r.json()).then(data => {
if (data.ok) { location.reload(); }
else { alert('Move failed: ' + (data.error || 'unknown')); }
}).catch(() => alert('Move failed'));
}
{% if chapter_id|string == '1' %}
function savePersonalization() {
const text = document.getElementById('personalization-text').value;

249
umbraaudio/web/templates/index.html
View File

@@ -62,6 +62,49 @@
{{ model_management(qwen_health=qwen_health, lm_status=lm_status, lm_models=lm_models) }}
<div class="card" id="pipeline-activity-section">
<div style="display:flex; justify-content:space-between; align-items:center; margin-bottom:0.75rem">
<div>
<h3 style="margin-top:0">Pipeline Activity</h3>
<p style="color:#94a3b8; font-size:0.85rem; margin:0.25rem 0 0 0">
Live status of background tasks — chapter builds, speaker creation, audio generation.
</p>
</div>
<button class="btn btn-sm" onclick="updatePipelineActivity()">Update</button>
</div>
<div id="pipeline-activity">
{% if pipeline_activity and pipeline_activity.idle %}
<div style="color:#94a3b8; font-size:0.9rem; padding:0.5rem 0">
<span class="status-indicator gray"></span> Idle — no background tasks running
</div>
{% elif pipeline_activity %}
{% for task in pipeline_activity.tasks if task.state != 'idle' %}
<div style="border:1px solid #334155; border-radius:8px; padding:0.75rem 1rem; margin-bottom:0.5rem; background:rgba(51,65,85,0.2)">
<div style="display:flex; justify-content:space-between; align-items:center; gap:1rem">
<div style="flex:1">
<div style="display:flex; align-items:center; gap:0.5rem; flex-wrap:wrap">
<span style="font-weight:600; color:#e2e8f0; font-size:0.9rem">{{ task.name }}</span>
<span class="badge" style="background:{% if task.state == 'active' %}#7c3aed{% elif task.state == 'error' %}#ef4444{% else %}#059669{% endif %}; color:#f8fafc; font-size:0.7rem">{{ task.state }}</span>
</div>
<div style="color:#94a3b8; font-size:0.8rem; margin-top:0.35rem">{{ task.detail }}</div>
{% if task.progress > 0 and task.state == 'active' %}
<div class="progress-bar" style="margin-top:0.5rem; height:4px">
<div class="progress-fill" style="width:{{ task.progress }}%"></div>
</div>
{% endif %}
</div>
{% if task.progress > 0 %}
<span style="color:#94a3b8; font-size:0.8rem">{{ task.progress }}%</span>
{% endif %}
</div>
</div>
{% endfor %}
{% else %}
<div style="color:#94a3b8; font-size:0.9rem; padding:0.5rem 0">Loading…</div>
{% endif %}
</div>
</div>
{% if chapter_count > 0 %}
<div class="card" style="border: 1px solid #7c3aed; background: linear-gradient(135deg, #1e1b4b, #0f172a)">
<div style="display:flex; justify-content:space-between; align-items:center">
@@ -420,6 +463,10 @@
<span class="badge" style="background:#ef4444; color:#fff">Failed</span>
{% elif exp and exp.state.value == 'completed' %}
<span class="badge" style="background:#10b981; color:#fff">Complete</span>
{% if m4b_download %}
<a href="/api/export/download/{{ m4b_download }}" class="btn btn-generate"
style="font-size:0.8rem; text-decoration:none" download>Download .m4b</a>
{% endif %}
{% elif exp and exp.state.value == 'running' %}
<span class="badge" style="background:#38bdf8; color:#fff">Building</span>
{% endif %}
@@ -459,6 +506,98 @@
</div>
<div id="m4b-warn-err" style="display:none; margin-top:0.75rem"></div>
{# --- Step 6c: Build Author's Republic MP3 package --- #}
<hr style="border-color:#1e293b; margin:1.25rem 0">
<h4 style="color:#e2e8f0; margin-bottom:0.4rem">Build Final Audiobook (MP3)</h4>
<p style="color:#94a3b8; font-size:0.85rem; margin-bottom:0.75rem">
Author's Republic-ready MP3 package, one file per chapter you edit:
sample (no number), opening intro <b>i</b>, closing <b>ii</b>, and
story chapters numbered 1&ndash;27 from Salvage. Files are 44.1&nbsp;kHz
mono, 192&nbsp;kbit CBR with 2&nbsp;s silence at each end. Uses only
your chapter audio &mdash; no narration is synthesized.
</p>
{% set aard = ar_status or {'state': 'idle', 'detail': '', 'progress': 0, 'error': ''} %}
<div style="display:flex; gap:0.5rem; align-items:center; margin-bottom:0.5rem">
<button class="btn btn-generate" id="mp3-export-btn"
onclick="exportMp3()"
{% if aard.state == 'running' %}disabled{% endif %}>
{% if aard.state == 'running' %}Building...{% else %}{% if ar_manifest %}Rebuild{% else %}Build{% endif %} MP3 Package{% endif %}
</button>
{% if aard.state == 'failed' %}
<span class="badge" style="background:#ef4444; color:#fff">Failed</span>
{% elif aard.state == 'completed' %}
<span class="badge" style="background:#10b981; color:#fff">Complete</span>
{% elif aard.state == 'running' %}
<span class="badge" style="background:#38bdf8; color:#fff">Building</span>
{% elif ar_manifest %}
<span class="badge" style="background:#10b981; color:#fff">Built</span>
{% endif %}
</div>
{% if aard.state == 'running' %}
<div style="margin-top:0.5rem">
<div class="progress-bar">
<div class="progress-fill" id="mp3-progress-fill" style="width:{{ aard.progress }}%"></div>
</div>
<p style="color:#94a3b8; font-size:0.8rem; margin-top:0.5rem" id="mp3-status-text">{{ aard.detail }}</p>
</div>
<script>pollMp3Status();</script>
{% elif ar_manifest %}
<p style="color:#94a3b8; font-size:0.8rem; margin-top:0.5rem">
{{ ar_manifest.track_count }} tracks &middot;
{{ "%.1f"|format(ar_manifest.total_duration_s / 60) }} min audio &middot;
built {{ ar_manifest.built_at }}
{% if ar_manifest.warnings %}&middot; <span style="color:#eab308">{{ ar_manifest.warnings|length }} warning(s)</span>{% endif %}
</p>
<div style="overflow-x:auto; margin-top:0.5rem">
<table style="width:100%; border-collapse:collapse; font-size:0.78rem">
<thead>
<tr style="color:#94a3b8; text-align:left">
<th style="padding:0.3rem 0.5rem">#</th>
<th style="padding:0.3rem 0.5rem">Role</th>
<th style="padding:0.3rem 0.5rem">Title / File</th>
<th style="padding:0.3rem 0.5rem; text-align:right">Min</th>
<th style="padding:0.3rem 0.5rem; text-align:right">MB</th>
<th style="padding:0.3rem 0.5rem; text-align:right">RMS dB</th>
<th style="padding:0.3rem 0.5rem; text-align:right">Peak dB</th>
<th style="padding:0.3rem 0.5rem">Status</th>
<th style="padding:0.3rem 0.5rem"></th>
</tr>
</thead>
<tbody>
{% for t in ar_manifest.tracks %}
<tr style="border-top:1px solid #1e293b; color:#e2e8f0">
<td style="padding:0.3rem 0.5rem">{{ t.number or "&ndash;" }}</td>
<td style="padding:0.3rem 0.5rem; color:#94a3b8">{{ t.role }}</td>
<td style="padding:0.3rem 0.5rem">
{{ t.name }}
<span style="color:#64748b">({{ t.source }})</span>
</td>
<td style="padding:0.3rem 0.5rem; text-align:right">{{ "%.1f"|format(t.duration_s / 60) }}</td>
<td style="padding:0.3rem 0.5rem; text-align:right">{{ "%.1f"|format(t.size_bytes / 1048576) }}</td>
<td style="padding:0.3rem 0.5rem; text-align:right">{{ "%.1f"|format(t.rms_db) }}</td>
<td style="padding:0.3rem 0.5rem; text-align:right">{{ "%.1f"|format(t.peak_db) }}</td>
<td style="padding:0.3rem 0.5rem">
{% if t.warning %}<span style="color:#eab308">{{ t.warning[:60] }}</span>
{% else %}<span style="color:#10b981">ok</span>{% endif %}
</td>
<td style="padding:0.3rem 0.5rem">
<a href="/api/export/download/ar_package/{{ t.file }}" class="btn btn-generate"
style="font-size:0.72rem; text-decoration:none" download>MP3</a>
</td>
</tr>
{% endfor %}
</tbody>
</table>
</div>
{% endif %}
<div id="mp3-export-progress" style="display:none; margin-top:0.75rem">
<div class="progress-bar">
<div class="progress-fill" id="mp3-progress-fill-live" style="width:0%"></div>
</div>
<p style="color:#94a3b8; font-size:0.8rem; margin-top:0.5rem" id="mp3-status-live"></p>
</div>
<div id="mp3-warn-err" style="display:none; margin-top:0.75rem"></div>
<div id="batch-activity" style="display:none; margin-top:1rem; padding:0.75rem; border:1px solid #1f2937; border-radius:0.5rem; background:#0b1220">
<div style="display:flex; justify-content:space-between; align-items:center">
<span style="font-weight:600; color:#38bdf8; font-size:0.9rem">Background rebuild in progress</span>
@@ -672,10 +811,10 @@ function runAnalysis() {
.then(function(data) {
if (data.ok) {
if (status) {
status.textContent = 'Discovery started. Redirecting to speakers...';
status.textContent = 'Discovery started — monitoring progress...';
status.style.color = '#10b981';
}
setTimeout(function() { window.location.href = '/speakers'; }, 1500);
pollAnalysisStatus();
} else {
if (btn) { btn.disabled = false; btn.textContent = 'Re-run Discovery'; }
if (status) {
@@ -960,6 +1099,69 @@ function pollM4bStatus() {
}, 2000);
}
var _mp3Poll = null;
function exportMp3() {
const btn = document.getElementById('mp3-export-btn');
if (!confirm('Build the Author\'s Republic MP3 package now?\nNeeds the Qwen TTS server for credits/announcements (~1-2h total).')) return;
const prog = document.getElementById('mp3-export-progress');
const fill = document.getElementById('mp3-progress-fill-live');
const status = document.getElementById('mp3-status-live');
const we = document.getElementById('mp3-warn-err');
if (btn) { btn.disabled = true; btn.textContent = 'Building...'; }
if (prog) prog.style.display = 'block';
if (fill) fill.style.width = '2%';
if (status) { status.textContent = 'Starting...'; status.style.color = '#94a3b8'; }
if (we) we.style.display = 'none';
fetch('/api/audiobook/export-mp3', { method: 'POST' })
.then(r => r.json().catch(() => ({ error: 'bad response' })))
.then(d => {
if (d.error) {
if (status) { status.textContent = d.error; status.style.color = '#ef4444'; }
if (btn) { btn.disabled = false; btn.textContent = 'Build MP3 Package'; }
} else {
pollMp3Status();
}
})
.catch(() => {
if (status) { status.textContent = 'Could not start build'; status.style.color = '#ef4444'; }
if (btn) { btn.disabled = false; btn.textContent = 'Build MP3 Package'; }
});
}
function pollMp3Status() {
if (_mp3Poll) clearInterval(_mp3Poll);
const btn = document.getElementById('mp3-export-btn');
const prog = document.getElementById('mp3-export-progress');
const fillLive = document.getElementById('mp3-progress-fill-live');
const statusLive = document.getElementById('mp3-status-live');
_mp3Poll = setInterval(() => {
fetch('/api/audiobook/ar-status')
.then(r => r.json())
.then(d => {
if (d.state === 'running' || d.state === 'queued') {
if (prog) prog.style.display = 'block';
if (fillLive) fillLive.style.width = (d.progress || 0) + '%';
if (statusLive) { statusLive.textContent = d.detail || 'Building...'; statusLive.style.color = '#94a3b8'; }
} else if (d.state === 'completed') {
clearInterval(_mp3Poll); _mp3Poll = null;
if (prog) prog.style.display = 'block';
if (fillLive) fillLive.style.width = '100%';
if (statusLive) { statusLive.textContent = d.detail || 'Package built'; statusLive.style.color = '#10b981'; }
if (btn) { btn.disabled = false; btn.textContent = 'Rebuild MP3 Package'; }
setTimeout(() => { location.reload(); }, 1500);
} else if (d.state === 'failed') {
clearInterval(_mp3Poll); _mp3Poll = null;
if (prog) prog.style.display = 'block';
if (fillLive) fillLive.style.width = '0%';
if (statusLive) { statusLive.textContent = d.error || d.detail || 'Build failed'; statusLive.style.color = '#ef4444'; }
if (btn) { btn.disabled = false; btn.textContent = 'Build MP3 Package'; }
}
})
.catch(() => { /* transient */ });
}, 2000);
}
function renderWarnErr(d) {
const we = document.getElementById('m4b-warn-err');
if (!we) return;
@@ -1295,6 +1497,49 @@ function pollFullPipelineStatus() {
}, 5000);
}
function updatePipelineActivity() {
const container = document.getElementById('pipeline-activity');
const btn = document.getElementById('pipeline-activity-section').querySelector('button');
if (!container) return;
btn.disabled = true;
btn.textContent = 'Updating...';
fetch('/api/pipeline/activity')
.then(r => r.json())
.then(data => {
if (data.idle) {
container.innerHTML = '<div style="color:#94a3b8; font-size:0.9rem; padding:0.5rem 0">' +
'<span class="status-indicator gray"></span> Idle — no background tasks running</div>';
return;
}
const active = data.tasks.filter(t => t.state !== 'idle');
container.innerHTML = active.map(t => {
let badgeBg = '#059669';
if (t.state === 'active') badgeBg = '#7c3aed';
else if (t.state === 'error') badgeBg = '#ef4444';
const bar = (t.progress > 0 && t.state === 'active') ?
'<div class="progress-bar" style="margin-top:0.5rem; height:4px">' +
'<div class="progress-fill" style="width:' + t.progress + '%"></div></div>' : '';
const pct = t.progress > 0 ? '<span style="color:#94a3b8; font-size:0.8rem">' + t.progress + '%</span>' : '';
return '<div style="border:1px solid #334155; border-radius:8px; padding:0.75rem 1rem; margin-bottom:0.5rem; background:rgba(51,65,85,0.2)">' +
'<div style="display:flex; justify-content:space-between; align-items:center; gap:1rem">' +
'<div style="flex:1">' +
'<div style="display:flex; align-items:center; gap:0.5rem; flex-wrap:wrap">' +
'<span style="font-weight:600; color:#e2e8f0; font-size:0.9rem">' + t.name + '</span>' +
'<span class="badge" style="background:' + badgeBg + '; color:#f8fafc; font-size:0.7rem">' + t.state + '</span>' +
'</div>' +
'<div style="color:#94a3b8; font-size:0.8rem; margin-top:0.35rem">' + (t.detail || '') + '</div>' +
bar + '</div>' + pct + '</div></div>';
}).join('');
})
.catch(e => {
container.innerHTML = '<div style="color:#ef4444; font-size:0.9rem; padding:0.5rem 0">Failed to load activity: ' + e.message + '</div>';
})
.finally(() => {
btn.disabled = false;
btn.textContent = 'Update';
});
}
pollGenStatus();
pollBatchStatus();
</script>

24
umbraaudio/web/templates/m4b_meta.html
View File

@@ -84,6 +84,7 @@
<div style="display:flex; gap:0.5rem; align-items:center">
<button type="submit" class="btn btn-approve" style="font-size:0.95rem">Save Metadata</button>
<button type="button" class="btn btn-generate" onclick="importEpub()" style="font-size:0.95rem">Import from uploaded EPUB</button>
<span id="save-status" style="color:#94a3b8; font-size:0.85rem"></span>
</div>
</form>
@@ -125,6 +126,29 @@ async function uploadCover() {
alert('Upload failed: ' + (result.error || 'unknown'));
}
}
async function importEpub() {
var status = document.getElementById('save-status');
status.textContent = 'Importing metadata + cover from EPUB…';
var resp;
try {
resp = await fetch('/api/m4b-meta/import-epub', { method: 'POST' });
} catch (e) {
status.textContent = 'Import error: ' + e;
return;
}
var result = await resp.json();
if (result.ok) {
var names = Object.keys(result.filled || {});
var msg = names.length
? 'Imported ✓ ' + names.join(', ') + (result.cover ? ' + cover' : '')
: 'Imported ✓ (all fields already filled)';
status.textContent = msg;
} else {
status.textContent = 'Import failed: ' + (result.error || 'unknown');
}
setTimeout(function(){ location.reload(); }, 900);
}
</script>
{% endblock %}

38
umbraaudio/web/templates/speakers.html
View File

@@ -185,9 +185,41 @@ function mergeSpeaker(sourceIdx) {
});
}
function playSpeakerSample(filename) {
const audio = new Audio('/voice-test-audio/' + filename + '?t=' + Date.now());
audio.play();
var activeSpeakerAudio = null;
var activeSpeakerBtn = null;
function toggleSpeakerPlay(idx, filename) {
var audio = document.getElementById('audio-' + idx);
var btn = document.getElementById('play-btn-' + idx);
if (!audio || !btn) return;
if (activeSpeakerAudio && activeSpeakerAudio !== audio) {
activeSpeakerAudio.pause();
activeSpeakerAudio.currentTime = 0;
if (activeSpeakerBtn) activeSpeakerBtn.innerHTML = '▶ Play';
}
if (audio.paused) {
audio.src = '/voice-test-audio/' + filename + '?t=' + Date.now();
audio.currentTime = 0;
audio.onended = function () {
btn.innerHTML = '▶ Play';
if (activeSpeakerAudio === audio) {
activeSpeakerAudio = null;
activeSpeakerBtn = null;
}
};
audio.play().catch(function () {});
btn.innerHTML = '■ Stop';
activeSpeakerAudio = audio;
activeSpeakerBtn = btn;
} else {
audio.pause();
audio.currentTime = 0;
btn.innerHTML = '▶ Play';
activeSpeakerAudio = null;
activeSpeakerBtn = null;
}
}
function previewSpeakerEffects(idx, name) {