UmbraAudio/AGENTS.md

UmbraAudio — Development Conventions

Commit Cadence (do not skip)

• Commit at least once every Sunday — never let a week of work sit uncommitted. On 2026-09-20 a single accidental git checkout -- umbraaudio/web/routes.py destroyed ~2,300 lines of a month's uncommitted work; the file only survived because OpenCode keeps its own editor snapshots, not because of git. Do not rely on that again.
• End-of-Sunday ritual: git status → stage the code (source, tests, specs, scripts, config) → git commit. Book projects, tmp_out/, logs and the one-off umbraaudio/web/static/samples/ A/B demo are intentionally ignored (see .gitignore); never git add -A without checking, and never leave a dirty umbraaudio/web/routes.py for days.
• Mechanical nudge: scripts/commit_check.sh exits non-zero (1) when the tree has been dirty longer than a week — it measures the oldest uncommitted edit, ignores submodule working-tree dirt, and prints a reminder block. Options: --days N, --any (warn on any dirt, ignoring age), --quiet, --help. Run it by hand or via the reminder below.
• Weekly reminder (installed): LaunchAgent com.umbraaudio.commitcheck fires Sunday 18:00 and posts a macOS notification when the tree is dirty. Plist loaded from /Users/worstauthor/Library/LaunchAgents/com.umbraaudio.commitcheck.plist and mirrored to /Volumes/Dock/worstauthor/Library/LaunchAgents/ (login-scanned). It runs /bin/zsh /Users/worstauthor/.umbraaudio/umbra_commit_check.sh → cd project → exec .venv/bin/python scripts/commit_check_notify.py. As with the web server, a shell never opens an external-volume file: the venv python reads the repo, pipes scripts/commit_check.sh to bash -s (so the script itself is never opened on the volume), and shells osascript for the notification. Logs to logs/commit_check.log; exit 1 = dirty (deliberate). Test with launchctl kickstart -k gui/$(id -u)/com.umbraaudio.commitcheck.

System & Deployment Notes

• macOS 26.6.2, home relocated: user worstauthor's home dir is /Volumes/Dock/worstauthor (dscl + $HOME); /Users/worstauthor still exists as a legacy/boot-volume path and is fine for scratch scripts. launchd scans the relocated home's Library/LaunchAgents.
• Web UI access: umbraudio.bdmurphy.me is served by a Caddy reverse proxy on ANOTHER machine on the LAN, forwarding to this Mac — so the web server MUST bind 0.0.0.0 (it does, port 8420). LAN IPs: 192.168.123.12 (en0) and 192.168.123.154 (en1). macOS App Firewall is disabled (inbound 8420 open). Any new firewall/user grant must keep 0.0.0.0:8420 open.
• launchd PATH is bare (/usr/bin:/bin:/usr/sbin:/sbin): anything depending on Homebrew tools (notably ffmpeg -> m4b/MP3 export) fails with [Errno 2] No such file or directory: 'ffmpeg' when run under the web server. umbraweb_login.sh now exports PATH="/opt/homebrew/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin" before exec'ing python — keep that export if editing the launcher. A server started before 2026-09-09 needs a restart to pick it up.
• macOS 26 external-volume restriction (critical): processes NOT launched from an approved context (everything spawned by launchd/cron — shells like /bin/zsh/cat, system python3) get EPERM: Operation not permitted on any read/write of /Volumes/1TB_APL_Storage. Only the venv python binary (.venv/bin/python) is volume-approved, and only in its own process — verified: under launchd it can import umbraaudio, open()/write project files, but a shell cannot even open a script on that volume (/bin/zsh: can't open input file), and launchd dies with EX_CONFIG 78 if a job's StandardOut/ErrorPath points at the volume. Consequence: auto-start must keep ALL project I/O inside python -c '...' — never let a shell open an external-volume script, never point launchd stdio at the volume.
• Login auto-start (not boot): LaunchAgent com.umbraaudio.web (RunAtLoad + KeepAlive, crash → auto-restart) runs /bin/zsh /Users/worstauthor/.umbraaudio/umbraweb_login.sh, which cds into the project and exec .venv/bin/python -c "<inline uvicorn>", with python dup2- ing stdout/stderr into logs/web_server.log. Plist lives at /Volumes/Dock/worstauthor/ Library/LaunchAgents/com.umbraaudio.web.plist (login-scanned) AND /Users/worstauthor/ Library/LaunchAgents/com.umbraaudio.web.plist (manual launchctl load). Starts when worstauthor logs in. For headless/pre-login (boot) start: a root LaunchDaemon (sudo launchctl bootstrap system) using the same python-inline pattern is the proven route.
• Rejected/unused: boot-time LaunchAgent/LaunchDaemon with shell-opened external scripts or external-volume stdio (TCC-blocked, EX_CONFIG/127); cron (same TCC denial); a sudoers.d NOPASSWD-to-worstauthor bridge was trialed and removed — unnecessary once everything stays in the venv python. Test plists/probes were cleaned up.
• Qwen TTS server (port 8881) is MANUAL start only — the login LaunchAgent starts only the web server. After any reboot the Qwen server is down and speaker "re-voice" silently degrades to Edge-TTS (web log: Qwen3-TTS voice reference failed, falling back to Edge-TTS: Cannot connect to Qwen TTS server at http://localhost:8881), which cannot do raspy/texture and may pick an arbitrary voice (e.g. a male for a female-low request). Restore with nohup .venv/bin/python scripts/qwen_tts_server.py (matches the prover-2821 pattern; LLM and Qwen must not be co-resident on the 16 GB Mac — wait until any LLM attribution batch is done).
• GPU Qwen3-TTS box (192.168.123.14) + Local Network TCC (critical): macOS 15+'s Local Network privacy blocks launchd-spawned user processes (incl. the web server) from ANY LAN IP — they get EHOSTUNREACH ("No route to host") while GUI/terminal processes reach the LAN fine. Loopback is NOT blocked. Workaround (deployed): a root LaunchDaemon relay (/Library/LaunchDaemons/com.umba.qwen-relay.plist + qwen_relay.py, sudo launchctl bootstrap system ...) forwards 127.0.0.1:18881 -> 192.168.123.14:8881; the web app targets UMBRA_TTS__QWEN_TTS_URL=http://127.0.0.1:18881. Root system daemons DO bypass Local Network TCC (verified LAN_OK 200). Box runs scripts/qwen_tts_server.py with QWEN_DEVICE=cuda on an RTX 3060 (12 GiB), models at /home/bm/umba/models/…, prompt cache at /home/bm/umba/voice_prompts, deps in /home/bm/umba/venv (torch 2.11.0+cu128, accelerate==1.12.0, python-multipart + onnxruntime REQUIRED), started manually via nohup ./gpu_server_env.sh. Box GPU shared with an LM Studio LLM (~6.6 GiB); when TTS runs, the LLM must be unloaded first (curl -X POST http://192.168.123.14:1234/api/v1/models/unload -d '{"instance_id":"qwen/qwen3.5-9b"}') or model swaps OOM and the connection drops ("Server disconnected without sending a response." → Edge fallback). Relay bug history: a socket.create_connection(..., timeout=10) left a 10s recv timeout on upstream — any TTS call

  10s was killed mid-response while /health and /load survived; fixed with up.settimeout(None) after connect. Test plists/probes (com.umba.netcheck, netpost, rootnetcheck) were bootstrapped into launchd and cleaned up; after any of these env changes, restart web via launchctl kickstart -k "gui/$(id -u)/com.umbraaudio.web".

Project Overview

Quality-first multi-pass audiobook generation from English manuscripts. Each pass has quality gates. Quality is the primary concern; generation speed is secondary.

Architecture

• 5-pass pipeline: Analysis → Voice Design → Mapping → Test → Generate
• Multi-engine TTS: Edge-TTS (primary for voice variety), Chatterbox (secondary for cloning)
• LLM-enhanced: google/gemma-4-12b-qat for character discovery and profiling
• Web review: FastAPI + HTMX at checkpoints for human approval

Voice System

• Narrator: Dedicated soft-spoken male voice (en-US-GuyNeural:-10), deep, radio speaker style
• Characters: 59 unique voices from Edge-TTS with pitch shifting
• Voice Assignment: Stored in .umbra/voice_assignments.json
• Test Samples: Verification phrase uses the Rainbow Passage (phonetically balanced paragraph covering all English phonemes)

Tech Stack

• Python 3.11+, asyncio throughout
• SQLite via aiosqlite for project state
• FastAPI + HTMX for web review UI
• Click + Rich for CLI
• Edge-TTS for voice generation
• LM Studio (google/gemma-4-e4b) for character discovery and speaker attribution

Commands

• Lint: ruff check umbraaudio/ scripts/
• Format: ruff format umbraaudio/ scripts/
• Type check: mypy umbraaudio/
• Test: pytest tests/ -v
• Run: python -m umbraaudio.cli
• Web: python -c "from pathlib import Path; from umbraaudio.web import create_app; from umbraaudio.config import load_config; import uvicorn; config = load_config(Path('.')); app = create_app(config); uvicorn.run(app, host='0.0.0.0', port=8420)"
• Mechanical re-segment: python scripts/resegment_all.py --mechanical-only
• LLM speaker attribution: python scripts/reattribute_all.py --model google/gemma-4-e4b
• LLM character discovery: 4-pass pipeline via /api/analyze endpoint (see specs/02-analysis.md)

Code Conventions

• Type hints on all function signatures
• Docstrings on public classes/methods
• from __future__ import annotations in every file
• No comments unless explaining why, not what
• Use logging module, never print() for debug output
• Async for all I/O-bound operations
• Dataclasses or Pydantic models for structured data
• Paths via pathlib.Path, never string paths

Testing

• Tests mirror source structure: umbraaudio/parsers/ → tests/test_parsers.py
• Use pytest fixtures for database, config, mock TTS engine
• Mock external services (LLM APIs, TTS engines) in tests
• Integration tests for full pipeline with sample manuscript

File Naming

• Modules: snake_case.py
• Classes: PascalCase
• Functions: snake_case
• Constants: UPPER_SNAKE_CASE
• Private: _leading_underscore

Dependencies

• Check pyproject.toml before adding new dependencies
• Never add a dependency if stdlib suffices
• Prefer: stdlib → third-party → custom implementation

Current Session Summary

Session: Xylia hiss fixed (post-TTS white-noise) + GPU box live

• Xylia's "hissing at the end" was NOT the TTS: her vocal_traits pile in speakers.json (LLM-discovery dump incl. "raspy"/"gravelly") triggered TRAIT_EFFECTS["raspy"] = noise_inject (uniform white noise @0.04) + pitch_jitter applied POST-TTS to every utterance — most audible as hiss when speech stops. Measured E>6 kHz: ref 18%, her clip 33.8%, raw clone w/o effects 19.4%, after fix 16.6% (tail rms 0.001 = calm stop). Fixed by cleaning Xylia's vocal_traits to ["deep","measured","ancient","highly articulate"] (+ kept distortion_effects: []) — no effect matched; the Qwen ref/ICL already carries the gravel. She was the ONLY speaker whose traits matched effects (audit over all speakers). Existing Xylia utterances across ch24–46 still carry the old noise layer until regenerated; ch1 sample utt_0012 regenerated clean (16.6% HF). Backup: /tmp/speakers.json.bak.

Session: Dramatic utterance variations (emotion/pace/custom instruct)

Goal: The per-utterance Emotion/Pace/Custom-instruct controls in the chapter UI didn't "change from neutral". Root cause: variations were dropped, not weak — every voice with a ref clones via generate_voice_clone, which never received the instruct, and pace was read nowhere.

Root cause chain: _remote_generate (qwen_tts.py) only sent voice_description in the no-ref branch → any speaker with a _ref.wav (narrator + all 50 characters) ignored emotion/ instruct; generate_voice_clone (emb = _clone) and ICL (_icl) in qwen_tts_server.py had no instruct slot; pace never consumed. Model already supports instruct+clone simultaneously via instruct_ids prepended to the talker input (modeling_qwen3_tts.py:2076).

Fixes:

• scripts/qwen_tts_server.py: _icl + _clone accept voice_description → build instruct_ids via model._tokenize_texts([model._build_instruct_text(s)])[0] and pass through **kwargs (flows via _merge_generate_kwargs → model.generate(instruct_ids=...)).
• qwen_tts.py _remote_generate: sends voice_description alongside voice_ref/prompt_slug (server keeps speaker identity, instruct bends delivery).
• generation/chapter.py: 26 emotions (EMOTION_OPTIONS: original 18 + tense/urgent/ horror + drama tiers furious/joyful/terrified/griefstricken/shocked), all presets rewritten as short directorial directives (≤100 char, ruff E501); PACE_OPTIONS slow/moderate/ fast + PACE_INSTRUCT; new compose_utterance_instruct(instruct, emotion, pace) — neutral/ moderate contribute NOTHING so no-variation clones stay natural; _build_utterance_instruct composes base profile + custom + emotion + pace.
• All three generation sites use the composer and pass instruct through every voice mode: chapter_builder.py (embedding/prompt_slug/ref branches), scripts/generate_all_audio.py, routes.py single-utt /api/utterance/{id}/generate. write_marker now records the instruct (staleness regenerates on variation changes).
• UI chapter_audio_detail.html: emotion dropdown now lists all 26 options.
• Verification: A/B on narrator clone (/tmp/var_{neutral,angry,furious_fast,slow}.wav) — angry hotter/louder (mean −20.6 vs −23.5 dB, peak −3.6 vs −6.5), furious+fast ~8% shorter + darker, slow distinct; audible delivery change while voice identity holds. New tests/test_variations.py (14 tests: composer, sync checks, remote payload carries voice_description with ref). Full suite green; ruff clean; mypy only pre-existing env noise.
• Full regen — COMPLETE: cleared all 4101 *.wav.txt markers (+ ch*_full.wav/mp3 and ch*_build_status.json) because newer JSON markers lack instruct and would have SKIPPED regen. Ran nohup .venv/bin/python scripts/generate_all_audio.py --project DealHunterShip_4.0e_whole-to-audio. Result (09-09 04:18 local): === COMPLETE: 4304 generated, 0 failed, 678.6 min === (11.3 h). All 32 chapters re-stitched fresh; markers now carry instruct (variation edits re-trigger regen).
• Vocals project (expression showcase) rebuilt: Rainbow Passage × all 26 emotions × 2 voices (Narrator utt0–17, Deal Hunter utt18–35). Cleared its 36 markers and ran ChapterBuilder(config, chapter_id=1).run() in a fresh process (the :8420 web server still held pre-wiring imports). 36/36 generated, quality passed, 6m12s. Every marker now records the composed instruct; delivery measurably distinct (angry +4.9 dB mean vs neutral, romantic 8.7 s vs 6.2 s, whisper soft peak, shock longer/wilder). User confirmed "usable".
• Transient TTS stall episode (was NOT content): regen and a probe both stalled ~20 min at ch4 utt_0002, blocking _model_lock. Suspected ©/℗ — falsified: direct /v1/tts with raw text (27.7 s), exact batch params + ref_text + phonetics (18.7 s) both fine. It's the known random MPS/orphan-generation class (per-token _local_scalar_dense_mps→MPSStream::synchronize crawl). Recovery per docs: kill -9 qwen server + restart (scripts/qwen_tts_server.py); batch resumed marker-safe at utt_0002 and completed. If a stall recurs: restart the server, don't rebuild the batch.
• AR package rebuilt + verified (auto-watcher): logs/ar_pre_rebuild_snapshot.json captured the stale package (built_at 2026-09-08T02:29:59Z + 32 MP3 md5s); on regen completion the one-shot watcher /tmp/auto_rebuild_ar.py ran ARPackageExporter(data_dir).build(discover_chapters(data_dir), load_meta(data_dir), force=True). Verified (logs/ar_rebuild_report.json): built_at 2026-09-09T11:21:11Z, 32 tracks, 0 warnings, all 32 tracks changed vs snapshot, loudness_ok/peak_ok all true, ch4 on-demand MP3 spot check RMS −20.5 / peak −3.0. Dashboard shows the new built {{ ar_manifest.built_at }} automatically.
• Operational notes: web POST /api/audiobook/export-mp3 only guards web-side generation, NOT external batch processes — do not click AR export while a generate_all_audio.py batch runs (ch12+ would be silently dropped while their ch*_full.wav are missing). Watcher-side, completion gate = batch PID gone + === COMPLETE: … 0 failed === in logs/regen_variations.log + all 32 ch{id}_full.wav mtime ≥ regen start; on failure it aborts without rebuilding.

Goal

Fix speaker-attribution errors so the user can review/accept voices with confidence. Reported failures in chapter "Salvage" (ch6): utt35 "We have strict orders…" (after "she asks the lead guard… He replies") was tagged Kabir instead of the lead guard, and utt156–157 "The voice calls again" / "Help me, please." was tagged Deal Hunter instead of the female space-survivor (Kainda).

Root Cause Analysis

1. No rule for unnamed role tags: narration "the lead guard", "the robot" → LLM guessed a named character present in the scene (Kabir) instead of the role.
2. Old prompt rule 3e over-applied: any "voice"/comm → Deal Hunter, so a human voice ("Help me, please.") became the ship AI.
3. Context window too narrow: _build_position_context only looked 8 utterances back → "It sounds female," (13 lines before) fell outside, so a female survivor's voice went to Deal Hunter.
4. No verification pass: attribution was trust-on-faith; wrong speakers were never caught mechanically.
5. ModelCoordinator.load_lm was not idempotent: re-entering CHAPTER_ANALYSIS always re-POSTed /v1/models/load → a second copy of a 9 GB model failed to allocate on the 16 GB Mac.

Fixes Implemented (this session)

• umbraaudio/analysis/llm_attributor.py prompt: new rule 3e (Deal Hunter/AI-1 ONLY when narration explicitly names a computer/AI/comm system; a bare "a voice is heard"/"the voice calls again" over radio is HUMAN — use gender cues like "it sounds female"), 3f (pronoun coreference; never resolve to an absent character or a named char when the antecedent is an unnamed role), 3g (unnamed roles → the role speaker's EXACT primary_name e.g. "Chief of the Guard", else a Title Case role name; NEVER a random named character), 3h (repeated lines — "The voice calls again, 'Help me, please.'" — keep the first occurrence's speaker). Context widened: _extract_scene_narration 12→20 lines; _build_position_context 8→40 utterances back, labelled dialogue, 4200-char / 45-part caps.
• New umbraaudio/analysis/attribution_qa.py (mechanical cross-check): check_attribution() flags speech_tag_mismatch (conf 0.9 resolved / 0.7 unresolved-name), narrator_dialogue (0.95), role_tag_guessed (0.5). Tag applied only from the IMMEDIATELY preceding narration (never across a dialogue line — prevents shared-tag bleed like "Kainda asks," + reply). Pronoun subjects skipped (coreference is the LLM's job). _names_match() falls back to word-overlap for unresolved titles ("Wakam" vs "Trader Wakam", "Admiral Wake" vs "Admiral Wilbur Wake"). fix_high_confidence() writes back only resolved exact-name mismatches. Role regex capped at 2 role words and rejects trailing conjunctions ("the robot and tells").
• New scripts/qa_attribution.py CLI: --chapters N, --fix, --type. _build_alias_map includes primary, aliases, and first-word mappings.
• umbraaudio/voices/model_coordinator.py: load_lm (sync + async) now no-ops when the requested model already has a loaded instance — re-entering CHAPTER_ANALYSIS no longer tries to allocate a second copy.
• .umbra/speakers.json aliases added: Admiral Wake/The Admiral→Admiral Wilbur Wake, Captain Antlion/Antlion→Captain Wolfgang Antlion, Wakam/Wakam the Trader→Trader Wakam, Vicky→Duchess Victoria, Pirate Leader/The Pirate Leader→Pirate Clan Leader, Niñera/Nini/The Nanny→Nanny (Niñera = the nanny robot, Support Robot 74532). This session: Lead Guard→Chief of the Guard, Lead Trainer→Training Officer, Bill→Worker Robot (Bill), Captain Alec Weston→Captain Weston.

Completed Work

• Full 35-chapter LLM re-attribute with the new prompt (google/gemma-4-e4b, ~21 tok/s at ~1.3 min/batch; total ~2 h): 1878 dialogue utterances, 0 left as narrator. Chapters 0–4 and 33–35 are front/back matter with no dialogue (skipped as "no utterances" or 0 dialogue).
• QA + auto-fix loop: book-wide QA fell from 38 → 10 issues. 21 auto-fixes applied (resolved tag mismatches) plus 5 evidence-driven role fixes.
• Fixed speakers: ch6 utt157→Kainda Walters Foster Murphy ("Help me, please." = female survivor); ch6 utt44/49/52/56/211→Nanny (the "robot"/Niñera who rushes in and explains the attack is the nanny bot); ch8 utt18/33→Trader Wakam; ch9 utt22→Duchess Victoria (tag "Vicky says assertively"); ch16 utt120→Pirate Clan Leader (the "modified voice" on the call); ch17 utt94→Deal Hunter; ch18 utt154→Xu Ming, utt168→Kainda; ch20 utt230/utt345→Warbots / Pirate Clan Leader; ch23 utt63→Deal Hunter; ch24 utt15→Kainda; ch25 utt131/utt224→Kainda, utt133→Warbots; ch30 utt82/115→Admiral Wilbur Wake, utt113/133→Duchess Victoria, utt162/170→Kabir, utt194/241→Captain Wolfgang Antlion; ch32 utt68/87→Captain Wolfgang Antlion.
• Ghost-speaker sweep (post-review, driven by "Kabir isn't in ch6"): script cross-checked every assigned speaker against all name/alias tokens present in the chapter text (stopwords filtered, alias-aware — Willamina's "Captain" alias counts). Found and fixed:
  • ch6 utt38/40→Nanny (were Kabir; the "service robot … named Niñera" who enters with the suit says these; Kabir appears only once in ch6, in narration as the off-screen brother).
  • ch9 utt2/8/11/13→Training Officer (were Admiral Wilbur Wake; narration: "She is talking to the lead trainer for the young royals" / "the argument with the training officer"; profile: [9]).
  • ch16 utt122/123/124→Pirate Clan Leader (were Admiral Wilbur Wake; the "modified voice" scolding the failing pirate, same voice as utt120).
  • ch24 utt16→Pilot (Freighter) (were Alec; same freighter pilot as utt18/20; Alec is a ch8-only Wakam associate).
  • ch26 utt128/139→Frigate Captain (were Admiral Wilbur Wake; "the pirate captain … yells" on the frigate bridge), ch26 utt135→Frigate Captain (were Willamina; "Seal the bridge" order after the captain sees the video). Profile: [24,26]. ch26 utt194→Kainda Walters Foster Murphy (were Admiral Wilbur Wake; she's the one who made it to 12.5G in the pilot chair — Warbots reply "Yes, Captain").
• Remaining QA flags are deliberate assignments the checker can't see intent behind (Nanny=robot, Kainda="voice", Warbots, Pirate Clan Leader) — treat as clean.
• pytest: 181 passed, 3 warnings (17 in tests/test_attribution_qa.py covering tag mismatch, alias resolution, tag isolation, pronouns, word-overlap fallback, conjunction rejection, auto-fix gating). ruff check + mypy clean on attribution_qa.py, qa_attribution.py, model_coordinator.py, tests.
• Hardware reality (16 GB Mac): gemma-4-12b needs 16.88 GB → cannot fit; qwen3.5-9b fits but generates ~1 tok/s under swap (~15 h/book) → not viable; gemma-4-e4b is the practical best (~21 tok/s). LLM and Qwen TTS must not be resident simultaneously (coordinator enforces).

Flagged for User Review (ambiguous, left as-is)

Loc	Line	Currently	Question
ch16 utt121/125	"The ship landed with a different designation…" / "Yes."	Adonai Maul	The unnamed pirate subordinate failing the Clan Leader (Adonai Maul is a ch27 female bounty — wrong). No role profile; suggest a new "Pirate Commander"
ch20 utt223	"You are taking out multiple air pressure areas…"	Deal Hunter	Ship AI warning — likely already correct
ch24 utt108	"We don't know. Checking."	Deal Hunter	Probably a bridge/crew voice, not Deal Hunter
ch26 utt129	"Captain, it is a real alarm. Warbots just boarded…"	Willamina	Frigate bridge crew reporting to the pirate captain (not Willamina, she's on Deal Hunter); no crew role profile
ch26 utt155	"Four minutes and fifty seconds."	Warbots	Who is "the speaker"? A leader/announcer?
ch26 utt158	"Captain, they have taken the bridge…"	Willamina	Narration: "Status info is radioed to Deal Hunter, who tells Kainda" — could be Deal Hunter the AI, not Willamina

Known Pre-existing Bugs (reported, not fixed)

• scripts/generate_chapter_audio.py:132 references undefined phonetic (NameError on first utterance; superseded by generate_all_audio.py).
• start_server.sh launches the Gradio qwen-tts-demo on port 8000 — NOT the API server scripts/qwen_tts_server.py (port 8881) the app actually uses. Use qwen_tts_server.py for restarts.
• Orphaned Qwen generations: a client that times out does NOT cancel the server-side thread; the model keeps generating under _model_lock for minutes, blocking subsequent requests. Recovery = kill -9 the qwen server PID and restart scripts/qwen_tts_server.py.
• dictionaryapi.dev free tier rate-limits; residual bad records no longer matter for audio (dictionaryapi records are not applied to speech), would need bulk re-lookup to clean review panel.

Audio-Generation Prep (voice resolution + staleness)

• umbraaudio/generation/staleness.py: markers are now JSON {"speaker", "text"} instead of plain text. is_current(text, wav, speaker=...) only returns True when BOTH match. Legacy plain-text markers are ALWAYS stale → any existing ch1–7 wavs regenerate with corrected attribution on the next run. All call sites updated (generate_all_audio.py, routes.py ×3), speaker default normalized to "narrator".
• scripts/generate_all_audio.py: load_voice_references now uses rglob (finds nested refs like human_staff_person/messenger_ref.wav); resolve_reference now does case-insensitive canonical alias lookup, unique first-word match, and longest-substring fallback (was exact/casefold only → silent narrator fallback). 50/50 distinct speakers now resolve to a voice ref (0 fallback).
• New speakers added (speakers.json + voice_profiles.json + generated refs): Crew Member (ch24 bridge crew, male, clipped urgency; crew_member_ref.wav) and Supervisor (ch10 female production supervisor, stressed/assertive; supervisor_ref.wav). Both generated via scripts/generate_voice_references.py.
• All 184 pytest tests pass; ruff clean on staleness.py, generate_all_audio.py, tests.
• Next: run scripts/generate_all_audio.py (nohup) — Qwen TTS server up (port 8881); all 4406 utterances will regenerate since legacy markers are stale (~6.4 s/utt, multi-hour run).

Remaining Edge Cases

Chapter-5 audio verified end-to-end (files, markers, stitch, ASR). A/B samples: /tmp/author_note_before.wav, /tmp/author_note_natural.wav, /tmp/ch5_combined.wav. Optional future tooling: a per-utterance Whisper-based QA transcription pass over ch*_full.wav.

Previous Session (chapter-5 audio fix)

Complete — ch5 audio (markers, staleness, pronunciation, curly punctuation) verified. History below this section documents the model-coordinator session and speaker-attribution completion.

Session: Edge-TTS fallback visibility + GPU Qwen3 box probe

Goal: the user had no way to know when a voice silently fell back to Edge-TTS, wants Qwen3 used almost always, and asked whether a GPU box's Qwen3-TTS (192.168.123.14:1234) could serve generation.

Done (this session):

• Fallback is no longer silent. VoiceDesigner.generate_voice_reference (voice_designer.py) now retries Qwen up to TTSConfig.max_retries (default 3, backoff retry_delay, tests used 1-2) and on exhausted attempts logs ERROR "FALLING BACK to Edge-TTS" and records it durably in a new per-project ledger {project}/.umbra/voice_engines.json: {speaker: {engine: qwen|edge, reason, at, ref_mtime}}. Ledger written atomically (tmp+replace) next to the voices/ dir.
• Web UI indication (routes.py _load_speakers + templates): each speaker row shows a red "⚠ Edge-TTS fallback" badge (title = reason) when the ledger says edge AND ref_mtime matches the current ref wav (so stale entries don't flag), else a green "Qwen3-TTS ✓" after a Qwen generation. /speakers/partial also runs a live health check on config.tts.qwen_tts_url (via QwenTTSEngine. check_health) and _speakers_list.html renders an amber banner before generating when Qwen is unreachable ("will fall back to Edge-TTS") or a red N-voices banner when fallbacks exist.
• Verified: real Qwen gen records engine=qwen; forced bogus-URL run retries 2×, falls back, records engine=edge with connection reason + matching ref_mtime; both project /speakers pages render (200). ruff clean on changed regions (routes.py has 116 pre-existing errors, none from these edits); mypy only pre-existing stubs noise (soundfile/parselmouth/numpy pyi).
• Note: existing refs predate the feature (no ledger) → no badge until next regen. Regenerate any speaker flagged "edge" after restarting Qwen to get the real Qwen3 voice.

GPU box (192.168.123.14:1234) probe — findings:

• It is an LM Studio server. /v1/chat/completions is a real endpoint; every audio path (/v1/audio/speech, /tts, /v1/tts, /audio/speech) returns LM Studio's catch-all {"error":"Unexpected endpoint or method. (POST ...)"} → this LM Studio build cannot synthesize audio. Catalog only has the qwen3-tts-tokenizer GGUF (Q8_0, 291 MB) downloaded — the 12 Hz codec/tokenizer component, NOT a usable full TTS model.
• Our app does NOT speak LM Studio's protocol anyway: QwenTTSEngine (qwen_tts.py) talks the custom FastAPI /v1/tts + /v1/voice-prompt protocol of scripts/qwen_tts_server.py (transformers Qwen3-TTS-12Hz-1.7B-VoiceDesign, QWEN_DEVICE env selects cpu/mps/cuda, default mps).
• To use the GPU for real: deploy scripts/qwen_tts_server.py on the GPU box (venv with CUDA torch + qwen_tts pip package + the HF Qwen3-TTS-12Hz-1.7B-VoiceDesign model — local copies already exist under models/; if using LM Studio instead, a TTS-capable LM Studio version + full model set would still need a client rewrite, so it's NOT worth it). Then point the Mac at it via TTSConfig.qwen_tts_url (env UMBRA_TTS__QWEN_TTS_URL=http://192.168.123.14:8881). SSH from this Mac to the box is denied (key+password) — needs user-provided access or manual provisioning.
• Xylia (Last Dwarf): regenerated lower — median F0 141.6 → 111.9 Hz via a pinned voice_description in speakers.json ("lowest baritone register possible, much deeper than any man… light Irish accent"). Pin overrides auto-built description verbatim (voice_profile_manager.py ~619) and is preserved on future regenerations.