Files
UmbraAudio/AGENTS.md
B.D. Murphy 829a03b89f Action the controller alignment proposal: source_unit_key + phase reports
The controller (oldbox) sent a data-model alignment proposal and asked for six
additive changes on this node, the validation protocol, and an honest report.
Six of the seven changes are done; the seventh is declared missing rather than faked.

Stage 0 - reachability verified exactly as documented: health 200, heartbeat 401
(the correct answer - auth is refusing), missing Host 400. Nothing else was sent.

source_unit_key (controller 5.1, the blocking item)
  umbraaudio/source_units.py. sha256("uk1" | chapter_source_text_sha256 |
  occurrence | text_sha256). Nothing positional in the key: "occurrence"
  disambiguates repeated text in document order.
  - 18,181/18,181 units keyed across all four projects, 0 collisions
  - chapter 1 renumbered +100 -> 0 keys changed (the load-bearing proof)
  - backfill 10,015 rows in 0.11s, second run stamps 0 (idempotent)
  - alignment phase 3: 200/200 probe units resolved with used_index: false
  - tolerates the older projects that have no chapters.chapter_index
  Recorded at generation time in chapter_builder, BEFORE the rows are read and
  BEFORE the marker is written, so every path that renders audio also produces
  identity and the clip can name its source unit.

5.7 recorded: utterance_index is KEPT as a display ordinal and made
  non-load-bearing. spec 21 §10.

Blob hashes and effect chain (5.2)
  Markers now carry the full sha256 of the delivered bytes and the effects that
  produced them. Verified the existing order is already correct - write_marker
  runs AFTER apply_effects - and closed the remaining hole: a changed trait list
  changed the hash with no recorded cause. sha256 verification is opt-in, since
  hashing every file on every validation pass would re-read gigabytes.

Vocabularies published (5.4) - umbraaudio/definitions.py, 19 sets, 126 values,
  each value carrying whether it is 'declared' (spec/controller) or 'extracted'
  (read out of code), so a diff shows a difference of meaning, not of wording.

Alignment report tool (6.6) - scripts/alignment_report.py --phase all. Emits the
  controller's envelope with the per-phase evidence shapes, and obeys the two
  rules they set: no evidence means blocked, never pass; a field we cannot produce
  is null PLUS a finding, never omitted. Phase 2 is deliberately 'fail' - the
  three-way store disagreement is real, and a pass there would be a check that
  passed by doing nothing.

Fingerprint discipline (5.5) - the drift detector reported its OWN fingerprint
  change when source_unit_key was added rather than hiding it. Node profile
  updated with the new fingerprints and an honest note that they moved.

Node report extended - per-project source-unit-key coverage, per-voice-asset
  sha256 + object_key, definition_set_version, controller stage reached.

Six gaps found in the proposal (spec 21 §11), including the one they invited plus
five more. The sharpest: "hash at generation" ignores that audio is
post-processed afterwards, and our key is anchored to chapters.source_text - a
second copy of the manuscript - so any change to chapter storage silently
invalidates every key in that chapter.

5.6 deliberately NOT faked: personality has no versioned artifact here, only
  unversioned LLM prose in speakers.json. Reported as phase 6b finding B-03
  instead, because wrapping unversioned inputs in a version field would produce
  artifacts claiming a lineage they do not have.

Answered the controller's two open questions (infra/controller-handoff-2026-10-03.md):
forward-only link for a revealed recurring character, and yes - a new edition may
contradict a frozen state, since this node's correction path IS a regeneration.

Full suite 484 passed / 8 failed - the same 8 pre-existing and unrelated failures.
ruff clean on every file touched. The parallel session's chapter_builder hunk
(helper_url) was filtered out of the commit and left intact in the working tree.
2026-10-02 20:16:06 -07:00

53 KiB
Raw Blame History

UmbraAudio — Development Conventions

Commit Cadence (do not skip)

  • Commit at least once every Sunday — never let a week of work sit uncommitted. On 2026-09-20 a single accidental git checkout -- umbraaudio/web/routes.py destroyed ~2,300 lines of a month's uncommitted work; the file only survived because OpenCode keeps its own editor snapshots, not because of git. Do not rely on that again.
  • End-of-Sunday ritual: git status → stage the code (source, tests, specs, scripts, config) → git commit. Book projects, tmp_out/, logs and the one-off umbraaudio/web/static/samples/ A/B demo are intentionally ignored (see .gitignore); never git add -A without checking, and never leave a dirty umbraaudio/web/routes.py for days.
  • Mechanical nudge: scripts/commit_check.sh exits non-zero (1) when the tree has been dirty longer than a week — it measures the oldest uncommitted edit, ignores submodule working-tree dirt, and prints a reminder block. Options: --days N, --any (warn on any dirt, ignoring age), --quiet, --help. Run it by hand or via the reminder below.
  • Weekly reminder (installed): LaunchAgent com.umbraaudio.commitcheck fires Sunday 18:00 and posts a macOS notification when the tree is dirty. Plist loaded from /Users/worstauthor/Library/LaunchAgents/com.umbraaudio.commitcheck.plist and mirrored to /Volumes/Dock/worstauthor/Library/LaunchAgents/ (login-scanned). It runs /bin/zsh /Users/worstauthor/.umbraaudio/umbra_commit_check.sh → cd project → exec .venv/bin/python scripts/commit_check_notify.py. As with the web server, a shell never opens an external-volume file: the venv python reads the repo, pipes scripts/commit_check.sh to bash -s (so the script itself is never opened on the volume), and shells osascript for the notification. Logs to logs/commit_check.log; exit 1 = dirty (deliberate). Test with launchctl kickstart -k gui/$(id -u)/com.umbraaudio.commitcheck.

System & Deployment Notes

  • macOS 26.6.2, home relocated: user worstauthor's home dir is /Volumes/Dock/worstauthor (dscl + $HOME); /Users/worstauthor still exists as a legacy/boot-volume path and is fine for scratch scripts. launchd scans the relocated home's Library/LaunchAgents.
  • Web UI access: umbraudio.bdmurphy.me is served by a Caddy reverse proxy on ANOTHER machine on the LAN, forwarding to this Mac — so the web server MUST bind 0.0.0.0 (it does, port 8420). LAN IPs: 192.168.123.12 (en0) and 192.168.123.154 (en1). macOS App Firewall is disabled (inbound 8420 open). Any new firewall/user grant must keep 0.0.0.0:8420 open.
  • launchd PATH is bare (/usr/bin:/bin:/usr/sbin:/sbin): anything depending on Homebrew tools (notably ffmpeg -> m4b/MP3 export) fails with [Errno 2] No such file or directory: 'ffmpeg' when run under the web server. umbraweb_login.sh now exports PATH="/opt/homebrew/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin" before exec'ing python — keep that export if editing the launcher. A server started before 2026-09-09 needs a restart to pick it up.
  • macOS 26 external-volume restriction (critical): processes NOT launched from an approved context (everything spawned by launchd/cron — shells like /bin/zsh/cat, system python3) get EPERM: Operation not permitted on any read/write of /Volumes/1TB_APL_Storage. Only the venv python binary (.venv/bin/python) is volume-approved, and only in its own process — verified: under launchd it can import umbraaudio, open()/write project files, but a shell cannot even open a script on that volume (/bin/zsh: can't open input file), and launchd dies with EX_CONFIG 78 if a job's StandardOut/ErrorPath points at the volume. Consequence: auto-start must keep ALL project I/O inside python -c '...' — never let a shell open an external-volume script, never point launchd stdio at the volume.
  • Login auto-start (not boot): LaunchAgent com.umbraaudio.web (RunAtLoad + KeepAlive, crash → auto-restart) runs /bin/zsh /Users/worstauthor/.umbraaudio/umbraweb_login.sh, which cds into the project and exec .venv/bin/python -c "<inline uvicorn>", with python dup2- ing stdout/stderr into logs/web_server.log. Plist lives at /Volumes/Dock/worstauthor/ Library/LaunchAgents/com.umbraaudio.web.plist (login-scanned) AND /Users/worstauthor/ Library/LaunchAgents/com.umbraaudio.web.plist (manual launchctl load). Starts when worstauthor logs in. For headless/pre-login (boot) start: a root LaunchDaemon (sudo launchctl bootstrap system) using the same python-inline pattern is the proven route.
  • Rejected/unused: boot-time LaunchAgent/LaunchDaemon with shell-opened external scripts or external-volume stdio (TCC-blocked, EX_CONFIG/127); cron (same TCC denial); a sudoers.d NOPASSWD-to-worstauthor bridge was trialed and removed — unnecessary once everything stays in the venv python. Test plists/probes were cleaned up.
  • Qwen TTS server (port 8881) is MANUAL start only — the login LaunchAgent starts only the web server. After any reboot the Qwen server is down and speaker "re-voice" silently degrades to Edge-TTS (web log: Qwen3-TTS voice reference failed, falling back to Edge-TTS: Cannot connect to Qwen TTS server at http://localhost:8881), which cannot do raspy/texture and may pick an arbitrary voice (e.g. a male for a female-low request). Restore with nohup .venv/bin/python scripts/qwen_tts_server.py (matches the prover-2821 pattern; LLM and Qwen must not be co-resident on the 16 GB Mac — wait until any LLM attribution batch is done).
  • GPU Qwen3-TTS box (192.168.123.14) + Local Network TCC (critical): macOS 15+'s Local Network privacy blocks launchd-spawned user processes (incl. the web server) from ANY LAN IP — they get EHOSTUNREACH ("No route to host") while GUI/terminal processes reach the LAN fine. Loopback is NOT blocked. Workaround (deployed): a root LaunchDaemon relay (/Library/LaunchDaemons/com.umba.qwen-relay.plist + qwen_relay.py, sudo launchctl bootstrap system ...) forwards 127.0.0.1:18881 -> 192.168.123.14:8881; the web app targets UMBRA_TTS__QWEN_TTS_URL=http://127.0.0.1:18881. Root system daemons DO bypass Local Network TCC (verified LAN_OK 200). Box runs scripts/qwen_tts_server.py with QWEN_DEVICE=cuda on an RTX 3060 (12 GiB), models at /home/bm/umba/models/…, prompt cache at /home/bm/umba/voice_prompts, deps in /home/bm/umba/venv (torch 2.11.0+cu128, accelerate==1.12.0, python-multipart + onnxruntime REQUIRED), started manually via nohup ./gpu_server_env.sh. Box GPU shared with an LM Studio LLM (~6.6 GiB); when TTS runs, the LLM must be unloaded first (curl -X POST http://192.168.123.14:1234/api/v1/models/unload -d '{"instance_id":"qwen/qwen3.5-9b"}') or model swaps OOM and the connection drops ("Server disconnected without sending a response." → Edge fallback). Relay bug history: a socket.create_connection(..., timeout=10) left a 10s recv timeout on upstream — any TTS call

    10s was killed mid-response while /health and /load survived; fixed with up.settimeout(None) after connect. Test plists/probes (com.umba.netcheck, netpost, rootnetcheck) were bootstrapped into launchd and cleaned up; after any of these env changes, restart web via launchctl kickstart -k "gui/$(id -u)/com.umbraaudio.web".

  • Relay correction (2026-09-27) — the two claims above were both wrong in practice; verified:
    1. The settimeout(None) fix was NOT in the deployed file. /Library/LaunchDaemons/qwen_relay.py (mtime Sep 10 06:33) had no settimeout call, so the 10s-mid-response-kill bug was still live. Patched (backup qwen_relay.py.bak.20260927) and the daemon restarted with sudo launchctl kickstart -k system/com.umba.qwen-relay. Re-check the deployed file before assuming a documented fix is present.
    2. The relay was never down. launchctl print system/com.umba.qwen-relay reports state = running and root holds 127.0.0.1:18881 LISTEN. An unprivileged lsof cannot see root processes, which makes the relay look absent; and curl to a relay whose upstream is down returns http_code=000 because the relay accepts then drops. So 18881 -> 000 means "relay up, upstream refused", not "relay down". Always confirm with sudo lsof/sudo pgrep.
    • Actual current failure: the GPU box 192.168.123.14:8881 refuses connections (it answers ping at 0.4 ms but the port is closed — its Qwen server is a manual nohup ./gpu_server_env.sh start and is down). qwen_relay.py has no upstream fallback, so every call is dropped and the app falls through to silent Edge-TTS. Restoring/keeping the relay cannot fix this; the box must be started. The working local alternative is http://127.0.0.1:8881 (loopback is exempt from Local Network TCC, which is why the relay is only needed for the LAN path).

Project Overview

Quality-first multi-pass audiobook generation from English manuscripts. Each pass has quality gates. Quality is the primary concern; generation speed is secondary.

Architecture

  • 5-pass pipeline: Analysis → Voice Design → Mapping → Test → Generate
  • Multi-engine TTS: Edge-TTS (primary for voice variety), Chatterbox (secondary for cloning)
  • LLM-enhanced: google/gemma-4-12b-qat for character discovery and profiling
  • Web review: FastAPI + HTMX at checkpoints for human approval

Voice System

  • Narrator: Dedicated soft-spoken male voice (en-US-GuyNeural:-10), deep, radio speaker style
  • Characters: 59 unique voices from Edge-TTS with pitch shifting
  • Voice Assignment: Stored in .umbra/voice_assignments.json
  • Test Samples: Verification phrase uses the Rainbow Passage (phonetically balanced paragraph covering all English phonemes)

Tech Stack

  • Python 3.11+, asyncio throughout
  • SQLite via aiosqlite for project state
  • FastAPI + HTMX for web review UI
  • Click + Rich for CLI
  • Edge-TTS for voice generation
  • LM Studio (google/gemma-4-e4b) for character discovery and speaker attribution

Commands

  • Lint: ruff check umbraaudio/ scripts/
  • Format: ruff format umbraaudio/ scripts/
  • Type check: mypy umbraaudio/
  • Test: pytest tests/ -v
  • Run: python -m umbraaudio.cli
  • Web: python -c "from pathlib import Path; from umbraaudio.web import create_app; from umbraaudio.config import load_config; import uvicorn; config = load_config(Path('.')); app = create_app(config); uvicorn.run(app, host='0.0.0.0', port=8420)"
  • Mechanical re-segment: python scripts/resegment_all.py --mechanical-only
  • LLM speaker attribution: python scripts/reattribute_all.py --model google/gemma-4-e4b
  • LLM character discovery: 4-pass pipeline via /api/analyze endpoint (see specs/02-analysis.md)

Code Conventions

  • Type hints on all function signatures
  • Docstrings on public classes/methods
  • from __future__ import annotations in every file
  • No comments unless explaining why, not what
  • Use logging module, never print() for debug output
  • Async for all I/O-bound operations
  • Dataclasses or Pydantic models for structured data
  • Paths via pathlib.Path, never string paths

Testing

  • Tests mirror source structure: umbraaudio/parsers/ → tests/test_parsers.py
  • Use pytest fixtures for database, config, mock TTS engine
  • Mock external services (LLM APIs, TTS engines) in tests
  • Integration tests for full pipeline with sample manuscript

File Naming

  • Modules: snake_case.py
  • Classes: PascalCase
  • Functions: snake_case
  • Constants: UPPER_SNAKE_CASE
  • Private: _leading_underscore

Dependencies

  • Check pyproject.toml before adding new dependencies
  • Never add a dependency if stdlib suffices
  • Prefer: stdlib → third-party → custom implementation

Current Session Summary

Session: Umbra system node declaration + cross-machine contract (spec 21)

  • Context: another machine/tab is moving UmbraTrans to the main Umbra system on oldbox (192.168.123.14). Four questions were raised for this machine: (1) align the data model and copy UmbraAudio data into the system, verify, transition later; (2) every machine becomes a node that receives any task, not just audio; (3) how changes are managed across products, with the controller owning requirements; (4) does a central server take over the UI, and how do code updates get deployed/tested.
  • New specs/21-umbra-system-nodes-tasks-and-changes.md — the shared cross-machine document. Four workstreams (WS1 data alignment, WS2 nodes/tasks, WS3 change management, WS4 UI topology), each with what already exists, what is missing, and a proposal. Proposals are drafts for the controller to ratify; nothing is decided here.
  • New infra/umbra-node.json — this machine's declared node profile (node_id: umbraaudio-m4, role worker, products [umbraaudio], services, capability list, resource pools, constraints, project list, lifecycle, schema fingerprints, and an honest "facts_for_the_controller" block). No secrets — ids, names, counts only.
  • New scripts/umbra_node_report.py — read-only node report. Proves what the controller needs to align: git state, listening ports, .env KEY NAMES only (never values), per-project row counts, and a schema fingerprint (sha256 over sorted table.column:type, so ALTER TABLE column reordering does not read as drift). --json for machine consumption; --check compares the declared profile to measured reality and exits 2 on drift. Verified: --check exits 0, ruff clean, --json parses. Deliberately stable-facts-only drift (contract version, project set, schema fingerprints) — git HEAD and live ports are reported but never drift, or every commit would "break" it.
  • Measured on this machine (2026-10-01): 4 projects, 2 distinct schema fingerprints — 0078569a0fa6bc70 (Last Dwarf, DealHunter: current shape) and 367d5df9e546dbba (Treasure Island, Vocals: no name_tokens, missing 9 utterance columns). Confirms the manifest's drift finding is real and machine-checkable. Speaker-count disagreement is now Last Dwarf 93/68/71, DealHunter 61/64/97 (93 after the designation merge; refs up from 94). Full suite re-verified: 461 passed / 8 failed — all 8 pre-existing/parallel (2 test_pronunciation helper_url mock, 6 voice_profile_manager accent).
  • manifest.md is now explicitly a dated measurement, not a live view, with an update log. Every other §1.1 table count re-verified unchanged. This is the single most important fix for cross-machine trust: another machine reading the manifest must know whether a number is current.
  • Key answers proposed in spec 21 (all need ratification):
    • Authority: controller owns identity/tenancy/requirements/definitions/task state/ lineage/release; the node owns its working area and keeps working when the controller is down (production availability outranks synchronization). Migration state is per project: legacy → mirroring → dual_write → authoritative → retired.
    • Nodes: stable node_id, declared node_contract_version, capability advertisement, task/result envelopes with report_back evidence, and refusal as a valid audited outcome. This machine must refuse any task needing LLM + local TTS co-resident.
    • Changes: requirement catalog in the controller; four classes (local_product, cross_product, shared_definition, operational); distributed, then confirmed with evidence, never pushed. Code ships by git tag per product repo; the controller carries the compatibility matrix and the requirement, not the code.
    • UI: hybrid. Controller gets the system portal (node health, task state, requirement conformance, definition versions, releases). The audio review UI stays on this machine: it is artifact-native (WAV players, per-utterance regen, markers) and has no authentication at all, so centralizing it would publish an unauthenticated mutation surface. Deep links are short-lived and signed; the worker footer shows node_id + definition_set_version + last confirmation time.
  • Highest-leverage local work identified (no controller dependency): add a content-addressed source_unit_key recorded at generation time (additive column, filenames untouched) — this is the one blocking identity concept that needs nothing from the controller; and extract the aligned definition set from code into versioned data (manifest.md §14.2). Both make every later migration phase cheaper.
  • No UmbraAudio behaviour changed. New files only, plus the manifest update log and a spec index row.

Session: Designation scanner — capture the ch5 speakers discovery missed

  • Discovery re-run on ch5 with the new over-detection prompt rules (gemma-4-e4b, ~22 tok/s; stopped Qwen TTS first per the no-co-residency rule, restarted it after). It improved — found "The big one", "The one on the left/right", "The Leader", "Girl helping" where it previously found none — but still dropped "The oldest" and "The tallest" from the same six-man crowd. The small model is not deterministic enough to guarantee coverage, so a mechanical backstop was added.
  • New umbraaudio/analysis/designation_scan.py (no LLM) reads the three canonical speech-attribution shapes and harvests every designation a speaker is referred to by: "The oldest says," / "I thought they died off," says the tallest / "No," the old smith said. Post-filters: pronouns, narration verbs (Hallr Grips His Axe And), finite-verb clauses (Treasure Is One), possessives (Hallr's→Hallr, Sheyla's voice→Sheyla), bare articles, indefinite collectives, and tags inside a quotation (someone being talked ABOUT, not speaking). Reads chapter source_text (not utterances) so it runs before attribution.
  • New scripts/merge_designations.py: adds uncovered designations as unidentified speakers with evidence. Never removes/rewrites existing entries, atomic tmp+replace, idempotent (0 on re-run). --dry-run first, always.
  • New scripts/run_discovery.py: runs discovery to a separate output file so a discovery run can never overwrite the curated speakers.json.
  • Result: Last Dwarf cast 70 → 93. 56 designations, 33 already covered, 23 genuinely missing — the ch5 crowd (The Oldest, The Tallest, The Young One, The One With The Biggest Sword, The Biggest One, The Leader, One Girl Helping) plus Manfred (a real missing speaker!), The Sergeant, One Witch, The Soldier, One Rider, One Lieutenant etc. All 23 land at 0.00 confidence / "needs check" so the reviewer triages them. Speakers page now: 93 rows, 31 clear / 62 need checking.
  • Regex lessons worth keeping: sings?|sang matched INSIDE gerunds ("Releasing", "Closing", "Sensing", "Passing") until a leading \b was added before the verb alternation; The Guard captured as just Guard until continuation words allowed capitals; appositives ("The biggest one, a healthy farm boy, says") need \s* not \s+ before the comma. A narration-verb blocklist in _is_designation is far more maintainable than a giant allow-list regex.
  • Commit 4775e3b. 14 new tests. Full suite 461 passed (8 pre-existing/parallel).
  • NEXT (Phase 5): re-run scripts/reattribute_all.py — the utterances currently labelled "Walking oldest"/"scum leader" should re-point to the new clean names now that they exist as profiles. Note "The Big One" vs "The Biggest One" are the same person and need a merge (the designation system flags the pair).

Previous Session: Speaker designation + discovery over-detection (spec 17 §3.1/§3.2)

  • Trigger: reviewer reported obvious speakers missing from Last Dwarf (the six men in ch5 identified only as "the oldest"/"the tallest"/"the young one"/"the one with the biggest sword"; the scum group), which they had to add by hand. And: "I would rather combine speakers than be missing them" + "show all possible speakers with designations for clear and needs to be checked."
  • New umbraaudio/analysis/speaker_confidence.py: deterministic (no LLM) identification confidence + clear/needs_review per speaker. Categorical/blocking defects always need a human: narration verb in the name, (Scene N) label, uncapitalised, name made only of role/comparative words, probable unmerged duplicate, unidentified, zero dialogue. Graded penalties: unknown gender, no traits/description, thin utterance share, high needs_review share, unevidenced aliases.
    • Duplicate detection tuned to avoid FALSE merges: "Lieutenant" vs "Lieutenant Stonesplitter" are NOT duplicates (extra word is a proper name); "Walking oldest" vs "Walking Tallest" are NOT (difflib raised to 0.92). Real pairs found: Guard 1/Guard 2, Rielan/Rieland, The Girl/The Girl helping, Officer/Older Officer.
  • UI (registered as a Jinja global in the clean __init__.py, so routes.py untouched): every speaker row gets ✓ clear / ⚠ needs check; hover a needs-check badge for the machine-readable reasons; collapsible "N speakers need checking" panel at top + header counts. Row template guards {% if designations is undefined %} so a standalone /speakers/partial-row/N still works without 66 extra queries. Page loads in ~60ms. Last Dwarf: 32 clear / 34 need checking — every hand-added speaker is flagged.
  • Discovery prompt (CHARACTER_DISCOVERY_PROMPT.md): new Rule 8 "Every Speaking Line Is a Speaker" (designations are valid names; if two may be the same person still list both + flag unsure — over-detect, because a bad merge is permanent and a duplicate costs one click) and Rule 9 "Name Hygiene" (no narration verbs in names, no "(Scene N)", consistent capitalisation). Rule 6 extended: a designation IS unidentified; never upgrade "the tallest" into a guessed proper name. tests/test_discovery_prompt_rules.py guards all of it.
  • NOTE: the new prompt rules only affect FUTURE discovery runs. Re-running discovery is required for them to change speakers.json; existing entries are untouched meanwhile (the designation panel flags the current mess).
  • Commits 6ef66f6 (confidence + UI), f7094e2 (prompt rules). Full suite 447 passed (8 pre-existing/parallel failures unchanged).

Previous Session: Name-token review UI (spec 17 Phase 3) — 3bc6945, abb9975

  • Phase 3 UI + 2 bug fixes (broken onclick quoting, scanner missing "Witch")

Previous Session: Orphan name-token scanner + resolution (spec 17 Phase 2) — c338370, 4d900d7

  • (scanner, resolution actions, CLI, name_tokens table; 3 bugs found by infra review and fixed)

Session: Orphan name-token scanner + resolution actions (spec 17 Phase 2)

  • Key move: a SEPARATE router module (umbraaudio/web/name_token_routes.py) instead of adding routes to the hot, concurrently-edited routes.py. Registered via a 2-line change to the clean umbraaudio/web/__init__.py (app.include_router(name_token_router)). This keeps the name-token UI fully independent of the parallel session's work.
  • Routes: GET /name-tokens (page), GET /name-tokens/partial (HTMX table body), POST /api/name-tokens/scan, POST /api/name-tokens/{token}/assign|create|dismiss ({token:path} so "The Kid" / "Nodding Tobias" with spaces work), and POST /api/speakers/merge-by-name. Templates: name_tokens.html + _name_tokens_list.html (token table: kind, uses, chapters, evidence, status, inline Assign/New speaker/Dismiss). base.html nav gained "Name Tokens" + "Review".
  • Refactor: added scan_project() + build_alias_map() to name_tokens.py so the CLI and the web scan share one code path (CLI simplified to call them).
  • Fixed a pre-existing 500 on /exception-review (blocking because I linked it in the nav): it used the old Starlette TemplateResponse(name, context) signature, so the context dict was passed as the template name → TypeError: unhashable type: 'dict'. Now TemplateResponse(request, name, context), and the hardcoded "DealHunter" project name replaced with config.project_dir.name.
  • Commit discipline under concurrency (worked): routes.py had 7 diff hunks — 2 mine (name_tokens schema @250, exception_review fix @5960) and 5 the parallel session's (helper_url). Used a Python filter to build a patch with only the non-helper_url hunks and git apply --cached it, so the commit carried only my 20 lines. The parallel session's 5 hunks are still uncommitted and intact in the working tree.
  • Verified live (server restarted, PID healthy on :8420): /name-tokens 200 with 8 rows; scan API persisted 9 tokens into Last Dwarf's DB; assign error paths 400; dismiss round-tripped via HTTP with scan evidence preserved (proves the Phase 2 fix) then reverted; all nav pages (/, /speakers, /name-tokens, /exception-review, /proper-names, /chapters-audio, /quality) return 200. Full suite 424 passed (8 failures are the pre-existing/parallel voice_profile_manager + test_pronunciation ones).

Previous Session: Orphan name-token scanner + resolution (spec 17 Phase 2) — c338370, 4d900d7

  • (scanner, resolution actions, CLI, name_tokens table; 3 bugs found by infra review and fixed)

Session: Orphan name-token scanner + resolution actions (spec 17 Phase 2)

  • Web UI restarted (launchctl kickstart com.umbraaudio.web): new PID healthy on :8420, and its _ensure_utterance_columns migrated the real Last Dwarf DB with the Phase 1 clarity columns. / returns a soft 400 "no active project selected" in hub mode — normal.
  • New umbraaudio/analysis/name_tokens.py: scans only two high-signal speaker-evidence positions — (1) unresolved proper-noun speech tags via attribution_qa._match_speech_tag, (2) un-profiled speaker_name values — NOT a whole-text capitalized sweep (that produced 594 mostly-noise tokens). Real result on Last Dwarf: 9 orphan tokens (8 role speakers "The first/second guard", "The Kid", "Advisor", "Royal Guard", "The voice", "Nodding Tobias" + 1 name-tag "Tobias Teppo" → suggests "Tobias"). Groups book-wide with occurrence count, chapters, sample lines. Fuzzy suggestion = max(word-overlap, difflib, soundex, 4-char common-prefix misspelling). Suppresses personal-name suggestions for role nouns (so "The Kid" no longer suggests "The King"). speaker_name tokens use the RAW stored value so the assign action (exact speaker_name match) actually re-normalizes.
  • New umbraaudio/analysis/orphan_resolution.py: assign-to-speaker (adds alias + alias_evidence, re-normalizes utterances case-insensitively), create-speaker (profile with sample_dialogue), dismiss (records decision only), merge-speakers (folds aliases, idempotent). Speakers written atomically (tmp+replace). All record status in name_tokens.
  • New scripts/scan_name_tokens.py: scan CLI (--write) + one-shot actions (--assign/--create/--dismiss/--merge). --dismiss/--assign/--merge verified end-to-end on COPIES of the Last Dwarf DB/speakers.json (alias + 2 re-normalized utts).
  • DB: name_tokens table (project_id, token, kind, status, target_speaker, occurrences, chapters_json, evidence_json, suggestion, suggestion_score, updated_at; UNIQUE project+token) added to database.py SCHEMA and routes.py SCHEMA_SQL; created on demand by ensure_name_tokens_table(). Migration smoke-tested via Database.connect.
  • CRITICAL — concurrent work in this repo (do NOT clobber): a parallel session is actively editing config.py, pronunciation.py, character_discovery.py, character_profiler.py, llm_attributor.py, llm_segmenter.py, chapter_builder.py, emotion.py, voices/qwen_tts.py, .env, and adding infra/, backup/, voices/lmstudio_gate.py, specs/19-enterprise-control-plane.md, project.db. It introduced TTSConfig.qwen_tts_helper_url and threaded it through pronunciation.py + routes.py; the test mock in tests/test_pronunciation.py was NOT updated, causing 2 failures (SimpleNamespace has no attribute qwen_tts_helper_url) — NOT ours. 6 voice_profile_manager accent failures are also pre-existing.
  • Commit discipline under concurrency: routes.py has BOTH my name_tokens schema hunk AND the parallel session's helper_url edits in the same file. I deliberately did NOT commit routes.py to avoid capturing/colliding with their in-flight work; my hunk stays in the working tree (table is created on demand, so nothing is blocked). database.py was purely mine, so it was committed. Full suite: 420 passed; the only failures are the parallel-session/pre-existing ones above.
  • Committed c338370. Remaining Phase 2→3 work: web UI for the token review panel (assign/create/dismiss/merge buttons on the speakers tab) — intentionally deferred since it would edit the contested routes.py + templates.

Previous Session: Utterance clarity signals (spec 17 Phase 1) — committed 1bbbe5d

  • (see below for full detail)

Session: Utterance clarity signals + review queue (spec 17 Phase 1)

  • Goal: persist per-utterance attribution transparency so the system auto-accepts high-confidence lines and surfaces only a small review queue (~1–5%); data feeds a downstream translation system and must be accurate for books the reviewer didn't write.
  • DB: utterances gains attribution_evidence_kind, attribution_tag_name, attribution_uncertain, needs_review, review_reason; attribution_confidence default → NULL. Idempotent migrations in database.py._migrate, routes.py _ensure_utterance_columns, and a new _ensure_utterance_columns in scripts/reattribute_all.py (that driver uses raw sqlite3 and would otherwise fail on the new UPDATE columns). Migrations smoke-tested against a copy of the real Last Dwarf DB.
  • llm_attributor.py: attribute_speakers accumulates scene evidence across batches (evidence_by_idx) and stamps attribution_evidence_kind + attribution_tag_name on attributed dicts.
  • New umbraaudio/analysis/clarity.py: deterministic, LLM-free post-pass that annotates every dialogue line. extract_mechanical_evidence/evidence_kind_for classify evidence name|role|pronoun|address_only|none (addressee counts only for known names/titles — guards the shared detector's "But where…" false positive). Pronoun resolvability via gender-filtered in-scene candidate count (exactly 1 → auto-accept). Composite needs_review: any QA issue (except long_monologue), non-unique pronoun referent, role tag with no profile entry, none/address_only evidence < 0.75, unresolved pass-1 attribution_uncertain (< 0.9), narrator-left dialogue, confidence < 0.7. review_reason is a "; "-joined readable summary.
  • Drivers persist all 8 fields: reattribute_all.py (runs assess_utterance_clarity per chapter), run_llm_attribution.py (via extended db.update_utterance_speaker), both web attribution paths in routes.py. run_llm_attribution._populate_exceptions now keys off needs_review instead of confidence<0.7 + keyword sniffing and maps reason → ambiguity_type; fixed a latent NameError (pid was not in scope).
  • Measured on Last Dwarf (3918 dialogue, pre-re-run confidences): review queue = 173 (4.42%) — within the 1–5% target. Top signals: none-evidence low-conf (73), pronoun ambiguity (53), narration-speaker mismatch (18), address mismatch (5), role-no-profile (6), narrator-left (3). long_monologue runs (43) deliberately NOT auto-queued (weak 0.5 heuristic; intentional 4+ line speeches are common) — still visible in QA reports.
  • Tests: tests/test_clarity.py (25 tests) green; full suite 380 passed / 6 pre-existing voice_profile_manager accent failures. ruff + mypy clean on clarity.py and deltas; routes.py stays at its 73 pre-existing errors (0 new).
  • Committed 1bbbe5d. Next phase: Phase 2 (orphan-alias scanner + assignment) or a reattribute run on Last Dwarf ch11–49 to refresh the clarity fields on real data (web server restart needed to pick up schema + code; qa_accepted rows skip as today).

Previous Session: Re-attribute + speaker-confidence evaluation (spec 17 written)

  • ch11–49 re-attribute run: 3,192 dialogue utts, 39 chapters, QA 157→65; evaluation + phased plan written to specs/17-speaker-confidence-review.md (5 phases: clarity persistence → orphan-alias scanner → UI → translation export → validation).

Session: Xylia hiss fixed (post-TTS white-noise) + GPU box live

  • Xylia's "hissing at the end" was NOT the TTS: her vocal_traits pile in speakers.json (LLM-discovery dump incl. "raspy"/"gravelly") triggered TRAIT_EFFECTS["raspy"] = noise_inject (uniform white noise @0.04) + pitch_jitter applied POST-TTS to every utterance — most audible as hiss when speech stops. Measured E>6 kHz: ref 18%, her clip 33.8%, raw clone w/o effects 19.4%, after fix 16.6% (tail rms 0.001 = calm stop). Fixed by cleaning Xylia's vocal_traits to ["deep","measured","ancient","highly articulate"] (+ kept distortion_effects: []) — no effect matched; the Qwen ref/ICL already carries the gravel. She was the ONLY speaker whose traits matched effects (audit over all speakers). Existing Xylia utterances across ch24–46 still carry the old noise layer until regenerated; ch1 sample utt_0012 regenerated clean (16.6% HF). Backup: /tmp/speakers.json.bak.

Session: Dramatic utterance variations (emotion/pace/custom instruct)

Goal: The per-utterance Emotion/Pace/Custom-instruct controls in the chapter UI didn't "change from neutral". Root cause: variations were dropped, not weak — every voice with a ref clones via generate_voice_clone, which never received the instruct, and pace was read nowhere.

Root cause chain: _remote_generate (qwen_tts.py) only sent voice_description in the no-ref branch → any speaker with a _ref.wav (narrator + all 50 characters) ignored emotion/ instruct; generate_voice_clone (emb = _clone) and ICL (_icl) in qwen_tts_server.py had no instruct slot; pace never consumed. Model already supports instruct+clone simultaneously via instruct_ids prepended to the talker input (modeling_qwen3_tts.py:2076).

Fixes:

  • scripts/qwen_tts_server.py: _icl + _clone accept voice_description → build instruct_ids via model._tokenize_texts([model._build_instruct_text(s)])[0] and pass through **kwargs (flows via _merge_generate_kwargs → model.generate(instruct_ids=...)).
  • qwen_tts.py _remote_generate: sends voice_description alongside voice_ref/prompt_slug (server keeps speaker identity, instruct bends delivery).
  • generation/chapter.py: 26 emotions (EMOTION_OPTIONS: original 18 + tense/urgent/ horror + drama tiers furious/joyful/terrified/griefstricken/shocked), all presets rewritten as short directorial directives (≤100 char, ruff E501); PACE_OPTIONS slow/moderate/ fast + PACE_INSTRUCT; new compose_utterance_instruct(instruct, emotion, pace) — neutral/ moderate contribute NOTHING so no-variation clones stay natural; _build_utterance_instruct composes base profile + custom + emotion + pace.
  • All three generation sites use the composer and pass instruct through every voice mode: chapter_builder.py (embedding/prompt_slug/ref branches), scripts/generate_all_audio.py, routes.py single-utt /api/utterance/{id}/generate. write_marker now records the instruct (staleness regenerates on variation changes).
  • UI chapter_audio_detail.html: emotion dropdown now lists all 26 options.
  • Verification: A/B on narrator clone (/tmp/var_{neutral,angry,furious_fast,slow}.wav) — angry hotter/louder (mean −20.6 vs −23.5 dB, peak −3.6 vs −6.5), furious+fast ~8% shorter + darker, slow distinct; audible delivery change while voice identity holds. New tests/test_variations.py (14 tests: composer, sync checks, remote payload carries voice_description with ref). Full suite green; ruff clean; mypy only pre-existing env noise.
  • Full regen — COMPLETE: cleared all 4101 *.wav.txt markers (+ ch*_full.wav/mp3 and ch*_build_status.json) because newer JSON markers lack instruct and would have SKIPPED regen. Ran nohup .venv/bin/python scripts/generate_all_audio.py --project DealHunterShip_4.0e_whole-to-audio. Result (09-09 04:18 local): === COMPLETE: 4304 generated, 0 failed, 678.6 min === (11.3 h). All 32 chapters re-stitched fresh; markers now carry instruct (variation edits re-trigger regen).
  • Vocals project (expression showcase) rebuilt: Rainbow Passage × all 26 emotions × 2 voices (Narrator utt0–17, Deal Hunter utt18–35). Cleared its 36 markers and ran ChapterBuilder(config, chapter_id=1).run() in a fresh process (the :8420 web server still held pre-wiring imports). 36/36 generated, quality passed, 6m12s. Every marker now records the composed instruct; delivery measurably distinct (angry +4.9 dB mean vs neutral, romantic 8.7 s vs 6.2 s, whisper soft peak, shock longer/wilder). User confirmed "usable".
  • Transient TTS stall episode (was NOT content): regen and a probe both stalled ~20 min at ch4 utt_0002, blocking _model_lock. Suspected ©/℗ — falsified: direct /v1/tts with raw text (27.7 s), exact batch params + ref_text + phonetics (18.7 s) both fine. It's the known random MPS/orphan-generation class (per-token _local_scalar_dense_mps→MPSStream::synchronize crawl). Recovery per docs: kill -9 qwen server + restart (scripts/qwen_tts_server.py); batch resumed marker-safe at utt_0002 and completed. If a stall recurs: restart the server, don't rebuild the batch.
  • AR package rebuilt + verified (auto-watcher): logs/ar_pre_rebuild_snapshot.json captured the stale package (built_at 2026-09-08T02:29:59Z + 32 MP3 md5s); on regen completion the one-shot watcher /tmp/auto_rebuild_ar.py ran ARPackageExporter(data_dir).build(discover_chapters(data_dir), load_meta(data_dir), force=True). Verified (logs/ar_rebuild_report.json): built_at 2026-09-09T11:21:11Z, 32 tracks, 0 warnings, all 32 tracks changed vs snapshot, loudness_ok/peak_ok all true, ch4 on-demand MP3 spot check RMS −20.5 / peak −3.0. Dashboard shows the new built {{ ar_manifest.built_at }} automatically.
  • Operational notes: web POST /api/audiobook/export-mp3 only guards web-side generation, NOT external batch processes — do not click AR export while a generate_all_audio.py batch runs (ch12+ would be silently dropped while their ch*_full.wav are missing). Watcher-side, completion gate = batch PID gone + === COMPLETE: … 0 failed === in logs/regen_variations.log + all 32 ch{id}_full.wav mtime ≥ regen start; on failure it aborts without rebuilding.

Goal

Fix speaker-attribution errors so the user can review/accept voices with confidence. Reported failures in chapter "Salvage" (ch6): utt35 "We have strict orders…" (after "she asks the lead guard… He replies") was tagged Kabir instead of the lead guard, and utt156–157 "The voice calls again" / "Help me, please." was tagged Deal Hunter instead of the female space-survivor (Kainda).

Root Cause Analysis

  1. No rule for unnamed role tags: narration "the lead guard", "the robot" → LLM guessed a named character present in the scene (Kabir) instead of the role.
  2. Old prompt rule 3e over-applied: any "voice"/comm → Deal Hunter, so a human voice ("Help me, please.") became the ship AI.
  3. Context window too narrow: _build_position_context only looked 8 utterances back → "It sounds female," (13 lines before) fell outside, so a female survivor's voice went to Deal Hunter.
  4. No verification pass: attribution was trust-on-faith; wrong speakers were never caught mechanically.
  5. ModelCoordinator.load_lm was not idempotent: re-entering CHAPTER_ANALYSIS always re-POSTed /v1/models/load → a second copy of a 9 GB model failed to allocate on the 16 GB Mac.

Fixes Implemented (this session)

  • umbraaudio/analysis/llm_attributor.py prompt: new rule 3e (Deal Hunter/AI-1 ONLY when narration explicitly names a computer/AI/comm system; a bare "a voice is heard"/"the voice calls again" over radio is HUMAN — use gender cues like "it sounds female"), 3f (pronoun coreference; never resolve to an absent character or a named char when the antecedent is an unnamed role), 3g (unnamed roles → the role speaker's EXACT primary_name e.g. "Chief of the Guard", else a Title Case role name; NEVER a random named character), 3h (repeated lines — "The voice calls again, 'Help me, please.'" — keep the first occurrence's speaker). Context widened: _extract_scene_narration 12→20 lines; _build_position_context 8→40 utterances back, labelled dialogue, 4200-char / 45-part caps.
  • New umbraaudio/analysis/attribution_qa.py (mechanical cross-check): check_attribution() flags speech_tag_mismatch (conf 0.9 resolved / 0.7 unresolved-name), narrator_dialogue (0.95), role_tag_guessed (0.5). Tag applied only from the IMMEDIATELY preceding narration (never across a dialogue line — prevents shared-tag bleed like "Kainda asks," + reply). Pronoun subjects skipped (coreference is the LLM's job). _names_match() falls back to word-overlap for unresolved titles ("Wakam" vs "Trader Wakam", "Admiral Wake" vs "Admiral Wilbur Wake"). fix_high_confidence() writes back only resolved exact-name mismatches. Role regex capped at 2 role words and rejects trailing conjunctions ("the robot and tells").
  • New scripts/qa_attribution.py CLI: --chapters N, --fix, --type. _build_alias_map includes primary, aliases, and first-word mappings.
  • umbraaudio/voices/model_coordinator.py: load_lm (sync + async) now no-ops when the requested model already has a loaded instance — re-entering CHAPTER_ANALYSIS no longer tries to allocate a second copy.
  • .umbra/speakers.json aliases added: Admiral Wake/The Admiral→Admiral Wilbur Wake, Captain Antlion/Antlion→Captain Wolfgang Antlion, Wakam/Wakam the Trader→Trader Wakam, Vicky→Duchess Victoria, Pirate Leader/The Pirate Leader→Pirate Clan Leader, Niñera/Nini/The Nanny→Nanny (Niñera = the nanny robot, Support Robot 74532). This session: Lead Guard→Chief of the Guard, Lead Trainer→Training Officer, Bill→Worker Robot (Bill), Captain Alec Weston→Captain Weston.

Completed Work

  • Full 35-chapter LLM re-attribute with the new prompt (google/gemma-4-e4b, ~21 tok/s at ~1.3 min/batch; total ~2 h): 1878 dialogue utterances, 0 left as narrator. Chapters 0–4 and 33–35 are front/back matter with no dialogue (skipped as "no utterances" or 0 dialogue).
  • QA + auto-fix loop: book-wide QA fell from 38 → 10 issues. 21 auto-fixes applied (resolved tag mismatches) plus 5 evidence-driven role fixes.
  • Fixed speakers: ch6 utt157→Kainda Walters Foster Murphy ("Help me, please." = female survivor); ch6 utt44/49/52/56/211→Nanny (the "robot"/Niñera who rushes in and explains the attack is the nanny bot); ch8 utt18/33→Trader Wakam; ch9 utt22→Duchess Victoria (tag "Vicky says assertively"); ch16 utt120→Pirate Clan Leader (the "modified voice" on the call); ch17 utt94→Deal Hunter; ch18 utt154→Xu Ming, utt168→Kainda; ch20 utt230/utt345→Warbots / Pirate Clan Leader; ch23 utt63→Deal Hunter; ch24 utt15→Kainda; ch25 utt131/utt224→Kainda, utt133→Warbots; ch30 utt82/115→Admiral Wilbur Wake, utt113/133→Duchess Victoria, utt162/170→Kabir, utt194/241→Captain Wolfgang Antlion; ch32 utt68/87→Captain Wolfgang Antlion.
  • Ghost-speaker sweep (post-review, driven by "Kabir isn't in ch6"): script cross-checked every assigned speaker against all name/alias tokens present in the chapter text (stopwords filtered, alias-aware — Willamina's "Captain" alias counts). Found and fixed:
    • ch6 utt38/40→Nanny (were Kabir; the "service robot … named Niñera" who enters with the suit says these; Kabir appears only once in ch6, in narration as the off-screen brother).
    • ch9 utt2/8/11/13→Training Officer (were Admiral Wilbur Wake; narration: "She is talking to the lead trainer for the young royals" / "the argument with the training officer"; profile: [9]).
    • ch16 utt122/123/124→Pirate Clan Leader (were Admiral Wilbur Wake; the "modified voice" scolding the failing pirate, same voice as utt120).
    • ch24 utt16→Pilot (Freighter) (were Alec; same freighter pilot as utt18/20; Alec is a ch8-only Wakam associate).
    • ch26 utt128/139→Frigate Captain (were Admiral Wilbur Wake; "the pirate captain … yells" on the frigate bridge), ch26 utt135→Frigate Captain (were Willamina; "Seal the bridge" order after the captain sees the video). Profile: [24,26]. ch26 utt194→Kainda Walters Foster Murphy (were Admiral Wilbur Wake; she's the one who made it to 12.5G in the pilot chair — Warbots reply "Yes, Captain").
  • Remaining QA flags are deliberate assignments the checker can't see intent behind (Nanny=robot, Kainda="voice", Warbots, Pirate Clan Leader) — treat as clean.
  • pytest: 181 passed, 3 warnings (17 in tests/test_attribution_qa.py covering tag mismatch, alias resolution, tag isolation, pronouns, word-overlap fallback, conjunction rejection, auto-fix gating). ruff check + mypy clean on attribution_qa.py, qa_attribution.py, model_coordinator.py, tests.
  • Hardware reality (16 GB Mac): gemma-4-12b needs 16.88 GB → cannot fit; qwen3.5-9b fits but generates ~1 tok/s under swap (~15 h/book) → not viable; gemma-4-e4b is the practical best (~21 tok/s). LLM and Qwen TTS must not be resident simultaneously (coordinator enforces).

Flagged for User Review (ambiguous, left as-is)

Loc Line Currently Question
ch16 utt121/125 "The ship landed with a different designation…" / "Yes." Adonai Maul The unnamed pirate subordinate failing the Clan Leader (Adonai Maul is a ch27 female bounty — wrong). No role profile; suggest a new "Pirate Commander"
ch20 utt223 "You are taking out multiple air pressure areas…" Deal Hunter Ship AI warning — likely already correct
ch24 utt108 "We don't know. Checking." Deal Hunter Probably a bridge/crew voice, not Deal Hunter
ch26 utt129 "Captain, it is a real alarm. Warbots just boarded…" Willamina Frigate bridge crew reporting to the pirate captain (not Willamina, she's on Deal Hunter); no crew role profile
ch26 utt155 "Four minutes and fifty seconds." Warbots Who is "the speaker"? A leader/announcer?
ch26 utt158 "Captain, they have taken the bridge…" Willamina Narration: "Status info is radioed to Deal Hunter, who tells Kainda" — could be Deal Hunter the AI, not Willamina

Known Pre-existing Bugs (reported, not fixed)

  • scripts/generate_chapter_audio.py:132 references undefined phonetic (NameError on first utterance; superseded by generate_all_audio.py).
  • start_server.sh launches the Gradio qwen-tts-demo on port 8000 — NOT the API server scripts/qwen_tts_server.py (port 8881) the app actually uses. Use qwen_tts_server.py for restarts.
  • Orphaned Qwen generations: a client that times out does NOT cancel the server-side thread; the model keeps generating under _model_lock for minutes, blocking subsequent requests. Recovery = kill -9 the qwen server PID and restart scripts/qwen_tts_server.py.
  • dictionaryapi.dev free tier rate-limits; residual bad records no longer matter for audio (dictionaryapi records are not applied to speech), would need bulk re-lookup to clean review panel.

Audio-Generation Prep (voice resolution + staleness)

  • umbraaudio/generation/staleness.py: markers are now JSON {"speaker", "text"} instead of plain text. is_current(text, wav, speaker=...) only returns True when BOTH match. Legacy plain-text markers are ALWAYS stale → any existing ch1–7 wavs regenerate with corrected attribution on the next run. All call sites updated (generate_all_audio.py, routes.py ×3), speaker default normalized to "narrator".
  • scripts/generate_all_audio.py: load_voice_references now uses rglob (finds nested refs like human_staff_person/messenger_ref.wav); resolve_reference now does case-insensitive canonical alias lookup, unique first-word match, and longest-substring fallback (was exact/casefold only → silent narrator fallback). 50/50 distinct speakers now resolve to a voice ref (0 fallback).
  • New speakers added (speakers.json + voice_profiles.json + generated refs): Crew Member (ch24 bridge crew, male, clipped urgency; crew_member_ref.wav) and Supervisor (ch10 female production supervisor, stressed/assertive; supervisor_ref.wav). Both generated via scripts/generate_voice_references.py.
  • All 184 pytest tests pass; ruff clean on staleness.py, generate_all_audio.py, tests.
  • Next: run scripts/generate_all_audio.py (nohup) — Qwen TTS server up (port 8881); all 4406 utterances will regenerate since legacy markers are stale (~6.4 s/utt, multi-hour run).

Remaining Edge Cases

Chapter-5 audio verified end-to-end (files, markers, stitch, ASR). A/B samples: /tmp/author_note_before.wav, /tmp/author_note_natural.wav, /tmp/ch5_combined.wav. Optional future tooling: a per-utterance Whisper-based QA transcription pass over ch*_full.wav.

Previous Session (chapter-5 audio fix)

Complete — ch5 audio (markers, staleness, pronunciation, curly punctuation) verified. History below this section documents the model-coordinator session and speaker-attribution completion.

Session: Edge-TTS fallback visibility + GPU Qwen3 box probe

Goal: the user had no way to know when a voice silently fell back to Edge-TTS, wants Qwen3 used almost always, and asked whether a GPU box's Qwen3-TTS (192.168.123.14:1234) could serve generation.

Done (this session):

  • Fallback is no longer silent. VoiceDesigner.generate_voice_reference (voice_designer.py) now retries Qwen up to TTSConfig.max_retries (default 3, backoff retry_delay, tests used 1-2) and on exhausted attempts logs ERROR "FALLING BACK to Edge-TTS" and records it durably in a new per-project ledger {project}/.umbra/voice_engines.json: {speaker: {engine: qwen|edge, reason, at, ref_mtime}}. Ledger written atomically (tmp+replace) next to the voices/ dir.
  • Web UI indication (routes.py _load_speakers + templates): each speaker row shows a red "⚠ Edge-TTS fallback" badge (title = reason) when the ledger says edge AND ref_mtime matches the current ref wav (so stale entries don't flag), else a green "Qwen3-TTS ✓" after a Qwen generation. /speakers/partial also runs a live health check on config.tts.qwen_tts_url (via QwenTTSEngine. check_health) and _speakers_list.html renders an amber banner before generating when Qwen is unreachable ("will fall back to Edge-TTS") or a red N-voices banner when fallbacks exist.
  • Verified: real Qwen gen records engine=qwen; forced bogus-URL run retries 2×, falls back, records engine=edge with connection reason + matching ref_mtime; both project /speakers pages render (200). ruff clean on changed regions (routes.py has 116 pre-existing errors, none from these edits); mypy only pre-existing stubs noise (soundfile/parselmouth/numpy pyi).
  • Note: existing refs predate the feature (no ledger) → no badge until next regen. Regenerate any speaker flagged "edge" after restarting Qwen to get the real Qwen3 voice.

GPU box (192.168.123.14:1234) probe — findings:

  • It is an LM Studio server. /v1/chat/completions is a real endpoint; every audio path (/v1/audio/speech, /tts, /v1/tts, /audio/speech) returns LM Studio's catch-all {"error":"Unexpected endpoint or method. (POST ...)"} → this LM Studio build cannot synthesize audio. Catalog only has the qwen3-tts-tokenizer GGUF (Q8_0, 291 MB) downloaded — the 12 Hz codec/tokenizer component, NOT a usable full TTS model.
  • Our app does NOT speak LM Studio's protocol anyway: QwenTTSEngine (qwen_tts.py) talks the custom FastAPI /v1/tts + /v1/voice-prompt protocol of scripts/qwen_tts_server.py (transformers Qwen3-TTS-12Hz-1.7B-VoiceDesign, QWEN_DEVICE env selects cpu/mps/cuda, default mps).
  • To use the GPU for real: deploy scripts/qwen_tts_server.py on the GPU box (venv with CUDA torch + qwen_tts pip package + the HF Qwen3-TTS-12Hz-1.7B-VoiceDesign model — local copies already exist under models/; if using LM Studio instead, a TTS-capable LM Studio version + full model set would still need a client rewrite, so it's NOT worth it). Then point the Mac at it via TTSConfig.qwen_tts_url (env UMBRA_TTS__QWEN_TTS_URL=http://192.168.123.14:8881). SSH from this Mac to the box is denied (key+password) — needs user-provided access or manual provisioning.
  • Xylia (Last Dwarf): regenerated lower — median F0 141.6 → 111.9 Hz via a pinned voice_description in speakers.json ("lowest baritone register possible, much deeper than any man… light Irish accent"). Pin overrides auto-built description verbatim (voice_profile_manager.py ~619) and is preserved on future regenerations.