The "Current Session Summary" heading had run into the first session's
heading on one line, so the first session read as part of the header and
never appeared in the outline. Split them, and add this session's entry
above.
A re-delivery could leave a stale row behind for an utterance the new
delivery does not cover: the upsert only touches rows the delivery mentions.
--replace clears the language's draft rows first, so what the reviewer sees
is what the delivery actually says.
Confirmed rows are never deleted. A confirmed translation is an approval, and
the import skips confirmed rows anyway, so the approved text survives a
re-delivery.
2 new tests.
UmbraTrans delivers one row per utterance per target language, in the shape
translations.import_translations already accepts. There was no script to run
that ingest, so the delivery side of the contract existed only as a function.
scripts/import_translations.py runs it: report-only by default, --write to
store, --confirm to store as confirmed rather than draft. It prints the
coverage before and after, so a delivery can be checked without writing it.
Two bugs in the ingest, both found by running a real delivery:
* import_translations folded the target language with lang.normalize, which
does not know script subtags. A delivery labelled "zh-Hans" - which is
what UmbraTrans writes - matched nothing, so every row was dropped and
reported as a problem. It now folds with canonical_code, and an unknown
language is still stored: the text is reviewable even when the render
will be refused.
* coverage and not_voiced_rows compared the stored code to the requested
code as strings, so a row stored as "zh-Hans" read as "nothing delivered"
when the reviewer asked about "zh". The coverage report was wrong in the
direction that makes a reviewer distrust it.
Measured on the real Deal Hunter Chinese delivery: 4,300 rows ingested, the
review UI's resolver then renders Chinese for 12/12 utterances of chapter 1
and 415/416 of chapter 6 - the missing one being the spoken chapter
announcement, which is deliberately not part of the reading edition.
spec 27 records what the shipped samples actually are: single-pass drafts
made against a hand-cleaned paraphrase of the English, matching the
speaker-verified database text in 0 of 12 utterances. The production Chinese
edition on oldbox does use the intent-based process; what it lacked was the
alignment back onto the utterances, which is now implemented in UmbraTrans.
7 new tests. Suite: 890 passed, 8 failed (the same 8 pre-existing failures).
The earlier note blamed git-over-HTTP on Local Network privacy and used a
url.insteadOf workaround. The real finding: HTTP genuinely does not
work for git (per-binary entitlement - curl reaches it), while SSH on
2222 works but the connect is intermittently refused with 'Undefined
error: 0'. The Host mini-eq ssh block now sets BatchMode, ConnectTimeout
and ConnectionAttempts 5, so ssh retries the refused connect; plain
git-over-ssh is 5/5 reliable and needs no GIT_SSH_COMMAND.
Also recorded both pinned Mini-EQ host keys - :22 and :2222 keep separate
keys - and that ssh-keyscan without -p 2222 scans the wrong port.
Git cannot reach Mini-EQ over HTTP on this machine - it fails with 'No
route to host' while curl to the same address returns 200, because the
Local Network entitlement is per-binary. The remote URL is unchanged and
now transparently rides SSH via url.insteadOf. Also recorded that the
remote main can move without the local branch.
The remote had one commit this branch did not (03f8a85), which adds the
Apache-2.0 LICENSE, the NOTICE and SPDX headers to every .py and .sh
file. The merge was clean - the headers sit at the top of each file and
my changes are further down.
The new files this branch added (languages.py, translations.py, the
cluster package, the new tests and scripts) were written after the
merge base, so the licence commit never saw them. They now carry the
same header, so the convention holds across the whole tree.
Nothing else changed. 883 tests passed / 8 failed - the same 8
pre-existing failures.
The UmbraAudio side of the cross-machine contract: this node registers
with the UmbraTrans controller, polls for jobs, claims them under a
lease, and writes artifacts back over NFS.
* `umbraaudio/cluster/` - configuration, PostgreSQL access, node
registration and heartbeat, the job queue, artifact storage and NFS
mount management.
* `scripts/cluster_worker.py` - the CLI: `worker`, `check-cluster`,
`register-node`.
* `scripts/backup_system.sh` / `scripts/verify_backup.sh` - whole-system
backup and integrity verification.
* `.env` - the `UMBRA_CLUSTER__*` settings. `UMBRA_CLUSTER__PG_PASSWORD`
is deliberately empty; the password comes from the environment, not
from a committed file.
`k3s-agent.yaml` is NOT committed: it contains a K3s server join token.
It is now gitignored, and the token should be rotated if that file has
ever left this machine.
ruff clean on the whole package.
The review UI's language dropdown did nothing: every selection kept
rendering the English text. Two independent causes, both fixed.
1. The translations were never being read. The pipeline's
`utterance_translations` table holds one row in Deal Hunter, and that
row stores the *English* text under a `zh-Hans` code. The real
translations live in the on-disk sample files under
`samples/translations/`. `umbraaudio/translations.py` now resolves both
sources and merges them by (chapter_id, utterance_index): the table wins
where it holds a real translation, the sample files fill the gaps. A
candidate whose text is identical to the source is a placeholder, not a
translation, so it is ignored - otherwise the page would show English
under a Chinese heading and claim it was translated.
2. The query that read them was broken. `language_routes.py` selected
`u.index` and `u.speaker`; the columns are `utterance_index` and
`speaker_name`, so every read raised `sqlite3.OperationalError: near
"index": syntax error` and `/language-mockup` returned 500. It also
opened `<project>/project.db` - a 0-byte decoy - instead of
`<project>/.umbra/project.db`, the file that actually holds the data.
Also in this commit:
* The chapter review page honours `?lang=`, remembers the choice in a
cookie (set by a middleware, so it survives on every page), and shows the
selected language's *name* in the "has not been translated yet" message.
* The page now shows the translation, not the source, and the edit box is
bound to the source text so saving cannot overwrite the English with the
Chinese.
* The stray language selector that sat after `</html>` in base.html is
gone; the chapters-list page's dropdown had zero options because the
route never passed the language list.
* The language tag is wired into generation: `utterance.language_code`
(NULL = project source language) reaches `engine.generate()` and the
staleness marker, so a retagged line is re-rendered instead of keeping
English phonology behind an unchanged text. The marker records the
language; a marker written before the tag existed means English, so the
~4,300 existing clips are not invalidated.
* `project.source_language` (spec 31 section 13.1) records the book's own
language, with a migration for existing project DBs.
* A scene-break marker deliberately records no language: it is silence,
identical in every language, and tagging it would claim a phonology the
audio does not have.
Tests: 883 passed / 8 failed - the same 8 pre-existing failures
(2 test_pronunciation, 6 voice_profile_manager accent). 26 new tests
cover the resolver and the generation wiring. ruff introduces no new
findings.
- Add active_language cookie tracking in routes.py
- _active_language_for_template() resolves language cookie, defaults to 'en'
- _languages_for_template() returns supported languages list
- chapter_audio_review() route passes active_language and languages to template
- Add language selector in chapter_audio_detail.html header_extra
- 10 languages from spec 30 SUPPORTED list
- JavaScript handles URL param-based language switching (?lang=de, ?lang=en)
- Persists selection across page reloads
- Label: 'Applies to the current chapter — navigation, buttons and statuses stay English.'
- Add language selector in chapters_audio.html header_extra
- Same dropdown for selecting language across all chapters
- Label: 'Applies to all chapters — navigation, buttons and statuses stay English.'
Closes umbratrans UI gaps for per-language chapter audio review.
OpenCode 2.0.27 withholds the built-in Shell tool regardless of config,
client (desktop app vs TUI) or model -- all three were eliminated by test.
MCP is the one channel demonstrably under user control, so add a stdio
server that restores it.
Stdlib-only and deliberately unfiltered so every line is auditable: a
third-party package with unrestricted command execution is an opaque
trust decision.
Two choices taken from AGENTS.md:
- PATH correction. OpenCode spawns MCP servers that can inherit a bare
PATH exactly as launchd does, which is the documented cause of
ffmpeg failing for m4b/MP3 export. /opt/homebrew/bin is prepended to
the child only.
- background=true detaches into its own session (start_new_session) so
long-running processes -- TTS server, web UI -- survive both the MCP
server exiting and an OpenCode restart, and log to a file rather than
a pipe that would eventually block the writer.
Selftest drives a full handshake and exits non-zero on failure, so a
misbehaving server is caught before an OpenCode restart rather than
after. It found two real bugs during development: a double-encoding
blind spot in an exit-code assertion (tool output is JSON nested in
JSON-RPC, so its quotes are escaped), and an f-string escape that is a
syntax error on Python 3.11, the project's stated floor.
Verified: py_compile clean, ruff clean, selftest 8/8, background mode
detached and logged, exit codes and stdout/stderr separation correct.
Full suite unchanged at 8 failed (pre-existing) / 803 passed.
Same reasoning as the lab renders: the WAVs are copies of files that live in
~/Documents/Audacity4/ and are mirrored at
~/.umbra/library/author_voice_matrix/20261009T191636Z/. Git does not need a
third copy to be safe.
What stays is the part that is not duplicated anywhere else in a form git can
restore: author_v2.pt and author_v1.pt, 62 KB, the actual voice identity.
manifest.json keeps a SHA-256 for each WAV, so any copy can be verified against
what was captured rather than trusted.
RESTORE.md was wrong after this change and is fixed. It listed the WAVs as
files of the backup and told you to `cp BDMurphy_Voice_NOBG_v2.wav` from that
directory, which no longer works. It now states the WAVs are untracked and where
they actually are, and adds the verified matrix->WAV decode as a first-class
restore path, because that is the point: the WAV is derived from the matrix, not
a prerequisite for it.
Note the asymmetry this leaves in place, deliberately. Loss of the matrix means
losing the captured voice, and no WAV in existence could reconstruct the exact
one. Loss of the WAVs costs nothing, because they can be rebuilt from the
matrix. So the durable, unreproducible artifact is the thing that stays tracked.
.gitignore alone does not untrack anything already committed, so the previous
commit stopped future tracking but left all 55 WAVs in the index and in history.
This removes them from the tree so a fresh clone no longer carries 25 MB of
audio.
The files themselves are untouched: moved to tmp_out/author_lab_wav/ and
tmp_out/author_lab_wav_cliff/, which is git-ignored, so the samples are still
listenable on this machine. They are regenerable at any time from the tracked
scripts:
backup/author_voice_lab/scripts/voice_lab.py # the 45-sample sweep
backup/author_voice_lab/scripts/verify_cliff.py # the 10 cliff samples
Nothing load-bearing was removed. Still tracked:
backup/author_voice_matrix/**/*.pt author_v1.pt + author_v2.pt (62 KB)
backup/author_voice_matrix/manifest.json SHA-256 + provenance per file
backup/author_voice_matrix/RESTORE.md
backup/author_voice_lab/FINDINGS.md, *.json, scripts/
The .pt files stay tracked on purpose: they are 62 KB, they are the actual
voice identity, and the matrix is the thing that must survive. The WAVs are
derived from it.
Caveat worth stating: the 25 MB still exists in git history from the earlier
commit, so the repository object store does not shrink until that commit is
garbage-collected. A future push carries them over the wire once; after that
no new clone pays for them. Rewriting published history to purge them was
deliberately not attempted.
The Author voice lab committed 45 sample renders (25 MB) plus the six-file
reference chain. Those are binaries and every one of them is regenerable from
the scripts sitting beside them, so tracking them buys nothing and costs a
git-ls-files full of audio.
Tracked deliberately:
backup/author_voice_lab/FINDINGS.md the measurements and the reasoning
backup/author_voice_lab/*.json results + cliff verification
backup/author_voice_lab/scripts/*.py everything that produces the audio
backup/author_voice_matrix/*.pt the vmatrix itself (62 KB, load-bearing)
backup/author_voice_matrix/manifest.json SHA-256 + provenance per file
backup/author_voice_matrix/RESTORE.md how to put it back
Ignored:
backup/author_voice_lab/wav/ wav_cliff/ 25 MB of renders
backup/author_voice_matrix/**/*.wav the reference chain, 25 MB
Note this only stops future tracking. The 45 WAVs are already in history from
the earlier commit, so they stay in the tree until removed with
`git rm -r --cached backup/author_voice_lab/wav backup/author_voice_lab/wav_cliff`
and committed -- which is deliberately left as a separate, explicit step rather
than folded in here, since it changes what a checkout contains.
The dry run passes since routes_compute.py:1792 was aligned with read_bundle
(separators + ensure_ascii=False), so three tests had to catch up. All three
carried narratives written when the bundle was rejected, and each kept asserting
a reason that had stopped being true — a BLOCKED test is a finding, so a stale
BLOCKED reason is a finding about the wrong world.
T06 re-runs the dry run and compares responses, proving validation is
deterministic. It says plainly that re-import suppressing stored rows is NOT
covered: that path needs rows that persist, and the import still builds
InMemoryAudioImportStore.
T07 reconciles the controller's counts against the bytes sent. 8884 records in,
8883 accounted for, plus the book record it does not count — so a controller
that silently dropped a record kind now fails here instead of passing quietly.
T08 keeps BLOCKED, but for the reason that is actually true: no Postgres
AudioImportStore adapter exists, so even a successful commit would return a
uuid4 project_id while translation_utterances stayed empty. Its text names
InMemoryAudioImportStore on purpose, so wiring an adapter fails a test and
forces the verdict to be re-decided.
Suite: 9 PASS / 0 FAIL / 1 BLOCKED (T08). Tests 30 -> 40.
The controller's ingest endpoint came up between runs and this tool misreported
it three more times, each in the same direction — naming the state it was in
when the check was written rather than the state it was reading.
* POST was probed without a credential, so a live route kept answering 401 and
the verdict stayed AUTH_REQUIRED after auth was configured and T02 passed.
The probe now sends whatever headers exist, because an unauthenticated probe
answers "is there a door", never "does it open".
* The rule for reading a status was wrong twice. First it accepted only
200/201/405 as routed, so a 401 on a live route read as absent. Then it was
widened to "anything but 404", which would have read a CSRF 403 as a route:
the controller checks CSRF in middleware before routing (app.py:246-248) and
exempts only /api/node (app.py:89), so POST /api/compute/audio/import answers
403 whether or not a route exists — and none does. Now: on a safe method, 404
means unrouted and anything else means routed; on an unsafe method a 403 off
an exempt prefix is discarded and recorded as csrf_masked_probe.
* T04 dropped csrf_masked_probe from its FAIL evidence, which is exactly the
case where the next reader needs it. Found by the new tests, not by reading.
* T05 returned BLOCKED unconditionally, so the one test that mattered was the
least willing to run. It now executes a real dry run against --bundle.
dry_run short-circuits before store.commit(), so running it cannot write.
* authenticate() sent X-Node-Key, but require_node parses only "bearer "
(routes_compute.py:477) — a valid key reported as rejected. It also probed
/api/auth/me, which resolves a browser session, so no node credential could
ever pass it. Now Bearer on /api/node/agent, and 403 there counts as
authenticated-but-unscoped, since the scope check runs after resolution.
* _summarise reduced a plain-text body to "<str>", discarding "Internal Server
Error" and every handler message — the detail worth reading.
* Four verdict names, including INGEST_ROUTED_NEEDS_RESTART, kept describing a
restart that had already happened. _readiness now derives the name from
evidence held by this run, and a real dry run outranks every transport
observation: the current verdict is DRY_RUN_REJECTED.
Measured against the controller: migration 0008 applied (27 -> 31 tables, guard
proven to fire, idempotent on re-run), service restarted to PID 392511, endpoint
routed and parsing. The dry run executes and is rejected on content_hash,
because the HTTP path re-serializes records (json.dumps default separators,
ensure_ascii=True) while their own read_bundle() hashes the file bytes — the
same bundle yields d0011ae6 over HTTP and 21ce5687 on disk, and their validator
reports 0 problems on disk. Reported to the controller; bundle hash unchanged.
Also reported: routes_compute.py:1809 constructs InMemoryAudioImportStore,
whose commit() appends to a list, and no PostgreSQL adapter exists — so a
successful commit would return a uuid4() project_id while all four
translation_* tables stayed empty. Read from source, not reproduced.
30 new tests pin the routing rule, the CSRF ordering, the credential header,
and every verdict branch. Full suite 799 passed / 8 failed (pre-existing
voice_profile_manager + test_pronunciation). ruff and mypy clean.
T04 said MISSING_INGEST_ENDPOINT. It was wrong about the codebase and right
about the running system, and both halves were avoidable.
The route is POST /api/node/audio/import, on agent_router mounted at /api/node,
guarded by node.require("dataset:ingest"), backed by audio_import.py and
migration 0008_audio_source_import.sql. I found it by reading app.py's
include_router calls - which I had already grepped earlier in this same session
and which printed `include_router(node_router, prefix="/api/node")` on screen
while I was writing the test that declared the path missing.
Two failures, not one:
* I probed six guessed paths when I held SSH access to the source that names the
router prefixes. A 404 to a guessed path proves only that the guess was wrong.
* I probed with GET only. The route is POST-only, so it answers GET with 404 and
POST with 405. That is why it looked absent for a whole session even to a
careful prober.
Both are fixed. T04 probes GET and POST and treats 405 as routed, because 405
means the path is in the table under another method. The failure case is now
UNCONFIRMED_INGEST_ENDPOINT rather than MISSING, and carries a note saying to
confirm against the route table rather than against its 404s. A new verdict,
INGEST_ROUTED_NEEDS_RESTART, distinguishes "routed but not serving" from
"absent".
Why it is routed but not serving: the process on :8442 started 2026-10-14 20:06.
audio_import.py was written 2026-10-15 11:15 and routes_compute.py 12:35, all
uncommitted. /health has been green throughout, because it probes liveness and
not the route table - so "healthy" was never evidence the new work was live.
Suite is 5 pass / 0 fail / 5 blocked now, verdict INGEST_ROUTED_NEEDS_RESTART.
48 tests pass. Nothing pushed.
The controller will not write a `speak` work function against a description, so
this measures instead of describing. scripts/build_tts_facts.py emits
docs/tts/tts-engine.{md,json} from the running system and the repository at run
time; anything unmeasurable says "unknown" with a reason. The content hash
excludes the timestamp, so it is stable across runs and answers "did anything
change?" rather than "was it run again?". 18 tests enforce that it does not
estimate -- including one that fails if a missing weights file is rendered as
"None", and one that fails if the Edge-voice answer is inverted, which it was, in
the first draft.
Four things that change the controller's plans:
* The fork is ONE commit ahead of upstream and it is cosmetic -- a logo and the
web demo. The contract being written against is not the fork, it is
umbraaudio/scripts/qwen_tts_server.py, where the HTTP surface and the four
generation paths were written.
* Synthesis is not deterministic. temperature 0.9, do_sample true, no seed. A
translation re-voiced because one line changed will not reproduce the other
lines byte-for-byte, so comparing takes needs an embedding measure, not a hash.
* The TTS server has NO authentication and binds 0.0.0.0:8881, and concurrency
is 1 behind a single lock. The generator names the seven auth mechanisms it
searched for so the claim can be checked rather than believed.
* assigned_edge_voice is the fallback engine for creating reference audio, not
the engine that renders utterances.
Not measured, and said so: working set / VRAM (the one that decides the 48 GB
CPU node question -- 9.5 GB of weights on disk is not the memory the process
holds, and on CPU that gap is the whole question); whether a seeded call is
reproducible; per-language quality; the model's own maximum text length; CUDA
failure modes.
scripts/build_scale_totals.py measures across all projects rather than handing
over a typed number: 4 projects, 131 chapters, 18,181 utterances, 14,526 audio
files, 6.98 GB, 38.58 hours, and 2 distinct schema fingerprints -- matching the
two already recorded in manifest.md. Markers are counted separately from audio,
because "files" that means both is a number nobody can use. Treasure Island
contributes 49 chapters and zero audio: that is a fact about it, not a failed
measurement.
Also delivered to the controller: their accepted-format validator accepts the
bundle, exit 0, 8884 records, and I confirmed it fails on a broken copy before
believing the pass.
The UmbraTrans controller asked (commit 70c23fd) for machine-readable metadata
about the audio system, in a specified layout, with a validator that ships with
the request so the sender can check its own work. That request is answered here.
scripts/build_controller_bundle.py emits that bundle: one real book (Deal
Hunter), 8,884 records, no media, no credentials, no hostnames. It passes the
controller's own validator, exit 0.
Three things in this system will break a mapping that does not know them, and
the bundle's real purpose is to say so precisely rather than to look tidy:
* A chapter has THREE numbers. The database row id is what filenames use,
chapter_index is reading order, and the distribution track number is a third
thing again. Only the row id survives an edit. Front and back matter are
numbered with roman numerals in the MP3 filenames.
* The narrator is written three ways, and the most common way is nothing at all.
1,803 utterances have a NULL speaker_name, meaning narrator; 614 say
"narrator" and 15 say "Narrator". A join that skips nulls loses most of the
narration silently. Utterances now carry speaker_name_raw, the resolved name,
and speaker_resolution, and the two spellings are not merged - that is a
decision for whoever owns speaker identity, not something an export should do
quietly.
* A .wav file is not always a WAV. 30 of the 91 voice reference files are MPEG.
Every render reports the container measured from the bytes, with
extension_matches_container, so a reader cannot be misled by the name.
A validator proves the bundle is internally consistent; it cannot prove the
bundle is true. scripts/audit_controller_bundle.py re-measures every hash, byte
size, duration and count against the project on disk - 66,011 claims, all
holding. It caught two real defects in the first draft of this bundle, including
one of my own: produced_at was stamped with the build time, which is a claim
about the audio that nothing supported.
The validator was also run against seven deliberately broken bundles to confirm
it can fail. A validator nobody has seen fail is not evidence of anything; those
cases are now tests.
scripts/audio_controller_connect.py runs the T01-T10 suite: 4 pass, 1 fails,
5 are blocked. T04 fails honestly - there is no ingest endpoint, and nothing was
pushed. T05-T08 are blocked because the controller's own plan sequences ingest
after the metadata arrives; a proposed contract is included rather than an
invented endpoint. T02 is blocked: no node key or service token is configured
here, and an operator has to supply one.
Two corrections to the node profile, which the controller reads: it claimed the
web UI has no authentication (a session gate was added 2026-10-09) and that the
controller host is offline (it is reachable and serving umbratrans-web 0.1.0).
Full suite 751 passed, 8 failed - the same pre-existing pronunciation and
accent failures, unchanged. ruff and mypy clean on everything added.
The dropdown was in the HTML all along and still could not be seen. Three
separate causes, found by measuring the rendered element rather than
trusting that the markup existed:
1. Clipped off-screen. The h1 is a flex row with no wrap. Measured at an
800px viewport: the language span started at x=620 and ended at x=810,
past the viewport edge, so only its first few pixels were painted. The
project badge alone took 309px. Adding flex-shrink and truncating the
badge did not fix it - there is genuinely no room in that row. The
selector now gets its own bar directly under the header, and the
project name ellipsises. Verified visible: x=181, right=341 in an
800px viewport, 34px tall.
2. Empty on every other page. base.html rendered the select from a
`languages` variable that almost no page supplies, so the login page
and the rest of the app showed a 38px-wide dropdown with zero options -
effectively invisible, and a dead control. Pages now opt in by
overriding a header_bar block. Verified /, /speakers, /chapters-audio
and /login all still render.
3. Browser cache made my own screenshot lie. It showed a stale Japanese
string containing the word "decisiveness" that is in no file on disk.
A cache-busted fetch confirmed the server is clean. Worth remembering:
a screenshot taken through a warm cache is not evidence.
The screenshot also exposed a data bug. All 12 sample utterances carried
type "narration" - including Kainda, Duchess Victoria, Deal Hunter, Nanny
and Trader Romano, who are all speaking. Every row rendered a narration
badge and the file told UmbraTrans that six characters were not speaking.
SHAPE in build_sample_translations.py now marks 6 dialogue / 6 narration,
and the generated JSONs are regenerated to match.
Also removed a misleading issue. It reported "12 rendered clips have no
staleness marker" as a translation problem; the markers are absent because
render_lang_test.py does not write them, so it was sending the reader after
a defect that does not exist. Replaced with the honest one: only 12 of
4,319 translated and none confirmed - sample only, not publishable.
Suite: 721 passed, 8 pre-existing. ruff clean on changed regions.
Rework per operator feedback that v1 was cluttered.
Language selection moves into the header beside Active project, as a
dropdown, replacing the large dedicated card. It belongs there rather than
on one page because it changes what every page shows. base.html now
carries the selector, guarded by {% if languages is defined %} so no other
page is affected until spec 30 is real. Verified the other pages still
render (dashboard, speakers, chapters, library, lang-samples all 200).
Coverage is now two independent numbers rather than one:
text 12 / 4319 utterances translated 0.28%
audio 12 / 4319 utterances voiced 0.28%
They fail independently: text can arrive before any audio exists, and
audio can be missing because a render failed rather than because the
translation does. One combined percentage hides which of the two is stuck,
which is the question you actually need answered before a release.
Needs attention replaces the old static coverage table. Up to 25 issues
list inline; above that it groups by chapter and links to /chapters-audio
rather than scrolling to unusable length. Verified the grouping branch
directly with 2,592 synthetic issues: 32 chapter rows, worst first, and
the per-chapter counts sum back to the total. Nothing is hidden - the
grouped view still accounts for every issue.
Also fixed a real measurement bug carried in from v1: rendered clips are
named utt<NN>_<Speaker>.wav, not utt<NNNN>.wav, so my original
exists() check matched nothing and reported 0 of 12 voiced. Now matches
by index prefix; the 12/12 figure is read from disk, not assumed.
Dropped the English reference column and the long explanatory prose;
the design notes collapsed into a <details> block so the page reads as a
page rather than a specification.
Suite: 721 passed, 8 pre-existing. ruff clean.
Proposal plus a working read-only mockup at /language-mockup for operator
review. German selected by default, as asked.
Measured first, so the proposal reflects reality:
utterances 4,319 rows, no language column - one text per utterance
language absent from the DB entirely; exists only as a filename
convention in the experimental lang-samples page
/v1/tts language: str = Form("english") - free-form, unvalidated
marker utt_XXXX.wav.txt JSON has speaker/text/voice_fp, no language
The load-bearing finding is that audio MUST be namespaced per language
before any second language is generated. staleness.is_current() compares
the marker's recorded text against the utterance text, so German and
English sharing ch6/utt_0001.wav would each mark the other stale and
overwrite it in a loop. That is a correctness requirement, not a tidy-up.
Schema choice: a new utterance_translations table rather than a language
column on utterances. One row can hold only one language, so a column
either forces 10x duplication of speaker/emotion/pace/attribution or
overwrites the source text. UNIQUE(utterance_id, language_code) makes
UmbraTrans re-delivery an update instead of a second competing German.
English stays in utterances.text and stays editable; translations are
confirm-only, matching the agreed asymmetry.
Audio: chapter_audio/ (English, path untouched, zero regression risk)
gains siblings chapter_audio_de/ etc. Marker JSON gains "language" so an
export can prove what it shipped.
The mockup is a separate router for the reason name_token_routes.py exists
- it keeps proposed work out of the hot routes.py until it is agreed. It
writes nothing, generates nothing, and touches the English pipeline not at
all. Verified: 200 on all 10 languages, real German text rendered,
coverage read from the real 4,319-row DB, unknown codes fall back to
German without a traceback, and the auth gate still 303s to /login.
I initially invented a routes.py helper (resolve_project_dir) that does
not exist; the real one is _get_active_config(request).project_dir. I also
removed a silent cwd fallback that would have rendered English under a
German heading and looked like a working page, and made an unreadable DB
show 'could not measure' rather than a misleading 0%.
Suite: 721 passed, 8 pre-existing. ruff clean on new files.
The Korean render sounded formal because the Korean translation was in
합쇼체 (-입니다/-합니다/-습니다) - the written formal register, like a
news anchor. Every line used it.
Korean politeness is grammatical, not stylistic: it lives in the verb
endings. 합쇼체 -습니다, 해요체 -이에요/-예요, 해체 -아/-어 are different
morphemes, so no TTS instruction can convert them. The fix has to be in
the text. The same applies to Japanese です/ます and to the T/V languages,
where Sie->du changes the words themselves.
This was not a Korean-only problem. All 9 translated languages were in
formal register while the English source is neutral and friendly:
DE 12 markers FR 13 PT 5 ES 4 IT 3 RU 3 JA polite KO 합쇼체
ZH 3 honorific 您 EN 0 (source)
Applied:
KO 합쇼체 -> 해요체. 해체/반말 deliberately NOT used: speech level is
social, and Nanny speaking 반말 to Kainda would be wrong.
DE/FR/ES/IT/PT Sie/vous/usted/Lei/você -> du/tu/tú/tu/tu
RU kept вы; ты is too intimate for narration. Softened instead.
JA polite-but-warm, fewer ください
ZH 您 -> 你
Re-verified against the formal-pronoun probe (not the earlier loose one,
which flagged possessives: su/suo/seu are 'his', and Japanese です is the
copula while います is existential - both register-neutral).
This is Pass 3 from specs/27 (register and voice consistency), which was
specified and then never applied.
Also fixed a drift trap: build_sample_translations.py wrote to
scripts/translations, a directory nothing reads and nothing tracks, while
the tracked record is samples/translations and the renderer reads
<project>/samples/translations. One regeneration silently left both stale.
It now writes the repo copy and, with --project, the renderer's copy in
the same run.
Re-rendered 120 clips, 0 failed. All 120 bitwise-different from the
formal render, confirming the new text was synthesized rather than a
replay. Korean total 117.8s -> 110.1s, consistent with the shorter 해요체
endings. Previous render kept as Lang_test_formalRegister.
Suite: 721 passed, 8 pre-existing.
The per-language narrator references were wrong. Generating a VoiceDesign
voice per language invents a different narrator each time: measured
0.77-0.96 band-energy similarity against the speaker page's own
narrator_ref.wav, so the voice changed with the language. That is the
opposite of using the vmatrix, and it is why the narrator did not sound
the same across languages.
The narrator now uses narrator_ref.wav in every language. The language
parameter supplies phonology; the single reference supplies identity. That
reference is what the vmatrix was built from, so it is the same voice - just
at x_vector resolution rather than the 8k matrix, which is the only form
this machine can run (one prompt_slug request takes the server 5.4 GB ->
19.1 GB and never returns).
Cross-language narrator similarity, same utterance, all 10 languages:
per-language designed refs : mean 0.945, min 0.795
single speaker-page ref : mean 0.973, min 0.910
The previous render is kept in Lang_test_perLangRefs for comparison. 120
clips re-rendered, 0 failed. Note the proxy is a band-energy fingerprint,
not a true speaker embedding, and it cannot judge accent - only timbre
consistency. Suite: 721 passed, 8 pre-existing.
Specification change. Two different rules, and they need different
mechanisms.
Narrator: accent = translation language. Its reference was generated by
VoiceDesign from a description that hardcoded '{accent} English', baking an
American-English timbre into the WAV; cloning that into German gives German
phonology from an American voice. Ten per-language references now exist
(narrator_EN.wav ... narrator_ZH.wav), each designed from the narrator's
timbre plus 'Speaking {language} as a native {language} speaker'.
Characters: keep the accent on the speaker page. Their accents were already
encoded in their existing refs - what destroyed them was the language
parameter forced onto every utterance, which made every voice adopt the
target language's prosody. That is why Deal Hunter came out French-native.
Phonology and accent are now separate controls: language carries phonology,
an explicit voice_description carries accent character. Deal Hunter gets
'with a strong Australian accent' in every language.
Measured A/B, French line from Deal Hunter: with and without the Australian
instruction the waveforms differ by 152% of signal amplitude and duration
moves 4.24s -> 4.08s, so the instruction is reaching the model. Whether it is
convincingly Australian is an ear test, not something measurable here.
Re-rendered all 120 clips under the new spec, 0 failures. The previous
render is kept in Lang_test_preSpec for A/B. 24 speakers carry a non-default
accent; the narrator is deliberately excluded from that set.
Suite: 721 passed, 8 pre-existing.
Every generation branch hardcoded language="english", and the endpoint
accepted no language parameter at all. German, French and the rest were
synthesised with English phonology and prosody - which is why the German
samples sounded like an English narrator reading German.
- /v1/tts now accepts a language form field, passed to all four branches.
Default stays "english" so the existing English pipeline is unchanged;
the renderer sends the target language from the model's own
codec_language_id keys (exactly ten: chinese english french german
italian japanese korean portuguese russian spanish).
- A/B on one German line with the same cloned voice: language=english
gives 4.56s, language=german 4.08s, waveforms differing by 151% of
signal amplitude. Audibly different renditions, not a no-op.
Also documented the ICL memory finding in the server comments: the branch
that consumes a cached prompt leaves x_vector_only_mode at its default of
False, so it decodes the 8k ref_code matrix on every request. One such
request takes the footprint from 5.4GB to 19.1GB and does not return. The
ref_audio branch passes x_vector_only_mode=True and is stable - 120 clips,
0 failures, 13.1GB.
Note the language fix does NOT address that; it was tested and still
blows up with language=german. Separate problem.
qwen_tts_server.py stays at its 8 pre-existing ruff errors. Suite: 721
passed, 8 pre-existing.
The rendered sample chapters existed on disk but had no way to be listened
to through the UI. New /lang-samples page, reachable from a new SLang nav
tab, lists every language chapter with an inline player plus the combined
all-languages file.
- Serves from <project>/Lang_test, which is where render_lang_test.py writes.
Deliberately project_dir and not data_dir: .umbra is gitignored state and
the render output lives beside the rest of the project.
- Languages are discovered from disk in a fixed order rather than hardcoded,
so an unrendered language simply does not appear instead of offering a
player that 404s.
- The audio path validates a two-to-three letter code and builds the
filename server-side, so it cannot be walked out of the sample directory.
Verified: %2e%2e and over-long codes are rejected 400, unauthenticated
requests 303 to login.
- A missing language now 404s rather than falling back to the combined file.
It did before, so an unrendered language would silently play the wrong
audio - the same class of bug as a stale speaker name resolving to no
voice at all.
Verified in a browser: 11 players, durations resolve (EN 92.3s, ZH 92.0s,
combined 1067.8s), no load errors, range requests 206 so seeking works.
routes.py stays at its 76 pre-existing ruff errors. Suite: 721 passed,
8 pre-existing.
Writes _ALL_LANGUAGES_full_sample.wav alongside the per-language chapters,
English first then the operator's list order, so the whole 10-language set
can be heard in one pass (17.8 min for the sample chapter).
Root cause found while rendering with 1.7B: the ICL prompt path is what
blows up memory, not model size or uptime. A single prompt_slug request
added ~14 GB (7.2 -> 21.6 GB) and wedged the server, after which the
otherwise-stable ref_audio path also hung. A fresh 1.7B loads in 8s at
5.4 GB, generates in 9-16s per clip, and settles at ~37 MB per additional
clip (13.1 GB after 120) with no restart needed between languages.
The 0.6B render is kept in Lang_test_0p6B for comparison; the 1.7B render
is in Lang_test.
Suite: 721 passed, 8 pre-existing.
Two bugs that would have failed on all 120 clips:
- The script parsed {"audio_base64": ...}. /v1/tts returns raw WAV bytes
with media_type audio/wav. Now validated with a RIFF check so a bad
response says so instead of raising a JSON error.
- It preferred the cached ICL prompt. Those are MODEL-SPECIFIC: narrator.pt
was built by the 1.7B model and the 0.6B rejects its ref_code with
'Expected size 1024 but got size 2048'. --voice-mode now defaults to ref,
sending ref_audio + ref_text (the Rainbow Passage the refs were generated
from) so the model builds its own prompt; --voice-mode prompt is available
and documented as valid only for the model that built the cache.
Measured on a reboot with swap at 0: the 1.7B model still fails to generate
(footprint 23.4G on a 16G machine, zero POSTs completing). The 0.6B model
loads in 5s and renders 6-21s per clip.
Rendered 120 clips across 10 languages plus a stitched file each: no silent
clips, RMS -22 to -23 dB throughout. Duration ordering is itself a useful
signal - ZH 92s and EN 81s against DE 98s, FR/IT 102-107s.
Suite: 721 passed, 8 pre-existing.
Operator direction: English is the source and stays editable; translations
are confirm-only.
- samples/translations/: Deal Hunter sample chapter 1 in all 10 Qwen3-TTS
languages, named deal_hunter_sample_ch1_<LANG>.json. Single-pass LLM
drafts, labelled as such in every file - they are the failure mode the
multi-pass design exists to fix, and they double as a bad-case corpus for
the Pass 4 critic.
- scripts/build_sample_translations.py, with a validator that refuses to
write on stray Latin residue, duplicated utterances or a wrong count. It
caught three defects in my own draft before it would ship.
- specs/27-translation-pipeline-multipass.md - six passes, the point being
that Pass 4 must be a separate critic call, not the same call that drafted.
- specs/28-ui-language-selection.md - confirm-only translations, English
editable, selector scoped to the chapter page, language named on the page
because the cookie is per browser.
- scripts/render_lang_test.py - renders the translations with each speaker's
own voice into <project>/Lang_test/, plus a stitched file per language.
Dry run resolves all 12 utterances x 10 languages; 11 of 12 use a cached
ICL prompt, Trader Romano falls back to ref_audio.
The renderer resolves display names through the alias map rather than by
munging strings: the translations say 'Kainda' and the speaker record says
'Kainda Walters Foster Murphy'. String-munging silently produced no voice for
Kainda in the first draft - the same failure mode as the speaker-review
audit.
NOT DONE: no audio rendered. Qwen3-TTS generation on this machine is stalled
- /v1/tts accepts the connection and never completes (zero POSTs logged across
17 minutes after a clean restart), while /v1/voice-prompt and /load return 422
immediately, so requests do reach the handler and the block is inside
generation. Swap is at 16GB of 17.4GB. Left as a known blocker rather than
worked around.
Suite: 721 passed, 8 pre-existing.
Survey of cross-lingual voice cloning, as a companion to the speaker-review
audit that gates the translation handoff. Recorded as research, not a
decision.
Terminology correction up front: this is a TTS problem, not an LLM problem.
Reproducing a voice in another language is cross-lingual voice cloning; an
LLM is still needed for the translation itself but plays no part in voice
fidelity.
Headline findings:
- Qwen3-TTS, already running locally, covers 10 of the 15 languages
(en zh ja ko de fr ru pt es it). The gaps are Dutch, Norwegian, Polish,
Arabic and Hindi - so no single model does this job.
- Licence is the deciding constraint, because the books are sold. XTTS v2
covers 14 of 15 and is the obvious capability choice, but its weights are
Coqui Public Model License (non-commercial) and Coqui shut down in Jan 2024,
so no commercial licence can be bought. Fish Audio S2 (83 languages) and
F5-TTS are likewise non-commercial weights.
- Recommendation is a two-engine split: Qwen3-TTS (Apache 2.0, verified from
the licence file) for its 10, Chatterbox Multilingual (MIT) for the 5 gaps.
Together they cover all 15.
The 'voice matrix' question, which the operator raised directly:
voice_registry stores two different things. voice_ref_path is the reference
WAV and is portable to any clone-from-audio engine; vmatrix_path is a
Qwen3-TTS ICL artefact (8k ref_code + speaker embedding, .pt) and is
engine-specific and unreadable by anything else. Deal Hunter already has 91
reference WAVs, so voice fidelity across languages needs no retraining and no
matrix translation - just handing the same WAV to whichever engine covers that
language.
Recorded honestly: the Chatterbox language list came from a third-party API
wrapper, not Resemble AI's own model card, and that build reports Chinese as
disabled. Verification against upstream is listed as open question 1.
Runs after a reviewer finishes a book and answers the two questions that
gate the next stage:
1. Are there utterances with no speaker?
2. Now that speakers have been defined, renamed, merged or deleted, are
there utterances that should change?
Read-only - it never edits the database, and a test asserts the files are
byte-identical afterwards. Output is a markdown table and a CSV, one row per
finding with chapter, utterance number and a text preview, so each item is
locatable rather than a count.
Deal Hunter, after the completed review (56 speakers, 0 still needing
review, 4319 utterances):
- Dialogue with no speaker: 0. The 1803 speakerless rows are narration
and scene breaks, which is correct and is reported as a number rather
than silently dropped.
- Utterances attributed to a deleted speaker: 0.
- One speaker stored under two spellings: 6 rows across 3 speakers
(narrator/Narrator, Kainda/Princess Kainda, Willamina Wake/Captain).
All resolve to one voice so the audio is correct, but a per-speaker
export would split them into six - caught before the translation stage,
not after.
- 6 speech-tag mismatches where the narration immediately before the line
names a different speaker.
- 10 dialogue rows voiced by the narrator: 9 are quoted labels ("Reactor.",
"operations") that are correctly read by the narrator, 1 (ch28 utt46) is
a real segmentation error. Classified advisory so the one real defect is
not buried under nine false positives.
build_alias_map moved from scripts/qa_attribution.py into attribution_qa.py
so the CLI and the audit cannot disagree about what a speech tag resolves to;
qa_attribution output verified byte-identical.
13 new tests. Suite: 721 passed, 8 pre-existing.
The handler read the name back out of the response, but the endpoint returns
the re-rendered list as HTML, so the JSON parse fell back to a placeholder
and the reviewer saw "Deleted speaker." instead of the name they clicked.
The name is already known at the call site - pass it through, and drop the
unused nameOf helper.
Verified in a browser: "Deleted ZZ Browser Delete."
The speakers page has carried a Delete button that called
DELETE /api/speakers/{index}. That endpoint did not exist - the server
answered 405 Method Not Allowed and the button silently did nothing.
Deleting is more than dropping an entry, so the endpoint refuses to orphan
dialogue: a speaker with utterances returns 409 with the count and the list
of possible targets, and the UI asks where that dialogue should go.
reassign_to then moves every utterance recorded under the speaker's name OR
any of its aliases - utterances are frequently stored under an alias - and
keeps the old name as an alias on the target so existing references still
resolve. A speaker with no dialogue is simply removed.
Also removes the speaker's own reference audio (never the shared narrator or
general ref) and its voice_status approval entry, so a speaker re-created
later cannot inherit an approval nobody gave for it.
The button is no longer hx-delete: htmx does not swap a 409, and deleting
renumbers every speaker after it, so the handler re-renders the whole list.
Verified live on Deal Hunter, reversibly: create throwaway -> delete (200,
gone) -> recreate with an utterance attached -> delete (409, count 1, 61
targets offered) -> reassign_to=Warbots (200, utterance restored, name kept
as alias) -> compared against a pre-test backup, speaker count and utterance
count identical.
10 new tests. Suite: 708 passed, 8 pre-existing.
Every item in Deal Hunter's voice_queue.json had failed with
error: "'primary_name'". _speaker_info matched only an exact primary_name,
so any other spelling returned {} and build_profile_from_speaker subscripts
that key directly - the job died before synthesising a single sample and the
reviewer saw only an error tag.
That is a live trap for Save & Re-voice specifically: renaming a speaker and
then re-voicing it leaves the queued name matching nothing.
- Case-insensitive, whitespace-tolerant match, then an alias pass.
- An unknown name now raises LookupError naming the speaker, instead of a
KeyError about a dictionary key three frames away.
Fixed in the queue layer rather than voice_profile_manager.py, which has 6
pre-existing failures in that area and should not be disturbed by this.
8 new tests. Suite: 698 passed, 8 pre-existing.
pollVoiceQueue began with 'if (!s || !s.items) return;'. On an unexpected
payload that returned before the code that clears the interval, so a page
left open polled /api/voices/queue every 2s forever. The web log shows 626
such polls - about 21 minutes of continuous requests from a browser through
the Caddy proxy, with CLOSE_WAIT sockets accumulating alongside.
An unexpected response now stops the poll rather than skipping the check.
Verified in a real browser against a deliberately malformed payload: one
request, then stopped, where it previously never stopped.
Reported: Save & Re-voice still gave no response after the previous fix.
That fix bound hx-on::after-request to the form. It never fires. htmx
reports a different detail.elt for each phase of this request - measured in
a real browser:
beforeRequest FORM
beforeSwap TR
afterRequest TBODY
Row swaps get re-parented into a table context, so afterRequest is
dispatched on the enclosing TBODY. A listener on the form never sees it,
because the event does not bubble downwards. An earlier probe suggested the
attribute worked, but that probe used a form outside a table, where elt
stays the FORM - it did not reproduce the real page.
So the profile was being saved (the PUT fired and swapped) while the
re-voice never ran and nothing on screen confirmed anything. That is
exactly "no response".
- Capture the row index at beforeRequest, where detail.elt is still the
form, and match it to the completing request by path at afterRequest so
a different response can never trigger a re-voice for the wrong speaker.
- Saving now shows a confirmation toast. The button previously looked dead
even when the edit did save.
- Tests now guard the mechanism, including that hx-on::after-request is not
reintroduced on this form.
Verified by clicking the button in a real browser on Deal Hunter's Dealer:
POST /api/voices/queue with {"speaker_name":"Dealer","row_index":14} and
the confirmation shown. Suite: 690 passed, 8 pre-existing.
Reported: editing a speaker, then "Save & Re-voice", did nothing.
This is a regression from e810b49 (the voice-queue commit), which deleted
the body of queueVoiceRegenOnSave but left the edit form calling it as
onsubmit="return queueVoiceRegenOnSave(...)". A JS handler that falls off
the end returns undefined, which is falsy, so the browser cancelled the
submit. The PUT never fired: no request, no error, no change. It affected
every field on the speaker edit form - name, gender, traits, accent - not
just re-voicing.
- Re-voicing is now hooked to hx-on::after-request, so the new profile is
on disk before the voice is rebuilt from it, and a failed save does not
queue work. The handler re-reads the name from the swapped row, so a
rename regenerates under the new name instead of the old one.
- The backend was never at fault: a full round-trip through
PUT /api/speakers/0 with the record's own values leaves every field
byte-identical.
- New tests/test_speaker_edit_submit.py guards the class of bug, not just
this instance: no onsubmit="return ..." in the row template, and any
onsubmit that returns a value anywhere must return a literal true/false.
Comments are stripped before scanning, since the hazard is documented in
one. Verified by reintroducing the bug - 3 of the 4 tests fail.
Suite: 688 passed, 8 pre-existing.
Reported: the speakers page showed 51 needing review in Chrome on Windows
and 20 in Safari on the Mac, and showed no names starting with D including
"Deal Hunter" - despite several reloads.
Neither browser was wrong. The active project is a per-browser cookie, so
each browser was on a different book and the two lists are unrelated:
Last Dwarf 91 speakers, 51 need checking, 0 D-names
DealHunterShip_4.0e 61 speakers, 19 need checking, includes
Deal Hunter, Dealer, Distorted Voice,
Duchess Victoria, Duty Nurse
"Deal Hunter" is the ship AI in the Deal Hunter book and is not a Last
Dwarf character, so its absence there was correct. The page gave no hint
that the book was even part of what was being displayed.
- The speakers page now shows a "Reviewing <project>" banner next to the
counts, so the number and the book it belongs to are read together.
- Added a project switcher directly on that page; previously changing
project meant navigating via the dashboard.
- activate_project accepts ?next=, restricted to same-site relative paths
so the switcher can stay put. Verified //evil.com and https://evil.com
both fall back to "/".
- list_projects registered as a template global, following the existing
active_project pattern; it degrades to [] on any error.
routes.py stays at its 76 pre-existing ruff errors. Suite: 684 passed, 8
pre-existing.
Follow-up to the badge fix: approving a speaker removed it from the
needs-check panel with no confirmation, which reads exactly like the
speaker being deleted. Reported as "the speaker advisor isn't shown now".
- A person-approved speaker now renders "✓ reviewed" with a tooltip saying
so. It previously rendered "✓ clear" whose tooltip claimed
"auto-accepted" - a false claim about a decision a machine never made.
- Approving confirms itself with a toast, drops the entry from the
worklist, and updates the header counts, all without a reload.
Separately, a pre-existing flake in tests/test_auth.py turned out to be a
real defect rather than test noise:
_session_secret() read the key file with .strip(), but bytes.strip()
removes ASCII whitespace from both ends. A secret beginning or ending
with one of those six bytes (4.35% of generated keys, measured) came back
one byte shorter on every read after the write - a different HMAC key,
silently, so a token signed before the flip could never be verified again.
Now read verbatim, with a length floor to reject a corrupt file.
That test also lacked the isolated_store fixture, so it read and rewrote the
real .umbra/session_secret - running the suite mutated production state.
This install's key is 32 bytes with no whitespace at either end, so no
existing session is invalidated by the fix.
Also verified: Advisor remains at display position 1 of 91 with its
utterance (utt 3668, ch15) intact - only the warning badge changed.
auth.py test flake 4/25 runs -> 0/30. Suite: 684 passed, 8 pre-existing.
Reviewer report: pressing Approve left the "needs check" badge in place and
appeared to navigate to a new page.
Root cause was a split brain, not a single bug. Approve writes the
voice_status.json ledger, but speaker_confidence read the voice_status key
embedded in speakers.json. That inline copy is stale - the web layer
overwrites it from the ledger on read (_load_speakers), so the page showed
the right value while the assessment computed from the old one. In Last
Dwarf, "Advisor" was 'pending' in speakers.json and 'approved' in the
ledger, so the reviewer's click had saved and still left it flagged.
- assess_speakers() takes the ledger as an explicit argument, authoritative
over the inline key; load_speaker_assessments() reads voice_status.json.
- New SpeakerAssessment.reviewed flag, surfaced in the serialized view.
- Approval clears the badge but keeps the reasons visible: the underlying
facts (role-only name, one utterance) stay inspectable on hover rather
than being silently discarded.
- Corrupt/ non-dict ledgers degrade to {} - this module never raises.
- Approve swapped #speakers-list with innerHTML, replacing the whole
reviewer list with a single <tr>; now swaps only its own row via
outerHTML on #speaker-row-N.
- The review panel explained only rename/merge and never mentioned Approve,
so the process was unclear; it now lists the three outcomes.
16 new tests. Suite: 681 passed, 8 pre-existing failures unchanged.
Reviewing speaker exceptions meant clicking Regenerate, watching a modal that
blocked the page, then navigating back to find out what happened. Worse, the
route refused a second request with a 409 while one was running, and
/api/generation/status exposed only a single global `active` flag - so there
was nowhere to put a second speaker and nothing to render per row.
Now: POST returns as soon as the task is captured, the row shows a status tag
(queued / working / done / failed), and further speakers can be queued while one
generates. No navigation, no modal.
umbraaudio/voices/voice_queue.py
- Ordered, serial queue. TTS is strictly serial here, so items run one at a
time in enqueue order; the second speaker waits rather than contending.
- Per-item state, persisted to voice_queue.json. An item left `running` by a
crash is marked `error` on load - the honest outcome, since silently
re-running could write audio nobody asked for twice.
- Enqueueing a speaker that is already pending is a no-op returning the
existing state, so a double-click cannot queue the same work twice.
- `cancel` withdraws a queued item but refuses one already running.
- Status vocabulary (queued/running/done/error) is deliberately small and
stable: it is what the controller will map onto node task states.
umbraaudio/web/voice_queue_routes.py - POST /api/voices/queue (capture the task),
GET /api/voices/queue (report per-item state), POST .../{speaker}/cancel. Kept
out of the hot routes.py, following name_token_routes.
/api/speakers/regenerate-voice is now a thin shim onto the queue so a
bookmarked call still works and the two paths cannot race. Net -24 lines in
routes.py; the blocking implementation is gone, not commented out.
The UI's modal, its 409 handler and the page-load resume handshake were removed
(67 lines) rather than left dormant - queue state is re-read from the snapshot
on load and after each HTMX row swap, so it survives navigation.
Unknown speaker names are rejected with a 404 at enqueue time. Previously they
reached the profile builder and surfaced as a bare KeyError('primary_name') in
the failed tag.
Verified live: two speakers enqueued back to back, first ran while the second
stayed queued, then the second started automatically on completion; both wrote
real ref WAVs. 16 new tests. Full suite 665 passed, 8 pre-existing failures.
The UI had no authentication at all: `active_project` is an unsigned cookie
and the server binds 0.0.0.0, so every page - including the mutation endpoints
and the audio downloads - was reachable by anything on the LAN and, through
the Caddy reverse proxy, from the internet. Spec 21/22 recorded this as an
unauthenticated mutation surface; this closes it locally until the controller
on oldbox takes over identity.
Gate is middleware, not per-route dependencies, so a newly added route is
protected by default and forgetting a decorator cannot expose the UI. It is
fail-CLOSED: an exception while checking a session denies the request, because
falling through to "no session required" would turn a bug into an open door.
- umbraaudio/auth.py: PBKDF2-HMAC-SHA256 (stdlib, per the project's
stdlib-before-third-party rule) with a random per-user salt. Sessions are
HMAC-signed and carry no secret, just username and expiry. All comparisons
use hmac.compare_digest. Credential writes are atomic (tmp + os.replace) so
an interrupted save cannot lock the user out, and both the store and the
session secret are chmod 0600 under .umbra/, which is gitignored.
- First use: seeds username `bdumbra` with the one-time password `First321`,
stored hashed and marked must_change, so the bootstrap password never lands
on disk in plaintext and is dead after the first login.
- Password rule: more than six characters, at least one letter and one special.
- Sessions last 14 days, sliding.
- Login failures are deliberately vague: a wrong username and a wrong
password produce the same message, so the form cannot be used to enumerate
accounts.
Verified live: unauthenticated requests to /, /speakers, /name-tokens, the
export endpoints and the ZIP download all 303 to /login; the first-use password
forces /change-password before any other page; weak, mismatched and reused
passwords are rejected; the old password stops working; tampered, forged and
expired session tokens are all rejected. Static assets remain reachable so
the login page renders. 19 new tests. Full suite: 649 passed, 8 pre-existing
failures.
The dashboard had per-track MP3 links but no way to get the package as one
artifact, which is what a distributor actually wants. New
GET /api/export/download-package.zip streams every track plus ar_package.json
so the download describes itself: order, roles, durations, measured RMS/peak.
Stored, not deflated - the MP3s are already compressed, so deflating 614 MB
would burn CPU for no gain.
Two bugs found while verifying rather than assuming:
- buffer.seek(0) sat INSIDE the ZipFile context manager, so it rewound before
the central directory was written. Every download came back 1,763 bytes
short and unzip could not find the end-of-central-directory record. Moved
the seek after the block closes; now "No errors detected" and all 32 MP3s
plus the manifest compare byte-for-byte identical to the originals.
- strict= is a zip() builtin parameter, not a ZipFile one. Ruff's B905 nudge
had me add it to ZipFile(), which raised TypeError and produced an empty
200. Reverted; the one remaining B905 is pre-existing elsewhere in the file.
Also surfaced the accepted_deviations from the manifest in the dashboard: a
collapsible table of the tracks below the RMS floor, with their measurement,
peak, crest factor and the rule they depart from, so the exception is visible
where the numbers are rather than only in the JSON.
Verified error paths: no active project -> 400, project without a built
package -> 404. Lint unchanged (77 before and after). Full suite: 630 passed,
8 pre-existing failures.
Single master MP3 set for both distributors, with the two sub-floor chapters
left as they are rather than limited.
accepted_deviations in the manifest, for tracks that are peak-compliant but
still under the RMS floor. Both sit at exactly -3.00 dB peak, so there is no
headroom to add the ~3 dB of gain they would need: a 20-21 dB crest factor
makes RMS -20.5 and peak -3.0 mutually exclusive at this gain. A limiter would
meet the number by squashing transients, which is a worse trade for a
narration-led book. Recorded as a decision with its measurement and reasoning
so a reviewer reads it instead of rediscovering it from a warning string.
Deal Hunter: "The Poker Game Distraction" (-23.80, crest 20.80) and
"Hunting Pirates" (-23.10, crest 20.10). The 30 compliant tracks span
crest 17.60-19.90, so the two separate cleanly and nothing marginal is swept
into the exception.
Also quarantined three orphan WAVs in ch35 (utt_0003/0004/0005, 40.9 s, dated
2026-08-29). They predate the closing track being rewritten to three
utterances, have no marker and no DB row, but the stitcher globs utt_*.wav -
so they were silently included in ch35_full.wav and in the shipped m4b, whose
closing chapter was 64.5 s where the tracked audio is 20.3 s. Moved to
chapter_audio/_orphans_ch35_20261009/ rather than deleted. Only ch35 was
affected; the book-wide orphan scan found no others.
Rebuilt both artifacts from the cleaned chapters: 32 MP3s all peak/rate/bitrate
compliant, and the m4b closing chapter is now 21.3 s.
_encode_and_enforce_peak measured RMS once, before any peak-correction trim,
then returned that stale figure. A track that needed peak correction was
therefore recorded louder than it actually is, and loudness_ok passed on a
number that described neither the intent nor the artifact.
ch15 took a -2.9 dB trim: the manifest said -20.6 dB, the delivered file
measured -23.8 dB, and the gate called it compliant. Measuring all 32 delivered
files independently found 2 real failures (ch15 -23.8, ch21 -23.1, both under
the -23 dB floor) that the manifest reported as 0.
Now both figures come from the encoded file after every trim, so the manifest
always describes the delivered bytes.
_measure_file gained a content_s argument that excludes head/tail room tone
from the RMS measurement via atrim. RMS is defined over "the entire program
(not including room tone)", and padding share varies enormously: 0.03 dB on a
30-minute chapter but ~7 dB on a 1-second fixture. Peak needs no such
treatment since padding is silence and cannot raise a peak, so one trimmed
pass serves both.
Rebuilt: 32 tracks, 7.45 h, 0 peak failures, loudness range -23.80..-20.70 -
which now matches independent measurement exactly.
The delivered m4b was 24,348s where the source chapters total 26,830s. Root
cause: libsndfile stops reading at 2**30 - 1 frames, and a 7.4-hour book at
44.1 kHz is ~1.18e9. ANY read through libsndfile was capped - including a
chunked read loop, because the limit is in the library's frame counter rather
than in the call. ffprobe read the same file correctly at 26,830s, which is
what made it look like the concat had lost data rather than the read.
Both ends now stream through ffmpeg instead:
- _concat_wavs_with_silence writes with sf.SoundFile per segment rather than
one np.concatenate() array, and as FLOAT rather than the PCM_16 default,
which had been clipping the book to 0.00 dBFS before normalisation saw it.
- _encode_aac_m4b measures RMS with ffmpeg and encodes with a
volume+alimiter filter chain, so libsndfile never reads the book at all.
alimiter looks ahead, so it holds the ceiling without the flat-topped
distortion hard clipping produces, and it survives the AAC encoder better.
The peak margin also moved from a post-hoc trim to the limiter ceiling. Trimming
after the fact cost ~3.2 dB and pushed RMS to -23.8, outside the band; clipping
to -6.0 dB instead lands at -3.10 peak / -20.70 RMS with no trim at all.
Verified on the real book: duration 26,799s (7.44 h), AAC 44.1 kHz mono
125 kbps, RMS -21.1, peak -5.2. 32 chapter markers, cover art embedded,
closing credit last. Full suite: 630 passed, 8 pre-existing failures.
export_m4b encoded with `-c:a aac -b:a 128k` and no `-ar`, so the delivered
file came out at 24 kHz / ~96 kbps mono while the AR MP3 masters were
44.1 kHz / 192 kbps CBR mono. Measured on the shipped Deal_Hunter.m4b:
24 kHz, ~96 kbps, and peak -2.0 dB - failing the 44.1 kHz and -3 dB
requirements at Author's Republic and Spotify. AudiobookExporter had zero
loudness or peak handling of any kind.
- _encode_aac_m4b now passes -ar 44100 and -ac 1 explicitly, and takes the
spec constants from ar_export so the two paths cannot drift apart again.
- Added normalisation to the RMS target and peak enforcement on the delivered
file, plus m4b_loudness_report() so the result can be verified.
- AAC overshoot is NOT monotonic. Measured +0.8, +0.9 then +1.6 dB across
three passes, so trimming by the exact excess oscillated and exhausted the
pass budget still breaching. Trimming by excess + 1.5 dB margin converges:
verified -2.20 -> -4.5 dB in two passes.
- _concat_wavs_with_silence now targets 44.1 kHz rather than the TTS-native
24 kHz.
- Closed spec 22 gap 1: _stitch_chapter hard-coded
silence_between_chapters_ms=0 while ChapterBuilder's AudioStitcher defaults
are 500/4000, so the packaged book had different silence than the audio a
reviewer approved.
Trade-off, flagged rather than hidden: the 1.5 dB overshoot margin costs ~1.3
dB of RMS, so a trimmed m4b measures about -22.8 dB against the -20.5 target.
Still inside the -23..-18 band, but only ~0.2 dB from the floor; a chapter
needing more than ~2.5 dB of trim would fall out of band and loudness_ok will
flag it.
Verified on real chapter audio: AAC 44100 Hz mono, RMS -22.8, peak -4.5.
6 new tests. Full suite: 630 passed, 8 pre-existing failures unchanged.
