main
27 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
B.D. Murphy
|
2cafbabe82 |
Measure whether Qwen3-TTS speaks Chinese with an English voice reference
This was the one open prerequisite for any target-language audio lane: the survey in spec 26 says the model covers Chinese, but that is a model-card claim, and our whole voice catalogue is English references. Nothing had been measured about what happens when an English reference is asked to speak Chinese. scripts/probe_cross_language_tts.py now also takes an optional second English reference and makes the identity question differential rather than a transcription question - a transcript has no timbre. Two English references reading the same text should stay as far apart when they read Chinese as when they read English; if they do, the reference is carrying identity into the Chinese output rather than being ignored. The measurement is made with ECAPA-TDNN cosine similarity. The result, on the real Deal Hunter narrator and chief-of-the-guard references, is in logs/cross_language_tts_probe/probe-report.json: - the English reference speaks Chinese. Whisper hears zh on every Chinese-text clip and en on every English-text clip, recovering 12 of the source's 22 distinct characters. - the text's script decides the language, not the `language` parameter. Chinese text with language=english still comes back as Chinese, so a caller that trusted the parameter to select a language would get the wrong phonology without an error. - the reference is carried: different voices are 0.383 apart reading English and 0.369 apart reading Chinese. What the measurement does not establish is stated in the spec rather than glossed: ECAPA is an English VoxCeleb model, so its Chinese embeddings are out of domain and the same-speaker/different-speaker separation collapses in Chinese. It can show that two clips differ; it cannot certify that two clips are the same speaker across a language boundary. How much of the identity survives is still a listening question, and the clips are saved for it. Spec 26 gains the measurement and a sixth open question - whether the identity survives for all 61 speakers, not just the two measured. Spec 27 gains the parameterisation record (UmbraTrans 3f05489) and a measurement of what the Chinese edition actually contains: 4,305 delivery rows, of which the 90 not-voiced rows are 44 normal collapses and 46 drift-dropped paragraphs covering 33 paragraphs and 356 source words, about 2.4 minutes of speech. Two of the dropped paragraphs are upstream translation defects the aligner cannot fix - ch-15-p0100 and ch-25-p0089 carry text that belongs to a different sentence - so fixing them means re-translating. Suite: 913 passed, 8 failed (the same pre-existing pronunciation and voice_profile_manager accent failures), 6 skipped. |
||
|
Umbra Tools
|
b259060808 |
Record two more consequences of the mislabelled Spanish production
Measured on oldbox: * The UmbraTrans dashboard reports the Spanish lane as "complete". project_status.py derives each language's state from the presence and chapter count of deal-hunter-<tag>/runs/ch-N/translation.json, and deal-hunter-es has 32 of 32. It never looks at what the text says, so a directory name and a file count are standing in for a measurement. The other nine target languages correctly report not_started. * A real Spanish run would silently reuse the Chinese translations. pipeline.py caches by file existence and the run directory is chosen by the caller, so a Spanish run pointed at deal-hunter-es without --force would skip drafting and publish the Chinese under a Spanish name. Neither was fixed here: the UmbraTrans checkout on oldbox is behind and dirty, and the script check belongs in the same place as the ingest's. |
||
|
Umbra Tools
|
31cd9f7ce4 |
Record the paragraph-boundary drift and how it is now caught
The production Chinese re-flows the passage across paragraph ids in the places where Pass 0 split the original mid-sentence. Measured: 439 of 3,038 translated paragraphs (14.5%) are sentence fragments, and in ch-15 the drift runs p0094-p0104 - eleven paragraphs whose Chinese is good but attached to the wrong paragraph. The alignment pass cannot see this by construction: a one-to-one paragraph is a pure copy, so there is nothing to divide and nothing to disagree. The verification stage finds it, and the spec records how it was made precise. The first version asked two yes/no questions per paragraph and was wrong in both directions - five of fifteen flags were paragraphs it claimed were missing an adverb that was present, and it missed two paragraphs that are drifted. The stage now asks which paragraph a translation renders, shows the model its neighbours, and judges the paragraphs as a window rather than one at a time. On the real ch-15 window it now finds exactly the drift: ten paragraphs dropped, p0094 kept and flagged because its Chinese correctly renders its first two sentences. Also recorded: the cost of the choice (a gap where a fragment inside a broken window is dropped on an unreliable verdict) and the root cause (the drafter is not told that each id must cover only its own English). |
||
|
Umbra Tools
|
3db6ac3b8b |
Refuse a delivery whose text is not written in the language it claims
The production directory oldbox calls "deal-hunter-es" declares target_language "es-ES", its glossary says Spanish, and its 32-chapter run finished - and all 3,038 translated paragraphs are in Chinese. It is a second Chinese draft filed under a Spanish name. The cause is in the pipeline: the drafter's instructions are hard-coded to Simplified Chinese in three places (SYSTEM_DRAFTER, draft_prompt, SYSTEM_CRITIC), so the glossary's target language was ignored. No non-Chinese translation has ever been produced by that pipeline. The danger at this end is that the rows would be stored under "es", the review lane would look populated, and a reviewer would approve Chinese as Spanish. So the ingest now checks the script of the delivered text against the declared language, and refuses the delivery before writing a single row. The check is about the script, not the language: it cannot tell Spanish from Portuguese, but it can tell Chinese from Spanish, which is the mistake that actually happens. Japanese needs its kana and Korean its Hangul, and a Chinese delivery carrying kana is a Japanese translation filed in the wrong lane. An unknown language is not checked - there is no expectation to check it against. Proven on the real data: the es delivery is refused with "the text is mostly Han characters, so it is not Spanish", and the 4,300-row Chinese delivery still imports unchanged. 19 new tests. Suite: 909 passed, 8 failed (the same 8 pre-existing failures). |
||
|
Umbra Tools
|
c476cfea1d |
Ingest the translation delivery, and stop mis-reporting locale codes
UmbraTrans delivers one row per utterance per target language, in the shape translations.import_translations already accepts. There was no script to run that ingest, so the delivery side of the contract existed only as a function. scripts/import_translations.py runs it: report-only by default, --write to store, --confirm to store as confirmed rather than draft. It prints the coverage before and after, so a delivery can be checked without writing it. Two bugs in the ingest, both found by running a real delivery: * import_translations folded the target language with lang.normalize, which does not know script subtags. A delivery labelled "zh-Hans" - which is what UmbraTrans writes - matched nothing, so every row was dropped and reported as a problem. It now folds with canonical_code, and an unknown language is still stored: the text is reviewable even when the render will be refused. * coverage and not_voiced_rows compared the stored code to the requested code as strings, so a row stored as "zh-Hans" read as "nothing delivered" when the reviewer asked about "zh". The coverage report was wrong in the direction that makes a reviewer distrust it. Measured on the real Deal Hunter Chinese delivery: 4,300 rows ingested, the review UI's resolver then renders Chinese for 12/12 utterances of chapter 1 and 415/416 of chapter 6 - the missing one being the spoken chapter announcement, which is deliberately not part of the reading edition. spec 27 records what the shipped samples actually are: single-pass drafts made against a hand-cleaned paraphrase of the English, matching the speaker-verified database text in 0 of 12 utterances. The production Chinese edition on oldbox does use the intent-based process; what it lacked was the alignment back onto the utterances, which is now implemented in UmbraTrans. 7 new tests. Suite: 890 passed, 8 failed (the same 8 pre-existing failures). |
||
|
B.D. Murphy
|
2b294d7fa4 |
Per-utterance language flag + a working language selector (spec 31)
The review UI's language dropdown did nothing: every selection kept rendering the English text. Two independent causes, both fixed. 1. The translations were never being read. The pipeline's `utterance_translations` table holds one row in Deal Hunter, and that row stores the *English* text under a `zh-Hans` code. The real translations live in the on-disk sample files under `samples/translations/`. `umbraaudio/translations.py` now resolves both sources and merges them by (chapter_id, utterance_index): the table wins where it holds a real translation, the sample files fill the gaps. A candidate whose text is identical to the source is a placeholder, not a translation, so it is ignored - otherwise the page would show English under a Chinese heading and claim it was translated. 2. The query that read them was broken. `language_routes.py` selected `u.index` and `u.speaker`; the columns are `utterance_index` and `speaker_name`, so every read raised `sqlite3.OperationalError: near "index": syntax error` and `/language-mockup` returned 500. It also opened `<project>/project.db` - a 0-byte decoy - instead of `<project>/.umbra/project.db`, the file that actually holds the data. Also in this commit: * The chapter review page honours `?lang=`, remembers the choice in a cookie (set by a middleware, so it survives on every page), and shows the selected language's *name* in the "has not been translated yet" message. * The page now shows the translation, not the source, and the edit box is bound to the source text so saving cannot overwrite the English with the Chinese. * The stray language selector that sat after `</html>` in base.html is gone; the chapters-list page's dropdown had zero options because the route never passed the language list. * The language tag is wired into generation: `utterance.language_code` (NULL = project source language) reaches `engine.generate()` and the staleness marker, so a retagged line is re-rendered instead of keeping English phonology behind an unchanged text. The marker records the language; a marker written before the tag existed means English, so the ~4,300 existing clips are not invalidated. * `project.source_language` (spec 31 section 13.1) records the book's own language, with a migration for existing project DBs. * A scene-break marker deliberately records no language: it is silence, identical in every language, and tagging it would claim a phonology the audio does not have. Tests: 883 passed / 8 failed - the same 8 pre-existing failures (2 test_pronunciation, 6 voice_profile_manager accent). 26 new tests cover the resolver and the generation wiring. ruff introduces no new findings. |
||
|
B.D. Murphy
|
3168b51e7c |
Spec 30: language as a first-class dimension, with a UI mockup
Proposal plus a working read-only mockup at /language-mockup for operator
review. German selected by default, as asked.
Measured first, so the proposal reflects reality:
utterances 4,319 rows, no language column - one text per utterance
language absent from the DB entirely; exists only as a filename
convention in the experimental lang-samples page
/v1/tts language: str = Form("english") - free-form, unvalidated
marker utt_XXXX.wav.txt JSON has speaker/text/voice_fp, no language
The load-bearing finding is that audio MUST be namespaced per language
before any second language is generated. staleness.is_current() compares
the marker's recorded text against the utterance text, so German and
English sharing ch6/utt_0001.wav would each mark the other stale and
overwrite it in a loop. That is a correctness requirement, not a tidy-up.
Schema choice: a new utterance_translations table rather than a language
column on utterances. One row can hold only one language, so a column
either forces 10x duplication of speaker/emotion/pace/attribution or
overwrites the source text. UNIQUE(utterance_id, language_code) makes
UmbraTrans re-delivery an update instead of a second competing German.
English stays in utterances.text and stays editable; translations are
confirm-only, matching the agreed asymmetry.
Audio: chapter_audio/ (English, path untouched, zero regression risk)
gains siblings chapter_audio_de/ etc. Marker JSON gains "language" so an
export can prove what it shipped.
The mockup is a separate router for the reason name_token_routes.py exists
- it keeps proposed work out of the hot routes.py until it is agreed. It
writes nothing, generates nothing, and touches the English pipeline not at
all. Verified: 200 on all 10 languages, real German text rendered,
coverage read from the real 4,319-row DB, unknown codes fall back to
German without a traceback, and the auth gate still 303s to /login.
I initially invented a routes.py helper (resolve_project_dir) that does
not exist; the real one is _get_active_config(request).project_dir. I also
removed a silent cwd fallback that would have rendered English under a
German heading and looked like a working page, and made an unreadable DB
show 'could not measure' rather than a misleading 0%.
Suite: 721 passed, 8 pre-existing. ruff clean on new files.
|
||
|
B.D. Murphy
|
d7fc0236bc |
Narrator speaks the translation language; characters keep their own accent
Specification change. Two different rules, and they need different
mechanisms.
Narrator: accent = translation language. Its reference was generated by
VoiceDesign from a description that hardcoded '{accent} English', baking an
American-English timbre into the WAV; cloning that into German gives German
phonology from an American voice. Ten per-language references now exist
(narrator_EN.wav ... narrator_ZH.wav), each designed from the narrator's
timbre plus 'Speaking {language} as a native {language} speaker'.
Characters: keep the accent on the speaker page. Their accents were already
encoded in their existing refs - what destroyed them was the language
parameter forced onto every utterance, which made every voice adopt the
target language's prosody. That is why Deal Hunter came out French-native.
Phonology and accent are now separate controls: language carries phonology,
an explicit voice_description carries accent character. Deal Hunter gets
'with a strong Australian accent' in every language.
Measured A/B, French line from Deal Hunter: with and without the Australian
instruction the waveforms differ by 152% of signal amplitude and duration
moves 4.24s -> 4.08s, so the instruction is reaching the model. Whether it is
convincingly Australian is an ear test, not something measurable here.
Re-rendered all 120 clips under the new spec, 0 failures. The previous
render is kept in Lang_test_preSpec for A/B. 24 speakers carry a non-default
accent; the narrator is deliberately excluded from that set.
Suite: 721 passed, 8 pre-existing.
|
||
|
B.D. Murphy
|
6b85e48b55 |
Multilingual samples, multi-pass translation design, UI language plan, renderer
Operator direction: English is the source and stays editable; translations are confirm-only. - samples/translations/: Deal Hunter sample chapter 1 in all 10 Qwen3-TTS languages, named deal_hunter_sample_ch1_<LANG>.json. Single-pass LLM drafts, labelled as such in every file - they are the failure mode the multi-pass design exists to fix, and they double as a bad-case corpus for the Pass 4 critic. - scripts/build_sample_translations.py, with a validator that refuses to write on stray Latin residue, duplicated utterances or a wrong count. It caught three defects in my own draft before it would ship. - specs/27-translation-pipeline-multipass.md - six passes, the point being that Pass 4 must be a separate critic call, not the same call that drafted. - specs/28-ui-language-selection.md - confirm-only translations, English editable, selector scoped to the chapter page, language named on the page because the cookie is per browser. - scripts/render_lang_test.py - renders the translations with each speaker's own voice into <project>/Lang_test/, plus a stitched file per language. Dry run resolves all 12 utterances x 10 languages; 11 of 12 use a cached ICL prompt, Trader Romano falls back to ref_audio. The renderer resolves display names through the alias map rather than by munging strings: the translations say 'Kainda' and the speaker record says 'Kainda Walters Foster Murphy'. String-munging silently produced no voice for Kainda in the first draft - the same failure mode as the speaker-review audit. NOT DONE: no audio rendered. Qwen3-TTS generation on this machine is stalled - /v1/tts accepts the connection and never completes (zero POSTs logged across 17 minutes after a clean restart), while /v1/voice-prompt and /load return 422 immediately, so requests do reach the handler and the block is inside generation. Swap is at 16GB of 17.4GB. Left as a known blocker rather than worked around. Suite: 721 passed, 8 pre-existing. |
||
|
B.D. Murphy
|
43a2853ee3 |
Research: multilingual TTS models for the 15-language catalogue
Survey of cross-lingual voice cloning, as a companion to the speaker-review audit that gates the translation handoff. Recorded as research, not a decision. Terminology correction up front: this is a TTS problem, not an LLM problem. Reproducing a voice in another language is cross-lingual voice cloning; an LLM is still needed for the translation itself but plays no part in voice fidelity. Headline findings: - Qwen3-TTS, already running locally, covers 10 of the 15 languages (en zh ja ko de fr ru pt es it). The gaps are Dutch, Norwegian, Polish, Arabic and Hindi - so no single model does this job. - Licence is the deciding constraint, because the books are sold. XTTS v2 covers 14 of 15 and is the obvious capability choice, but its weights are Coqui Public Model License (non-commercial) and Coqui shut down in Jan 2024, so no commercial licence can be bought. Fish Audio S2 (83 languages) and F5-TTS are likewise non-commercial weights. - Recommendation is a two-engine split: Qwen3-TTS (Apache 2.0, verified from the licence file) for its 10, Chatterbox Multilingual (MIT) for the 5 gaps. Together they cover all 15. The 'voice matrix' question, which the operator raised directly: voice_registry stores two different things. voice_ref_path is the reference WAV and is portable to any clone-from-audio engine; vmatrix_path is a Qwen3-TTS ICL artefact (8k ref_code + speaker embedding, .pt) and is engine-specific and unreadable by anything else. Deal Hunter already has 91 reference WAVs, so voice fidelity across languages needs no retraining and no matrix translation - just handing the same WAV to whichever engine covers that language. Recorded honestly: the Chatterbox language list came from a third-party API wrapper, not Resemble AI's own model card, and that build reports Chinese as disabled. Verification against upstream is listed as open question 1. |
||
|
B.D. Murphy
|
9bda553311 |
Add a speaker-review audit: is the attribution data fit for translation?
Runs after a reviewer finishes a book and answers the two questions that
gate the next stage:
1. Are there utterances with no speaker?
2. Now that speakers have been defined, renamed, merged or deleted, are
there utterances that should change?
Read-only - it never edits the database, and a test asserts the files are
byte-identical afterwards. Output is a markdown table and a CSV, one row per
finding with chapter, utterance number and a text preview, so each item is
locatable rather than a count.
Deal Hunter, after the completed review (56 speakers, 0 still needing
review, 4319 utterances):
- Dialogue with no speaker: 0. The 1803 speakerless rows are narration
and scene breaks, which is correct and is reported as a number rather
than silently dropped.
- Utterances attributed to a deleted speaker: 0.
- One speaker stored under two spellings: 6 rows across 3 speakers
(narrator/Narrator, Kainda/Princess Kainda, Willamina Wake/Captain).
All resolve to one voice so the audio is correct, but a per-speaker
export would split them into six - caught before the translation stage,
not after.
- 6 speech-tag mismatches where the narration immediately before the line
names a different speaker.
- 10 dialogue rows voiced by the narrator: 9 are quoted labels ("Reactor.",
"operations") that are correctly read by the narrator, 1 (ch28 utt46) is
a real segmentation error. Classified advisory so the one real defect is
not buried under nine false positives.
build_alias_map moved from scripts/qa_attribution.py into attribution_qa.py
so the CLI and the audit cannot disagree about what a speech tag resolves to;
qa_attribution output verified byte-identical.
13 new tests. Suite: 721 passed, 8 pre-existing.
|
||
|
B.D. Murphy
|
0c9bea72e0 |
Add enterprise/infra tooling and repoint Qwen3-TTS submodule to internal fork
Two groups of changes, committed together as the follow-up to the initial push: 1. .gitmodules — submodule Qwen3-TTS now points at the internal Gitea fork (ssh://git@mini-eq:2222/umbragit/Qwen3-TTS.git) instead of upstream GitHub, so a clone actually gets the UmbraAudio branding/integration commit 012e452 rather than vanilla upstream. 2. New infra/ tooling for the cross-machine work in spec 21: agent_bridge, remote_credential_broker (Keychain-backed, secrets read at runtime and generated via read_secret/random_secret — no hardcoded credentials), the enterprise bootstrap scripts, and the oldbox SSH/auth prompts. Plus backup/ restore scripts, two specs (18 UmbraTrans, 19 enterprise control plane), umbraaudio/voices/lmstudio_gate.py, and in-flight edits to the discovery/attribution/profiling modules, config, chapter_builder, emotion, pronunciation, qwen_tts and routes. Also adds *.db to .gitignore: SQLite project databases are runtime state and no .db file was tracked before. Verified before staging: .env's only change is UMBRA_TTS__QWEN_TTS_HELPER_URL and the committed copy has zero secret-looking values; the broker ships only config.example.json. |
||
|
B.D. Murphy
|
f19ada75fb |
Mechanical speech-evidence gate: catches the ch6 utt29 miss (spec 24 A)
The operator's chapter 6 review found a line attributed to Hallr that the text
gives to a scum boy, and no part of the system had flagged it. Spec 24 §2
diagnosed why: the duplicate scan only compares names that already exist, so it
cannot see a line given to the wrong EXISTING speaker. Spec 24 §4A proposed a
mechanical evidence gate. This is that gate, measured.
New umbraaudio/analysis/speech_evidence.py, model-free, plus
scripts/report_speech_evidence.py (read-only) and 44 tests.
For every dialogue line: extract the speaker the narration actually names, build
the candidate set from the cast rather than from the guess, and return
supported / contradicted / unsupported / unconstrained.
Last Dwarf, all 49 chapters, 3,918 dialogue lines:
supported 1,844 47.06%
unconstrained 2,032 51.86%
contradicted 34 0.87%
unsupported 8 0.20%
review queue 42 1.07%
It catches the chapter 6 defect exactly: "The leader on the right for Hallr
says," / "We want to be the ones to claim we killed the last dwarf" assigned to
Hallr, with the two leaders who could own it. attribution_qa reports 11 findings
book-wide and this one is not among them.
Two design decisions carried the precision, and both were forced by measurement.
First, "no tag" is not a finding: most dialogue has no tag because a tag
introduces only the first line of a conversation, so unconstrained and
unsupported are separate verdicts. The first working version queued 60% of the
book for this reason, which is the same as having no gate. Second, "ambiguous"
was removed as a verdict - it hid the case that matters, since a line assigned to
Hallr under "The leader on the right" is wrong no matter how many leaders exist.
Candidate count now decides whether a suggested fix is confident or a question.
Two corrections to spec 24, both from measurement. The 20 role-tag violations
reported in section 3 are really 11; my ad-hoc probe was worse than the checker
it was meant to replace. And identity coverage is already 100% - every dialogue
line resolves to a profile or alias - so the defect is never a missing speaker,
it is a line given to the wrong existing one.
Six extractor bugs the measurement exposed are now regression tests: a role
phrase with a spatial qualifier produced no tag at all; a coordinated predicate
spoke as its first subject instead of its last; a possessive voice resolved to a
profile called Voice; irregular plurals and superlatives never matched; and the
subject was read off trailing participial clauses. Two generalisable rules came
out of them - a speech tag's subject is a noun phrase and a prepositional
qualifier is still part of it, and a subordinating conjunction opens a clause of
its own ("The dragon swallows when Hallr says" is Hallr).
Reports carry chapter_id AND title because chapter_id and chapter_index differ
by one in every project here; a bare "ch8 utt264" sent the operator to the wrong
chapter, and the line they asked about is chapter 9 utt264.
Full suite 619 passed / 8 failed, the same 8 pre-existing failures. ruff clean.
|
||
|
B.D. Murphy
|
60564cc056 |
Analysis-quality plan: why the ch6 speakers were missed (spec 24)
The operator reviewed ch6 with author knowledge and found real defects the
duplicate scan could not see. Two corrections and one structural diagnosis.
I was wrong about two pairs. I wrote "probably one person" about The Officer /
Officer, and "probably two different riders" about Border Soldier / Soldier
(Rider). Both are two different characters. Measured, The Officer (113.3 Hz) and
Officer (104.5 Hz) are 1.4 SEMITONES apart - two distinct characters with
near-identical voices, which is a genuine cast-clarity risk that a merge would
have hidden. Border Soldier and Soldier (Rider) are 12.2 semitones apart, so
their risk is narrative proximity, not vocal.
The real lesson is the principle behind it: because the author wrote the book he
knows these facts, and on a book he did not write nobody does. So every merge
decision must be resolvable from evidence in the text, and the default when
evidence is absent must be SEPARATION. Keeping two speakers separate costs one
redundant voice; merging two people costs a wrong character for the rest of the
book. Reworded the scan so a blocked group is reported as "same name-space,
different characters unproven" and never as a probable merge.
Why the scan missed ch6: it can only compare names that already exist. It cannot
see a line attributed to the wrong EXISTING speaker. ch6 utt29 - "The leader on
the right for Hallr says," / "We want to be the ones to claim we killed the last
dwarf" - is a scum boy attributed to Hallr. No duplicate existed to find. The
scum group was also missed on identity: only the case variants were caught, and
nothing compares a designation against the text that produced it.
Diagnosis, measured book-wide: a role speech-tag must not resolve to a distinct
named character. 20 dialogue utterances violate it. True positives include ch11
utt191-193 where ONE tag ("The young man replies,") was split across Kaleb,
Tobias and Hallr, and ch13 utt45 where "The girl replies," became The King. But
40% are false positives because ROLE_WORDS is incomplete - royal, king, witch,
doctor and others are missing - and a gate that noisy would be ignored. The
vocabulary must come from definitions.py, not a hand-written list.
Plan, model-independent first: (A) mechanical evidence-coverage gate, which
catches this class with certainty; (B) constrained assignment - choose from the
evidence-supported candidates only, not all 93 names, which is the actual cause
of utt29; (C) role-phrase preservation so "the leader ON THE RIGHT" stops
acquiring a new name per mention; (D) tag-run consistency, never split one
introduction across three characters; (E) a segmentation gate for narration typed
as dialogue (ch6 utt26); (F) confusability signals, a casting concern distinct
from identity; (G) an evaluation harness with a ch6 gold set and five metrics,
because without a number "use a better model" is a guess.
On a better model, honestly: 12b cannot fit in 16 GB, and the GPU box at
192.168.123.14:1234 refused a connection today (curl exit 7), so it is blocked.
And B removes the search space first, which is where the error actually was.
Full suite unchanged at 575 passed / 8 failed, the same 8 pre-existing failures.
|
||
|
B.D. Murphy
|
216bc78a3a |
Remove the dead narrator profiles; add --force for decisions a scan cannot make
The operator confirmed Narratorold and OrigNarrator are dead profiles. --force exists for exactly this: a decision the scan is not entitled to make. It prints the overridden conflict before acting, and never deletes a reference file. Applied to Last Dwarf: profiles 93 -> 91 approved voices 13 -> 13 (unchanged) voice ref files 72 -> 72 (unchanged, both WAVs left on disk) dialogue rows labelled 'narrator' 3 -> 0 (now the approved Narrator) narration rows labelled 'narrator' 5978 -> 5978 (untouched) The dead names survive as aliases of Narrator so no stray utterance can resurrect a profile, and the 3 re-pointed rows are a genuine attribution fix: ch5 utt26 and ch27 utt125 are narration mis-segmented as dialogue; ch42 utt290 is a first-person line that may be intended inner thought. Rescan: 10 auto, 2 review, 3 blocked, unresolved spoken names 3 -> 2. Full suite 575 passed / 8 failed, the same 8 pre-existing failures. ruff clean. |
||
|
B.D. Murphy
|
714b11367a |
Duplicate-speaker scan that refuses to eat an approved voice (spec 23)
Last Dwarf has accumulated duplicate speakers from successive discovery and
designation scans: The Girl / The Girl helping / One Girl Helping, Guard 1 / The
first guard, Walking oldest / The Oldest, plus three lowercase spoken names that
differ from their profiles by one letter. The operator wants them eliminated and
was explicit that created and approved voices must not be lost.
umbraaudio/analysis/speaker_dedup.py
Read-only fact gathering (profiles, ledger, voice refs, chapter spread) and a
matcher that explains itself: determiner and ordinal normalization, superlative
stemming, and three VETO rules, because string similarity alone would merge
people who are not the same person:
- distinguishing modifiers: The Oldest is not The Tallest; Guard 1 is not Guard 2
- role nouns: Soldier (Rider) is not One Rider
- proper names need ratio >= 0.88: Anfisa (0.77 from Tanisha) is a different
character; Rielan/Rieland at 0.93 is a misspelling
Verdicts auto / review / blocked. A group with two different voice references is
blocked, because merging would discard a generated voice. Survivor selection
prefers an approval, then a voice reference, then utterance volume.
Two rules were written and then removed, recorded in the spec so nobody re-adds
them: a "shared token plus role noun" rule proposed The Oldest = The Old Woman,
and a "single shared token" rule proposed Scum leader = The Leader. A missed
duplicate costs one manual merge; a wrong one silently rewrites a speaker.
Two real bugs, both found by running against a COPY of Last Dwarf rather than by
reading code:
1. orphan_resolution.merge_speakers never touches voice_status.json, so merging
an approved speaker into an unapproved target would have deleted the approval
record silently. merge_group() now carries the approval to the survivor,
drops the loser's entry, and reports approvals_carried.
2. The first re-point was unscoped and moved 5,981 rows instead of 3: `narrator`
is the labelling convention for ~5,978 NARRATION rows too, and renaming them
would invalidate thousands of clips because every marker records the speaker
exactly. The re-point is now dialogue-only. Narration casing is a separate,
deliberate decision and is flagged as such, not taken by a duplicate scan.
Verified on a copy: 93 -> 84 profiles, 23 utterances re-pointed, narration
untouched, 72 -> 72 voice ref files with zero removed, 13 -> 13 approvals with
zero lost.
Last Dwarf scan: 10 auto, 2 review, 4 blocked. Blocked groups are exactly the
ones the operator must decide - The Girl (two voices for one person), Officer,
the three-way Narrator family, and Border Soldier vs Soldier (Rider).
Nothing was applied to the live project. tests/test_speaker_dedup.py: 37 tests,
vetoes weighted as heavily as positives. Full suite 575 passed / 8 failed, the
same 8 pre-existing failures. ruff clean.
|
||
|
B.D. Murphy
|
9f3a0a4b18 |
Release comment as a release step; releasing grants implicit voice approval
Two corrections from the operator, and I had both wrong. 1. "I didn't explicitly approve all minor character voices. By releasing the audiobook I have given implicit approval those voices are ready and should be tagged." The previous session reported "24 of 61 speakers have an approval record; 38 do not" as if it were a gap. It is not a gap. Releasing IS the approval, and those voices were waiting to be tagged. chapter_approved and audiobook_approved now read true, with approved_by: "this publication record" - there is no separate audiobook approval in this system, and pretending otherwise understated the release. The boundary matters, so it is measured rather than assumed: implicit approval covers voices that HAVE LINES in the released chapters, computed from the database. A character who never speaks is not covered, because releasing a book does not approve a voice it never uses. On Deal Hunter edition 1 that is 24 explicitly approved, 29 implicitly approved, and 10 in-profile voices with no lines. The three categories are never summed - an explicitly approved voice can also be one that stopped speaking. grant_implicit_voice_approval() writes plain status strings, the shape the web UI already reads, and keeps the provenance in the immutable publication record rather than the mutable ledger, so it cannot be edited away. 2. The m4b inter-chapter gap should be the same 2 s as the lead and trail padding. export.py had gap_ms=4000 in one path and a default of 1000 in the other, while ar_export.SILENCE_MS = 2000 governed lead and trail: three drifting values for one concept. The gap now imports SILENCE_MS as CHAPTER_GAP_MS, so all three are one number that changes in one place. New umbraaudio/release_comments.py - the comment is a gate, not a caption. designator 2026-10-03 #2 14:32:05 slug release-20261003-143205-02 The sequence is per date, not global, which is the whole point: a problem found and corrected on the same day is unambiguous about which was last. Two comments authored in the same second still get distinct slugs, because the sequence is allocated by scanning the directory. First line is the summary, the rest is the body, like a commit message. An approved comment is FROZEN - changing an approved note is a different claim about a different state, so it needs a new comment and therefore a new publication. require_approved() blocks --publish: passing a draft exits 2 and says what to do next. Verified live. draft_comment_text() composes the comment from the measured build rather than from memory. Deal Hunter's draft is written and waiting: release-20261003-133508-01, 32 chapters, the 24/29/10 voice split, 32/32 loudness and peak gates green, and the post-approval padding note. New invariants R-8 through R-11 in spec 22. Two of my own test premises were wrong and the implementation was right - recorded in AGENTS.md because the tests were the weaker artefact. Full suite 538 passed / 8 failed, the same 8 pre-existing failures. ruff clean. Nothing was published: the Deal Hunter comment is a draft awaiting review, and voice_status.json is untouched because publishing never ran. |
||
|
B.D. Murphy
|
ed3e249d0b |
Add the publish flag: pinned release sets, immutability, the Deal Hunter lesson
The operator clarified the release model: the system keeps current and previous
state, there must be an explicit publish flag the user sets, there are approvals
for a speaker voice but none for the audiobook or chapters, the chapters that
make the m4b are the versions to flag, and a fix found after assembly causes a
change rather than editing what shipped.
Deal Hunter is the precedent, and it is measurable rather than theoretical.
Against the real package (built_at 2026-09-09T11:21:11Z), chapter 6:
ch6_full.wav 2227.26 s @ 24000 Hz what a reviewer listens to
Deal_Hunter_01_Salvage.mp3 2231.26 s @ 44100 Hz what a listener received
+4.00 s = 2 s lead + 2 s trail
plus a resample and a -20.5 dB RMS normalization. Found on 32 of 32 chapters.
The approved artifact and the published artifact were different files, and
nothing in the system recorded either fact. ar_package.json is rewritten by
every rebuild, so it is a build status, not a release record.
umbraaudio/release.py
DeliveryProfile.from_ar_package reads the profile the exporter actually used,
so the record follows the code instead of a declaration that can rot.
build_publication is read-only and pins every chapter wav, every mp3 and the
package by full SHA-256, plus a per-chapter source_unit_key_digest so a release
pins which source revision produced it, not merely which bytes.
verify_publication returns ok / drift / warnings and never silently accepts a
changed artifact. write_publication is atomic and refuses to overwrite an
existing publication id - immutability enforced by the writer, not by
convention. A later fix is edition n+1 with supersedes pointing at n.
The rule that prevents a repeat: you cannot approve a chapter and then pad it.
An approval must name the artifact a listener receives. Every record states
chapter_approved: false and audiobook_approved: false in its own words, because
that is what would have caught Deal Hunter.
The record also refuses to overstate approval. voice_status.json is an approval
LEDGER - it only holds speakers someone explicitly touched - so it has 24 entries
against 61 speakers in the profile. Reporting "24 approved" as if it meant "all
approved" would claim an approval nobody gave, so the counts are separate and the
38-speaker gap is named.
scripts/publish_release.py: dry run by default; --publish, --verify, --list,
--edition/--supersedes. Exits 2 while blockers exist.
Also found, reported and NOT fixed (specs/22 section 7 gap 1): export_m4b's
_stitch_chapter hard-codes 500 ms/0 ms silence while ChapterBuilder uses
AudioStitcher defaults and AudioConfig holds its own values, so the m4b path can
stitch a chapter with different silence than the approved one. Until that is
closed an m4b release record would pin the wrong bytes, so it is better not to
offer one yet.
And projects.status is never advanced to 'released' in any of the four live
projects - all sit at parsed/analyzed - so the publication record is currently the
only release signal in the system.
tests/test_release.py: 24 tests. Full suite 512 passed / 8 failed, the same 8
pre-existing failures. ruff clean. No export, TTS or project data was modified.
|
||
|
B.D. Murphy
|
829a03b89f |
Action the controller alignment proposal: source_unit_key + phase reports
The controller (oldbox) sent a data-model alignment proposal and asked for six
additive changes on this node, the validation protocol, and an honest report.
Six of the seven changes are done; the seventh is declared missing rather than faked.
Stage 0 - reachability verified exactly as documented: health 200, heartbeat 401
(the correct answer - auth is refusing), missing Host 400. Nothing else was sent.
source_unit_key (controller 5.1, the blocking item)
umbraaudio/source_units.py. sha256("uk1" | chapter_source_text_sha256 |
occurrence | text_sha256). Nothing positional in the key: "occurrence"
disambiguates repeated text in document order.
- 18,181/18,181 units keyed across all four projects, 0 collisions
- chapter 1 renumbered +100 -> 0 keys changed (the load-bearing proof)
- backfill 10,015 rows in 0.11s, second run stamps 0 (idempotent)
- alignment phase 3: 200/200 probe units resolved with used_index: false
- tolerates the older projects that have no chapters.chapter_index
Recorded at generation time in chapter_builder, BEFORE the rows are read and
BEFORE the marker is written, so every path that renders audio also produces
identity and the clip can name its source unit.
5.7 recorded: utterance_index is KEPT as a display ordinal and made
non-load-bearing. spec 21 §10.
Blob hashes and effect chain (5.2)
Markers now carry the full sha256 of the delivered bytes and the effects that
produced them. Verified the existing order is already correct - write_marker
runs AFTER apply_effects - and closed the remaining hole: a changed trait list
changed the hash with no recorded cause. sha256 verification is opt-in, since
hashing every file on every validation pass would re-read gigabytes.
Vocabularies published (5.4) - umbraaudio/definitions.py, 19 sets, 126 values,
each value carrying whether it is 'declared' (spec/controller) or 'extracted'
(read out of code), so a diff shows a difference of meaning, not of wording.
Alignment report tool (6.6) - scripts/alignment_report.py --phase all. Emits the
controller's envelope with the per-phase evidence shapes, and obeys the two
rules they set: no evidence means blocked, never pass; a field we cannot produce
is null PLUS a finding, never omitted. Phase 2 is deliberately 'fail' - the
three-way store disagreement is real, and a pass there would be a check that
passed by doing nothing.
Fingerprint discipline (5.5) - the drift detector reported its OWN fingerprint
change when source_unit_key was added rather than hiding it. Node profile
updated with the new fingerprints and an honest note that they moved.
Node report extended - per-project source-unit-key coverage, per-voice-asset
sha256 + object_key, definition_set_version, controller stage reached.
Six gaps found in the proposal (spec 21 §11), including the one they invited plus
five more. The sharpest: "hash at generation" ignores that audio is
post-processed afterwards, and our key is anchored to chapters.source_text - a
second copy of the manuscript - so any change to chapter storage silently
invalidates every key in that chapter.
5.6 deliberately NOT faked: personality has no versioned artifact here, only
unversioned LLM prose in speakers.json. Reported as phase 6b finding B-03
instead, because wrapping unversioned inputs in a version field would produce
artifacts claiming a lineage they do not have.
Answered the controller's two open questions (infra/controller-handoff-2026-10-03.md):
forward-only link for a revealed recurring character, and yes - a new edition may
contradict a frozen state, since this node's correction path IS a regeneration.
Full suite 484 passed / 8 failed - the same 8 pre-existing and unrelated failures.
ruff clean on every file touched. The parallel session's chapter_builder hunk
(helper_url) was filtered out of the commit and left intact in the working tree.
|
||
|
B.D. Murphy
|
893dc57574 |
Spec 21 §4.8: reuse the in-flight agent-comm mailbox as the node transport
A parallel session is building an agent-comm stdio MCP server with an append-only NAS mailbox (infra/agent_bridge/, uncommitted). Rather than proposing a second channel, spec 21 now maps the task/result/requirement envelopes onto its message kinds and adopts its sanitization rules as system rules. Also states the boundary the parallel work should not cross: a mailbox is a transport, not a scheduler. It carries tasks; leases and heartbeats must not become 'a file in a folder'. |
||
|
B.D. Murphy
|
507ed95424 |
Declare this machine an Umbra node + cross-machine system contract (spec 21)
A parallel effort is moving UmbraTrans into the main Umbra system on oldbox
(192.168.123.14). This commit makes this machine describable to that system and
records what the system still needs to decide. Everything is a proposal for the
controller to ratify; no architecture is decided here.
specs/21-umbra-system-nodes-tasks-and-changes.md
Four workstreams, each with what exists, what is missing, and a proposal:
- WS1 data model alignment / copy / verify / transition. Design is far along
already (manifest.md + spec 20); the missing piece is ratification.
- WS2 every machine is a node that gets assigned any task. Job types, leases
and pools exist (spec 19 §12); the node half does not. Proposes node
identity, capability advertisement, task/result envelopes, and refusal as a
valid audited outcome.
- WS3 cross-product change management. Nothing exists today. Proposes a
requirement catalog owned by the controller, four change classes, and a
distribute-then-confirm-with-evidence loop. Includes the worked example of a
controller-owned entity (designation, pronunciation, voice attribute) that
must apply across every product.
- WS4 UI topology. Recommends hybrid: controller gets the system portal,
the audio review UI stays on the worker because it is artifact-native and
has no authentication at all. Adds a git-tag deploy + verify + confirm flow.
Also: 25 questions grouped by who can answer, each naming what it blocks, and
a task list for this machine (NOW / NEXT / LATER) where every task has a
machine-checkable acceptance criterion.
infra/umbra-node.json
This machine's declared profile: node_id umbraaudio-m4, role worker, services,
capabilities, resource pools, constraints, project list with lifecycle, schema
fingerprints, and an honest facts_for_the_controller block (empty tables, no
stable utterance id, no auth, no shared-postgres client). No secrets - ids,
names, labels and counts only.
scripts/umbra_node_report.py
Read-only node report. Git state, listening ports, .env KEY NAMES only, per
project row counts, and a schema fingerprint hashed over sorted
table.column:type so ALTER TABLE reordering does not read as drift. --json for
machine consumption; --check compares declared profile to measured reality and
exits 2 on drift. Drift is deliberately limited to stable facts (contract
version, project set, schema fingerprints) - git HEAD and live ports are
reported but never drift.
manifest.md
Now explicitly a dated measurement with an update log, because other machines
read it as current. 2026-10-01 refresh: Last Dwarf speakers.json 70 -> 93
after the designation merge, DealHunter ref WAVs 94 -> 97. Every other §1.1
count re-verified unchanged.
Measured this machine: 4 projects, 2 distinct schema fingerprints (real
per-project drift), only 4 of 11 tables populated, 461 tests passing / 8 known
pre-existing failures. No UmbraAudio behaviour changed.
|
||
|
B.D. Murphy
|
6ca8f80087 |
Document the scene-break rule and verify the 2 s silence
Scene-break rows must carry speaker_name='Narrator'. The system requires a speaker on every utterance row, so 'scene_break' was never a valid authorial value - it was a generator artifact. Normalized 74 rows book-wide (81 total). The value is inert for rendering: is_scene_break() keys off the <<<>>> text and the scene_break type, never speaker_name, and both generation paths short-circuit before voice resolution. Verified 62/62 existing scene-break WAVs are exactly 2.000 s, matching SCENE_BREAK_SILENCE_MS. Recorded so the transient 'scene_break' speaker value is not mistaken for a deliberate authorial choice again. |
||
|
B.D. Murphy
|
e8944fcd88 |
Spec 20: enterprise DB integration contract (discovery/design only)
Design-phase contract for moving UmbraAudio + the future UmbraTrans review UI
onto the shared enterprise PostgreSQL database without interrupting audiobook
production. No database contacted, no migration SQL, no production data written.
Verified against the real schema (infra/postgres/001_control_plane.sql, 13
tables, all identity/tenancy/audit) and the full current audio codebase.
Key findings:
- No local accounts exist to migrate; the only identity traces are the literal
string 'user' in two review tables. One canonical enterprise user.
- No durable utterance identity: artifacts are named by mutable
utterance_index (ch{N}/utt_{idx:04d}.wav), which spec 18 forbids as a
durable ID. Proposes a content-addressed source_unit_key recorded at
generation time, leaving on-disk names untouched for continuity.
- BLOCKER: umbracontrol_worker matches zero RLS rows and cannot attribute an
audit event, so worker identities are unrepresentable. Needs
service_identities + job_leases, not a GUC workaround.
- RLS machinery is owned by the postgres superuser because the bootstrap pipes
the schema through psql -U postgres. FORCE ROW LEVEL SECURITY would cause
42P17 infinite recursion; recorded as a load-bearing invariant.
- 10 verified application defects, incl. download traversal via string-prefix
containment and raw upload filenames.
No UmbraAudio behaviour changes. Adapter boundary keeps SQLite+filesystem as
the default repository so cutover is a flag flip and always reversible.
|
||
|
B.D. Murphy
|
1bbbe5d92c |
Add utterance clarity signals + review queue (spec 17 Phase 1)
Persist per-utterance attribution transparency so the system can auto-accept high-confidence lines and surface only a small review queue: - DB: utterances gains attribution_evidence_kind, attribution_tag_name, attribution_uncertain, needs_review, review_reason; attribution_confidence default -> NULL. Idempotent migrations in database._migrate, routes _ensure_utterance_columns, and a new ensure in reattribute_all (raw-sqlite driver must add columns itself). - llm_attributor: accumulates scene-level mechanical evidence across batches and stamps evidence kind + tag name on attributed utterances. - New umbraaudio/analysis/clarity.py: deterministic evidence classification (name/role/pronoun/address_only/none), in-scene pronoun resolvability (gender-filtered candidate count), and composite needs_review rule: QA issues (long_monologue excluded), non-unique pronoun referent, role tag with no profile, none/address_only evidence < 0.75, unresolved uncertain flag, narrator-left dialogue, confidence < 0.7. Addressee counts only for known names/titles (guards 'But where...' false positives). - Drivers persist all fields: reattribute_all (per-chapter clarity pass), run_llm_attribution (via update_utterance_speaker), both web attribution paths. _populate_exceptions keys off needs_review (was confidence<0.7 + keyword sniff) and fixes a latent NameError (pid not in scope). - tests/test_clarity.py: 25 tests. Eval on Last Dwarf: 3918 dialogue lines -> 173 needs_review (4.4%), within the 1-5% target. Full suite 380 passed; 6 pre-existing voice_profile_manager accent failures. ruff/mypy clean on new code; routes.py baseline unchanged (73 pre-existing). |
||
|
B.D. Murphy
|
df36dad8aa |
Weekly commit: attribution QA, dramatic variations, GPU TTS, AR export
Squashes ~3 weeks of uncommitted work (2026-09-03 → 09-21). The repo is the
only durable copy; a single accidental `git checkout` on routes.py nearly lost
a month of work (recovered from an editor snapshot, not git). See AGENTS.md →
"Commit Cadence" — commit every Sunday.
Attribution & QA
- New umbraaudio/analysis/attribution_qa.py: mechanical check_attribution /
fix_high_confidence / verify_conversation_flow (speech-tag mismatch, narrator
dialogue, role-tag guessing) plus scripts/qa_attribution.py CLI.
- llm_attributor prompt rules 3e-3h (unnamed roles, pronoun coreference,
repeated lines) with wider scene/position context.
- Per-utterance QA box in chapter audio detail UI: hover issues, Recheck
(auto-fix loop) and Accept (persisted qa_accepted* + text/speaker signature,
auto re-surfaces when either changes). New POST /api/utterance/{id}/recheck
and /api/utterance/{id}/qa-accept. _ensure_utterance_columns migration fixes
legacy DBs that lacked the columns.
Generation
- Dramatic utterance variations: 26 emotions + pace + custom instruct,
composed into every TTS voice mode (clone/icl/ref/embedding) and all
generation sites; markers record the instruct so edits re-trigger.
- Staleness markers are JSON {speaker,text}; legacy plain-text markers always
stale.
- Scene-break segmentation (generation/scene_break.py) + resegment merge.
- AR package exporter (assembly/ar_export.py) with tests.
- Voice ref resolution: rglob refs, canonical alias / first-word fallback.
- Edge-TTS fallback visibility: durable voice_engines.json ledger + UI badge.
GPU / TTS server
- qwen_tts_server: instruct + clone/icl support (voice_description →
instruct_ids prepended to the talker input).
- qwen_tts.py remote path sends voice_description alongside voice_ref/slug.
Config / ops
- .env: UMBRA_TTS__QWEN_TTS_URL=http://localhost:8881.
- .gitignore: book projects, tmp_out, logs, one-off samples A/B demo.
- AGENTS.md + README.md: Commit Cadence (commit at least weekly, on Sunday).
New tests: test_qa_acceptance, test_variations, test_ar_export,
test_scene_break, test_resegment_merge, test_vocals_project. Suite: 327 pass
(6 pre-existing voice-profile accent-wording failures, unrelated).
|
||
|
B.D. Murphy
|
a172610b0e |
Add voice library, embedding-mode generation, and webUI improvements
- Add cross-project voice/name library (library.py + webUI library tabs) - chapter_builder: embedding-mode generation + auto-load TTS base model in preflight - qwen_tts: vmatrix capture, embedding/voice-prompt refinements - registry: vmatrix_path column + migration for approved voices - webUI: speakers list, chapter audio detail, proper names updates - docs: update voice generation and tts_server README |
||
|
B.D. Murphy
|
c86e323d7a |
Initial commit: UmbraAudio audiobook generation program
Quality-first multi-pass audiobook generation from English manuscripts. Includes: - umbraaudio/ core package (pipeline, config, voices, web review UI) - scripts/ CLI utilities + model download script - tts_server/ remote TTS server - specs/ design docs, tests/, pyproject.toml + uv.lock - Qwen3-TTS git submodule (pinned 022e286) with local demo.py UI patch saved at scripts/qwen3_tts_demo_umbra_patch.patch - .env with runtime config/secrets Excluded (see .gitignore): models/ (downloaded via scripts/download_models.sh), output/, logs/, .venv, .umbra runtime data, and book projects (DealHunterShip_*), which are committed separately. |