The pipeline was hard-coded to Simplified Chinese in three places: the
drafter was told to translate into Chinese, the critic was told to compare
English with Chinese, and the glossary's translated forms were read from a
"zh" key. A production that declared target_language: es-ES was therefore
translated into Chinese and filed under a Spanish name - measured on oldbox,
deal-hunter-es held 3,038 paragraphs of Chinese and was reported complete.
A new registry, tools/translation/languages.py, is the single source of
truth for what a target language is: its names, its script, its punctuation,
its glossary key, its typography and its chapter-numbering convention. Ten
languages are registered. The prompt scaffolding stays in Chinese - that is
what the models read best - and the target is named with its Chinese, English
and native names so there is no room to misread it.
The registry now reaches every stage:
- prompts.py: the analyst, drafter, critic and glossary blocks take the
target. The hard rules about this book - Deal Hunter is the ship's name,
the AI's name and the book's title, so it stays in English - are the
language's own table for Chinese and the global rules for everything else,
so a language with no table does not lose them. The Chinese rules are
byte-for-byte what they were.
- pipeline.py: --target-language, a run-directory guard that refuses to
reuse a directory belonging to another language (from the manifest, or by
measuring the text it already holds), a draft-keyed-by-target reader, and
a refusal of a draft that came back in the wrong script.
- book.py: per-language chapter-title and subtitle tables. A language with no
table falls back to the English title, never to another language's; a
language with a table and a missing chapter gets a visible marker in its
own language.
- qa.py: the target's quote marks, punctuation, script count and forbidden
title forms. The Latin-residue check is skipped for a Latin target, where
it would flag the whole book.
- align.py: the aligner and verifier prompts name the target, and the
sentence and clause splitting uses the target's own terminators.
- render.py: the fonts, the language and country codes, the writing
direction, the chapter headings and the fixed editorial strings around the
translation are the target's. The Latin-run span is skipped for a Latin
target, where marking the text as English would tell the renderer it is
English.
- project_status.py: the dashboard measures a sample of a production's text
and reports a production whose text is not in the language it declares as
"mismatched", not "complete". A chapter count is a claim about the files;
the script check is a measurement of the text.
Also fixes: the dashboard gated every language's production on the source
production existing, so a Spanish production could never be reported even
when it was there.
The Chinese render is byte-identical to before on the real book, and the
Chinese rules are unchanged. Suite: 620 passed, the same 68 pre-existing
environment failures, +49 new tests.
The drift in the production Chinese is the drafter's, not the aligner's: the
drafter re-flows the passage across paragraph ids wherever Pass 0 split the
original mid-sentence, and the alignment pass cannot see it by construction.
So the prevention belongs in the prompts.
The drafter is now told, explicitly, that each id's translation may cover
only that id's own English - even where the source is cut mid-sentence and
the Chinese therefore reads as a fragment - and that it must not move a
neighbour's content into this id or this id's content into the next. The
instruction gives the concrete case the model actually does: "The chair in
the middle has" followed by "\"Captain\"".
The critic is now asked to check the id-to-text correspondence, with a new
issue type "misalignment" and a block severity. It cannot repair the draft
- the pipeline records critic issues rather than revising the translation -
but it makes the drift visible in the run record instead of silent.
4 new tests, 63 total, ruff clean.
A paragraph whose translation could not be verified produced no rows at
all, which is indistinguishable from one that was never translated: the
review lane showed a missing utterance with no explanation.
The project's own rule for a gap is "the row is kept, the gap is visible,
no clip is generated", and it already applies to the collapse orphans. An
unverifiable paragraph is the same kind of gap, so it now gets the same
treatment: a voiced=false row carrying the reason.
Measured on the real ch-15 window: the ten drifted paragraphs now produce
thirteen explicit not-voiced rows instead of vanishing, and the reviewer
can see that the translation was dropped because it renders another
paragraph rather than because nobody translated it.
60 tests, ruff clean.
The first version of the drift check asked two yes/no questions per
paragraph: is anything missing, is anything extra. Measured on the real
ch-15 window it was wrong in both directions:
* It flagged five paragraphs that were fine. The model reported "the adverb
'then' is missing" and "the adverb 'Enthusiastically' is missing" on
translations that contain both, and it called a singular/plural difference
a missing sentence. A stage whose action is to delete a line cannot be
allowed to over-report at that rate: five of fifteen flags were wrong.
* It missed two paragraphs that are drifted, because those paragraphs are
short and their Chinese looks like a plausible rendering of their own
English when seen alone.
So the check now asks a different question - which paragraph does this
translation actually render - and shows the model the neighbouring
paragraphs, because that question cannot be answered without them. The
verdict is one of aligned / incomplete / drift, and only a translation that
belongs to another paragraph is dropped.
The paragraphs are then judged as a window. A re-flowed passage cannot be
judged one paragraph at a time: the model calls the two paragraphs before a
real drift "aligned" when their Chinese is demonstrably the neighbouring
paragraph's, and the start of a drift is not knowable from any single
paragraph. Inside a window that contains a confirmed drift, a fragment the
model called "aligned" is dropped as well; one it called "incomplete" keeps
its text and is flagged, because that verdict says the translation really
does render its own leading content.
On the real ch-15 window this now finds exactly the drift: ten paragraphs
(p0095-p0104) dropped, and p0094 kept and flagged because its Chinese is a
correct translation of its first two sentences. The earlier version dropped
p0094 too, losing two good lines, and kept the two drifted paragraphs the
model wrongly called aligned.
The cost of the choice is explicit: a fragment inside a broken window can be
dropped on an unreliable verdict, so the audiobook has a gap there. The
alternative is a line in the audiobook that belongs to a different sentence.
58 tests, ruff clean.
Measured on the real Chinese edition: the drafter sometimes re-flows the
passage across paragraph ids. Pass 0 splits the original passage at sentence
boundaries, so many paragraphs are sentence fragments ("The chair in the
middle has", ""operations""). A fragment is exactly where an intent-based
drafter is told to write natural target-language prose, and once it does
that, the id-to-text correspondence drifts: the Chinese is good, but it is
attached to the wrong paragraph.
In ch-15 the drift runs from p0095 to p0104 - ten paragraphs - and the
Chinese for p0102 ("Yes, Captain.") is the translation of p0101. The audio
would say the right words at the wrong place.
The alignment pass could not see it. A one-to-one paragraph is a pure copy,
so there is nothing to divide and nothing to disagree; the drift is invisible
to an alignment check by construction.
--verify now checks the paragraphs where the drift is possible: it starts at
a sentence fragment and follows the drift until the passage re-synchronises.
The check asks two questions - is content missing, and is content extra - and
a paragraph that carries content its source does not have is marked
"mismatch" and dropped from the delivery. A visible gap is better than a
confident wrong line in the audiobook.
6 new tests, 51 total, ruff clean.
The client writes each answer as it arrives, so a cache directory that was
never created turns every model call into an error - and the fallback would
quietly turn the whole book into a mechanical split while the run still
reported success.
The directory is now created by the command line entry point, and the client
is constructed with it rather than having it assigned afterwards. A test
pins it.
45 tests, ruff clean.
The translation pipeline works on paragraphs, because epub and pdf are
paragraph media. The audiobook needs utterances: UmbraAudio has verified who
speaks every one of them, and each clip is voiced separately by that
speaker's voice. A paragraph-level translation cannot be spoken.
This is the pass that was missing. tools/translation/README.md said so
plainly: "It does not produce a 1:1 utterance alignment for TTS ... splitting
or aligning back to the 4,319 utterances is a separate pass."
The division is by meaning, not by position. An intent-based translation
merges and splits sentences for the target language's sake, so the sentence
counts do not line up on their own - that is the whole point of Passes 1-2.
A model is given the source utterances and the translation and asked which
part of the translation renders which source utterance.
Two rules the model cannot break:
* a source utterance may only be absorbed into another utterance by the same
speaker - a line must not be handed to a different character;
* the partition must reproduce the translated text exactly, modulo
whitespace, so the alignment can never rewrite the translation.
An answer that drops, duplicates or invents text is rejected and the
paragraph falls back to a mechanical split, which is reported as needing
review. A source utterance the target does not voice is delivered as
voiced=false with the reason, which is what UmbraAudio's
utterance_translations.voiced flag is for: the row is kept so the gap stays
visible, and no clip is generated for it.
The output is a delivery payload in the shape UmbraAudio's
translations.import_translations already accepts, so neither side has to
invent a second contract.
Measured on the real book, 42 sampled multi-utterance paragraphs across seven
chapters: 41 aligned by the model, 1 fell back to the mechanical split, 0
partition mismatches. The one fallback was a model answer that did not
reproduce the translated text; the content check caught it.
44 tests, ruff clean.
Add REST endpoints and frontend wiring for per-language project actions:
- POST /api/workspace/{project}/translation/{lang}/queue — queue translation
- POST /api/workspace/{project}/audio/{lang}/queue — queue audio generation
- GET/POST /api/workspace/{project}/format/{lang} — format metadata (EPUB/PDF)
- GET /api/workspace/{project}/review/{lang}/chapters/{chapter} — utterance review
- GET/POST /api/workspace/{project}/front-matter/{lang} — front matter editor
- GET /api/workspace/{project}/chapters/{lang} — chapter list
- GET /api/workspace/{project}/chapter-content/{lang}/{chapter} — chapter text
Frontend fixes:
- Review page shows per-language text (English source, Chinese translation, etc.)
- Non-translated languages show 'Queue translation' button only
- Speaker data from English source matched by paragraph ID
- UmbraAudio column layout: Speaker, Variation, Text, Actions
- Chapter tree in left sidebar: Front matter, Back matter, Body
- Audio Format label at top of sidebar
- Pandoc formatting toolbar for front matter text editing
