996fe37f4f
The translation pipeline works on paragraphs, because epub and pdf are paragraph media. The audiobook needs utterances: UmbraAudio has verified who speaks every one of them, and each clip is voiced separately by that speaker's voice. A paragraph-level translation cannot be spoken. This is the pass that was missing. tools/translation/README.md said so plainly: "It does not produce a 1:1 utterance alignment for TTS ... splitting or aligning back to the 4,319 utterances is a separate pass." The division is by meaning, not by position. An intent-based translation merges and splits sentences for the target language's sake, so the sentence counts do not line up on their own - that is the whole point of Passes 1-2. A model is given the source utterances and the translation and asked which part of the translation renders which source utterance. Two rules the model cannot break: * a source utterance may only be absorbed into another utterance by the same speaker - a line must not be handed to a different character; * the partition must reproduce the translated text exactly, modulo whitespace, so the alignment can never rewrite the translation. An answer that drops, duplicates or invents text is rejected and the paragraph falls back to a mechanical split, which is reported as needing review. A source utterance the target does not voice is delivered as voiced=false with the reason, which is what UmbraAudio's utterance_translations.voiced flag is for: the row is kept so the gap stays visible, and no clip is generated for it. The output is a delivery payload in the shape UmbraAudio's translations.import_translations already accepts, so neither side has to invent a second contract. Measured on the real book, 42 sampled multi-utterance paragraphs across seven chapters: 41 aligned by the model, 1 fell back to the mechanical split, 0 partition mismatches. The one fallback was a model answer that did not reproduce the translated text; the content check caught it. 44 tests, ruff clean.
6.8 KiB
Deal Hunter Chinese translation tools
This directory contains the translation and rendering pipeline used to produce the Simplified Chinese edition of Deal Hunter. It is deliberately separate from the web application: the book text and the translations are copyrighted and are not committed to this repository.
Where the data lives
productions/deal-hunter-zh/ # git-ignored
source/book.json # extracted English source model
glossary/glossary.json # names, places, terms, style rules
runs/<chapter>/*.json # analyses, drafts, critiques, final text
runs/usage.jsonl # per-call token/latency log
output/*.asciidoc, *.odt, *.pdf # review artefacts
The source bundle is the UmbraAudio controller bundle:
/home/umbra/audio-read/bundles/dh-bundle/manifest.jsonl
Its sha256 is recorded in Status.txt and in source/book.json.
Passes
| Pass | What it does | Model |
|---|---|---|
| 0 | Paragraph/scene/chunk assembly; keeps source utterance indices | none |
| 1 | Scene intent: summary, register, tone, participants, risks | lfm2.5-2.6b |
| 2 | Context-aware draft translation with glossary and analysis | qwen3.5-9b |
| 3 | Chapter voice/term consistency (optional) | critic model |
| 4 | Bilingual critic: meaning, pronoun, irony, term, omission | deepseek-r1-8b, fallback qwen3.5-9b |
| 5 | Align the translation back onto the verified source utterances | drafter model |
Every analysis, draft and critique is a file under runs/<chapter>/. A run can
be stopped and resumed; existing artefacts are reused unless --force is
given. LLM responses are cached by prompt hash, so editing a prompt
invalidates the old answer.
The drafter is told to translate intent, not words, and is given a glossary, character voices, the scene analysis, and the previous chunk's summary. The critic uses a different model and returns a severity-sorted issue list; it does not rewrite the text.
Pass 5 - aligning back to the utterances
Passes 0-4 work on paragraphs, because epub and pdf are paragraph media. The audiobook needs utterances: UmbraAudio has already verified who speaks every one of them, and every clip is voiced separately by that speaker's voice. So the last pass divides each translated paragraph between the source utterances it was built from.
The division is by meaning, not by position. An intent-based translation merges and splits sentences for the target language's sake, so the sentence counts do not line up on their own - that is the whole point of Passes 1-2. A model is given the source utterances and the translation and asked which part of the translation renders which source utterance.
Two rules the model cannot break:
- a source utterance may only be absorbed into another utterance by the same speaker - a line must not be handed to a different character;
- a scene break is silence, and silence has no translation.
The partition is checked against the translated text: the segments must reproduce it exactly once whitespace is removed. A model answer that drops, duplicates or invents text is rejected and the paragraph falls back to a mechanical split, which is reported as needing review. The alignment never rewrites the translation - it only decides where the boundaries fall.
When the target absorbed a source utterance, that utterance is recorded as
not voiced with the reason. UmbraAudio's utterance_translations.voiced
flag is what carries it: the row is kept so the gap is visible and
reconstructable, and no clip is generated for it.
python3 tools/translation/align.py \
--book productions/deal-hunter-zh/source/book.json \
--run-dir productions/deal-hunter-zh/runs \
--bundle /home/umbra/audio-read/bundles/dh-bundle/manifest.jsonl \
--out productions/deal-hunter-zh/aligned/delivery.json \
--chapters all
The output is a delivery payload in the shape UmbraAudio's
translations.import_translations accepts, so the translation can be
ingested without either side inventing a second contract:
python scripts/import_translations.py --project "DealHunterShip_4.0e_whole-to-audio" \
--delivery productions/deal-hunter-zh/aligned/delivery.json --write
--no-llm runs the mechanical split only. It never loses text, but it cannot
tell which clause renders which English sentence, so everything it touches is
reported as needing review.
Commands
Extract the source:
python3 tools/translation/book.py \
--out productions/deal-hunter-zh/source/book.json
Translate a chapter:
python3 tools/translation/pipeline.py \
--book productions/deal-hunter-zh/source/book.json \
--glossary productions/deal-hunter-zh/glossary/glossary.json \
--run-dir productions/deal-hunter-zh/runs \
--chapters ch-6
Selectors: all, front, body, back, or a comma-separated list of
chapter indices / ids / titles (-4,-3,-2,-1 must be quoted or passed as
--chapters=-4,-3).
Render:
python3 tools/translation/render.py \
--book productions/deal-hunter-zh/source/book.json \
--run-dir productions/deal-hunter-zh/runs \
--out-dir productions/deal-hunter-zh/output \
--sample
Mechanical QA:
python3 tools/translation/qa.py \
--book productions/deal-hunter-zh/source/book.json \
--run-dir productions/deal-hunter-zh/runs \
--json-out productions/deal-hunter-zh/runs/qa.json \
--strict
Models
The default server is the LAN LM Studio instance at
http://192.168.123.16:1234/v1. Override with:
UMBRA_TRANSLATION_BASE_URL
UMBRA_TRANSLATION_MODEL
UMBRA_TRANSLATION_ANALYST_MODEL
UMBRA_TRANSLATION_CRITIC_MODEL
Qwen models emit reasoning tokens by default; the client sends
reasoning_effort: "none" so the budget is spent on the translation. The small
analyst model rejects LM Studio's JSON-schema grammar, so its answer is parsed
from text and repaired if necessary.
Rendering
The AsciiDoc file is the canonical text output. The ODT is produced from a Flat
ODF document (.fodt) so that page layout, styles and the table of contents are
under our control. LibreOffice converts it and updates the TOC through UNO; a
plain --convert-to leaves the index empty. A PDF is exported for quick review.
Front matter, body and back matter use different styles. Chapter headings use
Chinese numerals (第一章 打捞). Body text is 11 pt Noto Serif CJK SC with
1.5 line spacing and a two-character first-line indent.
What this is not
- It is not a native-speaker-reviewed translation. The critic and the QA script catch mechanical defects; a human editor is still required.
- It does not recover the original DOCX paragraphing. The bundle is sentence-level, so paragraphs are reconstructed and can be reviewed.
- It does not produce the audio. The alignment pass hands UmbraAudio one row per utterance; voicing, loudness and the audiobook masters are UmbraAudio's job.