Record the paragraph-boundary drift findings and the verification stage

Also records the negative result from the independent cross-check attempt:
names are not a drift signal, and the glossary-anchor detector conflates
drift with ordinary glossary non-compliance. Both would otherwise be
tried again.
This commit is contained in:
Umbra Tools
2026-11-06 21:40:37 -08:00
parent 31cd9f7ce4
commit 25605e2cad

View File

@@ -287,6 +287,49 @@ Each pass has quality gates. Quality is the primary concern; generation speed is
- **Spec 27 updated** with the measured findings, the production pipeline table, and the - **Spec 27 updated** with the measured findings, the production pipeline table, and the
alignment-pass description. UmbraAudio commits `c476cfe` + `301ff17`, pushed to alignment-pass description. UmbraAudio commits `c476cfe` + `301ff17`, pushed to
`minieq` (`ssh://git@mini-eq:2222/umbragit/UmbraAudio.git`). `minieq` (`ssh://git@mini-eq:2222/umbragit/UmbraAudio.git`).
- **THE PRODUCTION CHINESE HAS A PARAGRAPH-BOUNDARY DRIFT, and the alignment pass
cannot see it by construction.** Pass 0 splits the original at sentence boundaries, so
many paragraphs are **sentence fragments** ("The chair in the middle has", `"operations"`,
"on the console."): measured **439 of 3,038 translated paragraphs (14.5%)**. A fragment is
exactly where an intent-based drafter is told to write natural target-language prose, and
once it does that the id-to-text correspondence drifts - the Chinese is good, but attached
to the wrong paragraph. **In ch-15 the drift runs p0094-p0104 (11 paragraphs):** p0102's
Chinese ("上一次有人登上这艘舰桥...") is p0101's translation, and p0104's Chinese opens with
p0103's sentence. The alignment pass *cannot* see it: a one-to-one paragraph is a pure copy,
so there is nothing to divide and nothing to disagree.
- **The verification stage (`align.py --verify`) finds it, but only after two rounds of
measurement.** The first version asked two yes/no questions per paragraph ("is anything
missing", "is anything extra") and was wrong in **both** directions: it flagged 5 paragraphs
that were fine (it called the adverb `then` missing from a translation that has it, and
`Enthusiastically` missing from one that has it - the model was reading the *English* and
not the Chinese), and it missed 2 paragraphs that are drifted. **5 of 15 flags were wrong.**
The stage now asks **which paragraph does this translation render** (aligned / incomplete /
drift), shows the model its **neighbouring** paragraphs, and judges the paragraphs as a
**window** rather than one at a time - because the model calls the two paragraphs *before* a
real drift "aligned" when their Chinese is demonstrably the neighbour's, and the start of a
drift is not knowable from any single paragraph. On the real ch-15 window it now finds
exactly the drift: **10 paragraphs dropped (p0095-p0104)**, p0094 kept and flagged because
its Chinese correctly renders its first two sentences. Only `drift` deletes a line.
**The cost is explicit:** a fragment inside a broken window can be dropped on an unreliable
verdict, so the audiobook has a gap there. The alternative is a line that belongs to a
different sentence, and a gap is visible while a wrong line is silent.
- **A dropped paragraph still gets a row:** `voiced=false` + the reason, so the gap says *why*
rather than looking like a paragraph nobody translated. Verified end-to-end: the ch-15
delivery imports 154 rows, 13 of them not-voiced with the reason in `review_reason`.
- **An independent deterministic cross-check was tried and FAILED - names are not a drift
signal.** A glossary-anchor detector found 179 candidate paragraphs, and a name-anchor
detector found **1**, which was a false positive (word order: 公主凯因达 vs 凯因达公主).
The ch-15 drift window contains no names at all. Do not repeat the glossary-anchor
detector: it conflates drift with ordinary glossary non-compliance (the model renders
"Warbots" as 战博特 as often as 战斗机器人).
- **The root cause is upstream and is now fixed in the prompts** (UmbraTrans `5433653`): the
drafter is told each id may cover only its own English, even where the source is cut
mid-sentence, and must not move a neighbour's content in or its own content out; the critic
is asked to check the id-to-text correspondence (new type `misalignment`, severity
`block`). The critic cannot repair the draft - the pipeline records critic issues rather
than revising - but the drift becomes visible in the run record instead of silent.
- **UmbraTrans commits pushed to `minieq`:** `acab79d` (window judging), `11fd8d8` (gap rows
keep the reason), `5433653` (drafter + critic prompts). Master = `5433653`.
- **UmbraAudio suite: 890 passed / 8 failed / 6 skipped** - the same 8 pre-existing failures - **UmbraAudio suite: 890 passed / 8 failed / 6 skipped** - the same 8 pre-existing failures
(2 `test_pronunciation`, 6 `voice_profile_manager` accent); +9 new tests, no regressions. (2 `test_pronunciation`, 6 `voice_profile_manager` accent); +9 new tests, no regressions.
Web server restarted; all pages return 200 (auth gate intact, `/` -> 303 `/login`). Web server restarted; all pages return 200 (auth gate intact, `/` -> 303 `/login`).