From 25605e2cad4d2d46558189877e5c6410d86e9731 Mon Sep 17 00:00:00 2001 From: Umbra Tools Date: Fri, 6 Nov 2026 21:40:37 -0800 Subject: [PATCH] Record the paragraph-boundary drift findings and the verification stage Also records the negative result from the independent cross-check attempt: names are not a drift signal, and the glossary-anchor detector conflates drift with ordinary glossary non-compliance. Both would otherwise be tried again. --- AGENTS.md | 43 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 43 insertions(+) diff --git a/AGENTS.md b/AGENTS.md index 5893e75..2a5a5e5 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -287,6 +287,49 @@ Each pass has quality gates. Quality is the primary concern; generation speed is - **Spec 27 updated** with the measured findings, the production pipeline table, and the alignment-pass description. UmbraAudio commits `c476cfe` + `301ff17`, pushed to `minieq` (`ssh://git@mini-eq:2222/umbragit/UmbraAudio.git`). +- **THE PRODUCTION CHINESE HAS A PARAGRAPH-BOUNDARY DRIFT, and the alignment pass + cannot see it by construction.** Pass 0 splits the original at sentence boundaries, so + many paragraphs are **sentence fragments** ("The chair in the middle has", `"operations"`, + "on the console."): measured **439 of 3,038 translated paragraphs (14.5%)**. A fragment is + exactly where an intent-based drafter is told to write natural target-language prose, and + once it does that the id-to-text correspondence drifts - the Chinese is good, but attached + to the wrong paragraph. **In ch-15 the drift runs p0094-p0104 (11 paragraphs):** p0102's + Chinese ("上一次有人登上这艘舰桥...") is p0101's translation, and p0104's Chinese opens with + p0103's sentence. The alignment pass *cannot* see it: a one-to-one paragraph is a pure copy, + so there is nothing to divide and nothing to disagree. +- **The verification stage (`align.py --verify`) finds it, but only after two rounds of + measurement.** The first version asked two yes/no questions per paragraph ("is anything + missing", "is anything extra") and was wrong in **both** directions: it flagged 5 paragraphs + that were fine (it called the adverb `then` missing from a translation that has it, and + `Enthusiastically` missing from one that has it - the model was reading the *English* and + not the Chinese), and it missed 2 paragraphs that are drifted. **5 of 15 flags were wrong.** + The stage now asks **which paragraph does this translation render** (aligned / incomplete / + drift), shows the model its **neighbouring** paragraphs, and judges the paragraphs as a + **window** rather than one at a time - because the model calls the two paragraphs *before* a + real drift "aligned" when their Chinese is demonstrably the neighbour's, and the start of a + drift is not knowable from any single paragraph. On the real ch-15 window it now finds + exactly the drift: **10 paragraphs dropped (p0095-p0104)**, p0094 kept and flagged because + its Chinese correctly renders its first two sentences. Only `drift` deletes a line. + **The cost is explicit:** a fragment inside a broken window can be dropped on an unreliable + verdict, so the audiobook has a gap there. The alternative is a line that belongs to a + different sentence, and a gap is visible while a wrong line is silent. +- **A dropped paragraph still gets a row:** `voiced=false` + the reason, so the gap says *why* + rather than looking like a paragraph nobody translated. Verified end-to-end: the ch-15 + delivery imports 154 rows, 13 of them not-voiced with the reason in `review_reason`. +- **An independent deterministic cross-check was tried and FAILED - names are not a drift + signal.** A glossary-anchor detector found 179 candidate paragraphs, and a name-anchor + detector found **1**, which was a false positive (word order: 公主凯因达 vs 凯因达公主). + The ch-15 drift window contains no names at all. Do not repeat the glossary-anchor + detector: it conflates drift with ordinary glossary non-compliance (the model renders + "Warbots" as 战博特 as often as 战斗机器人). +- **The root cause is upstream and is now fixed in the prompts** (UmbraTrans `5433653`): the + drafter is told each id may cover only its own English, even where the source is cut + mid-sentence, and must not move a neighbour's content in or its own content out; the critic + is asked to check the id-to-text correspondence (new type `misalignment`, severity + `block`). The critic cannot repair the draft - the pipeline records critic issues rather + than revising - but the drift becomes visible in the run record instead of silent. +- **UmbraTrans commits pushed to `minieq`:** `acab79d` (window judging), `11fd8d8` (gap rows + keep the reason), `5433653` (drafter + critic prompts). Master = `5433653`. - **UmbraAudio suite: 890 passed / 8 failed / 6 skipped** - the same 8 pre-existing failures (2 `test_pronunciation`, 6 `voice_profile_manager` accent); +9 new tests, no regressions. Web server restarted; all pages return 200 (auth gate intact, `/` -> 303 `/login`).