Record the paragraph-boundary drift findings and the verification stage
Also records the negative result from the independent cross-check attempt: names are not a drift signal, and the glossary-anchor detector conflates drift with ordinary glossary non-compliance. Both would otherwise be tried again.
This commit is contained in:
43
AGENTS.md
43
AGENTS.md
@@ -287,6 +287,49 @@ Each pass has quality gates. Quality is the primary concern; generation speed is
|
|||||||
- **Spec 27 updated** with the measured findings, the production pipeline table, and the
|
- **Spec 27 updated** with the measured findings, the production pipeline table, and the
|
||||||
alignment-pass description. UmbraAudio commits `c476cfe` + `301ff17`, pushed to
|
alignment-pass description. UmbraAudio commits `c476cfe` + `301ff17`, pushed to
|
||||||
`minieq` (`ssh://git@mini-eq:2222/umbragit/UmbraAudio.git`).
|
`minieq` (`ssh://git@mini-eq:2222/umbragit/UmbraAudio.git`).
|
||||||
|
- **THE PRODUCTION CHINESE HAS A PARAGRAPH-BOUNDARY DRIFT, and the alignment pass
|
||||||
|
cannot see it by construction.** Pass 0 splits the original at sentence boundaries, so
|
||||||
|
many paragraphs are **sentence fragments** ("The chair in the middle has", `"operations"`,
|
||||||
|
"on the console."): measured **439 of 3,038 translated paragraphs (14.5%)**. A fragment is
|
||||||
|
exactly where an intent-based drafter is told to write natural target-language prose, and
|
||||||
|
once it does that the id-to-text correspondence drifts - the Chinese is good, but attached
|
||||||
|
to the wrong paragraph. **In ch-15 the drift runs p0094-p0104 (11 paragraphs):** p0102's
|
||||||
|
Chinese ("上一次有人登上这艘舰桥...") is p0101's translation, and p0104's Chinese opens with
|
||||||
|
p0103's sentence. The alignment pass *cannot* see it: a one-to-one paragraph is a pure copy,
|
||||||
|
so there is nothing to divide and nothing to disagree.
|
||||||
|
- **The verification stage (`align.py --verify`) finds it, but only after two rounds of
|
||||||
|
measurement.** The first version asked two yes/no questions per paragraph ("is anything
|
||||||
|
missing", "is anything extra") and was wrong in **both** directions: it flagged 5 paragraphs
|
||||||
|
that were fine (it called the adverb `then` missing from a translation that has it, and
|
||||||
|
`Enthusiastically` missing from one that has it - the model was reading the *English* and
|
||||||
|
not the Chinese), and it missed 2 paragraphs that are drifted. **5 of 15 flags were wrong.**
|
||||||
|
The stage now asks **which paragraph does this translation render** (aligned / incomplete /
|
||||||
|
drift), shows the model its **neighbouring** paragraphs, and judges the paragraphs as a
|
||||||
|
**window** rather than one at a time - because the model calls the two paragraphs *before* a
|
||||||
|
real drift "aligned" when their Chinese is demonstrably the neighbour's, and the start of a
|
||||||
|
drift is not knowable from any single paragraph. On the real ch-15 window it now finds
|
||||||
|
exactly the drift: **10 paragraphs dropped (p0095-p0104)**, p0094 kept and flagged because
|
||||||
|
its Chinese correctly renders its first two sentences. Only `drift` deletes a line.
|
||||||
|
**The cost is explicit:** a fragment inside a broken window can be dropped on an unreliable
|
||||||
|
verdict, so the audiobook has a gap there. The alternative is a line that belongs to a
|
||||||
|
different sentence, and a gap is visible while a wrong line is silent.
|
||||||
|
- **A dropped paragraph still gets a row:** `voiced=false` + the reason, so the gap says *why*
|
||||||
|
rather than looking like a paragraph nobody translated. Verified end-to-end: the ch-15
|
||||||
|
delivery imports 154 rows, 13 of them not-voiced with the reason in `review_reason`.
|
||||||
|
- **An independent deterministic cross-check was tried and FAILED - names are not a drift
|
||||||
|
signal.** A glossary-anchor detector found 179 candidate paragraphs, and a name-anchor
|
||||||
|
detector found **1**, which was a false positive (word order: 公主凯因达 vs 凯因达公主).
|
||||||
|
The ch-15 drift window contains no names at all. Do not repeat the glossary-anchor
|
||||||
|
detector: it conflates drift with ordinary glossary non-compliance (the model renders
|
||||||
|
"Warbots" as 战博特 as often as 战斗机器人).
|
||||||
|
- **The root cause is upstream and is now fixed in the prompts** (UmbraTrans `5433653`): the
|
||||||
|
drafter is told each id may cover only its own English, even where the source is cut
|
||||||
|
mid-sentence, and must not move a neighbour's content in or its own content out; the critic
|
||||||
|
is asked to check the id-to-text correspondence (new type `misalignment`, severity
|
||||||
|
`block`). The critic cannot repair the draft - the pipeline records critic issues rather
|
||||||
|
than revising - but the drift becomes visible in the run record instead of silent.
|
||||||
|
- **UmbraTrans commits pushed to `minieq`:** `acab79d` (window judging), `11fd8d8` (gap rows
|
||||||
|
keep the reason), `5433653` (drafter + critic prompts). Master = `5433653`.
|
||||||
- **UmbraAudio suite: 890 passed / 8 failed / 6 skipped** - the same 8 pre-existing failures
|
- **UmbraAudio suite: 890 passed / 8 failed / 6 skipped** - the same 8 pre-existing failures
|
||||||
(2 `test_pronunciation`, 6 `voice_profile_manager` accent); +9 new tests, no regressions.
|
(2 `test_pronunciation`, 6 `voice_profile_manager` accent); +9 new tests, no regressions.
|
||||||
Web server restarted; all pages return 200 (auth gate intact, `/` -> 303 `/login`).
|
Web server restarted; all pages return 200 (auth gate intact, `/` -> 303 `/login`).
|
||||||
|
|||||||
Reference in New Issue
Block a user