Compare commits
2 Commits
6bf0baa2f8
...
25605e2cad
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
25605e2cad | ||
|
|
31cd9f7ce4 |
43
AGENTS.md
43
AGENTS.md
@@ -287,6 +287,49 @@ Each pass has quality gates. Quality is the primary concern; generation speed is
|
||||
- **Spec 27 updated** with the measured findings, the production pipeline table, and the
|
||||
alignment-pass description. UmbraAudio commits `c476cfe` + `301ff17`, pushed to
|
||||
`minieq` (`ssh://git@mini-eq:2222/umbragit/UmbraAudio.git`).
|
||||
- **THE PRODUCTION CHINESE HAS A PARAGRAPH-BOUNDARY DRIFT, and the alignment pass
|
||||
cannot see it by construction.** Pass 0 splits the original at sentence boundaries, so
|
||||
many paragraphs are **sentence fragments** ("The chair in the middle has", `"operations"`,
|
||||
"on the console."): measured **439 of 3,038 translated paragraphs (14.5%)**. A fragment is
|
||||
exactly where an intent-based drafter is told to write natural target-language prose, and
|
||||
once it does that the id-to-text correspondence drifts - the Chinese is good, but attached
|
||||
to the wrong paragraph. **In ch-15 the drift runs p0094-p0104 (11 paragraphs):** p0102's
|
||||
Chinese ("上一次有人登上这艘舰桥...") is p0101's translation, and p0104's Chinese opens with
|
||||
p0103's sentence. The alignment pass *cannot* see it: a one-to-one paragraph is a pure copy,
|
||||
so there is nothing to divide and nothing to disagree.
|
||||
- **The verification stage (`align.py --verify`) finds it, but only after two rounds of
|
||||
measurement.** The first version asked two yes/no questions per paragraph ("is anything
|
||||
missing", "is anything extra") and was wrong in **both** directions: it flagged 5 paragraphs
|
||||
that were fine (it called the adverb `then` missing from a translation that has it, and
|
||||
`Enthusiastically` missing from one that has it - the model was reading the *English* and
|
||||
not the Chinese), and it missed 2 paragraphs that are drifted. **5 of 15 flags were wrong.**
|
||||
The stage now asks **which paragraph does this translation render** (aligned / incomplete /
|
||||
drift), shows the model its **neighbouring** paragraphs, and judges the paragraphs as a
|
||||
**window** rather than one at a time - because the model calls the two paragraphs *before* a
|
||||
real drift "aligned" when their Chinese is demonstrably the neighbour's, and the start of a
|
||||
drift is not knowable from any single paragraph. On the real ch-15 window it now finds
|
||||
exactly the drift: **10 paragraphs dropped (p0095-p0104)**, p0094 kept and flagged because
|
||||
its Chinese correctly renders its first two sentences. Only `drift` deletes a line.
|
||||
**The cost is explicit:** a fragment inside a broken window can be dropped on an unreliable
|
||||
verdict, so the audiobook has a gap there. The alternative is a line that belongs to a
|
||||
different sentence, and a gap is visible while a wrong line is silent.
|
||||
- **A dropped paragraph still gets a row:** `voiced=false` + the reason, so the gap says *why*
|
||||
rather than looking like a paragraph nobody translated. Verified end-to-end: the ch-15
|
||||
delivery imports 154 rows, 13 of them not-voiced with the reason in `review_reason`.
|
||||
- **An independent deterministic cross-check was tried and FAILED - names are not a drift
|
||||
signal.** A glossary-anchor detector found 179 candidate paragraphs, and a name-anchor
|
||||
detector found **1**, which was a false positive (word order: 公主凯因达 vs 凯因达公主).
|
||||
The ch-15 drift window contains no names at all. Do not repeat the glossary-anchor
|
||||
detector: it conflates drift with ordinary glossary non-compliance (the model renders
|
||||
"Warbots" as 战博特 as often as 战斗机器人).
|
||||
- **The root cause is upstream and is now fixed in the prompts** (UmbraTrans `5433653`): the
|
||||
drafter is told each id may cover only its own English, even where the source is cut
|
||||
mid-sentence, and must not move a neighbour's content in or its own content out; the critic
|
||||
is asked to check the id-to-text correspondence (new type `misalignment`, severity
|
||||
`block`). The critic cannot repair the draft - the pipeline records critic issues rather
|
||||
than revising - but the drift becomes visible in the run record instead of silent.
|
||||
- **UmbraTrans commits pushed to `minieq`:** `acab79d` (window judging), `11fd8d8` (gap rows
|
||||
keep the reason), `5433653` (drafter + critic prompts). Master = `5433653`.
|
||||
- **UmbraAudio suite: 890 passed / 8 failed / 6 skipped** - the same 8 pre-existing failures
|
||||
(2 `test_pronunciation`, 6 `voice_profile_manager` accent); +9 new tests, no regressions.
|
||||
Web server restarted; all pages return 200 (auth gate intact, `/` -> 303 `/login`).
|
||||
|
||||
@@ -185,6 +185,76 @@ the mechanical split, 0 partition mismatches**. The one fallback was a model
|
||||
answer that did not reproduce the translated text; the content check caught it
|
||||
and the paragraph was flagged rather than accepted.
|
||||
|
||||
### The drafter re-flows the passage, and the alignment cannot see it
|
||||
|
||||
There is a defect in the production Chinese that the alignment pass is
|
||||
structurally unable to find, and it has to be stated plainly because it
|
||||
changes what the delivery means.
|
||||
|
||||
Pass 0 splits the original passage at sentence boundaries, so many
|
||||
paragraphs are **sentence fragments** - `The chair in the middle has`,
|
||||
`"operations"`, `on the console.` Measured: **439 of 3,038 translated
|
||||
paragraphs (14.5%) are fragments**. A fragment is exactly where an
|
||||
intent-based drafter is told to write natural target-language prose, and
|
||||
once it does that, the id-to-text correspondence drifts: the Chinese is
|
||||
good, but it is attached to the wrong paragraph.
|
||||
|
||||
In ch-15 the drift runs from p0094 to p0104 - eleven paragraphs. The Chinese
|
||||
for p0102 (`"Yes, Captain."`) is the translation of p0101, and the Chinese
|
||||
for p0104 opens with p0103's sentence. The audiobook would say the right
|
||||
words at the wrong place.
|
||||
|
||||
The alignment pass could not see it. A one-to-one paragraph is a pure copy,
|
||||
so there is nothing to divide and nothing to disagree; the drift is invisible
|
||||
to an alignment check by construction.
|
||||
|
||||
The verification stage (`align.py --verify`) finds it. It starts at a
|
||||
sentence fragment - the only place the drift is possible - and follows the
|
||||
run until the passage re-synchronises. The verdict is one of three:
|
||||
|
||||
| verdict | meaning | action |
|
||||
|---|---|---|
|
||||
| `aligned` | the translation renders this paragraph | keep |
|
||||
| `incomplete` | it renders this paragraph but leaves part out | keep, flag for review |
|
||||
| `drift` | it renders another paragraph's content | drop, with a reason |
|
||||
|
||||
Only `drift` deletes a line, and the paragraphs are judged as a **window**
|
||||
rather than one at a time. Both of those choices are measured, not assumed:
|
||||
|
||||
- The first version asked two yes/no questions per paragraph ("is anything
|
||||
missing", "is anything extra") and was wrong in both directions. It
|
||||
flagged five paragraphs that were fine - it reported the adverb `then` as
|
||||
missing from a translation that contains it, and `Enthusiastically` as
|
||||
missing from a translation that contains it - and it missed two paragraphs
|
||||
that are drifted, because those paragraphs are short and their Chinese
|
||||
looks plausible when seen alone. **Five of fifteen flags were wrong.**
|
||||
- A re-flowed passage cannot be judged one paragraph at a time. The model
|
||||
calls the two paragraphs *before* a real drift "aligned" when their Chinese
|
||||
is demonstrably the neighbouring paragraph's, and the start of a drift is
|
||||
not knowable from any single paragraph. So inside a window that contains a
|
||||
confirmed drift, a fragment the model called `aligned` is dropped as well.
|
||||
|
||||
On the real ch-15 window this now finds exactly the drift: **ten paragraphs
|
||||
dropped (p0095-p0104)**, and p0094 kept and flagged because its Chinese is a
|
||||
correct translation of its first two sentences. An earlier version dropped
|
||||
p0094 too - losing two good lines - and kept the two drifted paragraphs the
|
||||
model wrongly called aligned.
|
||||
|
||||
The cost of the choice is explicit. A fragment inside a broken window can be
|
||||
dropped on an unreliable verdict, so the audiobook has a gap there. The
|
||||
alternative is a line in the audiobook that belongs to a different sentence,
|
||||
and a gap is visible while a wrong line is silent.
|
||||
|
||||
A dropped paragraph still produces a row: `voiced = false` with the reason,
|
||||
so the gap says *why* it is there rather than looking like a paragraph
|
||||
nobody translated.
|
||||
|
||||
**The root cause is upstream.** The drafter should be told that each id's
|
||||
translation must cover that id's English only, even where that leaves the
|
||||
Chinese reading as a fragment. Fixing the drafter is a re-translation; until
|
||||
then the verification stage is what stands between the drifted Chinese and
|
||||
the audiobook.
|
||||
|
||||
### The pipeline is Chinese-only, and one production says otherwise
|
||||
|
||||
Measured on oldbox (2026-11-06), the second production directory is
|
||||
|
||||
Reference in New Issue
Block a user