Refuse a delivery whose text is not written in the language it claims
The production directory oldbox calls "deal-hunter-es" declares target_language "es-ES", its glossary says Spanish, and its 32-chapter run finished - and all 3,038 translated paragraphs are in Chinese. It is a second Chinese draft filed under a Spanish name. The cause is in the pipeline: the drafter's instructions are hard-coded to Simplified Chinese in three places (SYSTEM_DRAFTER, draft_prompt, SYSTEM_CRITIC), so the glossary's target language was ignored. No non-Chinese translation has ever been produced by that pipeline. The danger at this end is that the rows would be stored under "es", the review lane would look populated, and a reviewer would approve Chinese as Spanish. So the ingest now checks the script of the delivered text against the declared language, and refuses the delivery before writing a single row. The check is about the script, not the language: it cannot tell Spanish from Portuguese, but it can tell Chinese from Spanish, which is the mistake that actually happens. Japanese needs its kana and Korean its Hangul, and a Chinese delivery carrying kana is a Japanese translation filed in the wrong lane. An unknown language is not checked - there is no expectation to check it against. Proven on the real data: the es delivery is refused with "the text is mostly Han characters, so it is not Spanish", and the 4,300-row Chinese delivery still imports unchanged. 19 new tests. Suite: 909 passed, 8 failed (the same 8 pre-existing failures).
This commit is contained in:
@@ -185,6 +185,50 @@ the mechanical split, 0 partition mismatches**. The one fallback was a model
|
||||
answer that did not reproduce the translated text; the content check caught it
|
||||
and the paragraph was flagged rather than accepted.
|
||||
|
||||
### The pipeline is Chinese-only, and one production says otherwise
|
||||
|
||||
Measured on oldbox (2026-11-06), the second production directory is
|
||||
`productions/deal-hunter-es`. It declares `target_language: es-ES`, its
|
||||
glossary says `"script": "Spanish"` with Spanish style rules and the Spanish
|
||||
title `Cazador de Ofertas`, and it has a complete 32-chapter run (39,083 s).
|
||||
|
||||
**Its text is Chinese.** All 3,038 translated paragraphs are Chinese, in the
|
||||
same proportion as the Chinese production (3,034 with Han characters, 4
|
||||
Latin-only). It is a second Chinese draft filed under a Spanish name - a
|
||||
different Chinese wording from the `deal-hunter-zh` run (2,792 of 3,038
|
||||
paragraphs differ), but Chinese.
|
||||
|
||||
The cause is in the pipeline, not in the run: the drafter's instructions are
|
||||
**hard-coded to Simplified Chinese** in three places.
|
||||
|
||||
- `prompts.py` `SYSTEM_DRAFTER`: *"把英文太空冒险小说译成简体中文"* - translate the
|
||||
English space adventure novel into Simplified Chinese.
|
||||
- `prompts.py` `draft_prompt`: *"请把下面英文译为简体中文"* and a JSON key of
|
||||
`"zh"`.
|
||||
- `prompts.py` `SYSTEM_CRITIC`: *"对照英文原文和中文译文"*.
|
||||
|
||||
The glossary is passed into the prompt, but the *instruction* says Chinese, so
|
||||
the model followed the instruction and ignored the glossary's target. The
|
||||
critic is hard-coded the same way, so it checked the Chinese text and found
|
||||
nothing wrong - there was nothing wrong with it, for the language it was
|
||||
actually asked about.
|
||||
|
||||
Two consequences, both measured rather than inferred:
|
||||
|
||||
1. **No non-Chinese translation has ever been produced by this pipeline.** The
|
||||
nine other Qwen3-TTS languages exist only as the single-pass sample files
|
||||
described above.
|
||||
2. **The English translation output would be wrong for any other language**,
|
||||
because the analyst, the drafter and the critic are all told they are
|
||||
working on Chinese.
|
||||
|
||||
The pipeline needs a target-language parameter before it can be run for
|
||||
anything but Chinese. Until then, a production directory's name and its
|
||||
`target_language` field are a *claim*, not a measurement - which is why the
|
||||
delivery ingest now checks the script of the delivered text against the
|
||||
declared language and refuses a delivery whose text is written in another
|
||||
language (`umbraaudio/languages.py:script_mismatch`).
|
||||
|
||||
## Open questions
|
||||
|
||||
1. Which model runs Passes 1-4? This is where the local LLM question returns,
|
||||
|
||||
Reference in New Issue
Block a user