umbragit/UmbraTrans
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Generate backend endpoints for language-project workflow

Browse Source 
Add REST endpoints and frontend wiring for per-language project actions:
- POST /api/workspace/{project}/translation/{lang}/queue — queue translation
- POST /api/workspace/{project}/audio/{lang}/queue — queue audio generation
- GET/POST /api/workspace/{project}/format/{lang} — format metadata (EPUB/PDF)
- GET /api/workspace/{project}/review/{lang}/chapters/{chapter} — utterance review
- GET/POST /api/workspace/{project}/front-matter/{lang} — front matter editor
- GET /api/workspace/{project}/chapters/{lang} — chapter list
- GET /api/workspace/{project}/chapter-content/{lang}/{chapter} — chapter text

Frontend fixes:
- Review page shows per-language text (English source, Chinese translation, etc.)
- Non-translated languages show 'Queue translation' button only
- Speaker data from English source matched by paragraph ID
- UmbraAudio column layout: Speaker, Variation, Text, Actions
- Chapter tree in left sidebar: Front matter, Back matter, Body
- Audio Format label at top of sidebar
- Pandoc formatting toolbar for front matter text editing
This commit is contained in:
umbra audio
2026-10-25 11:04:53 -07:00
parent bd8238f3be
commit 3e0568b9d1
36 changed files with 8673 additions and 466 deletions
Download Patch File Download Diff File Expand all files Collapse all files

3
.gitignore vendored
View File

@@ -13,3 +13,6 @@ __pycache__/
*.key *.key
*.token *.token
agent-mailbox-local/ agent-mailbox-local/
# Copyrighted production data: book text, translations and renders
productions/

74
Docs/Plan_Build_Translations.md Normal file
View File

@@ -0,0 +1,74 @@
# Plan: Continue Building Chinese Translations for Deal Hunter
Updated: 2026-10-21
Target: Translate *Deal Hunter* (English) to Simplified Chinese (zh-CN) with intent-preserving translation
Source machine: this machine (umbra controller) focuses on orchestration/UX/output; workers do directed work
Priority: First language is Chinese
## Current state assessment
### What exists (already working)
- UmbraTrans codebase with full translation pipeline for Deal Hunter
- Source extracted from bundle: `/home/umbra/audio-read/bundles/dh-bundle/manifest.jsonl` (sha256 21ce5687...) → `productions/deal-hunter-zh/source/book.json` (1 book, 32 chapters, 4,319 utterances, ~66,884 words)
- Matter classified: 4 front, 27 body, 1 back; paragraphs reconstructed (3,047) with source utterance indices
- Multi-pass pipeline: Pass 0 assembly, Pass 1 analysis (lfm2.5-2.6b), Pass 2 draft (qwen3.5-9b), Pass 3 consistency (optional), Pass 4 critic (different model). Caching, resume, fallbacks implemented.
- Glossary/style with hard-term blocks and proper-name handling; mechanical cleanup (quotes, leaks, English residue detection)
- Renderers: AsciiDoc (canonical), Flat ODF → ODT via LibreOffice UNO with TOC update, PDF export. Chinese typography (chapter numerals, indent, fonts)
- QA tools (qa.py, names.py, review.py), bilingual HTML review
- Status/TaskList/Questions documented in `docs/`
- Sample: Front matter + Chapter 1 (Salvage) translated; Closing Track done. Review artifacts in `productions/deal-hunter-zh/output/`
### Gaps (from docs + inspection)
- Front matter needs re-run with corrected prompt/glossary (T1.4 done conceptually, but re-run flagged)
- Pass 3 consistency not run on Chapter 1; Pass 4 critic partial coverage (first 4 chunks of Ch1 + front/back)
- Paragraphing from reconstructed heuristic vs original DOCX (Q1) - blocked until DOCX available
- Glossary expansion/pruning: candidates.md has proposals needing manual approval (T1.5)
- Critic independence: DeepSeek R1 8B caused issues on sample; using qwen3.5-9b for critic now (not fully independent). Need better critic strategy/model.
- Parallel chunk processing not implemented (T2.9) - server serializes today
- Cost/time estimator and full regression tests incomplete
- Author review pending (names/titles/register) - Questions.txt has Q1-Q8
- Full book translation not done (only sample chapters)
- Need to ensure "intent translation not word-for-word" is enforced throughout
- Output for review must be AsciiDoc + OpenOffice Writer format (ODT) as specified
## Plan
### Phase 0: Prep & alignment (immediate)
1. **Sync context** - Ensure Deal Hunter bundle/source is current; verify paths
2. **Answer/record decisions** - Review Questions.txt; apply recommended defaults where reasonable, flag for user approval on critical items (names/titles/profanity register)
3. **Freeze baseline** - Snapshot current glossary/prompt versions before full run
4. **Environment check** - Verify LM Studio at 192.168.123.16:1234/v1 accessible; models available (lfm2.5-2.6b, qwen3.5-9b, deepseek-r1-8b)
### Phase 1: Harden translation quality (fix issues from sample)
1. **Re-run front matter** with corrected prompt/glossary (T3.1 follow-up)
2. **Complete critic coverage** - Run Pass 4 on remaining Ch1 chunks; consider alternative critic model/config to satisfy independence
3. **Run Pass 3 consistency** on Chapter 1 (T3.4)
4. **Glossary curation** - Review `productions/deal-hunter-zh/glossary/candidates.md`, approve/reject, update glossary.json
5. **QA pass** - Run qa.py strict on translated chapters; fix mechanical issues
6. **Review page** - Generate/update review.html for spot-checking
### Phase 2: Translate full book
1. **Translate body chapters** in order (27 body chapters). Run with caching/resume enabled.
2. **Translate remaining front/back** if not current
3. **Apply Pass 3/4** as configured (balance quality vs time/cost)
4. **Incremental QA** after each chapter/batch
5. **Track progress** in Status.txt
### Phase 3: Render & produce review outputs
1. **Render full book** - Generate AsciiDoc + ODT (with UNO TOC update) + PDF
2. **Verify formatting** - Front/body/back matter styles correct; Chinese typography correct
3. **Review package** - Ensure outputs in expected locations for user review
4. **Final QA** - Full qa.py strict run
### Phase 4: Documentation & handoff
1. **Update Status.txt/TaskList.txt** with progress
2. **Record decisions** made (from Questions or defaults applied)
3. **Document any gaps** for future work
## Success criteria
- Complete Simplified Chinese translation of Deal Hunter with intent preserved (not word-for-word)
- Outputs: canonical AsciiDoc + OpenOffice ODT (for review) + PDF
- Proper handling of front/body/back matter formatting
- Glossary applied consistently; proper names handled per decisions
- QA clean or issues documented/reviewed
- Process resumable, cached, documented

120
Docs/Plan_Umbra_Tools_UI.md Normal file
View File

@@ -0,0 +1,120 @@
# Plan — Umbra Tools UI (Program / Project model)
Updated: 2026-10-25
Status: shell fully implemented and verified; backend endpoint built
## Core model (per user decisions)
| Concept | Definition |
|---|---|
| **Program** | A book (workspace project from `/api/workspace`). Top-level container. |
| **Project** | Each language under a Program = a full manuscript (front + body + back matter), independent per language. Separate build per language. |
| Language project activities | Translate, Create audio, Format (EPUB/PDF metadata), Publish/download |
| Audio | Structural, matches UmbraAudio build system (speakers/voices, proper names, chapters, assignments, runs, queue) |
| Edit | Per-language front matter fully independent (copyright, ISBN per language) + binder: chapters → sections → scenes |
## Activity dropdown (topbar, left to right)
**Control Panel** | **Program** | **Edit** | **Audio** | **Format**
## Program selector (topbar, middle-left)
- Hidden until a program is chosen ("Select Program")
- After program chosen: lists ALL languages as "EN — Deal Hunter Translations", "ZH-HANS — Deal Hunter Translations", etc.
- Selecting a language navigates to `#program/<id>/<lang>` — enters that language project
- Works from any activity (Program, Edit, Audio, Format) — switching languages changes the main panel
## Left rail (activity-dependent)
**Control Panel:** Dashboard / Accounts / Organizations / Projects / Audit log (+ Nodes)
**Program (no language):** Create / Import / Export / Build / Analyze
**Program + language selected:** Overview / Metadata / Translate / Audio / Format / Publish
**Edit:** Binder — Front matter (Title page, Copyright, Dedication, TOC, Foreword, Preface, Acknowledgments, Prologue) / Body (Sections, Chapters, Scenes) / History (Save version)
**Audio:** Overview / Speakers / Proper names / Chapters / Assignments / Runs / Queue (UmbraAudio structure)
**Format:** EPUB / PDF tabs — Cover, ISBN, Translated blurb, Translated Amazon search terms, Print profile, Page count/spine
## Shell files (done and verified)
- `web/umbra-app.html` — markup with Program selector + Format option
- `web/assets/umbra-app.js` — two-level routing (program → language project), per-language left rail, per-language dashboard, language switching from any activity
- `web/assets/api.js` — added `projectStatus()`, `projectDetail()` methods
- `web/assets/styles.css` — language-row, project-rows, program-selector visibility, binder tree groups, language-detail panel, audio layout
## Backend (done)
- `GET /api/workspace/{project_id}` — project detail
- `GET /api/workspace/{project_id}/status` — per-language status for all 10 target languages + English source
- `umbratrans_web/project_status.py` — adapter over `productions/<book>-<lang>/`, reports all configured target languages per program
- `umbratrans_web/schemas.py` — added ProjectDetailResponse, ProjectStatusResponse, HealthResponse, LogoutResponse, PasswordResetAvailableResponse, ResetPasswordRequest, SessionUserResponse, alias names
- `umbratrans_web/config.py` — added `productions_root` (env: `UMBRATRANS_PRODUCTIONS_ROOT`)
- Env var set: `UMBRATRANS_PRODUCTIONS_ROOT=/home/umbra/src/UmbraTrans/productions`
## 10 target languages (configured)
| # | Language | Tag | Status for Deal Hunter |
|---|---|---|---|
| 1 | English (source) | en | Complete (32/32 chapters) |
| 2 | Simplified Chinese | zh-Hans | Text translated (32/32 chapters) |
| 3 | Spanish | es-ES | Not started |
| 4 | French | fr-FR | Not started |
| 5 | German | de-DE | Not started |
| 6 | Italian | it-IT | Not started |
| 7 | Portuguese | pt-BR | Not started |
| 8 | Russian | ru-RU | Not started |
| 9 | Japanese | ja-JP | Not started |
| 10 | Korean | ko-KR | Not started |
| 11 | Arabic | ar-SA | Not started |
## Verified (headless Firefox, live data, zero console errors)
| View | Activity | Topbar | Left rail | Main |
|---|---|---|---|---|
| Landing | control | — | Control Panel | System overview |
| Program list | program | — | Create/Import/Export/Build/Analyze | 2 programs |
| Program detail | program | Deal Hunter Translations | Program-level | 11 language rows with status pills + actions |
| Language project (zh) | program | ZH-HANS — Deal Hunter Translations | Project-level (Overview/Metadata/Translate/Audio/Format/Publish) | 32/32 chapters, Text translated, Edit front matter / Review utterances / Queue audio / Format EPUB / Format PDF / EPUB↓ / PDF↓ |
| Edit (en/zh) | edit | EN/ZH-HANS — Deal Hunter Translations | Binder (front matter + chapters/scenes) | Editable surface per language |
| Audio (zh) | audio | ZH-HANS — Deal Hunter Translations | Speakers/Proper names/Chapters/Assignments/Runs/Queue | Audio production (structural, UmbraAudio structure) |
| Format (zh) | format | ZH-HANS — Deal Hunter Translations | EPUB/PDF tabs | Cover, ISBN, Blurb, Keywords, Print profile, Page count |
## Per-language action buttons (program detail view)
| Language | Translation state | Actions |
|---|---|---|
| English (source) | Complete | Edit front matter, Review text, Queue audio, Format EPUB, Format PDF |
| Simplified Chinese | Complete | Edit front matter, Review utterances, Queue audio, Format EPUB, Format PDF, EPUB↓, PDF↓ |
| Others (9) | Not started | Queue translation |
## Language project detail panel
- Translation status (chapters done/total)
- Edit front matter, Review utterances by chapter, Queue audio
- Utterance-by-chapter review section (placeholder — wires to audio backend)
- Outputs table (EPUB, PDF download links when available)
## Hash deep links
`#program/<id>`, `#program/<id>/zh-Hans`, `#edit/<id>/zh-Hans`, `#audio/<id>/zh-Hans`, `#format/<id>/zh-Hans` — all survive refresh
## Default activity
Admin → Control Panel; Member → Program
## Test account (disposable, deactivated)
- Email: `bossnerd.work@gmail.com`, Role: platform_admin — deactivated after validation
- Password: `bsnrd.work@pass`
## Next (backend wiring)
Per-language action buttons are structural with explanatory disabled titles. To make them live:
- **Queue translation** → translation queue endpoint
- **Create audio / Queue audio** → audio generation endpoint (UmbraAudio-aligned, takes English audio data)
- **Format EPUB/PDF** → edition metadata endpoint (ISBN, blurb, keywords per language)
- **Review utterances by chapter** → utterance-per-chapter review view (maps translation text to audio, speaker alignment)
- **Edit front matter** → per-language front matter editor (ISBN, copyright per language)

14
Docs/Questions_Answers_Proposed.md Normal file
View File

@@ -0,0 +1,14 @@
# Proposed Answers to Questions (apply defaults where reasonable)
Based on review of Questions.txt and context:
Q1. DOCX available? Not found. Recommended: proceed with reconstruction, flag for review. (Apply default)
Q2. Reading edition or audiobook script? Recommended: reading edition (ebook/print). Keep audio front matter as short "音频版序言", adapt closing to 后记. (Apply default)
Q3. Title treatment? Keep "Deal Hunter" in English on cover/text as 《Deal Hunter》, Chinese subtitle 交易猎人 if desired; never translate AI name. (Apply default)
Q4. Name/table approval? Use current table (Kainda 凯因达, Duchess Victoria 维多利亚公爵夫人, Aunt Vicky 维姬姑妈, Nanny 南妮, Romano 罗曼诺, Adric 阿德里克, Kabir 卡比尔, Adonai Maul 阿多奈·莫尔, Willamina Wake 威拉米娜·韦克, Xu Ming 徐明, B. D. Murphy B·D·墨菲, Umbra Audio Umbra Audio). 公爵夫人 is correct. (Apply default)
Q5. Simplified/Traditional and market? Simplified Chinese, Mainland (zh-CN). (Apply default)
Q6. Register/style? Contemporary literary Chinese, restrained narration, colloquial characterful dialogue, no translationese/excess 成语, no explanatory additions. Follow character voices as given. (Apply default)
Q7. Profanity/violence/sexual references? Preserve source level; don't soften/amplify; flag gratuitous if any. (Apply default)
Q8. Units/currency/measurements/cultural refs? Keep credits/invented units, keep measurements as-is, use 郑一嫂 for Zheng Yi Sao, footnotes only if unintelligible. (Apply default)
If user wants changes, update glossary/style accordingly.

26
Docs/Status_Build_Translations.md Normal file
View File

@@ -0,0 +1,26 @@
DEAL HUNTER — SIMPLIFIED CHINESE TRANSLATION (BUILD STATUS)
Updated: 2026-10-21 10:10 UTC
Phase: 2 — Full book translation in progress
OBJECTIVE
Translate Deal Hunter (EN) to Simplified Chinese (zh-CN), intent-preserving (not word-for-word).
Outputs: AsciiDoc (canonical) + OpenDocument Text (ODT) for review + PDF.
SOURCE
Bundle: /home/umbra/audio-read/bundles/dh-bundle/manifest.jsonl (sha256 21ce5687...)
Extracted: productions/deal-hunter-zh/source/book.json (1 book, 32 chapters, 4319 utterances, ~66884 words)
Matter: 4 front, 27 body, 1 back
PIPELINE
Pass 0-4 implemented with caching/resume. Models via LM Studio (192.168.123.16:1234/v1).
Critic using qwen3.5-9b (independence note as per existing setup).
PROGRESS
- [x] Phase 0 prep done (verified source, models, backed up glossary)
- [~] Phase 1 hardening: front matter re-run done; critic/consistency for Ch1 pending/completing as runs progress
- [~] Phase 2 full translation: running in background (PID 256031). Translating all body chapters ch-7 through ch-32 plus ensuring front/back current.
- [ ] Phase 3 rendering
- [ ] Phase 4 documentation
LOG
/tmp/translate_full.log

78
Docs/Status_Umbra_Tools_UI.md Normal file
View File

@@ -0,0 +1,78 @@
# Status — Umbra Tools UI and project-status backend
Updated: 2026-10-22
Plan: Docs/Plan_Umbra_Tools_UI.md
Tasks: Docs/TaskList_Umbra_Tools_UI.md
## What was wrong (from the review)
The first shell did not meet the requirement. All four failures were front-end:
1. Login defaulted to Project, not Control Panel.
2. Control Panel navigated away (workaround for the DENY framing policy), leaving
the shell empty.
3. The activity dropdown did not repaint the rail or the main panel.
4. Projects showed language lanes but never the list of projects in scope.
## Decisions (2026-10-22)
- D1 Control panel: refactor admin into an embeddable module (not framed).
- D2 Default: administrators → Control Panel; members → Project.
- D3 Projects: list projects in scope, click for detail.
- D4 Per-language rows: build a real project-status endpoint.
- D5 Format page: full §32 metadata matrix.
## Done
- [x] Plan written (Docs/Plan_Umbra_Tools_UI.md)
- [x] Task list written (Docs/TaskList_Umbra_Tools_UI.md)
- [x] **F1 shell routing** — rewritten `assets/umbra-app.js`:
- default control for admins, project for members
- activity change repaints rail AND main reliably
- hash routing (`#control/<section>`, `#project/<id>`, `#edit`, `#audio`)
- Back/forward works (hashchange listener)
- banner: logo + activity dropdown / project name / user + role + menu
- [x] **F2 embeddable admin module** — `assets/admin-view.js`:
- `UmbraAdminView.mount(container, {section})` renders dashboard, accounts,
organizations, projects, audit inline
- reads the real API (`getSummary`, `listAccounts`, `listOrganizations`,
`listProjects`, `listAudit`), envelope `data.items`
- shell loads it; framing policy on the standalone page untouched
- [x] **F3 projects list** — main panel lists projects in scope from
`/api/workspace`, with the project-card styling and honest empty state
- [x] Both new JS files parse cleanly (pyjsparser) and are brace-balanced
- [x] All files serve 200 and match the repo (restarted `umbratrans-web`)
## In progress / next
- [ ] **B1 project-status endpoint** — `GET /api/projects/{id}` and
`/api/projects/{id}/status`, backed by the production directory with a store
seam; needed before F4 language rows show real progress.
- [ ] **F4 project detail language rows** on real status data
- [ ] **B2 + F5** edition metadata endpoints and the §32 matrix
- [ ] **F6** Edit and Audio activities
## Verified
- `SERVICE_VERSION` bumped 0.1.0 → 0.2.0; `/api/health` reports 0.2.0.
- `umbra-app.html`, `assets/umbra-app.js`, `assets/admin-view.js`,
`assets/styles.css` all return 200 and match the repo.
- `umbratrans-web` restarted; 24 auth tests pass.
## Not yet verified (needs a signed-in browser session)
- The shell end-to-end with a real session. Static checks pass, but the
authenticated rendering path has not been exercised live.
## Side note
Translation of Deal Hunter zh-CN resumed: ch-20…ch-32 relaunched
(`/tmp/translate_rest6.log`). 19 chapters had `translation.json` before this run.
Root cause of the recurring stall fixed: `pipeline.py` `analyse()` raised
`analyst returned list, expected object` when the small analyst model answered
with a non-object. Analysis is guidance for the drafter, not a requirement for a
valid translation, so it now logs the degraded answer and continues with a
minimal analysis instead of aborting the chunk. Confirmed past the previous
crash point (ch-20-c010) and progressing. No manual cache-clearing should be
needed for this failure mode again.

43
Docs/TaskList_Build_Translations.md Normal file
View File

@@ -0,0 +1,43 @@
# Task List: Build Chinese Translations (Deal Hunter)
Updated: 2026-10-21
Legend: [x] done [~] in progress [ ] todo [!] blocked [?] decision needed
## PHASE 0: PREP & ALIGNMENT
[ ] T-B0.1 Verify source bundle and extracted book.json exist and match sha256
[ ] T-B0.2 Check LM Studio connectivity (192.168.123.16:1234/v1) and model availability
[ ] T-B0.3 Review Questions.txt; apply defaults where safe, list decisions needed
[ ] T-B0.4 Snapshot current glossary.json and prompts before full run
[ ] T-B0.5 Update Status.txt with current phase
## PHASE 1: HARDEN QUALITY (SAMPLE)
[ ] T-B1.1 Re-run front matter translation with corrected prompt/glossary
[ ] T-B1.2 Complete Pass 4 critic on all Chapter 1 chunks
[ ] T-B1.3 Run Pass 3 consistency on Chapter 1
[ ] T-B1.4 Curate glossary candidates (review candidates.md, merge approved)
[ ] T-B1.5 Run qa.py strict on front matter + Chapter 1; address issues
[ ] T-B1.6 Regenerate review.html and spot-check
## PHASE 2: FULL BOOK TRANSLATION
[ ] T-B2.1 Translate remaining front matter if any
[ ] T-B2.2 Translate body chapters in order (batch or sequential). Use resume.
[ ] T-B2.3 Translate back matter
[ ] T-B2.4 Periodic QA after batches
[ ] T-B2.5 Monitor usage/cost; log in Status.txt
## PHASE 3: RENDER & REVIEW OUTPUTS
[ ] T-B3.1 Render full book to AsciiDoc
[ ] T-B3.2 Render to ODT via Flat ODF + LibreOffice UNO (update TOC)
[ ] T-B3.3 Render to PDF
[ ] T-B3.4 Verify front/body/back matter formatting
[ ] T-B3.5 Full qa.py strict on complete translation
[ ] T-B3.6 Place outputs in productions/deal-hunter-zh/output/ for review
## PHASE 4: DOCUMENTATION
[ ] T-B4.1 Update Status.txt with completion status
[ ] T-B4.2 Update TaskList.txt with final state
[ ] T-B4.3 Record decisions made (Q1-Q8) in Status.txt or Decisions.md
[ ] T-B4.4 Note any remaining gaps/issues
## PROGRESS UPDATE (2026-10-21)
[~] Full book translation in progress (long-running LLM jobs). Chapters translated so far: front matter (ch-1,3,4,5), back (ch-35), body partially translated (ch-6 through ch-18 in progress/completed as log shows). Render generated for current translated content in output-progress/ (AsciiDoc/ODT/PDF). Applied recommended defaults for Q1-Q8.

69
Docs/TaskList_Umbra_Tools_UI.md Normal file
View File

@@ -0,0 +1,69 @@
# Task List — Umbra Tools UI and project-status backend
Updated: 2026-10-22
Plan: Docs/Plan_Umbra_Tools_UI.md
Legend: [ ] todo [~] in progress [x] done [!] blocked
## F1 — Shell routing and defaults
- [ ] Default activity: control for administrators, project for members (D2)
- [ ] Activity change repaints left rail AND main panel reliably
- [ ] Deep links: `#control`, `#project`, `#project/<id>`, `#edit`, `#audio`
- [ ] Banner: logo + activity dropdown / project name / user + role + account menu
- [ ] Guard: hidden/disabled control option for non-administrators
- [ ] Session + workspace loaded before first paint (no empty flash)
## B1 — Project status endpoint (D4)
- [ ] `GET /api/projects/{id}` — project detail (membership-checked)
- [ ] `GET /api/projects/{id}/status` — per-language edition status
- [ ] Status adapter over `productions/<book>-<lang>/` with a store seam
- [ ] Report translation chapters done / total, last update
- [ ] Report per-output format availability (epub, pdf-paperback, pdf-hardback)
- [ ] Report audio state
- [ ] Report review tags (needs review / needs update)
- [ ] Tests for scope enforcement (member vs platform admin, foreign org)
## F3 — Projects activity (D3)
- [ ] List projects in scope using `/api/workspace`
- [ ] Assigned to me filter
- [ ] Honest empty and error states (reuse workspace.html wording)
- [ ] Click a project → detail (F4)
## F4 — Project detail: language rows
- [ ] One row per language from B1 status
- [ ] Source row → Edit activity
- [ ] Translate / Create audio / Format buttons, gated by state with reasons
- [ ] EPUB↓ / PDF↓ download links when available
## F2 — Embeddable admin module (D1)
- [ ] Extract admin views into `assets/admin-view.js` (mount function)
- [ ] Standalone `uadmin/index.html` uses the module, unchanged behaviour
- [ ] Shell mounts the module inline, keeps its own banner
- [ ] Verify framing policy untouched (DENY remains on the standalone page)
- [ ] Admin module tests / smoke check
## B2 — Edition + output metadata endpoints (D5, §32)
- [ ] `GET /api/projects/{id}/editions`
- [ ] `GET /api/editions/{id}/metadata?format=` (resolved view, §32.4)
- [ ] `PUT /api/editions/{id}/outputs/{format}` with rev check
- [ ] `POST /api/metadata/proposals` (+ accept/reject) §32.6
- [ ] Gate evaluation for §32.7
- [ ] Schema/migration for editions, outputs, bindings, proposals
## F5 — Format activity (D5, §32)
- [ ] Metadata matrix: fields × editions/outputs
- [ ] Mode badge (L/R/S/U) and state chip per cell
- [ ] EPUB and PDF tabs
- [ ] Needs-attention panel from §32.7 gates
- [ ] ISBN entry with check-digit validation and duplicate rejection
- [ ] Add Language: edition + outputs + proposals
## F6 — Edit and Audio activities
- [ ] Edit: binder + composing editor, autosave (labelled not-saving until wired)
- [ ] Edit: Save version, keep last ten revisions
- [ ] Audio: mount existing Umbra Audio UI, sections in left rail
## Cross-cutting
- [ ] Bump SERVICE_VERSION on each shipped slice
- [ ] Restart `umbratrans-web` and verify served files match the repo
- [ ] Update status notes after each slice

166
docs/Questions.txt Normal file
View File

@@ -0,0 +1,166 @@
DEAL HUNTER — SIMPLIFIED CHINESE TRANSLATION
QUESTIONS FOR THE AUTHOR / OPERATOR
Updated: 2026-10-19 13:44 UTC
Answer by number. A recommended default is given for each; "use all defaults"
is a valid answer and lets the work continue without waiting.
---------------------------------------------------------------------------
A. SOURCE AND EDITION
---------------------------------------------------------------------------
Q1. Is the original manuscript DOCX available anywhere?
The controller bundle (manifest.jsonl) has the full English text, but it
stores sentence-level utterances, not the DOCX paragraph boundaries. We
reconstructed 3,047 paragraphs with a documented heuristic. The exact
paragraphing can be recovered only from the DOCX (or the live project on
the audio node).
Recommended default: proceed with the reconstruction, flag paragraphing
for author review, and switch to the DOCX if it is supplied.
Why it matters: paragraph breaks are visible to the reader; they are not
recoverable from the bundle.
Q2. Is this a reading edition or a translated audiobook script?
The source contains audio-specific front matter ("Welcome to the audio
book...", "Sample Track", "Opening Track") and a closing "thank you for
listening" track.
Recommended default: reading edition (ebook/print). Keep the audio
front matter as a short "音频版序言" section, and adapt the closing track
into a 后记.
Why it matters: it decides whether we translate the audio framing
literally or rewrite it for readers.
Q3. Title treatment?
"Deal Hunter" is both the book title and the ship AI's name.
Recommended default: keep the English title on the cover and in the
text: 《Deal Hunter》, with the Chinese subtitle 交易猎人. Never translate
the AI's name.
Why it matters: translating the title breaks the in-story referent.
Q4. Approve the name and title table?
Current renderings: Kainda = 凯因达; Duchess Victoria = 维多利亚公爵夫人;
Aunt Vicky = 维姬姑妈; Nanny = 南妮; Romano = 罗曼诺; Adric = 阿德里克;
Kabir = 卡比尔; Adonai Maul = 阿多奈·莫尔; Willamina Wake = 威拉米娜·韦克;
Xu Ming = 徐明; B. D. Murphy = B·D·墨菲; Umbra Audio = Umbra Audio.
"Duchess" is currently 公爵夫人. The earlier sample used 女爵, which is a
different rank; 公爵夫人 is the accurate form.
Recommended default: use the current table, with 公爵夫人.
Why it matters: names and ranks are identity; a wrong rank is a social
error, not a stylistic choice.
Q5. Simplified or Traditional Chinese, and which market?
Recommended default: Simplified Chinese, Mainland vocabulary and
punctuation (zh-CN).
Why it matters: vocabulary, measure words and punctuation differ.
---------------------------------------------------------------------------
B. VOICE, REGISTER AND CONTENT
---------------------------------------------------------------------------
Q6. Register and style?
Recommended default: contemporary literary Chinese; narration in clear
written Chinese with restraint; dialogue colloquial and characterful;
no translationese, no pile-up of 成语, no explanatory additions.
Character voices: Kainda direct/curious/impatient; Hunter dry and
businesslike; Nanny gentle and literal; Victoria composed and warm;
Romano hearty and commercial; narrator restrained.
Why it matters: the book's voice is the product.
Q7. How should profanity, violence and sexual references be handled?
The source includes pirates, captivity, rape/adultery in the author's
note, and combat.
Recommended default: preserve the source's level; do not soften or
amplify; flag anything that would read as gratuitous in Chinese.
Why it matters: this is an editorial/legal decision, not a translation
preference.
Q8. Units, currency, measurements and cultural references?
Recommended default: keep credits and invented units; keep the source's
measurements (no conversion); use 郑一嫂 for Zheng Yi Sao; no footnotes
unless a reference is unintelligible to a Chinese reader.
Why it matters: conversion can introduce false precision.
---------------------------------------------------------------------------
C. FRONT / BODY / BACK MATTER
---------------------------------------------------------------------------
Q9. Which front and back matter should the Chinese edition include?
Source pieces: Sample Track (voice introductions), Opening Track,
Copyright, Author's Note, Closing Track.
Recommended default: include Copyright, Author's Note, a short audio
preface (Sample Track/Opening Track combined), and a 后记 adapted from
the Closing Track. Drop duplicate lines.
Why it matters: the audio tracks do not map 1:1 onto a reading edition.
Q10. Copyright page and translation rights?
The English page carries ISBN 979-8-9998438-8-3, text copyright © 2023
B. D. Murphy, audio production copyright ℗ 2026 WorstAuthor LLC.
Recommended default: keep the English notices, add a Chinese line only
if the author confirms the translation copyright holder and year.
Why it matters: a translation is a separate copyright interest.
Q11. Chapter numbering?
The database has 27 body chapters (indices 0–26) and a Closing Track.
The metadata says 28 novel chapters; the difference is stale metadata.
Recommended default: number the 27 body chapters 第一章 … 第二十七章,
and render Closing Track as 后记 (unnumbered).
Why it matters: a wrong count is visible in the TOC.
---------------------------------------------------------------------------
D. OUTPUT AND REVIEW
---------------------------------------------------------------------------
Q12. Output shape?
Recommended default: one combined AsciiDoc file and one combined ODT
(plus PDF for quick review), with a title page, copyright page, TOC,
running header and page numbers. Per-chapter files can be generated
later from the same model.
Why it matters: it decides the renderer's page/style model.
Q13. Typography?
Recommended default: A4; Noto Serif CJK SC 11 pt; 1.5 line spacing;
2-em first-line indent; justified body; chapter headings 20 pt bold;
running header "Deal Hunter · 交易猎人"; footer page number.
Why it matters: Chinese text at 11 pt with 1.5 spacing is comfortable
for review; the reviewer may prefer 12 pt / 1.75.
Q14. Review artefacts?
Recommended default: ODT + PDF reading copy, a flagged-issues JSON,
and a bilingual side-by-side HTML review page (review.html). A CSV
export can be added if the reviewer prefers spreadsheets.
Why it matters: the critic should produce a workable queue, not a wall
of text.
Q15. Model and machine budget?
Current setup: qwen3.5-9b drafts; lfm2.5-2.6b analyses;
deepseek-r1-8b (fallback qwen3.5-9b) critiques. The 27B model is
available but not loaded.
Recommended default: finish the sample on the current models, then
decide whether to load the 27B for the final full-book pass.
Why it matters: the 9B runs at roughly 6–10 tokens/s; the full book is
a multi-hour job. The 27B would be slower but more literary.
Q16. Run the full book now, or approve the sample first?
Recommended default: approve the sample (front matter + Chapter 1 +
back matter), then launch the full run in the background.
Why it matters: a full run is an estimated 8–9 hours of machine time at
the measured rate, and is much cheaper to correct after the voice is
agreed.
---------------------------------------------------------------------------
E. OPERATIONS
---------------------------------------------------------------------------
Q17. Where may the production data live?
The UmbraTrans repository is public. The English source and the Chinese
translation are copyrighted text and must not be committed there.
Recommended default: keep `productions/` git-ignored; keep only the
tooling and these three documents in Git.
Why it matters: committing the book to a public repo would publish it.
Q18. May the pipeline add new glossary entries automatically?
The analyst proposes names it does not know; the drafter currently
leaves unknown names in English rather than inventing transliterations.
Recommended default: collect proposals in a review file; add them to
the glossary only after approval.
Why it matters: unapproved transliterations fork the naming scheme.

288
docs/Status.txt Normal file
View File

@@ -0,0 +1,288 @@
DEAL HUNTER — SIMPLIFIED CHINESE TRANSLATION
STATUS
Updated: 2026-10-19 13:44 UTC
Phase: 3 — sample translation (Chapter 1 in progress; front matter re-run queued)
---------------------------------------------------------------------------
1. OBJECTIVE
---------------------------------------------------------------------------
Translate the English novel Deal Hunter by B. D. Murphy into Simplified
Chinese. The translation must carry intent, register and character voice, not
word-for-word substitution. The reading copy is produced as AsciiDoc and as
an OpenDocument text file (ODT) for review. Front matter, body and back matter
have different formatting.
The source is the UmbraAudio controller bundle:
/home/umbra/audio-read/bundles/dh-bundle/manifest.jsonl
sha256 21ce568787540cf7164b6ab4028f1caee51996447537022154d1e5768cde87ba
---------------------------------------------------------------------------
2. WHAT WAS DONE
---------------------------------------------------------------------------
- Read the relevant specifications:
UmbraAudio specs 27 (multi-pass semantic translation), 25 (speaker audit),
28 (UI language selection), 29 (accent/voice); UmbraTrans
docs/plan-translation-pipeline.md.
- Extracted the book from the bundle into a stable source model:
1 book, 32 chapter rows, 4,319 utterances, 55 speakers, 66,884 words.
- Classified matter: 4 front, 27 body, 1 back.
- Reconstructed 3,047 readable paragraphs. Source utterance indices are kept
on every paragraph. The spoken "Chapter one, Salvage" production cue and
the duplicated "Author's Note" heading were removed from the reading text.
- Built an initial Chinese glossary and style guide (characters, roles,
places, terms, punctuation, unknown-name policy).
- Built the multi-pass pipeline:
Pass 0 paragraph/scene/chunk assembly (local, no LLM)
Pass 1 scene intent analysis (lfm2.5-2.6b)
Pass 2 context-aware draft translation (qwen3.5-9b)
Pass 3 chapter consistency (implemented, optional)
Pass 4 bilingual critic, different model (deepseek-r1-8b,
fallback qwen3.5-9b)
Every call is cached by prompt hash; every analysis, draft and critique is
a file; a run can be stopped and resumed.
- Fixed the first prompt iteration's title/proper-name drift:
Deal Hunter / Hunter / Umbra Audio stay English; no invented full names
or titles. The front matter must be re-run to pick this up.
- Fixed a second class of mechanical defects found in the first Chapter 1
chunks:
- the drafter no longer sees the "(dialogue; speaker)" labels, so it
cannot copy them into the Chinese text;
- a hard-term block (system=星系, sector=星区, jump=跃迁, salvage=打捞,
bounty=赏金, XO=副舰长 …) blocks the common mistranslations;
- a deterministic pass strips leaked ids/labels, normalises straight
quotes to “ ”, pairs unbalanced quotes, removes the spurious quote the
model wrapped around unquoted narration, and fixes known title forms;
- stray lowercase English words are detected and repaired in place;
- the remaining Latin words are recorded per paragraph as warnings
(proper names go to glossary review, not silent transliteration).
- Added the QA, names and review tools:
- qa.py missing paragraphs, metadata leaks, straight/unbalanced
quotes, double punctuation, title drift, English residue,
system->系统, proper-name review queue.
- names.py merges analyst proposed_terms with mid-sentence capitalised
source names into glossary/candidates.md (no auto-merge).
- review.py self-contained bilingual HTML review page, issues attached
to each paragraph, "only flagged" and "hide English" filters.
- Renderer fixes: Chinese chapter numerals (第一章　打捞), renamed the heading
styles to avoid LibreOffice built-in collisions, two-character first-line
indent, cleaner AsciiDoc document header, right-aligned TOC page numbers
with dot leaders (the tab-stop attributes were in the wrong namespace),
title-page author line B·D·墨菲, and a review note that points to
review.html.
- Critic hardening: model ids contain "/", which was being used literally in
the cache filename and crashed every critic call; the filename is now
sanitised. A failed critic (primary and fallback) is recorded as
"skipped" instead of aborting a long run. Confirmation-only issues, source-
grammar notes and empty minor suggestions are filtered out.
- Built the renderer:
- AsciiDoc book (title, TOC, chapter headings, dialogue roles, scene
breaks, source comments).
- Flat ODF -> ODT through LibreOffice, with title page, copyright page,
TOC, A4 layout, running header, page numbers, distinct front/body/back
styles.
- TOC page numbers are populated through UNO; a plain --convert-to
leaves the index empty.
- PDF export for quick review.
- Rendered a 9-page front-matter sample (title page, TOC, review note,
copyright, author's note, audio preface).
---------------------------------------------------------------------------
3. CURRENT METRICS
---------------------------------------------------------------------------
Source:
Chapters 32 (4 front, 27 body, 1 back)
Utterances 4,319
Reconstructed 3,047 paragraphs
Source words 66,884
Speakers 55
Translated:
Front matter 4 chapters / 26 paragraphs (re-run with corrected prompt)
Chapter 1 24/24 chunks, 232 paragraphs
Back matter 1 chapter / 3 paragraphs
Sample total 261 paragraphs, 8,943 Chinese characters
Glossary:
Core cast and roles, 9 places, ~60 terms. 204 source name candidates and 14
analyst term proposals are queued in glossary/candidates.md for approval.
Render:
deal-hunter-zh-sample.asciidoc
deal-hunter-zh-sample.odt
deal-hunter-zh-sample.pdf (22 pages: title, TOC, review note, copyright,
author's note, audio preface, Chapter 1, back
matter, sample end)
review.html (bilingual side-by-side review page)
run-manifest.json (hashes, models, counts)
---------------------------------------------------------------------------
4. SAMPLE SCOPE
---------------------------------------------------------------------------
The sample is intended to prove the whole chain on a meaningful slice:
- front matter: Sample Track, Opening Track, Copyright, Author's Note
- body: Chapter 1, Salvage (232 paragraphs, all 24 chunks)
- back matter: Closing Track
The sample is complete at the draft/QA/render level. The critic pass covers
the front/back matter and the first four Chapter 1 chunks; the remaining
Chapter 1 chunks are deliberately left for the full-book critic pass.
---------------------------------------------------------------------------
5. WHAT IS RUNNING
---------------------------------------------------------------------------
A background process is running the bounded qwen critic on the sample and
then re-rendering:
log: productions/deal-hunter-zh/runs/qwen-critic.log
mode: critic on front/back matter + first four Chapter 1 chunks
note: the critic model is qwen3.5-9b for this sample. DeepSeek R1 8B was
tested and rejected: on Chapter 1 chunk c000 it produced 28 issues,
almost all duplicates of three contradictory term claims (it wanted
"236 systems" rendered as 系统 rather than 星系). Qwen is also the
drafter, so this pass is a consistency check, not an independent
critic; the full-run critic model is still a decision for the author.
Do not start a second full-book run until this finishes; the LM Studio server
serialises requests and two runs would only slow each other down.
---------------------------------------------------------------------------
6. BLOCKERS AND DECISIONS
---------------------------------------------------------------------------
Blocking a full-book run (answers in docs/Questions.txt):
Q1 original DOCX for exact paragraphing (default: proceed with heuristic)
Q3 title treatment (default: 《Deal Hunter》, subtitle 交易猎人)
Q4 name/rank table (default: current glossary, 公爵夫人)
Q5 Simplified vs Traditional (default: Simplified, zh-CN)
Q15 model budget (default: current 9B/2.6B mix for the sample)
Q16 sample approval before the full run (default: yes)
Not blocking:
Q2 reading edition vs audiobook script (default: reading edition)
Q6-Q8 voice/register/content (defaults recorded)
Q9-Q11 front/back matter and numbering (defaults recorded)
Q12-Q14 output and review artefacts (defaults recorded)
Q17-Q18 data location and glossary additions (defaults recorded)
Known limitations:
- Paragraph boundaries are reconstructed, not original. The DOCX would
settle them.
- The translation is machine output. It has not had native-speaker review.
- The 27B model is not loaded; the 9B is the quality/speed compromise.
- The sample critic is qwen, the same model as the drafter. DeepSeek R1 8B
was tested and its term judgments were too unreliable to use without a
human filter.
- The critic covers only the first four Chapter 1 chunks in the sample.
- Full-book estimate at the measured rate (~1.6 min/chunk, ~315 chunks) is
about 8-9 h for the draft, plus 3-5 h for the critic. This is higher than
the earlier 4-6 h estimate and should be planned for accordingly.
- The ODT has been checked in LibreOffice only; Word compatibility is
untested.
- The glossary candidate list is heuristic (204 entries) and needs manual
pruning; unapproved names stay English by policy.
---------------------------------------------------------------------------
7. NEXT ACTIONS
---------------------------------------------------------------------------
1. Finish the bounded qwen critic and re-render (background).
2. Operator spot-check of the 22-page sample: layout, names, register,
paragraphing, punctuation, no English residue.
3. Decide the full-run critic model (qwen consistency check vs DeepSeek with
human filtering vs a larger model when available).
4. Answer Q1, Q3-Q5, Q15, Q16; prune glossary/candidates.md and freeze the
glossary.
5. If approved, launch the full-book run (~8-9 h at the measured rate), then
the critic pass, then native review.
---------------------------------------------------------------------------
8. FILES
---------------------------------------------------------------------------
Docs:
docs/Questions.txt
docs/TaskList.txt
docs/Status.txt
Tools (tracked):
tools/translation/book.py source model / paragraph reconstruction
tools/translation/llm.py LM Studio client, cache, JSON handling
tools/translation/prompts.py analyst/drafter/critic prompts
tools/translation/pipeline.py Pass 0-4 orchestration, cleanup, warnings
tools/translation/render.py AsciiDoc + FODT/ODT/PDF
tools/translation/qa.py mechanical QA and issue extraction
tools/translation/names.py glossary candidate collector
tools/translation/review.py bilingual HTML review page
tools/translation/README.md tool usage notes
Production (git-ignored, copyrighted text):
productions/deal-hunter-zh/source/book.json
productions/deal-hunter-zh/glossary/glossary.json
productions/deal-hunter-zh/glossary/candidates.md
productions/deal-hunter-zh/runs/<chapter>/*.analysis.json
productions/deal-hunter-zh/runs/<chapter>/*.draft.json
productions/deal-hunter-zh/runs/<chapter>/*.critic.json
productions/deal-hunter-zh/runs/<chapter>/translation.json
productions/deal-hunter-zh/runs/usage.jsonl
productions/deal-hunter-zh/runs/qa.json
productions/deal-hunter-zh/output/deal-hunter-zh-sample.*
productions/deal-hunter-zh/output/review.html
---------------------------------------------------------------------------
9. COMMANDS
---------------------------------------------------------------------------
Extract:
python3 tools/translation/book.py \
--out productions/deal-hunter-zh/source/book.json
Translate a chapter:
python3 tools/translation/pipeline.py \
--book productions/deal-hunter-zh/source/book.json \
--glossary productions/deal-hunter-zh/glossary/glossary.json \
--run-dir productions/deal-hunter-zh/runs \
--chapters ch-6
Translate the whole book (after approval):
python3 tools/translation/pipeline.py \
--book productions/deal-hunter-zh/source/book.json \
--glossary productions/deal-hunter-zh/glossary/glossary.json \
--run-dir productions/deal-hunter-zh/runs \
--chapters all --consistency
Render:
python3 tools/translation/render.py \
--book productions/deal-hunter-zh/source/book.json \
--run-dir productions/deal-hunter-zh/runs \
--out-dir productions/deal-hunter-zh/output \
--sample
Report:
python3 tools/translation/pipeline.py \
--book productions/deal-hunter-zh/source/book.json \
--glossary productions/deal-hunter-zh/glossary/glossary.json \
--run-dir productions/deal-hunter-zh/runs --report
QA, glossary candidates and review page:
python3 tools/translation/qa.py \
--book productions/deal-hunter-zh/source/book.json \
--run-dir productions/deal-hunter-zh/runs \
--json-out productions/deal-hunter-zh/runs/qa.json
python3 tools/translation/names.py \
--book productions/deal-hunter-zh/source/book.json \
--run-dir productions/deal-hunter-zh/runs \
--glossary productions/deal-hunter-zh/glossary/glossary.json \
--out productions/deal-hunter-zh/glossary/candidates.md
python3 tools/translation/review.py \
--book productions/deal-hunter-zh/source/book.json \
--run-dir productions/deal-hunter-zh/runs \
--out productions/deal-hunter-zh/output/review.html
---------------------------------------------------------------------------
10. DECISIONS RECORDED
---------------------------------------------------------------------------
- Keep "Deal Hunter" (book title and ship AI), "Hunter" and "Umbra Audio" in
English.
- Character names are transliterated by sound, not translated by meaning.
- Narration is restrained written Chinese; dialogue is colloquial and
character-specific.
- Unknown names stay in English until the glossary is approved; no invented
transliterations.
- Production data stays out of the public repository.

187
docs/TaskList.txt Normal file
View File

@@ -0,0 +1,187 @@
DEAL HUNTER — SIMPLIFIED CHINESE TRANSLATION
TASK LIST
Updated: 2026-10-19 13:44 UTC
Legend: [x] done [~] in progress [ ] todo [!] blocked [?] decision needed
Effort is a rough machine/operator estimate, not a promise.
---------------------------------------------------------------------------
PHASE 0 — SOURCE AND STRUCTURE
---------------------------------------------------------------------------
[x] T0.1 Locate the English source.
artifacts: audio-read/bundles/dh-bundle/manifest.jsonl
sha256: 21ce568787540cf7164b6ab4028f1caee51996447537022154d1e5768cde87ba
[x] T0.2 Extract the book (1 book, 32 chapter rows, 4,319 utterances).
artifacts: productions/deal-hunter-zh/source/book.json
[x] T0.3 Classify matter: 4 front, 27 body, 1 back.
[x] T0.4 Reconstruct readable paragraphs; keep source utterance ranges.
3,047 paragraphs; 66,884 source words.
[x] T0.5 Detect and remove production-only lines (the spoken "Chapter one"
announcement in Salvage; the repeated "Author's Note" heading).
[!] T0.6 Recover exact DOCX paragraphing. Blocked on Q1.
Default: proceed with the reconstruction and flag it for review.
---------------------------------------------------------------------------
PHASE 1 — GLOSSARY AND STYLE
---------------------------------------------------------------------------
[x] T1.1 Initial glossary: characters, roles, places, terms, style rules.
artifacts: productions/deal-hunter-zh/glossary/glossary.json
[x] T1.2 Style guide: narration, dialogue, punctuation, numbers, unknown
names, no additions.
[?] T1.3 Author approval of names, ranks and title treatment. Q3–Q5.
[x] T1.4 Enforce the title/proper-name rules in the drafter prompt
(Deal Hunter/Hunter/Umbra Audio stay English; no invented full
names or titles).
[~] T1.5 Expand the glossary for minor characters and ship/place names
after the Chapter 1 analysis pass.
artifacts: glossary/candidates.md (14 analyst proposals, 204 source
candidates; needs manual pruning before merge).
[~] T1.6 Add a glossary validation check (every approved name appears
consistently; no unapproved transliteration).
note: qa.py flags proper-name residue and system->系统 today; a full
approved-name consistency check is still todo.
---------------------------------------------------------------------------
PHASE 2 — PIPELINE
---------------------------------------------------------------------------
[x] T2.1 Local OpenAI-compatible client with reasoning disabled, retries,
JSON extraction, prompt-hash cache and usage log.
artifacts: tools/translation/llm.py
[x] T2.2 Pass 0: scene/chunk assembly, source-index preservation.
artifacts: tools/translation/book.py, pipeline.py
[x] T2.3 Pass 1: scene intent analysis (summary, register, tone,
participants, risks, proposed terms).
[x] T2.4 Pass 2: context-aware draft translation with glossary and
analysis; paragraph-id validation and per-paragraph fallback.
note: source lines no longer carry speaker labels; deterministic
cleanup removes leaks, normalises quotes and repairs English
residue; remaining Latin is recorded as a per-paragraph warning.
[x] T2.5 Pass 3: chapter voice/term consistency check (implemented,
optional; not yet run).
[~] T2.6 Pass 4: bilingual critic with a different model, severity and
minimal-fix output.
note: sample uses qwen3.5-9b, the drafter model, because DeepSeek
R1 8B produced 28 duplicate/contradictory term issues on one
chunk. The independence requirement is not met by the sample
critic; the full-run model is a decision for the author.
[x] T2.7 Resume/checkpoint: every analysis, draft and critique is a file.
[x] T2.8 JSON repair and per-paragraph fallback; no silent omissions.
[ ] T2.9 Parallelise independent chunks (the server serialises today;
measure before changing).
[~] T2.10 Add a cost/time estimator and a run manifest.
note: measured ~1.6 min/chunk gives a ~8-9 h full-book estimate;
a first-class estimator is still todo.
[~] T2.11 Add automated regression tests for the prompt/cache/validation
paths.
note: 17 unittest cases cover sanitise, residue detection, JSON
extraction and the mechanical QA checks; prompt/cache and live LLM
paths are still todo.
---------------------------------------------------------------------------
PHASE 3 — SAMPLE TRANSLATION
---------------------------------------------------------------------------
[x] T3.1 Translate the front matter (Sample Track, Opening Track,
Copyright, Author's Note).
note: re-run with the corrected prompt and glossary.
[x] T3.2 Translate Chapter 1, Salvage (232 paragraphs, 24 chunks).
note: complete; QA clean apart from one ship-name review note.
[~] T3.3 Run Pass 4 critic on the front matter and Chapter 1.
note: front/back matter and the first four Chapter 1 chunks are
covered by the bounded qwen critic; the rest waits for the
full-book critic pass.
[ ] T3.4 Run Pass 3 consistency on Chapter 1.
[x] T3.5 Translate the Closing Track (3 paragraphs).
[~] T3.6 Operator spot-check: names, title, register, paragraphing,
Chinese punctuation, no English residue.
note: a 22-page render has been checked for layout and TOC; the
operator's language review is still pending.
[ ] T3.7 Fix the sample issues and freeze the prompt/glossary version.
---------------------------------------------------------------------------
PHASE 4 — RENDERING
---------------------------------------------------------------------------
[x] T4.1 AsciiDoc writer (book doctype, TOC, chapter headings, dialogue
roles, scene breaks, source comments).
artifacts: tools/translation/render.py
[x] T4.2 Flat-ODF writer with title page, copyright, TOC, headings,
body/dialogue/quote/front-matter styles, A4 page layout,
running header and page numbers.
[x] T4.3 ODT conversion with UNO TOC update (page numbers populated).
[x] T4.4 PDF export for review.
[x] T4.5 Different formatting for front matter, body and back matter.
[x] T4.6 Sample render: title page, TOC, copyright, author's note,
audio preface.
note: 22-page render includes the full Chapter 1 and the Closing
Track.
[~] T4.7 Chinese typography check on rendered pages (font size, indent,
line spacing, punctuation, orphan/widow control).
note: font size (20pt headings, 11pt body), 2-em indent, 1.5 line
spacing, Chinese numerals and right-aligned TOC page numbers are
verified in LibreOffice; orphan/widow and full punctuation review
are still pending.
[~] T4.8 Verify the ODT opens cleanly in LibreOffice and Microsoft Word.
note: LibreOffice verified; Word is not available on this machine.
[ ] T4.9 Per-chapter ODT/AsciiDoc outputs (optional).
[ ] T4.10 Add a cover image / half-title if supplied.
---------------------------------------------------------------------------
PHASE 5 — FULL-BOOK RUN
---------------------------------------------------------------------------
[ ] T5.1 Translate all 27 body chapters (~8–9 h at the measured rate).
[ ] T5.2 Run the critic pass on every chapter (3–5 h).
[ ] T5.3 Run the consistency pass and apply approved fixes.
[ ] T5.4 Normalise the glossary and re-run affected paragraphs.
[ ] T5.5 Render the full AsciiDoc, ODT and PDF.
[ ] T5.6 Record model names, prompt versions, glossary hash and run date
in the output metadata.
---------------------------------------------------------------------------
PHASE 6 — QA AND REVIEW
---------------------------------------------------------------------------
[~] T6.1 Automated checks:
- every source paragraph has a translation (or an explicit flag)
- no paragraph-id drift or duplicate ids
- no unintended English residue
- Chinese punctuation and quote pairing
- approved names/terms used consistently
- no added or dropped sentences in flagged passages
note: qa.py covers missing paragraphs, id drift, metadata leaks,
straight/unbalanced quotes, double punctuation, title drift,
English residue and system->系统. Full approved-name consistency
and added/dropped-sentence detection are still todo.
[x] T6.2 Produce a bilingual side-by-side review table.
artifacts: tools/translation/review.py -> output/review.html
[~] T6.3 Produce a severity-sorted issues queue (block/major/minor).
note: qa.py --json-out and the critic issues are merged in
review.html; a standalone queue is still todo.
[ ] T6.4 Native-speaker review of the full book.
[ ] T6.5 Apply the review fixes and re-render.
[ ] T6.6 Final read-through of the ODT/PDF (TOC, page breaks, headers).
---------------------------------------------------------------------------
PHASE 7 — DELIVERY
---------------------------------------------------------------------------
[ ] T7.1 Final deliverables: AsciiDoc, ODT, PDF, glossary, issue log.
[ ] T7.2 Version and date the output; keep the source hash.
[ ] T7.3 Archive the production outside the public Git repository.
[ ] T7.4 Optional: EPUB and print-ready PDF.
---------------------------------------------------------------------------
PHASE 8 — AUDIO ALIGNMENT (FUTURE)
---------------------------------------------------------------------------
[x] T8.1 Preserve source utterance indices on every translated paragraph.
[ ] T8.2 1:1 utterance alignment for TTS (split or align the paragraph
translation back to the 4,319 source utterances).
[ ] T8.3 Update the UmbraAudio import format to accept paragraph-level
translations with source ranges.
[ ] T8.4 Re-render the audiobook in Chinese and compare timing.
---------------------------------------------------------------------------
CRITICAL PATH
---------------------------------------------------------------------------
Q1/Q3/Q4/Q5 (source, title, names, script)
-> T3.6 sample approval
-> T5.1 full-book draft
-> T5.2 critic
-> T6.4 native review
-> T7.1 delivery

2
pyproject.toml
View File

@@ -32,7 +32,7 @@ include = ["agent_comm*", "umbratrans_web*"]
umbratrans_web = ["migrations/*.sql", "migrations/*.md"] umbratrans_web = ["migrations/*.sql", "migrations/*.md"]
[tool.pytest.ini_options] [tool.pytest.ini_options]
testpaths = ["umbratrans_web/tests", "agent_comm/tests"] testpaths = ["umbratrans_web/tests", "agent_comm/tests", "tools/translation/tests"]
python_files = ["test_*.py"] python_files = ["test_*.py"]
[tool.ruff] [tool.ruff]

126
tools/translation/README.md Normal file
View File

@@ -0,0 +1,126 @@
# Deal Hunter Chinese translation tools
This directory contains the translation and rendering pipeline used to produce
the Simplified Chinese edition of *Deal Hunter*. It is deliberately separate
from the web application: the book text and the translations are copyrighted
and are **not** committed to this repository.
## Where the data lives
```
productions/deal-hunter-zh/ # git-ignored
source/book.json # extracted English source model
glossary/glossary.json # names, places, terms, style rules
runs/<chapter>/*.json # analyses, drafts, critiques, final text
runs/usage.jsonl # per-call token/latency log
output/*.asciidoc, *.odt, *.pdf # review artefacts
```
The source bundle is the UmbraAudio controller bundle:
```
/home/umbra/audio-read/bundles/dh-bundle/manifest.jsonl
```
Its sha256 is recorded in `Status.txt` and in `source/book.json`.
## Passes
| Pass | What it does | Model |
|---|---|---|
| 0 | Paragraph/scene/chunk assembly; keeps source utterance indices | none |
| 1 | Scene intent: summary, register, tone, participants, risks | `lfm2.5-2.6b` |
| 2 | Context-aware draft translation with glossary and analysis | `qwen3.5-9b` |
| 3 | Chapter voice/term consistency (optional) | critic model |
| 4 | Bilingual critic: meaning, pronoun, irony, term, omission | `deepseek-r1-8b`, fallback `qwen3.5-9b` |
Every analysis, draft and critique is a file under `runs/<chapter>/`. A run can
be stopped and resumed; existing artefacts are reused unless `--force` is
given. LLM responses are cached by prompt hash, so editing a prompt
invalidates the old answer.
The drafter is told to translate intent, not words, and is given a glossary,
character voices, the scene analysis, and the previous chunk's summary. The
critic uses a different model and returns a severity-sorted issue list; it does
not rewrite the text.
## Commands
Extract the source:
```bash
python3 tools/translation/book.py \
--out productions/deal-hunter-zh/source/book.json
```
Translate a chapter:
```bash
python3 tools/translation/pipeline.py \
--book productions/deal-hunter-zh/source/book.json \
--glossary productions/deal-hunter-zh/glossary/glossary.json \
--run-dir productions/deal-hunter-zh/runs \
--chapters ch-6
```
Selectors: `all`, `front`, `body`, `back`, or a comma-separated list of
chapter indices / ids / titles (`-4,-3,-2,-1` must be quoted or passed as
`--chapters=-4,-3`).
Render:
```bash
python3 tools/translation/render.py \
--book productions/deal-hunter-zh/source/book.json \
--run-dir productions/deal-hunter-zh/runs \
--out-dir productions/deal-hunter-zh/output \
--sample
```
Mechanical QA:
```bash
python3 tools/translation/qa.py \
--book productions/deal-hunter-zh/source/book.json \
--run-dir productions/deal-hunter-zh/runs \
--json-out productions/deal-hunter-zh/runs/qa.json \
--strict
```
## Models
The default server is the LAN LM Studio instance at
`http://192.168.123.16:1234/v1`. Override with:
```
UMBRA_TRANSLATION_BASE_URL
UMBRA_TRANSLATION_MODEL
UMBRA_TRANSLATION_ANALYST_MODEL
UMBRA_TRANSLATION_CRITIC_MODEL
```
Qwen models emit reasoning tokens by default; the client sends
`reasoning_effort: "none"` so the budget is spent on the translation. The small
analyst model rejects LM Studio's JSON-schema grammar, so its answer is parsed
from text and repaired if necessary.
## Rendering
The AsciiDoc file is the canonical text output. The ODT is produced from a Flat
ODF document (`.fodt`) so that page layout, styles and the table of contents are
under our control. LibreOffice converts it and updates the TOC through UNO; a
plain `--convert-to` leaves the index empty. A PDF is exported for quick review.
Front matter, body and back matter use different styles. Chapter headings use
Chinese numerals (`第一章　打捞`). Body text is 11 pt Noto Serif CJK SC with
1.5 line spacing and a two-character first-line indent.
## What this is not
- It is not a native-speaker-reviewed translation. The critic and the QA script
catch mechanical defects; a human editor is still required.
- It does not recover the original DOCX paragraphing. The bundle is
sentence-level, so paragraphs are reconstructed and can be reviewed.
- It does not produce a 1:1 utterance alignment for TTS. Source ranges are
preserved on every paragraph; splitting or aligning back to the 4,319
utterances is a separate pass.

409
tools/translation/book.py Normal file
View File

@@ -0,0 +1,409 @@
# Copyright 2026 Umbra Tools (umbratools.com)
# SPDX-License-Identifier: Apache-2.0
"""Load the Deal Hunter controller bundle into a book model.
The bundle is a JSONL manifest produced by UmbraAudio. One utterance is a
sentence or a short paragraph; the original DOCX paragraph boundaries are not
in the bundle. This module therefore reconstructs readable paragraphs with a
documented heuristic and keeps the source utterance indices on every
paragraph so a later audio-alignment step can still map back.
Nothing here calls an LLM.
"""
from __future__ import annotations
import argparse
import hashlib
import json
import re
from dataclasses import asdict, dataclass, field
from pathlib import Path
DEFAULT_BUNDLE = Path("/home/umbra/audio-read/bundles/dh-bundle/manifest.jsonl")
# Chinese chapter titles. Kept here, not inferred, so every render is stable.
# Reviewable in one place; the English title stays on the paragraph.
TITLE_ZH = {
"Sample Track": "音频版序言：声音介绍",
"Opening Track": "开篇说明",
"Copyright": "版权信息",
"Author's Note": "作者的话",
"Salvage": "打捞",
"Family Reunion": "家庭团聚",
"Selling Salvage": "出售打捞物",
"Frustration": "挫败",
"Next Bounty": "下一个赏金",
"Time for a Change": "改变的时刻",
"Trade Hub Vince-4": "贸易枢纽文斯-4",
"Can I Join the Crew": "我能加入船员吗",
"Salvage from Crashed Fighters": "坠毁战机的残骸",
"Where Is the Captain?": "船长在哪里？",
"Turn in Fugitives": "上交逃犯",
"Her New Ship": "她的新船",
"Introducing the new XO": "新任副舰长",
"A New Ship Salvage": "新船打捞",
"The Poker Game Distraction": "扑克牌局掩护",
"Deal Hunter Bounty": "Deal Hunter 的赏金",
"Selecting Bounties": "挑选赏金",
"Shuttle Scans": "穿梭机扫描",
"The Pirate Trap": "海盗陷阱",
"The Vid Interview": "视频采访",
"Hunting Pirates": "追猎海盗",
"The Friendly Pirate": "友好的海盗",
"Aunt Vicky": "维姬姑妈",
"Preparation": "准备",
"The Coronation": "加冕典礼",
"Long Live the King": "国王万岁",
"Let’s Make It Better": "让它变得更好",
"Let's Make It Better": "让它变得更好",
"Closing Track": "后记：感谢收听",
}
MAX_NARRATION_SENTENCES = 3
MAX_NARRATION_WORDS = 58
@dataclass
class Utterance:
index: int
speaker: str
type: str
text: str
@dataclass
class Paragraph:
id: str
speaker: str
type: str
text: str
source_indices: list[int] = field(default_factory=list)
source_ids: list[str] = field(default_factory=list)
@property
def words(self) -> int:
return len(self.text.split())
@dataclass
class Scene:
id: str
index: int
paragraphs: list[Paragraph] = field(default_factory=list)
@dataclass
class Chapter:
id: str
index: int
row: int
title: str
title_zh: str
matter: str
paragraphs: list[Paragraph] = field(default_factory=list)
scenes: list[Scene] = field(default_factory=list)
def scenes_json(self) -> list[dict]:
return [asdict(s) for s in self.scenes]
@dataclass
class Book:
title: str
title_zh: str
author: str
source_language: str
target_language: str
bundle: str
bundle_sha256: str
chapters: list[Chapter] = field(default_factory=list)
def chapter(self, ref: str | int) -> Chapter:
for c in self.chapters:
if str(c.index) == str(ref) or c.id == str(ref):
return c
raise KeyError(f"chapter not found: {ref}")
def to_json(self) -> dict:
return {
"title": self.title,
"title_zh": self.title_zh,
"author": self.author,
"source_language": self.source_language,
"target_language": self.target_language,
"bundle": self.bundle,
"bundle_sha256": self.bundle_sha256,
"chapters": [
{
**{k: v for k, v in asdict(c).items() if k != "scenes"},
"scenes": [asdict(s) for s in c.scenes],
}
for c in self.chapters
],
}
def _sentences(text: str) -> int:
# A conservative sentence count for grouping only. Quotes and ellipses are
# the places where a naive split goes wrong, so count terminators instead.
n = len(re.findall(r"[.!?]+(?:[”’\"']|$|\s)", text))
return max(1, n)
def _classify(index: int, title: str) -> str:
if index < 0:
if title in {"Sample Track", "Opening Track"}:
return "front_audio"
if title == "Copyright":
return "front_copyright"
if title == "Author's Note":
return "front_note"
return "front"
if title == "Closing Track" or index > 26:
return "back"
return "body"
def _clean_title(title: str) -> str:
return title.replace("’", "'").strip()
def _title_zh(title: str) -> str:
key = _clean_title(title)
if key in TITLE_ZH:
return TITLE_ZH[key]
# Unknown title: leave a visible marker rather than invent a translation.
return f"[待译] {title}"
def load_manifest(path: Path = DEFAULT_BUNDLE) -> tuple[list[dict], dict]:
raw = path.read_bytes()
digest = hashlib.sha256(raw).hexdigest()
records = [json.loads(line) for line in raw.decode("utf-8").splitlines() if line.strip()]
return records, {"path": str(path), "sha256": digest}
def build_book(path: Path = DEFAULT_BUNDLE) -> Book:
records, meta = load_manifest(path)
book_rec = next(r for r in records if r.get("kind") == "book")
chapter_recs = sorted(
(r for r in records if r.get("kind") == "chapter"),
key=lambda r: r["chapter_index"],
)
utterances: dict[str, list[dict]] = {}
for r in records:
if r.get("kind") == "utterance":
utterances.setdefault(r["chapter_id"], []).append(r)
for rows in utterances.values():
rows.sort(key=lambda r: r["utterance_index"])
book = Book(
title="Deal Hunter",
title_zh="交易猎人",
author="B. D. Murphy",
source_language="en",
target_language="zh-CN",
bundle=meta["path"],
bundle_sha256=meta["sha256"],
)
for cr in chapter_recs:
index = int(cr["chapter_index"])
title = cr["title"]
matter = _classify(index, title)
rows = utterances.get(cr["id"], [])
paragraphs = _paragraphs(cr["id"], title, matter, rows)
chapter = Chapter(
id=cr["id"],
index=index,
row=int(cr["database_row_id"]),
title=title,
title_zh=_title_zh(title),
matter=matter,
paragraphs=paragraphs,
)
chapter.scenes = _scenes(cr["id"], paragraphs)
book.chapters.append(chapter)
return book
def _paragraphs(chapter_id: str, title: str, matter: str, rows: list[dict]) -> list[Paragraph]:
if not rows:
return []
# The parser leaves the spoken chapter announcement in the first utterance
# of Salvage ("Deal Hunter. Chapter one, Salvage."). It is a production
# cue, not prose, so it must not be translated into the reading edition.
if matter == "body" and rows[0]["text"].strip().startswith("Deal Hunter. Chapter"):
rows = rows[1:]
# The author's-note heading is an utterance, not a paragraph.
if matter == "front_note" and rows and rows[0]["text"].strip().rstrip(".").lower() in {
"author's note",
"authors note",
"author’s note",
}:
rows = rows[1:]
paragraphs: list[Paragraph] = []
scene_no = 0
buf: list[dict] = []
buf_words = 0
buf_sentences = 0
def flush() -> None:
nonlocal buf, buf_words, buf_sentences
if not buf:
return
paragraphs.append(
Paragraph(
id=f"{chapter_id}-p{len(paragraphs):04d}",
speaker=buf[0].get("speaker_name_resolved") or "narrator",
type=buf[0].get("utterance_type") or "narration",
text=" ".join(r["text"].strip() for r in buf),
source_indices=[int(r["utterance_index"]) for r in buf],
source_ids=[r["id"] for r in buf],
)
)
buf = []
buf_words = 0
buf_sentences = 0
for row in rows:
utype = row.get("utterance_type") or "narration"
speaker = row.get("speaker_name_resolved") or "narrator"
text = (row.get("text") or "").strip()
if utype == "scene_break":
flush()
scene_no += 1
paragraphs.append(
Paragraph(
id=f"{chapter_id}-break{scene_no}",
speaker="",
type="scene_break",
text="* * *",
source_indices=[int(row["utterance_index"])],
source_ids=[row["id"]],
)
)
continue
if matter.startswith("front") or matter == "back":
# Front and back matter are line-oriented: a copyright line and an
# author's-note bullet are not the same thing as a prose paragraph.
flush()
paragraphs.append(
Paragraph(
id=f"{chapter_id}-p{len(paragraphs):04d}",
speaker=speaker,
type=utype,
text=text,
source_indices=[int(row["utterance_index"])],
source_ids=[row["id"]],
)
)
continue
if utype == "dialogue":
# Each speech turn is its own paragraph.
flush()
paragraphs.append(
Paragraph(
id=f"{chapter_id}-p{len(paragraphs):04d}",
speaker=speaker,
type=utype,
text=text,
source_indices=[int(row["utterance_index"])],
source_ids=[row["id"]],
)
)
continue
# Narration: group consecutive sentences into a readable paragraph.
if buf and (
buf[0].get("speaker_name_resolved") != speaker
or buf[0].get("utterance_type") != utype
):
flush()
sentences = _sentences(text)
if buf and (buf_sentences + sentences > MAX_NARRATION_SENTENCES or buf_words + len(text.split()) > MAX_NARRATION_WORDS):
flush()
buf.append(row)
buf_words += len(text.split())
buf_sentences += sentences
if buf_sentences >= MAX_NARRATION_SENTENCES or buf_words >= MAX_NARRATION_WORDS:
flush()
flush()
# Drop a trailing scene-break marker.
while paragraphs and paragraphs[-1].type == "scene_break":
paragraphs.pop()
return paragraphs
def _scenes(chapter_id: str, paragraphs: list[Paragraph]) -> list[Scene]:
scenes: list[Scene] = []
current = Scene(id=f"{chapter_id}-s{len(scenes)}", index=len(scenes))
for p in paragraphs:
if p.type == "scene_break":
if current.paragraphs:
scenes.append(current)
current = Scene(id=f"{chapter_id}-s{len(scenes)}", index=len(scenes))
continue
current.paragraphs.append(p)
if current.paragraphs:
scenes.append(current)
return scenes
def save(book: Book, out: Path) -> None:
out.parent.mkdir(parents=True, exist_ok=True)
out.write_text(json.dumps(book.to_json(), ensure_ascii=False, indent=2) + "\n", encoding="utf-8")
def load(path: Path) -> dict:
return json.loads(path.read_text(encoding="utf-8"))
def summary(book: Book) -> str:
lines = [
f"Book: {book.title} / {book.title_zh}",
f"Author: {book.author}",
f"Bundle: {book.bundle}",
f"Bundle sha256: {book.bundle_sha256}",
"",
f"{'idx':>4} {'matter':16} {'paras':>6} {'words':>7} title",
]
for c in book.chapters:
words = sum(p.words for p in c.paragraphs if p.type != "scene_break")
lines.append(
f"{c.index:>4} {c.matter:16} {len(c.paragraphs):>6} {words:>7} {c.title} / {c.title_zh}"
)
body = [c for c in book.chapters if c.matter == "body"]
lines += [
"",
f"Front matter chapters: {sum(1 for c in book.chapters if c.matter.startswith('front'))}",
f"Body chapters: {len(body)}",
f"Back matter chapters: {sum(1 for c in book.chapters if c.matter == 'back')}",
f"Paragraphs: {sum(len(c.paragraphs) for c in book.chapters)}",
f"Words: {sum(p.words for c in book.chapters for p in c.paragraphs if p.type != 'scene_break')}",
]
return "\n".join(lines)
def main() -> None:
ap = argparse.ArgumentParser(description="Extract the Deal Hunter bundle into a book model.")
ap.add_argument("--bundle", type=Path, default=DEFAULT_BUNDLE)
ap.add_argument("--out", type=Path, required=True)
args = ap.parse_args()
book = build_book(args.bundle)
save(book, args.out)
print(summary(book))
print(f"\nwrote {args.out}")
if __name__ == "__main__":
main()

266
tools/translation/llm.py Normal file
View File

@@ -0,0 +1,266 @@
# Copyright 2026 Umbra Tools (umbratools.com)
# SPDX-License-Identifier: Apache-2.0
"""Minimal OpenAI-compatible client for the local LM Studio server.
Only the standard library is used. The Qwen models on BigBox emit reasoning
tokens unless told not to; ``reasoning_effort="none"`` is what makes a
translation call return prose instead of a thinking trace.
"""
from __future__ import annotations
import hashlib
import json
import os
import re
import time
import urllib.error
import urllib.request
from dataclasses import dataclass
from pathlib import Path
from typing import Any
DEFAULT_BASE_URL = os.environ.get("UMBRA_TRANSLATION_BASE_URL", "http://192.168.123.16:1234/v1")
DEFAULT_MODEL = os.environ.get("UMBRA_TRANSLATION_MODEL", "qwen/qwen3.5-9b")
DEFAULT_ANALYST = os.environ.get(
"UMBRA_TRANSLATION_ANALYST_MODEL",
"lfm2.5-2.6b-qwen3.8-turbo-brilliance-power-x12-neo-max",
)
DEFAULT_CRITIC = os.environ.get("UMBRA_TRANSLATION_CRITIC_MODEL", "deepseek/deepseek-r1-0528-qwen3-8b")
@dataclass
class Usage:
model: str
prompt_tokens: int
completion_tokens: int
elapsed_s: float
class LLMError(RuntimeError):
pass
class LLM:
def __init__(
self,
base_url: str = DEFAULT_BASE_URL,
cache_dir: Path | None = None,
usage_log: Path | None = None,
timeout: int = 600,
) -> None:
self.base_url = base_url.rstrip("/")
self.cache_dir = cache_dir
self.usage_log = usage_log
self.timeout = timeout
if cache_dir:
cache_dir.mkdir(parents=True, exist_ok=True)
# ------------------------------------------------------------------ HTTP
def _post(self, payload: dict[str, Any]) -> dict[str, Any]:
body = json.dumps(payload, ensure_ascii=False).encode("utf-8")
req = urllib.request.Request(
self.base_url + "/chat/completions",
data=body,
headers={"Content-Type": "application/json"},
)
try:
with urllib.request.urlopen(req, timeout=self.timeout) as resp:
return json.loads(resp.read().decode("utf-8"))
except urllib.error.HTTPError as exc:
detail = exc.read().decode("utf-8", "replace")[:800]
raise LLMError(f"HTTP {exc.code}: {detail}") from exc
except urllib.error.URLError as exc:
raise LLMError(f"cannot reach {self.base_url}: {exc.reason}") from exc
def chat(
self,
system: str,
user: str,
*,
model: str = DEFAULT_MODEL,
temperature: float = 0.3,
max_tokens: int = 4096,
reasoning: bool = False,
json_mode: bool = False,
cache_key: str | None = None,
retries: int = 3,
) -> tuple[str, Usage]:
digest = hashlib.sha256(
json.dumps(
{
"model": model,
"temperature": temperature,
"system": system,
"user": user,
"json_mode": json_mode,
},
ensure_ascii=False,
sort_keys=True,
).encode("utf-8")
).hexdigest()
cache_path = None
if self.cache_dir:
# Tie the cache file to the prompt hash as well as the logical key.
# Otherwise a prompt edit silently reuses the old answer.
safe_key = re.sub(r"[^A-Za-z0-9_.-]+", "_", cache_key or "chat")
cache_path = self.cache_dir / f"{safe_key}.{digest[:12]}.json"
if cache_path.exists():
cached = json.loads(cache_path.read_text(encoding="utf-8"))
return cached["content"], Usage(
model=model,
prompt_tokens=cached.get("prompt_tokens", 0),
completion_tokens=cached.get("completion_tokens", 0),
elapsed_s=0.0,
)
payload: dict[str, Any] = {
"model": model,
"messages": [
{"role": "system", "content": system},
{"role": "user", "content": user},
],
"temperature": temperature,
"max_tokens": max_tokens,
}
if not reasoning:
payload["reasoning_effort"] = "none"
if json_mode:
# LM Studio rejects the OpenAI "json_object" shorthand but accepts a
# permissive json_schema. This keeps the model from wrapping the
# answer in prose, which used to cost a full repair call.
payload["response_format"] = {
"type": "json_schema",
"json_schema": {
"name": "translation_payload",
"schema": {"type": "object", "additionalProperties": True},
},
}
last: Exception | None = None
for attempt in range(retries):
started = time.time()
try:
data = self._post(payload)
message = data["choices"][0]["message"]
content = (message.get("content") or "").strip()
if not content:
# Reasoning models can spend the whole budget thinking.
raise LLMError(
"empty content "
f"(reasoning_tokens={data.get('usage', {}).get('completion_tokens_details', {}).get('reasoning_tokens')})"
)
usage = data.get("usage", {})
elapsed = time.time() - started
u = Usage(
model=model,
prompt_tokens=int(usage.get("prompt_tokens", 0)),
completion_tokens=int(usage.get("completion_tokens", 0)),
elapsed_s=elapsed,
)
if cache_path:
cache_path.write_text(
json.dumps(
{
"content": content,
"prompt_tokens": u.prompt_tokens,
"completion_tokens": u.completion_tokens,
"model": model,
},
ensure_ascii=False,
indent=2,
)
+ "\n",
encoding="utf-8",
)
if self.usage_log:
self.usage_log.parent.mkdir(parents=True, exist_ok=True)
with self.usage_log.open("a", encoding="utf-8") as fh:
fh.write(
json.dumps(
{
"ts": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime()),
"model": model,
"prompt_tokens": u.prompt_tokens,
"completion_tokens": u.completion_tokens,
"elapsed_s": round(elapsed, 2),
"cache_key": cache_key,
}
)
+ "\n"
)
return content, u
except Exception as exc: # noqa: BLE001 - retry boundary
last = exc
if attempt < retries - 1:
time.sleep(2**attempt)
raise LLMError(f"chat failed after {retries} attempts: {last}")
# ------------------------------------------------------------------ JSON
def chat_json(
self,
system: str,
user: str,
*,
model: str = DEFAULT_MODEL,
temperature: float = 0.2,
max_tokens: int = 4096,
cache_key: str | None = None,
json_mode: bool = True,
) -> tuple[Any, Usage]:
content, usage = self.chat(
system,
user,
model=model,
temperature=temperature,
max_tokens=max_tokens,
reasoning=False,
json_mode=json_mode,
cache_key=cache_key,
)
try:
return parse_json(content), usage
except ValueError:
# One repair attempt, with the malformed output in front of the model.
repair = (
"The previous answer was not valid JSON. Return only valid JSON, "
"no code fence, no commentary. The required shape was:\n"
f"{user}\n\nPrevious answer:\n{content[:6000]}"
)
fixed, usage2 = self.chat(
system,
repair,
model=model,
temperature=0.0,
max_tokens=max_tokens,
reasoning=False,
json_mode=json_mode,
cache_key=(cache_key + "-repair") if cache_key else None,
)
return parse_json(fixed), usage2
def parse_json(text: str) -> Any:
"""Parse a JSON object/array from a model answer.
Accepts a code fence, leading prose, or a trailing sentence. Raises
ValueError when no complete JSON value can be decoded.
"""
text = text.strip()
if text.startswith("```"):
text = re.sub(r"^```[a-zA-Z0-9_-]*\s*", "", text)
text = re.sub(r"\s*```$", "", text).strip()
try:
return json.loads(text)
except json.JSONDecodeError:
pass
decoder = json.JSONDecoder()
for i, ch in enumerate(text):
if ch in "{[":
try:
value, _ = decoder.raw_decode(text[i:])
return value
except json.JSONDecodeError:
continue
raise ValueError(f"no JSON value found in answer: {text[:300]!r}")

198
tools/translation/names.py Normal file
View File

@@ -0,0 +1,198 @@
# Copyright 2026 Umbra Tools (umbratools.com)
# SPDX-License-Identifier: Apache-2.0
"""Collect proper-name and term candidates for the glossary.
Two sources are merged:
1. Terms proposed by the scene analyst (``proposed_terms`` in every
``*.analysis.json``).
2. Capitalised words in the English source that appear mid-sentence, which is
a decent signal for a name but filters out sentence-initial function words.
Nothing is written into the glossary automatically. The output is a proposal
list for the operator to accept, edit or reject, because an unapproved
transliteration is worse than an English name left in place.
"""
from __future__ import annotations
import argparse
import json
import re
import sys
from collections import defaultdict
from pathlib import Path
sys.path.insert(0, str(Path(__file__).resolve().parent))
from pipeline import LATIN_ALLOW # noqa: E402
STOPWORDS = {
"a", "about", "after", "again", "against", "all", "also", "am", "an", "and",
"any", "are", "aren", "as", "at", "back", "be", "because", "been", "before",
"being", "below", "between", "both", "but", "by", "can", "could", "did",
"didn", "do", "does", "doesn", "doing", "don", "down", "during", "each",
"even", "ever", "every", "few", "for", "from", "get", "gets", "getting",
"give", "go", "going", "gone", "good", "got", "had", "has", "have",
"having", "he", "her", "here", "hers", "herself", "him", "himself", "his",
"how", "however", "i", "if", "in", "into", "is", "isn", "it", "its",
"itself", "just", "know", "let", "like", "look", "looking", "made", "make",
"many", "may", "maybe", "me", "might", "mine", "more", "most", "much",
"must", "my", "myself", "need", "no", "nor", "not", "now", "of", "off",
"on", "once", "one", "only", "or", "other", "ought", "our", "ours",
"ourselves", "out", "over", "own", "perhaps", "please", "put", "quite",
"rather", "really", "right", "said", "same", "say", "saying", "see",
"seen", "she", "should", "since", "so", "some", "something", "still",
"such", "sure", "take", "than", "thank", "thanks", "that", "the", "their",
"theirs", "them", "themselves", "then", "there", "these", "they", "thing",
"things", "think", "this", "those", "though", "through", "to", "too",
"under", "until", "up", "upon", "us", "very", "was", "wasn", "we", "well",
"were", "weren", "what", "when", "where", "whether", "which", "while",
"who", "whom", "why", "will", "with", "within", "without", "would",
"yes", "yet", "you", "your", "yours", "yourself", "yourselves",
"okay", "ok", "hi", "hello", "hey", "oh", "ah", "um", "uh", "hmm",
"mr", "mrs", "ms", "dr", "sir", "ma", "am", "captain", "princess",
"doctor", "aunt", "uncle", "nanny", "hunter",
"everyone", "everything", "someone", "something", "nothing", "anyone",
"anything", "people", "one", "two", "three", "four", "five", "six",
"seven", "eight", "nine", "ten", "eleven", "twelve", "first", "second",
"third", "fourth", "fifth", "next", "last", "inside", "outside", "based",
"turning", "checking", "looking", "going", "trying", "waiting",
"standing", "sitting", "walking", "talking", "several", "tell",
"additionally", "besides", "however", "therefore", "meanwhile",
"suddenly", "finally", "actually", "apparently", "obviously",
"unfortunately", "luckily", "instead", "as", "at", "if", "is", "it",
"its", "be", "by", "do", "does", "did", "for", "from", "had", "has",
"he", "her", "him", "his", "how", "into", "me", "my", "of", "on", "or",
"our", "she", "so", "than", "that", "the", "their", "them", "then",
"there", "these", "they", "this", "those", "to", "up", "us", "was",
"we", "were", "what", "when", "where", "which", "who", "why", "will",
"with", "would", "you", "your",
}
SENTENCE_BOUNDARY = set('.!?…"“”\n')
def _load(path: Path) -> dict:
return json.loads(path.read_text(encoding="utf-8"))
def _glossary_keys(glossary: dict) -> set[str]:
keys: set[str] = set()
for section in ("characters", "roles", "places", "terms"):
for key in glossary.get(section, {}) or {}:
keys.add(key.lower())
for extra in (glossary.get("title") or {}).values():
if isinstance(extra, str):
keys.add(extra.lower())
return keys
def _analyst_terms(run_dir: Path) -> dict[str, dict]:
out: dict[str, dict] = {}
for path in sorted(run_dir.glob("*/*.analysis.json")):
try:
data = _load(path)
except (OSError, json.JSONDecodeError):
continue
for term in data.get("proposed_terms", []) or []:
if not isinstance(term, dict):
continue
en = str(term.get("en") or "").strip().lower()
if not en:
continue
entry = out.setdefault(en, {"en": term.get("en"), "zh": term.get("zh"), "why": [], "count": 0})
entry["count"] += 1
why = term.get("why")
if why and why not in entry["why"]:
entry["why"].append(why)
if term.get("zh") and not entry.get("zh"):
entry["zh"] = term.get("zh")
return out
def _source_names(book: dict, only: set[str] | None) -> dict[str, dict]:
counts: dict[str, dict] = defaultdict(lambda: {"count": 0, "contexts": []})
pattern = re.compile(r"\b([A-Z][a-zA-Z'’-]+(?:\s+[A-Z][a-zA-Z'’-]+){0,2})\b")
for chapter in book["chapters"]:
if only and chapter["id"] not in only:
continue
for p in chapter["paragraphs"]:
text = p.get("text") or ""
for m in pattern.finditer(text):
name = m.group(1).strip()
name = re.sub(r"[’']s$", "", name).strip("-'")
if not name or "’" in name or "'" in name:
continue
words = name.split()
if all(w.lower() in STOPWORDS for w in words):
continue
if len(words) == 1 and words[0].lower().endswith("ly"):
continue
first = words[0].lower()
if first in STOPWORDS and len(words) == 1:
continue
start = m.start()
prev = (text[:start].rstrip() or " ")[-1]
mid_sentence = prev not in SENTENCE_BOUNDARY
if not mid_sentence and len(words) == 1:
# A single capitalised word at the start of a sentence is
# too weak a signal; require at least one mid-sentence use.
counts[name]["_initial"] = counts[name].get("_initial", 0) + 1
continue
counts[name]["count"] += 1
if len(counts[name]["contexts"]) < 3:
lo = max(0, start - 40)
hi = min(len(text), m.end() + 40)
counts[name]["contexts"].append("…" + text[lo:hi].replace("\n", " ") + "…")
return {k: v for k, v in counts.items() if v["count"] > 0}
def main() -> None:
ap = argparse.ArgumentParser(description="Propose glossary entries from the source and analyses.")
ap.add_argument("--book", type=Path, required=True)
ap.add_argument("--run-dir", type=Path, required=True)
ap.add_argument("--glossary", type=Path, required=True)
ap.add_argument("--out", type=Path, required=True)
ap.add_argument("--chapters", help="comma-separated chapter ids (default: all)")
args = ap.parse_args()
book = _load(args.book)
glossary = _load(args.glossary)
only = {c.strip() for c in args.chapters.split(",")} if args.chapters else None
known = _glossary_keys(glossary)
analyst = _analyst_terms(args.run_dir)
names = _source_names(book, only)
lines = ["# Deal Hunter glossary candidates", "", "未自动写入 glossary.json；请人工批准。", ""]
lines.append("## 分析器提议的术语 (proposed_terms)")
lines.append("")
lines.append("| en | zh | 次数 | 说明 |")
lines.append("|---|---|---|---|")
for en, entry in sorted(analyst.items()):
if en in known:
continue
why = " / ".join(entry["why"])[:120]
lines.append(f"| {entry['en']} | {entry.get('zh') or ''} | {entry['count']} | {why} |")
lines.append("")
lines.append("## 源码中的专名候选 (mid-sentence capitalised)")
lines.append("")
lines.append("| name | 次数 | 上下文 |")
lines.append("|---|---|---|")
for name, entry in sorted(names.items(), key=lambda kv: (-kv[1]["count"], kv[0])):
low = name.lower()
if low in known or low in LATIN_ALLOW:
continue
ctx = " <br> ".join(entry["contexts"][:2])
lines.append(f"| {name} | {entry['count']} | {ctx} |")
lines.append("")
args.out.parent.mkdir(parents=True, exist_ok=True)
args.out.write_text("\n".join(lines) + "\n", encoding="utf-8")
print(f"wrote {args.out}")
print(f" analyst proposals: {len(analyst)} source candidates: {len(names)}")
if __name__ == "__main__":
main()

704
tools/translation/pipeline.py Normal file
View File

@@ -0,0 +1,704 @@
# Copyright 2026 Umbra Tools (umbratools.com)
# SPDX-License-Identifier: Apache-2.0
"""Multi-pass translation orchestration.
Pass 0 is local: paragraphs are grouped and chunked with source indices kept.
Pass 1 analyses intent, Pass 2 drafts, Pass 4 checks meaning with a different
model. Pass 3 (chapter voice consistency) is optional and runs after the
chapter's chunks are drafted.
Every artefact is written to disk and reused on the next run, so a multi-hour
book run can be stopped and resumed.
"""
from __future__ import annotations
import argparse
import json
import re
import sys
import time
from dataclasses import dataclass
from pathlib import Path
sys.path.insert(0, str(Path(__file__).resolve().parent))
from llm import DEFAULT_ANALYST, DEFAULT_CRITIC, DEFAULT_MODEL, LLM, LLMError, Usage # noqa: E402
from prompts import ( # noqa: E402
SYSTEM_ANALYST,
SYSTEM_CRITIC,
SYSTEM_DRAFTER,
analysis_prompt,
consistency_prompt,
critic_prompt,
draft_prompt,
load_glossary,
)
CHARACTER_NOTES = """- 凯因达：聪明、直率、好奇，带一点叛逆；对权威不耐烦，对朋友温暖。不要把她写成娇滴滴的公主。
- Hunter / Deal Hunter：干练、简洁、有冷幽默；是船，也是生意人；不用敬语堆砌。
- 南妮：温和、认真、字面化，偶尔因为太认真而好笑；不要写成卖萌。
- 维多利亚公爵夫人：端庄、克制、关切；对凯因达既有长辈的威严，也有家人的柔软。
- 罗曼诺：热情、商人腔、爱谈价格，但不油滑。
- 旁白：克制、清晰、有画面感；不替人物抒情。"""
# Proper nouns that may legitimately stay in Latin script. Anything else that
# survives in the Chinese text is reported, not silently accepted.
LATIN_ALLOW = {
"ai", "audio", "authorbdmurphy", "b", "blazes", "brutus", "com", "d",
"deal", "dial", "eva", "felucca", "gisfrid", "goodreads", "hunter",
"http", "https", "isbn", "kabir", "kainda", "konrad", "llc", "l2",
"maul", "murphy", "nanny", "niñera", "romano", "sao", "umbra", "vince",
"wakam", "wake", "weston", "willamina", "witold", "worstauthors",
"worstauthor", "www", "zheng", "yi", "adonai", "adric", "albert",
}
def _normalise_quotes(text: str) -> str:
text = re.sub(r'"([^"\n]*)"', r"“\1”", text)
if '"' in text:
parts = text.split('"')
out = parts[0]
for i, part in enumerate(parts[1:]):
out += ("“" if i % 2 == 0 else "”") + part
text = out
if "“" not in text and text.endswith("”"):
text = text[:-1]
opens, closes = text.count("“"), text.count("”")
if opens > closes:
text += "”" * (opens - closes)
elif closes > opens:
for _ in range(closes - opens):
idx = text.rfind("”")
if idx > text.rfind("“"):
text = text[:idx] + text[idx + 1 :]
else:
break
return text
def sanitize(text: str) -> str:
text = text.strip()
text = re.sub(r"^\[[^\]]{1,40}\]\s*", "", text)
text = re.sub(r"^（(?:对话|旁白|叙述)[^）]{0,60}）\s*", "", text)
text = re.sub(r"^\((?:dialogue|narration)[^)]{0,60}\)\s*", "", text, flags=re.I)
text = text.replace("《交易猎人》", "《Deal Hunter》")
text = text.replace("交易猎人", "Deal Hunter")
text = text.replace("“猎人”", "Hunter")
text = text.replace("猎人号", "Deal Hunter")
text = text.replace("亨特", "Hunter")
text = text.replace("B.D. 墨菲", "B·D·墨菲")
text = text.replace("B. D. 墨菲", "B·D·墨菲")
text = text.replace("B.D.墨菲", "B·D·墨菲")
text = text.replace("郑义少", "郑一嫂")
text = text.replace("乌姆布拉音频系统", "Umbra Audio")
text = text.replace("乌姆布拉音频", "Umbra Audio")
text = text.replace("Umbra Audio 星系系统", "Umbra Audio 系统")
text = text.replace("Umbra Audio星系系统", "Umbra Audio 系统")
text = text.replace("音频星系系统", "音频系统")
text = text.replace("在小时前", "一小时前")
text = text.replace("...", "……").replace("--", "——")
return _normalise_quotes(text).strip()
def latin_residue(text: str) -> tuple[list[str], list[str]]:
words = re.findall(r"[A-Za-zÀ-ÿ][A-Za-zÀ-ÿ'’\-]{1,}", text)
common: set[str] = set()
proper: set[str] = set()
for word in words:
base = word.strip("'’-")
if not base or base.lower() in LATIN_ALLOW:
continue
if base.isupper() and len(base) <= 5:
continue
if word[0].isupper():
proper.add(word)
else:
common.add(word)
return sorted(common), sorted(proper)
@dataclass
class Chunk:
id: str
scene_index: int
chunks_in_scene: int
paragraphs: list[dict]
@property
def words(self) -> int:
return sum(len(p.get("text", "").split()) for p in self.paragraphs)
def load_book(path: Path) -> dict:
return json.loads(path.read_text(encoding="utf-8"))
def select_chapters(book: dict, selector: str) -> list[dict]:
chapters = book["chapters"]
if selector in {"all", "*"}:
return chapters
if selector == "front":
return [c for c in chapters if c["matter"].startswith("front")]
if selector == "body":
return [c for c in chapters if c["matter"] == "body"]
if selector == "back":
return [c for c in chapters if c["matter"] == "back"]
wanted = {s.strip() for s in selector.split(",") if s.strip()}
out = []
for c in chapters:
if str(c["index"]) in wanted or c["id"] in wanted or c["title"] in wanted:
out.append(c)
missing = wanted - {str(c["index"]) for c in out} - {c["id"] for c in out} - {c["title"] for c in out}
if missing:
raise SystemExit(f"chapter selector did not match: {sorted(missing)}")
return out
def chunk_chapter(chapter: dict, max_words: int = 260, max_paras: int = 14) -> list[Chunk]:
paras = chapter["paragraphs"]
# Assign scene numbers.
scene_of: dict[str, int] = {}
scene = 0
for p in paras:
if p.get("type") == "scene_break":
scene += 1
continue
scene_of[p["id"]] = scene
raw: list[list[dict]] = []
current: list[dict] = []
current_words = 0
current_scene = None
for p in paras:
if p.get("type") == "scene_break":
if current:
raw.append(current)
current, current_words, current_scene = [], 0, None
continue
p_scene = scene_of.get(p["id"], 0)
if current and (p_scene != current_scene or current_words + len(p["text"].split()) > max_words or len(current) >= max_paras):
raw.append(current)
current, current_words = [], 0
current.append(p)
current_words += len(p["text"].split())
current_scene = p_scene
if current:
raw.append(current)
# Count chunks per scene.
per_scene: dict[int, int] = {}
for group in raw:
s = scene_of.get(group[0]["id"], 0)
per_scene[s] = per_scene.get(s, 0) + 1
chunks: list[Chunk] = []
seen: dict[int, int] = {}
for i, group in enumerate(raw):
s = scene_of.get(group[0]["id"], 0)
seen[s] = seen.get(s, 0) + 1
chunks.append(
Chunk(
id=f"{chapter['id']}-c{i:03d}",
scene_index=s,
chunks_in_scene=per_scene[s],
paragraphs=group,
)
)
return chunks
class Runner:
def __init__(
self,
*,
book_path: Path,
glossary_path: Path,
run_dir: Path,
model: str = DEFAULT_MODEL,
analyst_model: str = DEFAULT_ANALYST,
critic_model: str = DEFAULT_CRITIC,
base_url: str | None = None,
force: bool = False,
critic: bool = True,
consistency: bool = False,
max_words: int = 260,
max_paras: int = 14,
critic_limit: int = 0,
) -> None:
self.book = load_book(book_path)
self.glossary = load_glossary(glossary_path)
self.run_dir = run_dir
self.run_dir.mkdir(parents=True, exist_ok=True)
self.model = model
self.analyst_model = analyst_model
self.critic_model = critic_model
self.force = force
self.do_critic = critic
self.do_consistency = consistency
self.max_words = max_words
self.max_paras = max_paras
self.critic_limit = critic_limit
kwargs = {}
if base_url:
kwargs["base_url"] = base_url
self.llm = LLM(
cache_dir=run_dir / "cache",
usage_log=run_dir / "usage.jsonl",
**kwargs,
)
# ----------------------------------------------------------------- files
def chapter_dir(self, chapter: dict) -> Path:
d = self.run_dir / chapter["id"]
d.mkdir(parents=True, exist_ok=True)
return d
def _read(self, path: Path) -> dict | None:
if path.exists() and not self.force:
return json.loads(path.read_text(encoding="utf-8"))
return None
def _write(self, path: Path, value: object) -> None:
path.write_text(json.dumps(value, ensure_ascii=False, indent=2) + "\n", encoding="utf-8")
# ----------------------------------------------------------------- passes
def analyse(self, chapter: dict, chunk: Chunk, previous_summary: str) -> dict:
path = self.chapter_dir(chapter) / f"{chunk.id}.analysis.json"
cached = self._read(path)
if cached is not None:
return cached
prompt = analysis_prompt(
chapter_title=f"{chapter['title']} / {chapter['title_zh']}",
scene_index=chunk.scene_index,
chunk_index=int(chunk.id.rsplit("c", 1)[-1]),
chunks_in_scene=chunk.chunks_in_scene,
paragraphs=chunk.paragraphs,
previous_summary=previous_summary,
glossary=self.glossary,
)
value, usage = self.llm.chat_json(
SYSTEM_ANALYST,
prompt,
model=self.analyst_model,
temperature=0.2,
max_tokens=1400,
cache_key=f"{chunk.id}.analysis.{self.analyst_model}",
# The small analyst model rejects LM Studio's JSON-schema grammar;
# its answer is parsed from the text instead.
json_mode=False,
)
if isinstance(value, list) and value and isinstance(value[0], dict):
value = value[0]
# Sometimes wrapped as {"analysis": {...}} or similar
if isinstance(value, dict):
for k in ("analysis", "result", "data"):
if isinstance(value.get(k), dict):
value = value[k]
break
if not isinstance(value, dict):
# The analyst is a small model asked for a JSON object, and it
# sometimes answers with a list or a bare string instead. Analysis is
# guidance for the drafter, not a requirement for a valid translation,
# so a malformed answer must not abort a chunk that could otherwise be
# translated. Record that the analysis is degraded and continue with a
# minimal one; the drafter falls back to the source and glossary.
print(
f" {chunk.id}: analyst returned {type(value).__name__}, "
"using a minimal analysis",
flush=True,
)
value = {"degraded": True, "note": "analyst returned non-object JSON"}
value["_usage"] = usage.__dict__
self._write(path, value)
return value
def draft(self, chapter: dict, chunk: Chunk, analysis: dict, previous_summary: str) -> dict:
path = self.chapter_dir(chapter) / f"{chunk.id}.draft.json"
cached = self._read(path)
if cached is not None:
return cached
expected = [p["id"] for p in chunk.paragraphs if p.get("type") != "scene_break"]
prompt = draft_prompt(
chapter_title=f"{chapter['title']} / {chapter['title_zh']}",
scene_index=chunk.scene_index,
chunk_index=int(chunk.id.rsplit("c", 1)[-1]),
chunks_in_scene=chunk.chunks_in_scene,
paragraphs=chunk.paragraphs,
analysis={k: v for k, v in analysis.items() if k != "_usage"},
previous_summary=previous_summary,
glossary=self.glossary,
)
value, usage = self.llm.chat_json(
SYSTEM_DRAFTER,
prompt,
model=self.model,
temperature=0.45,
max_tokens=6000,
cache_key=f"{chunk.id}.draft",
)
out = self._normalise_draft(value, expected)
missing = [i for i in expected if not out.get(i)]
if missing:
out.update(self._draft_missing(chapter, chunk, analysis, missing))
out = self._repair_residue(chapter, chunk, out)
out = self._fix_paragraph_quotes(chunk, out)
warnings: dict[str, dict[str, list[str]]] = {}
for pid, zh in out.items():
common, proper = latin_residue(zh)
if common or proper:
warnings[pid] = {"common_english": common, "proper_names": proper}
result = {
"paragraphs": out,
"analysis": analysis,
"warnings": warnings,
"_usage": usage.__dict__,
}
self._write(path, result)
return result
def _normalise_draft(self, value: object, expected: list[str]) -> dict[str, str]:
out: dict[str, str] = {}
if isinstance(value, dict) and isinstance(value.get("paragraphs"), list):
items = value["paragraphs"]
elif isinstance(value, list):
items = value
elif isinstance(value, dict) and value.get("id") and value.get("zh"):
items = [value]
elif isinstance(value, dict):
# maybe mapping id->zh
for k, v in value.items():
if isinstance(v, str) and k.startswith("ch-"):
out[k] = sanitize(v)
return {k: v for k, v in out.items() if k in expected and v}
else:
raise LLMError(f"drafter returned unexpected JSON: {str(value)[:200]}")
for item in items:
if isinstance(item, dict) and item.get("id") and item.get("zh"):
out[str(item["id"])] = sanitize(str(item["zh"]))
return {k: v for k, v in out.items() if k in expected and v}
def _fix_paragraph_quotes(self, chunk: Chunk, drafts: dict[str, str]) -> dict[str, str]:
"""Remove quotes the model wrapped around unquoted narration."""
for p in chunk.paragraphs:
zh = drafts.get(p["id"])
source = (p.get("text") or "").strip()
if (
zh
and p.get("type") == "narration"
and not source.startswith("“")
and zh.startswith("“")
and zh.endswith("”")
and zh.count("“") == 1
and zh.count("”") == 1
):
drafts[p["id"]] = zh[1:-1].strip()
return drafts
def _repair_residue(self, chapter: dict, chunk: Chunk, drafts: dict[str, str]) -> dict[str, str]:
"""Translate stray lowercase English words in place, without touching names."""
by_id = {p["id"]: p for p in chunk.paragraphs}
for pid, zh in list(drafts.items()):
common, _proper = latin_residue(zh)
if not common:
continue
source = by_id.get(pid, {}).get("text", "")
prompt = f"""下面这段中文译文里混入了英文单词：{", ".join(common)}。
请只把这些英文单词译成中文，其余内容尽量保持不变；不要解释，不要改写剧情。
输出 JSON：{{"zh": "修正后的中文译文"}}
英文原文：
{source}
当前译文：
{zh}"""
try:
value, _ = self.llm.chat_json(
SYSTEM_DRAFTER,
prompt,
model=self.model,
temperature=0.2,
max_tokens=1500,
cache_key=f"{chunk.id}.{pid}.residue",
)
fixed = value.get("zh") if isinstance(value, dict) else None
if fixed:
drafts[pid] = sanitize(str(fixed))
except LLMError as exc:
print(f" {chunk.id}/{pid}: residue repair failed: {exc}", flush=True)
return drafts
def _draft_missing(self, chapter: dict, chunk: Chunk, analysis: dict, missing: list[str]) -> dict[str, str]:
"""Last-resort one-paragraph calls. Keeps a chunk failure from losing the run."""
result: dict[str, str] = {}
by_id = {p["id"]: p for p in chunk.paragraphs}
for pid in missing:
p = by_id[pid]
try:
prompt = draft_prompt(
chapter_title=f"{chapter['title']} / {chapter['title_zh']}",
scene_index=chunk.scene_index,
chunk_index=0,
chunks_in_scene=1,
paragraphs=[p],
analysis=analysis,
previous_summary="",
glossary=self.glossary,
)
value, _ = self.llm.chat_json(
SYSTEM_DRAFTER,
prompt,
model=self.model,
temperature=0.4,
max_tokens=1500,
cache_key=f"{chunk.id}.{pid}.draft",
)
one = self._normalise_draft(value, [pid])
if one.get(pid):
result[pid] = one[pid]
continue
except Exception:
pass
# Fallback to source text or empty to avoid blocking
result[pid] = sanitize(str(p.get("text", "")))
return result
def criticise(self, chapter: dict, chunk: Chunk, draft: dict[str, str]) -> dict:
path = self.chapter_dir(chapter) / f"{chunk.id}.critic.json"
cached = self._read(path)
if cached is not None:
return cached
prompt = critic_prompt(
chapter_title=f"{chapter['title']} / {chapter['title_zh']}",
paragraphs=chunk.paragraphs,
draft=draft,
glossary=self.glossary,
)
try:
value, usage = self.llm.chat_json(
SYSTEM_CRITIC,
prompt,
model=self.critic_model,
temperature=0.15,
max_tokens=6000,
cache_key=f"{chunk.id}.critic.{self.critic_model}",
)
except LLMError as exc:
# A reasoning-only critic produces empty content. Fall back to the
# drafter model rather than silently skipping the check.
print(f" critic {self.critic_model} failed ({exc}); falling back to {self.model}", flush=True)
try:
value, usage = self.llm.chat_json(
SYSTEM_CRITIC,
prompt,
model=self.model,
temperature=0.15,
max_tokens=6000,
cache_key=f"{chunk.id}.critic.{self.model}",
)
except LLMError as exc2:
# Do not abort a multi-hour run because one critique failed.
print(f" critic fallback failed ({exc2}); recording a skipped critique", flush=True)
value = {"issues": [], "verdict": "skipped", "_error": str(exc2)}
usage = Usage(model=self.model, prompt_tokens=0, completion_tokens=0, elapsed_s=0.0)
if isinstance(value, list):
value = {"issues": value, "verdict": "revise" if value else "pass"}
if not isinstance(value, dict):
raise LLMError(f"{chunk.id}: critic returned {type(value).__name__}")
# DeepSeek sometimes invents its own key names. Normalise the common
# aliases so the report and the review queue stay machine-readable.
normalised = []
confirmations = (
"被正确翻译",
"翻译正确",
"已正确",
"无需修改",
"没有问题",
"不存在问题",
"未改变理解",
"英文原文存在",
"原文存在语法错误",
"建议英文",
"英文原文表达",
"原文错误",
"无需修改中文",
)
for raw in value.get("issues", []) or []:
if not isinstance(raw, dict):
continue
problem_text = str(raw.get("problem") or raw.get("explanation") or "")
if any(marker in problem_text for marker in confirmations):
continue
fix_text = str(raw.get("fix") or raw.get("suggestion") or "").strip()
if not fix_text and str(raw.get("severity") or "major").lower() == "minor":
continue
normalised.append(
{
"id": raw.get("id") or raw.get("paragraph_id") or raw.get("sentence_id") or "",
"severity": raw.get("severity") or "major",
"type": raw.get("type") or "meaning",
"problem": raw.get("problem") or raw.get("explanation") or "",
"source_excerpt": raw.get("source_excerpt") or raw.get("english_sentence_part") or "",
"target_excerpt": raw.get("target_excerpt") or raw.get("chinese_sentence_part") or "",
"fix": raw.get("fix") or raw.get("suggestion") or "",
}
)
value["issues"] = normalised
verdict = str(value.get("verdict", "")).lower()
if verdict == "skipped":
value["verdict"] = "skipped"
elif normalised or verdict in {"fail", "revise"}:
value["verdict"] = "revise"
else:
value["verdict"] = "pass"
value["_usage"] = usage.__dict__
self._write(path, value)
return value
def consistency(self, chapter: dict, drafts: dict[str, str], paragraphs: list[dict]) -> dict:
path = self.chapter_dir(chapter) / "consistency.json"
cached = self._read(path)
if cached is not None:
return cached
prompt = consistency_prompt(
chapter_title=f"{chapter['title']} / {chapter['title_zh']}",
paragraphs=paragraphs,
draft=drafts,
glossary=self.glossary,
character_notes=CHARACTER_NOTES,
)
value, usage = self.llm.chat_json(
SYSTEM_CRITIC,
prompt,
model=self.critic_model,
temperature=0.15,
max_tokens=6000,
cache_key=f"{chapter['id']}.consistency",
)
if isinstance(value, list):
value = {"issues": value}
if not isinstance(value, dict):
raise LLMError(f"{chapter['id']}: consistency returned {type(value).__name__}")
value["_usage"] = usage.__dict__
self._write(path, value)
return value
# ----------------------------------------------------------------- driver
def run(self, chapters: list[dict]) -> None:
started = time.time()
for ci, chapter in enumerate(chapters, 1):
chunks = chunk_chapter(chapter, self.max_words, self.max_paras)
print(
f"[{ci}/{len(chapters)}] {chapter['id']} {chapter['title']} / {chapter['title_zh']}: "
f"{len(chapter['paragraphs'])} paragraphs, {len(chunks)} chunks",
flush=True,
)
previous_summary = ""
drafts: dict[str, str] = {}
issues: list[dict] = []
for chunk in chunks:
analysis = self.analyse(chapter, chunk, previous_summary)
previous_summary = str(analysis.get("summary_zh") or previous_summary)
drafted = self.draft(chapter, chunk, analysis, previous_summary)
drafts.update(drafted["paragraphs"])
if self.do_critic and (
self.critic_limit <= 0 or int(chunk.id.rsplit("c", 1)[-1]) < self.critic_limit
):
crit = self.criticise(chapter, chunk, drafted["paragraphs"])
for issue in crit.get("issues", []) or []:
issue = dict(issue)
issue["chunk"] = chunk.id
issues.append(issue)
print(
f" {chunk.id}: {len(drafted['paragraphs'])}/{len([p for p in chunk.paragraphs if p.get('type') != 'scene_break'])} "
f"paras, {len(issues)} issue(s) so far",
flush=True,
)
if self.do_consistency:
cons = self.consistency(chapter, drafts, chapter["paragraphs"])
for issue in cons.get("issues", []) or []:
issue = dict(issue)
issue["type"] = "consistency:" + str(issue.get("type"))
issues.append(issue)
summary = {
"chapter_id": chapter["id"],
"chapter_index": chapter["index"],
"title": chapter["title"],
"title_zh": chapter["title_zh"],
"paragraphs": drafts,
"issues": issues,
"chunks": [c.id for c in chunks],
}
self._write(self.chapter_dir(chapter) / "translation.json", summary)
print(f" wrote {self.chapter_dir(chapter) / 'translation.json'}", flush=True)
print(f"done in {time.time() - started:.0f}s", flush=True)
def report(run_dir: Path) -> str:
lines = []
total_paras = 0
total_issues = 0
for d in sorted(p for p in run_dir.iterdir() if p.is_dir() and p.name.startswith("ch-")):
t = d / "translation.json"
if not t.exists():
lines.append(f"{d.name}: not started")
continue
data = json.loads(t.read_text(encoding="utf-8"))
n = len(data.get("paragraphs", {}))
issues = data.get("issues", [])
total_paras += n
total_issues += len(issues)
by_sev: dict[str, int] = {}
for i in issues:
by_sev[i.get("severity", "?")] = by_sev.get(i.get("severity", "?"), 0) + 1
lines.append(f"{d.name}: {n} paragraphs, {len(issues)} issues {by_sev}")
lines.append(f"TOTAL: {total_paras} paragraphs, {total_issues} issues")
return "\n".join(lines)
def main() -> None:
ap = argparse.ArgumentParser(description="Run the Deal Hunter Chinese translation pipeline.")
ap.add_argument("--book", type=Path, required=True)
ap.add_argument("--glossary", type=Path, required=True)
ap.add_argument("--run-dir", type=Path, required=True)
ap.add_argument("--chapters", default="all", help="all, front, body, back, or comma-separated indices/ids")
ap.add_argument("--model", default=DEFAULT_MODEL)
ap.add_argument("--analyst-model", default=DEFAULT_ANALYST)
ap.add_argument("--critic-model", default=DEFAULT_CRITIC)
ap.add_argument("--base-url", default=None)
ap.add_argument("--force", action="store_true")
ap.add_argument("--no-critic", action="store_true")
ap.add_argument("--consistency", action="store_true")
ap.add_argument("--max-words", type=int, default=260)
ap.add_argument("--max-paras", type=int, default=14)
ap.add_argument("--critic-limit", type=int, default=0, help="criticise only the first N chunks per chapter (0 = all)")
ap.add_argument("--report", action="store_true")
args = ap.parse_args()
if args.report:
print(report(args.run_dir))
return
book = load_book(args.book)
chapters = select_chapters(book, args.chapters)
runner = Runner(
book_path=args.book,
glossary_path=args.glossary,
run_dir=args.run_dir,
model=args.model,
analyst_model=args.analyst_model,
critic_model=args.critic_model,
base_url=args.base_url,
force=args.force,
critic=not args.no_critic,
consistency=args.consistency,
max_words=args.max_words,
max_paras=args.max_paras,
critic_limit=args.critic_limit,
)
runner.run(chapters)
if __name__ == "__main__":
main()

350
tools/translation/prompts.py Normal file
View File

@@ -0,0 +1,350 @@
# Copyright 2026 Umbra Tools (umbratools.com)
# SPDX-License-Identifier: Apache-2.0
"""Prompt construction for the multi-pass translation pipeline.
The prompts follow specs/27: analyse the scene before rendering it, give the
drafter the intent and the glossary, then check meaning with a different
model. They deliberately do not ask for word-for-word equivalence.
"""
from __future__ import annotations
import json
from pathlib import Path
CORE_CHARACTERS = [
"Kainda Walters Foster Murphy",
"Princess Kainda",
"Duchess Victoria",
"Aunt Vicky",
"Deal Hunter",
"Hunter",
"Nanny",
"Trader Romano",
"Romano",
"B. D. Murphy",
"Umbra Audio",
]
CORE_TERMS = [
"star system",
"sector",
"space station",
"salvage",
"bounty",
"bounty hunter",
"jump",
"wake cycle",
"frigate",
"shuttle",
"bridge",
"engineering",
"cargo bay",
"dock",
"XO",
"captain",
"pirate",
"fugitive",
"coronation",
]
def load_glossary(path: Path) -> dict:
return json.loads(path.read_text(encoding="utf-8"))
def _pick(mapping: dict, keys: list[str]) -> list[tuple[str, str]]:
out: list[tuple[str, str]] = []
for key in keys:
if key in mapping:
value = mapping[key]
if isinstance(value, dict):
zh = value.get("zh") or value.get("short") or ""
note = value.get("note")
if note:
zh = f"{zh}（{note}）"
else:
zh = str(value)
if zh:
out.append((key, zh))
return out
def glossary_block(glossary: dict, speakers: set[str], text: str) -> str:
chars = dict(glossary.get("characters", {}))
roles = dict(glossary.get("roles", {}))
places = dict(glossary.get("places", {}))
terms = dict(glossary.get("terms", {}))
keys = list(CORE_CHARACTERS)
for s in sorted(speakers):
if s and s not in keys:
keys.append(s)
char_pairs = _pick({**chars, **roles}, keys)
term_keys = list(CORE_TERMS)
low = text.lower()
for key in terms:
if key.lower() in low and key not in term_keys:
term_keys.append(key)
term_pairs = _pick(terms, term_keys)
place_pairs = _pick(places, list(places))
title = glossary.get("title", {})
lines = ["【书名与专名规则】"]
if title:
en = title.get("en", "Deal Hunter")
sub = title.get("subtitle_zh", "")
lines.append(f"- 书名：{en}；正文书名写作《{en}》" + (f"，副标题可写“{sub}”" if sub else ""))
lines += [
"- Deal Hunter 作为船名、AI 名或书名时保留英文，不译成中文；不要写成《交易猎人》",
"- Hunter 作为 AI 的简称保留英文，不译成“猎人”",
"- Umbra Audio 保留英文，不音译",
"- 只翻译原文出现的称谓；不要补全人物全名、头衔或身份",
"",
"【硬性术语：违反即为错译】",
"- 恒星系 system / star system = 星系；但 ship systems / audio system / computer system = 系统",
"- sector = 星区",
"- space station = 空间站；trade station = 贸易站",
"- jump = 跃迁（不译作“跳跃”）",
"- salvage = 打捞物 / 打捞（不译作“救援”）",
"- bounty = 赏金；bounty hunter = 赏金猎人",
"- XO = 副舰长；captain = 舰长",
"",
"【人物与称谓】",
]
lines += [f"- {en} = {zh}" for en, zh in char_pairs]
if place_pairs:
lines.append("【地名与舰名】")
lines += [f"- {en} = {zh}" for en, zh in place_pairs]
if term_pairs:
lines.append("【术语】")
lines += [f"- {en} = {zh}" for en, zh in term_pairs]
return "\n".join(lines)
def style_block(glossary: dict) -> str:
style = glossary.get("style", {})
return "\n".join(f"- {v}" for v in style.values() if isinstance(v, str))
SYSTEM_ANALYST = """你是一位双语文学编辑，负责在翻译前分析英文小说的场景。
你的任务不是翻译，而是读出这一段文字的意图、语气、人物关系和可能被直译毁掉的地方。
只输出 JSON，不要解释，不要 Markdown 代码围栏。"""
SYSTEM_DRAFTER = """你是一位资深文学译者，把英文太空冒险小说译成简体中文。
你的目标不是逐字对应，而是让中文读者读到和英文读者相同的场景、语气、人物关系和潜台词。
可以重组句子、合并或拆分分句，但不得增删事实、不得替作者解释、不得加译注。
对话要像中文人物在说话；叙述要流畅、克制、有画面感。
严格使用给出的术语表。未列入术语表的人名、地名、舰名保留英文原文，不要自行音译。
不要补全原文没有出现的全名、头衔或身份；原文说“Princess Kainda”，不要写成“凯因达·沃尔特斯·福斯特·墨菲公主”。
Deal Hunter 作为船名、AI 名或书名保留英文；Hunter 作为简称保留英文；Umbra Audio 保留英文。
只输出 JSON，不要解释，不要 Markdown 代码围栏。"""
SYSTEM_CRITIC = """你是一位严格的双语审校。对照英文原文和中文译文，只找意义、指代、语气、术语和遗漏方面的问题。
不要因为“风格不同”而报错；只有当读者会因此理解错、或人物声音被抹平、或事实被改变时才报。
不要改写全文，只报告问题并给出最小修正建议。
只输出 JSON，不要解释，不要 Markdown 代码围栏。"""
def _paragraph_lines(paragraphs: list[dict]) -> str:
lines = []
for p in paragraphs:
if p.get("type") == "scene_break":
lines.append(f'[{p["id"]}] (场景分隔)')
continue
speaker = p.get("speaker") or "narrator"
ptype = p.get("type") or "narration"
lines.append(f'[{p["id"]}] ({ptype}; {speaker}) {p["text"]}')
return "\n".join(lines)
def _draft_paragraph_lines(paragraphs: list[dict]) -> str:
"""Source lines for the drafter: id + text only.
Speaker/type labels are useful to the analyst but the drafter used to copy
them into the Chinese output ("（对话；凯因达）"). The analysis already
carries that information, so the draft prompt keeps the source clean.
"""
lines = []
for p in paragraphs:
if p.get("type") == "scene_break":
continue
lines.append(f'[{p["id"]}] {p["text"]}')
return "\n".join(lines)
def analysis_prompt(
*,
chapter_title: str,
scene_index: int,
chunk_index: int,
chunks_in_scene: int,
paragraphs: list[dict],
previous_summary: str,
glossary: dict,
) -> str:
speakers = {p.get("speaker") or "narrator" for p in paragraphs}
text = " ".join(p.get("text", "") for p in paragraphs)
return f"""书名：Deal Hunter（交易猎人）
本章：{chapter_title}
场景 {scene_index + 1}，第 {chunk_index + 1}/{chunks_in_scene} 段
上一段场景摘要：{previous_summary or "（无，本章开头）"}
{glossary_block(glossary, speakers, text)}
{style_block(glossary)}
请分析下面这段英文。输出 JSON：
{{
"summary_zh": "1-2句中文（80字以内），概括本段发生了什么、人物想要什么、语气如何；供下一段翻译参考",
"register": "正式/中性/口语（15字以内）",
"tone": "例如：紧张、戏谑、疲惫、克制（15字以内）",
"participants": ["本段实际说话或被点名的人物"],
"proposed_terms": [{{"en": "术语表里没有的专名", "zh": "建议译名", "why": "简短理由"}}],
"risks": [{{"id": "段落id", "issue": "直译会错在哪里", "guidance": "译者应注意什么"}}]
}}
要求：risks 最多 4 条，只报真正会被直译毁掉的地方；没有就返回空数组。
proposed_terms 最多 5 条，不要重复术语表已有的词条。
英文原文：
{_paragraph_lines(paragraphs)}"""
def draft_prompt(
*,
chapter_title: str,
scene_index: int,
chunk_index: int,
chunks_in_scene: int,
paragraphs: list[dict],
analysis: dict,
previous_summary: str,
glossary: dict,
) -> str:
speakers = {p.get("speaker") or "narrator" for p in paragraphs}
text = " ".join(p.get("text", "") for p in paragraphs)
analysis_text = json.dumps(analysis, ensure_ascii=False, indent=2)
return f"""书名：Deal Hunter（交易猎人）
本章：{chapter_title}
场景 {scene_index + 1}，第 {chunk_index + 1}/{chunks_in_scene} 段
上一段场景摘要：{previous_summary or "（无，本章开头）"}
{glossary_block(glossary, speakers, text)}
{style_block(glossary)}
翻译前的场景分析（必须遵守其中的语气、人物关系和风险提示）：
{analysis_text}
请把下面英文译为简体中文。输出 JSON：
{{"paragraphs": [{{"id": "段落id", "zh": "中文译文"}}]}}
规则：
- 段落 id 必须与原文一一对应，数量和顺序不变。
- 每个 id 只输出一段中文；不要把两段合并成一个 id。
- 只输出译文。绝对不要把段落 id、说话人、类型、括号注释写进译文。
- 不要输出英文原文，不要输出解释。
- 不要把英文单词留在译文里（术语表允许保留的专名和书名除外）；每个英文实词都必须译成中文。
- 对话用中文“”，并保证每个引号成对；不要把英文直引号 " 原样搬进译文。叙述中不要滥用引号。
- 如果原文有省略号、破折号、打断，保留其节奏；省略号用……，破折号用——。
- 专名严格按术语表；术语表没有的名字保留英文。
- 指恒星系时，system / star system 译作“星系”（如 Albert system = 阿尔伯特星系；236 systems = 236 个星系）。
- 指船载、音频、计算机等设备时，system 译作“系统”（如 ship systems = 飞船系统；Umbra Audio system = Umbra Audio 系统）。不要一律译作“星系”。
英文原文：
{_draft_paragraph_lines(paragraphs)}"""
def critic_prompt(
*,
chapter_title: str,
paragraphs: list[dict],
draft: dict[str, str],
glossary: dict,
) -> str:
speakers = {p.get("speaker") or "narrator" for p in paragraphs}
text = " ".join(p.get("text", "") for p in paragraphs)
pairs = []
for p in paragraphs:
if p.get("type") == "scene_break":
continue
pairs.append(
{
"id": p["id"],
"speaker": p.get("speaker"),
"type": p.get("type"),
"en": p["text"],
"zh": draft.get(p["id"], ""),
}
)
return f"""书名：Deal Hunter（交易猎人）
本章：{chapter_title}
{glossary_block(glossary, speakers, text)}
请逐段对照英文与中文，只报告会让读者理解错、或改变人物声音、或丢失/增加事实的问题。
注意：原文 “236 systems in the Dial space sector” 中的 systems 指恒星系，译作“星系”是正确的，不要改成“系统”。ship systems / audio system 才译作“系统”。
输出 JSON：
{{
"verdict": "pass" 或 "revise",
"issues": [
{{
"id": "段落id",
"severity": "block|major|minor",
"type": "meaning|pronoun|irony|term|omission|addition|register",
"problem": "中文说明问题",
"source_excerpt": "英文片段（不超过30词）",
"target_excerpt": "中文片段（不超过40字）",
"fix": "最小修正建议，不要重译整段"
}}
]
}}
严重度标准：
- block：事实被改变、整句遗漏或增加、说话人/指代错、关键术语错。
- major：读者会明显理解错、反讽或人物声音被抹平、重要细节丢失。
- minor：细微语气或可选润色；不要为了凑数报风格偏好。
id 必须使用输入中的段落 id（例如 ch-6-p0000），不要用句子序号。
键名必须与上面的 JSON 完全一致，不要自创 english_sentence_part 之类的字段。
没有问题时 issues 为空数组，verdict 为 pass。
对照材料：
{json.dumps(pairs, ensure_ascii=False, indent=2)}"""
def consistency_prompt(
*,
chapter_title: str,
paragraphs: list[dict],
draft: dict[str, str],
glossary: dict,
character_notes: str,
) -> str:
speakers = {p.get("speaker") or "narrator" for p in paragraphs}
text = " ".join(p.get("text", "") for p in paragraphs)
pairs = [
{"id": p["id"], "speaker": p.get("speaker"), "zh": draft.get(p["id"], "")}
for p in paragraphs
if p.get("type") != "scene_break"
]
return f"""书名：Deal Hunter（交易猎人）
本章：{chapter_title}
{glossary_block(glossary, speakers, text)}
人物声音约束：
{character_notes}
请检查整章的译名、称谓、术语和人物声音是否一致。只报告需要统一的问题，不要重译。
输出 JSON：
{{"issues": [{{"ids": ["段落id"], "type": "name|term|register|voice", "problem": "...", "fix": "..."}}]}}
译文：
{json.dumps(pairs, ensure_ascii=False, indent=2)}"""

172
tools/translation/qa.py Normal file
View File

@@ -0,0 +1,172 @@
# Copyright 2026 Umbra Tools (umbratools.com)
# SPDX-License-Identifier: Apache-2.0
"""Mechanical QA for the Deal Hunter Chinese translation.
This does not judge literary quality. It catches the defects that can be
detected without a human: missing paragraphs, leaked prompt metadata, stray
English, unbalanced quotes, duplicated punctuation and forbidden title forms.
"""
from __future__ import annotations
import argparse
import json
import re
import sys
from pathlib import Path
sys.path.insert(0, str(Path(__file__).resolve().parent))
from pipeline import latin_residue # noqa: E402
def _load(path: Path) -> dict:
return json.loads(path.read_text(encoding="utf-8"))
def check_chapter(chapter: dict, data: dict) -> tuple[list[dict], dict]:
drafts: dict[str, str] = data.get("paragraphs", {})
by_id = {p["id"]: p for p in chapter["paragraphs"] if p.get("type") != "scene_break"}
expected = list(by_id)
issues: list[dict] = []
for pid in expected:
if pid not in drafts or not str(drafts[pid]).strip():
issues.append({"id": pid, "level": "block", "kind": "missing", "detail": "no translation"})
for pid in drafts:
if pid not in by_id:
issues.append({"id": pid, "level": "major", "kind": "unknown_id", "detail": "id not in source"})
chars = 0
for pid, zh in drafts.items():
zh = str(zh)
chars += len(re.findall(r"[\u4e00-\u9fff]", zh))
if re.search(r"^\[ch-[^\]]+\]", zh) or re.search(r"（(?:对话|旁白|叙述)(?:；[^）]*)?）", zh):
issues.append({"id": pid, "level": "block", "kind": "metadata_leak", "detail": zh[:40]})
if '"' in zh:
issues.append({"id": pid, "level": "major", "kind": "straight_quote", "detail": zh[:60]})
if zh.count("“") != zh.count("”"):
issues.append(
{
"id": pid,
"level": "major",
"kind": "quote_balance",
"detail": f"{zh.count('“')} open / {zh.count('”')} close",
}
)
if re.search(r"[。，！？；：]{2,}", zh):
issues.append({"id": pid, "level": "minor", "kind": "double_punctuation", "detail": zh[:60]})
if "《交易猎人》" in zh:
issues.append({"id": pid, "level": "major", "kind": "title_translated", "detail": zh[:60]})
common, proper = latin_residue(zh)
if common:
issues.append(
{"id": pid, "level": "major", "kind": "english_residue", "detail": ", ".join(common)}
)
if proper:
issues.append(
{"id": pid, "level": "info", "kind": "proper_name_review", "detail": ", ".join(proper)}
)
source = by_id.get(pid, {}).get("text", "")
star_system = re.search(
r"\bstar system\b|\b(?!The\b)[A-Z][a-z]+\s+system\b|\b\d+\s+systems\b", source
)
if star_system:
phrase = star_system.group(0).lower()
if any(
x in phrase
for x in (
"audio system",
"ship system",
"computer system",
"operating system",
"weapon system",
"security system",
"navigation system",
"power system",
"life support system",
)
):
star_system = None
if star_system and "系统" in zh:
issues.append(
{"id": pid, "level": "major", "kind": "system_mistranslated", "detail": "star system -> 系统"}
)
metrics = {
"chapter_id": chapter["id"],
"chapter_index": chapter["index"],
"title": chapter["title"],
"expected": len(expected),
"translated": len(drafts),
"chinese_chars": chars,
"issues": len(issues),
}
return issues, metrics
def main() -> None:
ap = argparse.ArgumentParser(description="Mechanical QA for the Chinese translation.")
ap.add_argument("--book", type=Path, required=True)
ap.add_argument("--run-dir", type=Path, required=True)
ap.add_argument("--json-out", type=Path)
ap.add_argument("--strict", action="store_true", help="exit 1 on any block/major issue")
args = ap.parse_args()
book = _load(args.book)
all_issues: list[dict] = []
metrics: list[dict] = []
for chapter in book["chapters"]:
path = args.run_dir / chapter["id"] / "translation.json"
if not path.exists():
continue
data = _load(path)
issues, m = check_chapter(chapter, data)
for issue in issues:
issue["chapter_id"] = chapter["id"]
issue["chapter_title"] = chapter["title"]
all_issues.extend(issues)
metrics.append(m)
by_level: dict[str, int] = {}
by_kind: dict[str, int] = {}
for issue in all_issues:
by_level[issue["level"]] = by_level.get(issue["level"], 0) + 1
by_kind[issue["kind"]] = by_kind.get(issue["kind"], 0) + 1
print("CHAPTERS")
for m in metrics:
print(
f" {m['chapter_id']:6} {m['title'][:28]:28} "
f"{m['translated']:>4}/{m['expected']:<4} paras {m['chinese_chars']:>6} chars {m['issues']:>3} issues"
)
print()
print("ISSUES BY LEVEL:", dict(sorted(by_level.items())))
print("ISSUES BY KIND: ", dict(sorted(by_kind.items())))
print()
for issue in all_issues:
if issue["level"] in {"block", "major"}:
print(
f" [{issue['level']:5}] {issue['kind']:20} {issue['chapter_id']}/{issue['id']}: {issue['detail']}"
)
elif issue["level"] == "minor":
print(f" [minor] {issue['kind']:20} {issue['chapter_id']}/{issue['id']}: {issue['detail']}")
else:
print(f" [info ] {issue['kind']:20} {issue['chapter_id']}/{issue['id']}: {issue['detail']}")
if args.json_out:
args.json_out.parent.mkdir(parents=True, exist_ok=True)
args.json_out.write_text(
json.dumps({"metrics": metrics, "issues": all_issues}, ensure_ascii=False, indent=2) + "\n",
encoding="utf-8",
)
print(f"\nwrote {args.json_out}")
if args.strict and (by_level.get("block", 0) or by_level.get("major", 0)):
raise SystemExit(1)
if __name__ == "__main__":
main()

671
tools/translation/render.py Normal file
View File

@@ -0,0 +1,671 @@
# Copyright 2026 Umbra Tools (umbratools.com)
# SPDX-License-Identifier: Apache-2.0
"""Render the translated book as AsciiDoc and as an OpenDocument text file.
The AsciiDoc file is the canonical text output. The ODT is produced from a
Flat ODF (.fodt) document so that styles, page breaks and the table of
contents are under our control, then converted with LibreOffice. The TOC is
updated through UNO, because a plain ``--convert-to`` leaves an empty index.
"""
from __future__ import annotations
import argparse
import hashlib
import json
import re
import shutil
import subprocess
import sys
import tempfile
import time
from pathlib import Path
from xml.sax.saxutils import escape
from llm import DEFAULT_ANALYST, DEFAULT_CRITIC, DEFAULT_MODEL
FONT_SERIF = "Noto Serif CJK SC"
FONT_SANS = "Noto Sans CJK SC"
def _esc(text: str) -> str:
return escape(text, {'"': "&quot;"})
LATIN_RUN = re.compile(r"[A-Za-z][A-Za-z0-9'’.\-·]*(?:\s+[A-Za-z][A-Za-z0-9'’.\-·]*)*")
def _rich_esc(text: str) -> str:
"""Escape text and mark Latin runs as English.
A Chinese paragraph tells LibreOffice to use Chinese line-breaking for the
whole run, which can split an English name in the middle of a word
("Deal Hu / nter"). A character style with fo:language="en" keeps the
Latin run on Western line-breaking rules without changing its font.
"""
out: list[str] = []
last = 0
for m in LATIN_RUN.finditer(text):
out.append(_esc(text[last : m.start()]))
out.append(f'<text:span text:style-name="Latin">{_esc(m.group(0))}</text:span>')
last = m.end()
out.append(_esc(text[last:]))
return "".join(out)
def load_run(run_dir: Path, chapter_id: str) -> dict | None:
path = run_dir / chapter_id / "translation.json"
if not path.exists():
return None
return json.loads(path.read_text(encoding="utf-8"))
def chapter_drafts(run_dir: Path | None, chapter_id: str) -> dict[str, str]:
if not run_dir:
return {}
data = load_run(run_dir, chapter_id)
return dict(data.get("paragraphs", {})) if data else {}
def _ordered_chapters(book: dict, run_dir: Path | None, include_missing: bool = False) -> list[dict]:
chapters = book["chapters"]
out = []
for c in chapters:
if c["matter"].startswith("front") and c["matter"] != "front_audio":
out.append(c)
out += [c for c in chapters if c["matter"] == "front_audio"]
out += [c for c in chapters if c["matter"] == "body"]
out += [c for c in chapters if c["matter"] == "back"]
if include_missing:
return out
return [c for c in out if chapter_drafts(run_dir, c["id"])]
def _cn_num(n: int) -> str:
digits = "零一二三四五六七八九"
if n <= 0:
return str(n)
if n <= 10:
return "十" if n == 10 else digits[n]
if n < 20:
return "十" + digits[n - 10]
if n < 100:
return digits[n // 10] + "十" + (digits[n % 10] if n % 10 else "")
return str(n)
def _body_chapter_number(book: dict, chapter: dict) -> int:
body = [c for c in book["chapters"] if c["matter"] == "body"]
body.sort(key=lambda c: c["index"])
return body.index(chapter) + 1
def _paragraph_style(paragraph: dict, matter: str) -> str:
if matter == "front_copyright":
return "Copyright"
if matter == "front_note":
return "Quote" if _is_author_note_code(paragraph.get("text", "")) else "FrontText"
if matter.startswith("front"):
return "FrontText"
if paragraph.get("type") == "dialogue":
return "Dialogue"
return "BodyText"
def _is_author_note_code(text: str) -> bool:
keys = (
"One of those codes was",
"All women captives",
"Those who rape",
)
return any(text.startswith(k) for k in keys)
# --------------------------------------------------------------------- AsciiDoc
def build_asciidoc(book: dict, run_dir: Path, *, sample: bool = False, date: str = "") -> str:
chapters = _ordered_chapters(book, run_dir)
lines: list[str] = []
lines.append("= Deal Hunter: 交易猎人")
lines.append("B. D. Murphy")
lines.append(":doctype: book")
lines.append(":lang: zh-CN")
lines.append(":toc: left")
lines.append(":toclevels: 2")
lines.append(f":revdate: {date or time.strftime('%Y-%m-%d')}")
if sample:
lines.append(":revremark: 审阅样本")
lines.append(":icons: font")
lines.append("")
lines.append("// 译者：Umbra Tools")
if sample:
lines.append("")
lines.append("NOTE: 本文件是简体中文翻译的审阅样本。已译部分见目录；未列出的章节将在全文运行中补齐。")
lines.append("")
seen_audio = False
for c in chapters:
drafts = chapter_drafts(run_dir, c["id"])
if not drafts:
continue
if c["matter"] == "front_audio":
if seen_audio:
continue
seen_audio = True
lines.append(f"// source: {c['id']} (audio front matter)")
lines.append("== 音频版序言")
lines.append("")
lines.append("以下是《Deal Hunter》有声版的开场白。")
lines.append("")
for ac in [x for x in book["chapters"] if x["matter"] == "front_audio"]:
adrafts = chapter_drafts(run_dir, ac["id"])
if not adrafts:
continue
lines.append(f"=== {ac['title_zh']}")
lines.append("")
for p in ac["paragraphs"]:
zh = adrafts.get(p["id"])
if not zh:
continue
if p.get("type") == "scene_break":
lines.append("'''")
lines.append("")
continue
lines.append(zh)
lines.append("")
continue
if c["matter"] == "body":
n = _body_chapter_number(book, c)
heading = f"第{_cn_num(n)}章　{c['title_zh']}"
else:
heading = c["title_zh"]
lines.append(f"// source: {c['id']} (database row {c['row']}), {len(c['paragraphs'])} paragraphs")
lines.append(f"== {heading}")
lines.append("")
for p in c["paragraphs"]:
zh = drafts.get(p["id"])
if not zh:
continue
if p.get("type") == "scene_break":
lines.append("'''")
lines.append("")
continue
style = _paragraph_style(p, c["matter"])
if style in {"Quote", "Dialogue"}:
lines.append(f"[.{style.lower()}]" if style == "Dialogue" else "[quote]")
if style == "Quote":
lines.append("____")
lines.append(zh)
lines.append("____")
else:
lines.append(zh)
lines.append("")
return "\n".join(lines).rstrip() + "\n"
# --------------------------------------------------------------------- Flat ODF
STYLES = f"""
<office:font-face-decls>
<style:font-face style:name="{FONT_SERIF}" svg:font-family="'{FONT_SERIF}'" style:font-family-generic="roman" style:font-pitch="variable"/>
<style:font-face style:name="{FONT_SANS}" svg:font-family="'{FONT_SANS}'" style:font-family-generic="swiss" style:font-pitch="variable"/>
</office:font-face-decls>
<office:styles>
<style:default-style style:family="paragraph">
<style:paragraph-properties fo:orphans="2" fo:widows="2" fo:hyphenation-ladder-count="no-limit"/>
<style:text-properties style:font-name="{FONT_SERIF}" fo:font-size="11pt" fo:language="zh" fo:country="CN"/>
</style:default-style>
<style:style style:name="Title" style:family="paragraph" style:parent-style-name="Standard" style:class="text" style:master-page-name="FirstPage">
<style:paragraph-properties fo:text-align="center" fo:margin-top="2.1in" fo:margin-bottom="0.15in"/>
<style:text-properties fo:font-size="30pt" fo:font-weight="bold"/>
</style:style>
<style:style style:name="Subtitle" style:family="paragraph" style:parent-style-name="Standard" style:class="text">
<style:paragraph-properties fo:text-align="center" fo:margin-bottom="0.5in"/>
<style:text-properties fo:font-size="18pt" fo:color="#444444"/>
</style:style>
<style:style style:name="Author" style:family="paragraph" style:parent-style-name="Standard" style:class="text">
<style:paragraph-properties fo:text-align="center" fo:margin-top="0.4in" fo:margin-bottom="0.15in"/>
<style:text-properties fo:font-size="14pt"/>
</style:style>
<style:style style:name="Small" style:family="paragraph" style:parent-style-name="Standard" style:class="text">
<style:paragraph-properties fo:text-align="center" fo:margin-bottom="0.08in"/>
<style:text-properties fo:font-size="9.5pt" fo:color="#666666"/>
</style:style>
<style:style style:name="PageBreak" style:family="paragraph" style:parent-style-name="Standard" style:master-page-name="Standard">
<style:paragraph-properties fo:break-before="page"/>
</style:style>
<style:style style:name="ChapterHeading" style:family="paragraph" style:parent-style-name="Standard" style:class="text">
<style:paragraph-properties fo:break-before="page" fo:margin-top="0.25in" fo:margin-bottom="0.3in" fo:keep-with-next="always"/>
<style:text-properties fo:font-size="20pt" fo:font-weight="bold"/>
</style:style>
<style:style style:name="SectionHeading" style:family="paragraph" style:parent-style-name="Standard" style:class="text">
<style:paragraph-properties fo:margin-top="0.25in" fo:margin-bottom="0.12in" fo:keep-with-next="always"/>
<style:text-properties fo:font-size="14pt" fo:font-weight="bold"/>
</style:style>
<style:style style:name="BodyText" style:family="paragraph" style:parent-style-name="Standard" style:class="text">
<style:paragraph-properties fo:text-align="justify" fo:text-indent="0.31in" fo:margin-bottom="0.06in" fo:line-height="150%"/>
<style:text-properties fo:font-size="11pt"/>
</style:style>
<style:style style:name="Dialogue" style:family="paragraph" style:parent-style-name="Standard" style:class="text">
<style:paragraph-properties fo:text-align="justify" fo:text-indent="0.31in" fo:margin-bottom="0.06in" fo:line-height="150%"/>
<style:text-properties fo:font-size="11pt"/>
</style:style>
<style:style style:name="Latin" style:family="text">
<style:text-properties fo:language="en" fo:country="US"/>
</style:style>
<style:style style:name="FrontText" style:family="paragraph" style:parent-style-name="Standard" style:class="text">
<style:paragraph-properties fo:text-align="left" fo:margin-bottom="0.10in" fo:line-height="140%"/>
<style:text-properties fo:font-size="10.5pt"/>
</style:style>
<style:style style:name="Copyright" style:family="paragraph" style:parent-style-name="Standard" style:class="text">
<style:paragraph-properties fo:text-align="left" fo:margin-bottom="0.10in" fo:line-height="135%"/>
<style:text-properties fo:font-size="9.5pt" fo:color="#333333"/>
</style:style>
<style:style style:name="Quote" style:family="paragraph" style:parent-style-name="Standard" style:class="text">
<style:paragraph-properties fo:margin-left="0.5in" fo:margin-right="0.4in" fo:margin-bottom="0.12in" fo:line-height="145%" fo:background-color="#f4f4f4"/>
<style:text-properties fo:font-size="10.5pt"/>
</style:style>
<style:style style:name="SceneBreak" style:family="paragraph" style:parent-style-name="Standard" style:class="text">
<style:paragraph-properties fo:text-align="center" fo:margin-top="0.12in" fo:margin-bottom="0.12in"/>
<style:text-properties fo:font-size="12pt"/>
</style:style>
<style:style style:name="TOCTitle" style:family="paragraph" style:parent-style-name="Standard" style:class="index">
<style:paragraph-properties fo:text-align="center" fo:margin-bottom="0.3in"/>
<style:text-properties fo:font-size="18pt" fo:font-weight="bold"/>
</style:style>
<style:style style:name="Contents1" style:family="paragraph" style:parent-style-name="Standard" style:class="index">
<style:paragraph-properties fo:margin-left="0in" fo:margin-bottom="0.08in"/>
<style:text-properties fo:font-size="11pt"/>
</style:style>
<style:style style:name="Header" style:family="paragraph" style:parent-style-name="Standard" style:class="extra">
<style:paragraph-properties fo:text-align="center"/>
<style:text-properties fo:font-size="8.5pt" fo:color="#777777"/>
</style:style>
<style:style style:name="Footer" style:family="paragraph" style:parent-style-name="Standard" style:class="extra">
<style:paragraph-properties fo:text-align="center"/>
<style:text-properties fo:font-size="8.5pt" fo:color="#777777"/>
</style:style>
<text:outline-style style:name="Outline">
<text:outline-level-style text:level="1" style:num-format=""/>
<text:outline-level-style text:level="2" style:num-format=""/>
</text:outline-style>
</office:styles>
"""
def _odt_paragraph(style: str, text: str, outline: int | None = None) -> str:
if outline:
return f'<text:h text:style-name="{style}" text:outline-level="{outline}">{_rich_esc(text)}</text:h>'
return f'<text:p text:style-name="{style}">{_rich_esc(text)}</text:p>'
def build_fodt(book: dict, run_dir: Path, *, sample: bool = False, date: str = "") -> str:
chapters = _ordered_chapters(book, run_dir)
body: list[str] = []
# Title page
body.append(_odt_paragraph("Title", "Deal Hunter"))
body.append(_odt_paragraph("Subtitle", "交易猎人"))
body.append(_odt_paragraph("Author", "B·D·墨菲 著"))
body.append(_odt_paragraph("Small", "Umbra Tools 译"))
if sample:
body.append(_odt_paragraph("Small", "简体中文译本 · 审阅样本"))
body.append(_odt_paragraph("Small", date or time.strftime("%Y-%m-%d")))
body.append('<text:p text:style-name="PageBreak"/>')
# Table of contents
body.append(
'<text:table-of-content text:name="TableOfContents" text:style-name="Contents1" text:protected="false">'
'<text:table-of-content-source text:outline-level="1">'
'<text:index-title-template text:style-name="TOCTitle">目录</text:index-title-template>'
'<text:table-of-content-entry-template text:outline-level="1" text:style-name="Contents1">'
"<text:index-entry-text/>"
'<text:index-entry-tab-stop style:type="right" style:leader-char="."/>'
"<text:index-entry-page-number/>"
"</text:table-of-content-entry-template>"
"</text:table-of-content-source>"
"</text:table-of-content>"
)
if sample:
body.append(_odt_paragraph("ChapterHeading", "审阅样本说明", 1))
body.append(
_odt_paragraph(
"FrontText",
"本文件是《Deal Hunter》简体中文翻译的审阅样本。已译部分见目录；未列出的章节将在全文运行中补齐。",
)
)
body.append(
_odt_paragraph(
"FrontText",
"翻译流程：段落重建 → 意图分析 → 初译 → 双语审校。术语见 glossary.json，审校意见见 review.html。",
)
)
seen_audio = False
for c in chapters:
drafts = chapter_drafts(run_dir, c["id"])
if not drafts:
continue
if c["matter"] == "front_audio":
if seen_audio:
continue
seen_audio = True
body.append(_odt_paragraph("ChapterHeading", "音频版序言", 1))
body.append(_odt_paragraph("FrontText", "以下是《Deal Hunter》有声版的开场白。"))
for ac in [x for x in book["chapters"] if x["matter"] == "front_audio"]:
adrafts = chapter_drafts(run_dir, ac["id"])
if not adrafts:
continue
body.append(_odt_paragraph("SectionHeading", ac["title_zh"], 2))
for p in ac["paragraphs"]:
zh = adrafts.get(p["id"])
if not zh:
continue
if p.get("type") == "scene_break":
body.append(_odt_paragraph("SceneBreak", "* * *"))
else:
body.append(_odt_paragraph("FrontText", zh))
continue
if c["matter"] == "body":
n = _body_chapter_number(book, c)
heading = f"第{_cn_num(n)}章　{c['title_zh']}"
else:
heading = c["title_zh"]
body.append(_odt_paragraph("ChapterHeading", heading, 1))
for p in c["paragraphs"]:
zh = drafts.get(p["id"])
if not zh:
continue
if p.get("type") == "scene_break":
body.append(_odt_paragraph("SceneBreak", "* * *"))
continue
body.append(_odt_paragraph(_paragraph_style(p, c["matter"]), zh))
if sample:
body.append(_odt_paragraph("ChapterHeading", "样本结束", 1))
body.append(
_odt_paragraph(
"FrontText",
"以上为审阅样本。全文运行将覆盖其余章节；审校问题清单会随各章翻译记录一并提供。",
)
)
return f"""<?xml version="1.0" encoding="UTF-8"?>
<office:document xmlns:office="urn:oasis:names:tc:opendocument:xmlns:office:1.0"
xmlns:style="urn:oasis:names:tc:opendocument:xmlns:style:1.0"
xmlns:text="urn:oasis:names:tc:opendocument:xmlns:text:1.0"
xmlns:table="urn:oasis:names:tc:opendocument:xmlns:table:1.0"
xmlns:draw="urn:oasis:names:tc:opendocument:xmlns:drawing:1.0"
xmlns:fo="urn:oasis:names:tc:opendocument:xmlns:xsl-fo-compatible:1.0"
xmlns:xlink="http://www.w3.org/1999/xlink"
xmlns:dc="http://purl.org/dc/elements/1.1/"
xmlns:meta="urn:oasis:names:tc:opendocument:xmlns:meta:1.0"
xmlns:number="urn:oasis:names:tc:opendocument:xmlns:datastyle:1.0"
xmlns:svg="urn:oasis:names:tc:opendocument:xmlns:svg-compatible:1.0"
xmlns:of="urn:oasis:names:tc:opendocument:xmlns:of:1.2"
office:version="1.3" office:mimetype="application/vnd.oasis.opendocument.text">
{STYLES}
<office:automatic-styles>
<style:page-layout style:name="pm1">
<style:page-layout-properties fo:page-width="8.27in" fo:page-height="11.69in"
fo:margin-top="0.9in" fo:margin-bottom="0.9in" fo:margin-left="1.0in" fo:margin-right="1.0in"
style:print-orientation="portrait" style:first-page-number="1"/>
<style:header-style><style:header-footer-properties fo:min-height="0.3in" fo:margin-bottom="0.2in"/></style:header-style>
<style:footer-style><style:header-footer-properties fo:min-height="0.3in" fo:margin-top="0.2in"/></style:footer-style>
</style:page-layout>
</office:automatic-styles>
<office:master-styles>
<style:master-page style:name="Standard" style:page-layout-name="pm1" style:next-style-name="Standard">
<style:header><text:p text:style-name="Header">Deal Hunter · 交易猎人</text:p></style:header>
<style:footer><text:p text:style-name="Footer"><text:page-number text:select-page="current">1</text:page-number></text:p></style:footer>
</style:master-page>
<style:master-page style:name="FirstPage" style:page-layout-name="pm1"/>
</office:master-styles>
<office:body>
<office:text>
<text:sequence-decls>
<text:sequence-decl text:display-outline-level="0" text:name="Illustration"/>
<text:sequence-decl text:display-outline-level="0" text:name="Table"/>
<text:sequence-decl text:display-outline-level="0" text:name="Text"/>
<text:sequence-decl text:display-outline-level="0" text:name="Drawing"/>
</text:sequence-decls>
{chr(10).join(body)}
</office:text>
</office:body>
</office:document>
"""
# --------------------------------------------------------------------- LibreOffice
def _soffice_binary() -> str:
for name in ("soffice", "libreoffice"):
path = shutil.which(name)
if path:
return path
raise SystemExit("LibreOffice not found; cannot produce ODT/PDF")
def convert_with_uno(fodt: Path, odt: Path, pdf: Path | None, port: int = 2002) -> None:
"""Convert FODT -> ODT (and PDF) with an updated TOC.
``--convert-to`` leaves a text:table-of-content empty. Opening the document
through UNO lets us call update() on the index before storing.
"""
fodt = fodt.resolve()
odt = odt.resolve()
if pdf:
pdf = pdf.resolve()
soffice = _soffice_binary()
profile = Path(tempfile.mkdtemp(prefix="lo-profile-"))
proc = subprocess.Popen(
[
soffice,
"--headless",
"--invisible",
"--norestore",
"--nofirststartwizard",
f"-env:UserInstallation={profile.as_uri()}",
f"--accept=socket,host=127.0.0.1,port={port};urp;",
],
stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL,
)
try:
import uno # type: ignore
from com.sun.star.beans import PropertyValue # type: ignore
local = uno.getComponentContext()
resolver = local.ServiceManager.createInstanceWithContext(
"com.sun.star.bridge.UnoUrlResolver", local
)
ctx = None
for _ in range(80):
try:
ctx = resolver.resolve(
f"uno:socket,host=127.0.0.1,port={port};urp;StarOffice.ComponentContext"
)
break
except Exception:
time.sleep(0.25)
if ctx is None:
raise RuntimeError("could not connect to LibreOffice")
desktop = ctx.ServiceManager.createInstanceWithContext("com.sun.star.frame.Desktop", ctx)
doc = desktop.loadComponentFromURL(uno.systemPathToFileUrl(str(fodt)), "_blank", 0, ())
indexes = doc.getDocumentIndexes()
for i in range(indexes.getCount()):
indexes.getByIndex(i).update()
def prop(name: str, value: object) -> object:
p = PropertyValue()
p.Name = name
p.Value = value
return p
doc.storeToURL(uno.systemPathToFileUrl(str(odt)), (prop("FilterName", "writer8"),))
if pdf:
doc.storeToURL(uno.systemPathToFileUrl(str(pdf)), (prop("FilterName", "writer_pdf_Export"),))
doc.close(True)
finally:
proc.terminate()
try:
proc.wait(timeout=10)
except subprocess.TimeoutExpired:
proc.kill()
shutil.rmtree(profile, ignore_errors=True)
def render(
*,
book_path: Path,
run_dir: Path,
out_dir: Path,
sample: bool = False,
date: str = "",
pdf: bool = True,
) -> dict[str, Path]:
book = json.loads(book_path.read_text(encoding="utf-8"))
out_dir.mkdir(parents=True, exist_ok=True)
stem = "deal-hunter-zh-sample" if sample else "deal-hunter-zh"
asciidoc = out_dir / f"{stem}.asciidoc"
fodt = out_dir / f"{stem}.fodt"
odt = out_dir / f"{stem}.odt"
pdf_path = out_dir / f"{stem}.pdf" if pdf else None
asciidoc.write_text(build_asciidoc(book, run_dir, sample=sample, date=date), encoding="utf-8")
fodt.write_text(build_fodt(book, run_dir, sample=sample, date=date), encoding="utf-8")
convert_with_uno(fodt, odt, pdf_path)
out = {"asciidoc": asciidoc, "fodt": fodt, "odt": odt}
if pdf_path:
out["pdf"] = pdf_path
return out
def _sha256(path: Path) -> str:
h = hashlib.sha256()
with path.open("rb") as f:
for block in iter(lambda: f.read(1024 * 1024), b""):
h.update(block)
return h.hexdigest()
def _repo_commit(start: Path) -> str | None:
for p in [start, *start.parents]:
if (p / ".git").exists():
try:
return subprocess.check_output(
["git", "-C", str(p), "rev-parse", "HEAD"],
text=True,
stderr=subprocess.DEVNULL,
).strip()
except (OSError, subprocess.CalledProcessError):
return None
return None
def write_manifest(
*,
book_path: Path,
run_dir: Path,
out_dir: Path,
sample: bool,
files: dict[str, Path],
) -> Path:
"""Write the reproducibility manifest next to the rendered files.
It contains hashes, counts and model names, never the book text.
"""
book = json.loads(book_path.read_text(encoding="utf-8"))
chapters: list[dict] = []
total = translated = issues = scene_breaks = 0
for c in book["chapters"]:
data = load_run(run_dir, c["id"])
expected = len([p for p in c["paragraphs"] if p.get("type") != "scene_break"])
scene_breaks += len([p for p in c["paragraphs"] if p.get("type") == "scene_break"])
n = len(data.get("paragraphs", {})) if data else 0
total += expected
translated += n
if not data:
continue
chapter_issues = len(data.get("issues", []) or [])
issues += chapter_issues
chapters.append(
{
"id": c["id"],
"index": c["index"],
"matter": c["matter"],
"title": c["title"],
"title_zh": c["title_zh"],
"paragraphs_expected": expected,
"paragraphs_translated": n,
"critic_issues": chapter_issues,
}
)
glossary = run_dir.parent / "glossary" / "glossary.json"
manifest = {
"title": book.get("title"),
"title_zh": book.get("title_zh"),
"author": book.get("author"),
"generated_at": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime()),
"sample": sample,
"source": {
"bundle": book.get("bundle"),
"bundle_sha256": book.get("bundle_sha256"),
"book_json_sha256": _sha256(book_path),
},
"glossary_sha256": _sha256(glossary) if glossary.exists() else None,
"prompts_sha256": _sha256(Path(__file__).with_name("prompts.py")),
"git_commit": _repo_commit(book_path),
"models": {
"drafter": DEFAULT_MODEL,
"analyst": DEFAULT_ANALYST,
"critic": DEFAULT_CRITIC,
},
"counts": {
"chapters": len(chapters),
"paragraphs_expected": total,
"paragraphs_translated": translated,
"scene_breaks": scene_breaks,
"critic_issues": issues,
},
"chapters": chapters,
"outputs": {k: str(v) for k, v in files.items()},
}
path = out_dir / "run-manifest.json"
path.write_text(json.dumps(manifest, ensure_ascii=False, indent=2) + "\n", encoding="utf-8")
return path
def main() -> None:
ap = argparse.ArgumentParser(description="Render Deal Hunter Chinese AsciiDoc + ODT.")
ap.add_argument("--book", type=Path, required=True)
ap.add_argument("--run-dir", type=Path, required=True)
ap.add_argument("--out-dir", type=Path, required=True)
ap.add_argument("--sample", action="store_true")
ap.add_argument("--date", default="")
ap.add_argument("--no-pdf", action="store_true")
args = ap.parse_args()
out = render(
book_path=args.book,
run_dir=args.run_dir,
out_dir=args.out_dir,
sample=args.sample,
date=args.date,
pdf=not args.no_pdf,
)
out["manifest"] = write_manifest(
book_path=args.book,
run_dir=args.run_dir,
out_dir=args.out_dir,
sample=args.sample,
files=out,
)
for k, v in out.items():
print(f"{k:9} {v}")
if __name__ == "__main__":
main()

297
tools/translation/review.py Normal file
View File

@@ -0,0 +1,297 @@
# Copyright 2026 Umbra Tools (umbratools.com)
# SPDX-License-Identifier: Apache-2.0
"""Build a bilingual side-by-side review page for the translation.
The output is a single self-contained HTML file. It contains the English source
and the Chinese draft, so it must stay inside the git-ignored production
directory. It is meant for a human editor: every paragraph is on one row, the
critic and mechanical issues are attached to the row, and a filter hides the
paragraphs that have no issues.
"""
from __future__ import annotations
import argparse
import html
import json
import re
import sys
import time
from pathlib import Path
sys.path.insert(0, str(Path(__file__).resolve().parent))
from qa import check_chapter # noqa: E402
def _load(path: Path) -> dict:
return json.loads(path.read_text(encoding="utf-8"))
def _severity_rank(sev: str) -> int:
return {"block": 0, "major": 1, "minor": 2, "info": 3}.get(sev, 4)
def _critic_issues(data: dict) -> list[dict]:
out = []
for raw in data.get("issues", []) or []:
if not isinstance(raw, dict):
continue
out.append(
{
"id": str(raw.get("id") or ""),
"severity": str(raw.get("severity") or "major"),
"type": str(raw.get("type") or "meaning"),
"problem": str(raw.get("problem") or ""),
"source_excerpt": str(raw.get("source_excerpt") or ""),
"target_excerpt": str(raw.get("target_excerpt") or ""),
"fix": str(raw.get("fix") or ""),
"origin": "critic",
}
)
return out
def _mechanical_issues(chapter: dict, data: dict) -> list[dict]:
issues, _metrics = check_chapter(chapter, data)
out = []
for raw in issues:
out.append(
{
"id": str(raw.get("id") or ""),
"severity": str(raw.get("level") or "major"),
"type": str(raw.get("kind") or "mechanical"),
"problem": str(raw.get("detail") or ""),
"source_excerpt": "",
"target_excerpt": "",
"fix": "",
"origin": "qa",
}
)
return out
def _badge(sev: str) -> str:
label = {"block": "BLOCK", "major": "MAJOR", "minor": "minor", "info": "info"}.get(sev, sev)
return f'<span class="badge {html.escape(sev)}">{html.escape(label)}</span>'
def _issue_html(issue: dict) -> str:
parts = [
f'<div class="issue {html.escape(issue["severity"])}">',
_badge(issue["severity"]),
f' <span class="kind">{html.escape(issue["type"])}</span> ',
f'<span class="origin">[{html.escape(issue["origin"])}]</span>',
]
if issue["problem"]:
parts.append(f'<div class="problem">{html.escape(issue["problem"])}</div>')
if issue["source_excerpt"]:
parts.append(f'<div class="excerpt">EN: {html.escape(issue["source_excerpt"])}</div>')
if issue["target_excerpt"]:
parts.append(f'<div class="excerpt">ZH: {html.escape(issue["target_excerpt"])}</div>')
if issue["fix"]:
parts.append(f'<div class="fix">建议：{html.escape(issue["fix"])}</div>')
parts.append("</div>")
return "".join(parts)
def build_html(book: dict, run_dir: Path, out_path: Path, only: set[str] | None = None) -> None:
sections: list[str] = []
stats = {"chapters": 0, "paragraphs": 0, "flagged": 0, "chars": 0}
total_issues = {"block": 0, "major": 0, "minor": 0, "info": 0}
for chapter in book["chapters"]:
if only and chapter["id"] not in only:
continue
path = run_dir / chapter["id"] / "translation.json"
if not path.exists():
continue
data = _load(path)
drafts: dict[str, str] = data.get("paragraphs", {})
if not drafts:
continue
issues = _critic_issues(data) + _mechanical_issues(chapter, data)
by_id: dict[str, list[dict]] = {}
unmatched: list[dict] = []
for issue in issues:
total_issues[issue["severity"]] = total_issues.get(issue["severity"], 0) + 1
if issue["id"] and issue["id"] in drafts:
by_id.setdefault(issue["id"], []).append(issue)
else:
unmatched.append(issue)
rows: list[str] = []
chapter_chars = 0
flagged = 0
heading = chapter.get("heading_zh") or chapter["title_zh"]
for p in chapter["paragraphs"]:
pid = p["id"]
if p.get("type") == "scene_break":
rows.append('<tr class="scene"><td colspan="3">* * *</td></tr>')
continue
zh = drafts.get(pid)
if zh is None:
rows.append(
f'<tr id="{pid}" class="row missing" data-flagged="1">'
f'<td class="pid">{pid}</td>'
f'<td class="en">{html.escape(p.get("text") or "")}</td>'
f'<td class="zh"><em>（未译）</em></td></tr>'
)
stats["flagged"] += 1
continue
zh = str(zh)
chapter_chars += len(re.findall(r"[\u4e00-\u9fff]", zh))
row_issues = sorted(by_id.get(pid, []), key=lambda i: _severity_rank(i["severity"]))
if row_issues:
flagged += 1
row_class = "row flagged" if row_issues else "row"
detail = "".join(_issue_html(i) for i in row_issues)
rows.append(
f'<tr id="{pid}" class="{row_class}" data-flagged="{1 if row_issues else 0}">'
f'<td class="pid">{html.escape(pid)}</td>'
f'<td class="en">{html.escape(p.get("text") or "")}</td>'
f'<td class="zh"><div class="zh-text">{html.escape(zh)}</div>{detail}</td>'
f"</tr>"
)
stats["chapters"] += 1
stats["paragraphs"] += len(drafts)
stats["chars"] += chapter_chars
stats["flagged"] += flagged
unmatched_html = ""
if unmatched:
unmatched_html = (
'<div class="unmatched"><h4>未能定位到段落的审校意见</h4>'
+ "".join(_issue_html(i) for i in unmatched)
+ "</div>"
)
sections.append(
f"""
<section class="chapter" id="{html.escape(chapter['id'])}">
<h2>{html.escape(heading)} <span class="meta">{html.escape(chapter['title'])} · {chapter['id']}</span></h2>
<p class="chapstats">{len(drafts)} 段 · {chapter_chars} 汉字 · {flagged} 段有意见</p>
{unmatched_html}
<table>
<thead><tr><th class="pid">ID</th><th class="en">English source</th><th class="zh">中文译文 / 意见</th></tr></thead>
<tbody>
{''.join(rows)}
</tbody>
</table>
</section>"""
)
doc = f"""<!doctype html>
<html lang="zh-CN">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>Deal Hunter 中译审校表</title>
<style>
:root {{ --bg:#fbfaf7; --ink:#222; --muted:#666; --line:#d9d4c8; --flag:#fff4e5; --block:#c62828; --major:#b26a00; --minor:#5c6bc0; --info:#607d8b; }}
* {{ box-sizing: border-box; }}
body {{ margin:0; background:var(--bg); color:var(--ink); font:15px/1.65 "Noto Serif CJK SC","Source Han Serif SC",serif; }}
header {{ position:sticky; top:0; z-index:10; background:rgba(251,250,247,.96); border-bottom:1px solid var(--line); padding:14px 22px; backdrop-filter:blur(6px); }}
header h1 {{ margin:0 0 4px; font-size:22px; }}
header .sub {{ color:var(--muted); font-size:13px; }}
header .controls {{ margin-top:8px; display:flex; gap:16px; align-items:center; flex-wrap:wrap; font-size:13px; }}
main {{ max-width:1500px; margin:0 auto; padding:20px 22px 80px; }}
.chapter {{ margin:28px 0 40px; }}
.chapter h2 {{ font-size:20px; border-bottom:2px solid var(--ink); padding-bottom:6px; margin:0 0 4px; }}
.chapter h2 .meta {{ font-size:12px; color:var(--muted); font-weight:normal; margin-left:8px; }}
.chapstats {{ margin:0 0 10px; color:var(--muted); font-size:12px; }}
table {{ width:100%; border-collapse:collapse; table-layout:fixed; background:#fff; }}
th,td {{ border:1px solid var(--line); vertical-align:top; padding:8px 10px; }}
thead th {{ position:sticky; top:87px; background:#efe9dc; z-index:5; text-align:left; font-size:12px; letter-spacing:.04em; }}
col.pid, th.pid, td.pid {{ width:92px; }}
th.en, td.en {{ width:48%; color:#333; font-family:"Noto Serif","Times New Roman",serif; font-size:14px; }}
th.zh, td.zh {{ width:calc(52% - 92px); }}
td.pid {{ font:11px/1.4 ui-monospace,monospace; color:var(--muted); }}
td.zh-text {{ white-space:pre-wrap; }}
tr.flagged {{ background:var(--flag); }}
tr.missing td.zh {{ color:var(--block); }}
tr.scene td {{ text-align:center; color:var(--muted); letter-spacing:.5em; background:#f6f2e9; }}
.issue {{ margin-top:8px; padding:6px 8px; border-left:4px solid var(--line); background:#fffdf8; font-size:12.5px; }}
.issue.block {{ border-color:var(--block); }}
.issue.major {{ border-color:var(--major); }}
.issue.minor {{ border-color:var(--minor); }}
.issue.info {{ border-color:var(--info); }}
.badge {{ display:inline-block; padding:0 5px; border-radius:3px; color:#fff; font:10px/1.6 ui-monospace,monospace; }}
.badge.block {{ background:var(--block); }} .badge.major {{ background:var(--major); }}
.badge.minor {{ background:var(--minor); }} .badge.info {{ background:var(--info); }}
.kind {{ font:11px/1.4 ui-monospace,monospace; color:var(--muted); }}
.origin {{ font-size:10px; color:#999; }}
.problem {{ margin:3px 0 0; }}
.excerpt {{ color:#555; font-size:11.5px; }}
.fix {{ color:#1b5e20; font-size:11.5px; }}
.unmatched {{ border:1px dashed var(--line); padding:8px 10px; margin:8px 0; background:#fff; }}
.unmatched h4 {{ margin:0 0 6px; font-size:13px; }}
@media print {{
header {{ position:static; }}
thead th {{ position:static; }}
tr.flagged {{ background:#fff; }}
.chapter {{ page-break-before:always; }}
}}
</style>
</head>
<body>
<header>
<h1>Deal Hunter · 交易猎人 — 中译审校表</h1>
<div class="sub">生成 {time.strftime('%Y-%m-%d %H:%M')} · {stats['chapters']} 章 · {stats['paragraphs']} 段 · {stats['chars']} 汉字 · {stats['flagged']} 段有意见 · block {total_issues['block']} / major {total_issues['major']} / minor {total_issues['minor']}</div>
<div class="controls">
<label><input type="checkbox" id="onlyFlagged"> 只看有意见的段落</label>
<label><input type="checkbox" id="hideEnglish"> 隐藏英文</label>
<span id="count"></span>
</div>
</header>
<main>
{''.join(sections)}
</main>
<script>
const rows = [...document.querySelectorAll('tr.row')];
const only = document.getElementById('onlyFlagged');
const hideEn = document.getElementById('hideEnglish');
const count = document.getElementById('count');
function apply() {{
let shown = 0;
for (const r of rows) {{
const flagged = r.dataset.flagged === '1';
const visible = !only.checked || flagged;
r.style.display = visible ? '' : 'none';
if (visible) shown++;
}}
document.querySelectorAll('td.en, th.en').forEach(el => el.style.display = hideEn.checked ? 'none' : '');
count.textContent = `显示 ${{shown}} / ${{rows.length}} 段`;
}}
only.addEventListener('change', apply);
hideEn.addEventListener('change', apply);
apply();
</script>
</body>
</html>
"""
out_path.parent.mkdir(parents=True, exist_ok=True)
out_path.write_text(doc, encoding="utf-8")
print(f"html {out_path}")
print(
f" {stats['chapters']} chapters, {stats['paragraphs']} paragraphs, "
f"{stats['chars']} chars, {stats['flagged']} flagged"
)
def main() -> None:
ap = argparse.ArgumentParser(description="Build a bilingual HTML review page.")
ap.add_argument("--book", type=Path, required=True)
ap.add_argument("--run-dir", type=Path, required=True)
ap.add_argument("--out", type=Path, required=True)
ap.add_argument("--chapters", help="comma-separated chapter ids (default: all translated)")
args = ap.parse_args()
book = _load(args.book)
only = {c.strip() for c in args.chapters.split(",")} if args.chapters else None
build_html(book, args.run_dir, args.out, only)
if __name__ == "__main__":
main()

137
tools/translation/tests/test_translation.py Normal file
View File

@@ -0,0 +1,137 @@
# Copyright 2026 Umbra Tools (umbratools.com)
# SPDX-License-Identifier: Apache-2.0
"""Unit tests for the translation cleanup and QA helpers.
The pipeline calls an LLM, so these tests deliberately cover only the
deterministic parts: prompt-leak removal, quote normalisation, English
residue detection, JSON extraction and the mechanical QA checks.
"""
from __future__ import annotations
import sys
import unittest
from pathlib import Path
HERE = Path(__file__).resolve().parent
sys.path.insert(0, str(HERE.parent))
from llm import parse_json # noqa: E402
from pipeline import latin_residue, sanitize # noqa: E402
from qa import check_chapter # noqa: E402
class SanitizeTests(unittest.TestCase):
def test_strips_dialogue_label(self) -> None:
self.assertEqual(sanitize("（对话；凯因达）“我在游艇上。”"), "“我在游艇上。”")
def test_strips_narration_label(self) -> None:
self.assertEqual(sanitize("（旁白）凯因达登上了游艇。"), "凯因达登上了游艇。")
def test_strips_paragraph_id(self) -> None:
self.assertEqual(sanitize("[ch-6-p0000] “你好。”"), "“你好。”")
def test_title_and_brand(self) -> None:
self.assertEqual(sanitize("《交易猎人》"), "《Deal Hunter》")
self.assertEqual(sanitize("由乌姆布拉音频系统制作"), "由Umbra Audio制作")
def test_missing_hour(self) -> None:
self.assertEqual(sanitize("她在小时前登船。"), "她一小时前登船。")
def test_straight_quotes_become_chinese(self) -> None:
self.assertEqual(sanitize('"你好"，她说。'), "“你好”，她说。")
def test_unmatched_closing_quote_removed(self) -> None:
self.assertEqual(sanitize("凯因达登上了游艇。”"), "凯因达登上了游艇。")
def test_unmatched_opening_quote_closed(self) -> None:
self.assertEqual(sanitize("“你好"), "“你好”")
def test_ellipsis_and_dash(self) -> None:
self.assertEqual(sanitize("等等... 然后--"), "等等…… 然后——")
class ResidueTests(unittest.TestCase):
def test_lowercase_english_is_flagged(self) -> None:
common, proper = latin_residue("她被 escorted 到了舱室。")
self.assertEqual(common, ["escorted"])
self.assertEqual(proper, [])
def test_allowed_names_are_ignored(self) -> None:
common, proper = latin_residue("Deal Hunter 和 Hunter 在 Umbra Audio。")
self.assertEqual(common, [])
self.assertEqual(proper, [])
def test_unknown_capitalised_name_is_reported(self) -> None:
common, proper = latin_residue("这是 Foo 的船。")
self.assertEqual(common, [])
self.assertEqual(proper, ["Foo"])
class JsonTests(unittest.TestCase):
def test_plain_json(self) -> None:
self.assertEqual(parse_json('{"a": 1}'), {"a": 1})
def test_fenced_json(self) -> None:
self.assertEqual(parse_json('```json\n{"a": 1}\n```'), {"a": 1})
def test_json_with_prose(self) -> None:
self.assertEqual(parse_json('结果如下：\n{"a": 1}\n完毕。'), {"a": 1})
class QaTests(unittest.TestCase):
def test_missing_paragraph_is_block(self) -> None:
chapter = {
"id": "ch-x",
"index": 1,
"title": "Test",
"paragraphs": [
{"id": "ch-x-p0000", "type": "narration", "text": "One."},
{"id": "ch-x-p0001", "type": "narration", "text": "Two."},
],
}
data = {"paragraphs": {"ch-x-p0000": "第一段。"}}
issues, metrics = check_chapter(chapter, data)
kinds = {i["kind"] for i in issues}
self.assertIn("missing", kinds)
self.assertEqual(metrics["translated"], 1)
def test_metadata_leak_and_residue(self) -> None:
chapter = {
"id": "ch-x",
"index": 1,
"title": "Test",
"paragraphs": [
{"id": "ch-x-p0000", "type": "narration", "text": "One."},
{"id": "ch-x-p0001", "type": "narration", "text": "The Albert system is far."},
],
}
data = {
"paragraphs": {
"ch-x-p0000": "（旁白）第一段 escorted。",
"ch-x-p0001": "阿尔伯特系统很远。",
}
}
issues, _ = check_chapter(chapter, data)
kinds = {i["kind"] for i in issues}
self.assertIn("metadata_leak", kinds)
self.assertIn("english_residue", kinds)
self.assertIn("system_mistranslated", kinds)
def test_ship_systems_are_not_star_systems(self) -> None:
chapter = {
"id": "ch-x",
"index": 1,
"title": "Test",
"paragraphs": [
{"id": "ch-x-p0000", "type": "narration", "text": "Keep away from the ship systems."},
],
}
data = {"paragraphs": {"ch-x-p0000": "远离飞船系统。"}}
issues, _ = check_chapter(chapter, data)
self.assertNotIn("system_mistranslated", {i["kind"] for i in issues})
if __name__ == "__main__":
unittest.main()

14
umbratrans_web/config.py
View File

@@ -34,7 +34,7 @@ from .security import (
) )
SERVICE_NAME = "umbratrans-web" SERVICE_NAME = "umbratrans-web"
SERVICE_VERSION = "0.1.0" SERVICE_VERSION = "0.2.0"
DEFAULT_SESSION_COOKIE_NAME = "umbratrans_session" DEFAULT_SESSION_COOKIE_NAME = "umbratrans_session"
DEFAULT_CSRF_COOKIE_NAME = "umbratrans_csrf" DEFAULT_CSRF_COOKIE_NAME = "umbratrans_csrf"
@@ -155,6 +155,12 @@ class Settings:
database_pool_min: int = 1 database_pool_min: int = 1
database_pool_max: int = 10 database_pool_max: int = 10
static_root: Path | None = None static_root: Path | None = None
#: Where production directories live (``productions/<book>-<lang>/``). Read
#: by the project-status adapter so the workspace can report real translation
#: and output state before the master store holds those rows. Unset means
#: "no production data is reachable", which the adapter reports honestly
#: rather than inventing progress.
productions_root: Path | None = None
mailer_transport: str = "log" mailer_transport: str = "log"
smtp_host: str = "" smtp_host: str = ""
#: Set only to bring a staging deployment up before SMTP exists. It is #: Set only to bring a staging deployment up before SMTP exists. It is
@@ -346,6 +352,11 @@ def load_settings(environ: Mapping[str, str] | None = None) -> Settings:
if raw_static: if raw_static:
static_root = Path(raw_static).expanduser() static_root = Path(raw_static).expanduser()
productions_root: Path | None = None
raw_productions = env.get("UMBRATRANS_PRODUCTIONS_ROOT")
if raw_productions:
productions_root = Path(raw_productions).expanduser()
# The DSN is read from the environment and is never logged, echoed by an # The DSN is read from the environment and is never logged, echoed by an
# endpoint, or written to a development data file. repr/compare are off on # endpoint, or written to a development data file. repr/compare are off on
# the field so it cannot leak through a settings dump. # the field so it cannot leak through a settings dump.
@@ -400,6 +411,7 @@ def load_settings(environ: Mapping[str, str] | None = None) -> Settings:
database_pool_min=pool_min, database_pool_min=pool_min,
database_pool_max=pool_max, database_pool_max=pool_max,
static_root=static_root, static_root=static_root,
productions_root=productions_root,
mailer_transport=env.get("UMBRATRANS_MAILER", "log").strip().lower(), mailer_transport=env.get("UMBRATRANS_MAILER", "log").strip().lower(),
smtp_host=env.get("UMBRATRANS_SMTP_HOST", "").strip(), smtp_host=env.get("UMBRATRANS_SMTP_HOST", "").strip(),
smtp_port=_int_from(env.get("UMBRATRANS_SMTP_PORT", "587"), name="UMBRATRANS_SMTP_PORT", minimum=1), smtp_port=_int_from(env.get("UMBRATRANS_SMTP_PORT", "587"), name="UMBRATRANS_SMTP_PORT", minimum=1),

207
umbratrans_web/project_status.py Normal file
View File

@@ -0,0 +1,207 @@
# Copyright 2026 Umbra Tools (umbratools.com)
# SPDX-License-Identifier: Apache-2.0
"""Project-status adapter backed by the ``productions/`` directory.
The master store does not yet hold translation/audio/format state, so this
adapter reads ``productions/<book>-<lang>/`` directly. It is a seam: when the
store gains those rows, this module can be replaced without changing the API.
For each program, the adapter returns every configured target language plus the
English source row. Languages without a production directory are reported as
``not_started``; languages with a production directory report real chapter
progress from the runs directory.
"""
from __future__ import annotations
import json
from dataclasses import dataclass
from datetime import datetime
from pathlib import Path
from typing import Any
# slug -> productions directory name stem
_SLUG_TO_PROD = {
"dealhunter-trans": "deal-hunter-zh",
}
# Configured target languages per program slug.
# Each entry: language_tag -> (label, role)
_TARGET_LANGUAGES = {
"dealhunter-trans": {
"en": ("English (source)", "source"),
"zh-Hans": ("Simplified Chinese (zh-CN)", "translation"),
"es-ES": ("Spanish (es-ES)", "translation"),
"fr-FR": ("French (fr-FR)", "translation"),
"de-DE": ("German (de-DE)", "translation"),
"it-IT": ("Italian (it-IT)", "translation"),
"pt-BR": ("Portuguese (pt-BR)", "translation"),
"ru-RU": ("Russian (ru-RU)", "translation"),
"ja-JP": ("Japanese (ja-JP)", "translation"),
"ko-KR": ("Korean (ko-KR)", "translation"),
"ar-SA": ("Arabic (ar-SA)", "translation"),
},
}
@dataclass(slots=True)
class LanguageStatus:
language_tag: str
language_label: str
source_present: bool
translation: dict[str, Any]
audio: dict[str, Any]
outputs: dict[str, Any]
review: dict[str, Any]
updated_at: str | None
def _load_json(path: Path) -> dict[str, Any] | None:
if not path.is_file():
return None
try:
with path.open("r", encoding="utf-8") as fh:
data = json.load(fh)
return data if isinstance(data, dict) else None
except Exception:
return None
def _max_mtime(paths: list[Path]) -> str | None:
mtimes = [p.stat().st_mtime for p in paths if p.exists()]
if not mtimes:
return None
return datetime.fromtimestamp(max(mtimes)).isoformat()
def _count_translated_chapters(runs_dir: Path) -> tuple[int, int]:
"""Return (done, total) chapter counts from the runs directory."""
done = 0
chapter_dirs: list[Path] = []
if runs_dir.is_dir():
for ch in sorted(runs_dir.iterdir()):
if ch.is_dir() and ch.name.startswith("ch-"):
chapter_dirs.append(ch)
if (ch / "translation.json").is_file():
done += 1
return done, len(chapter_dirs)
def _read_outputs(out_progress: Path, out_final: Path) -> dict[str, Any]:
"""Read available output files from progress and final directories."""
outputs: dict[str, Any] = {}
candidates: list[tuple[Path, str]] = []
if out_progress.is_dir():
candidates.append((out_progress, "output-progress"))
if out_final.is_dir():
candidates.append((out_final, "output"))
for subdir, key in candidates:
for p in subdir.iterdir():
if not p.is_file():
continue
name = p.name
if name.endswith(".pdf"):
outputs.setdefault("pdf-paperback", {"available": True, "path": str(p), "source": key})
elif name.endswith(".odt") or name.endswith(".asciidoc"):
outputs.setdefault("epub", {"available": True, "path": str(p), "source": key})
return outputs
class ProjectStatusAdapter:
def __init__(self, productions_root: Path | None = None) -> None:
if productions_root is not None:
self.productions_root = Path(productions_root)
else:
self.productions_root = Path(__file__).resolve().parents[1] / "productions"
def status_for(self, project) -> dict[str, Any]:
prod_name = _SLUG_TO_PROD.get(getattr(project, "slug", ""))
prod_dir = self.productions_root / prod_name if prod_name else None
runs_dir = prod_dir / "runs" if prod_dir else None
out_progress = prod_dir / "output-progress" if prod_dir else None
out_final = prod_dir / "output" if prod_dir else None
# Chapter totals come from the source book when available; fall back
# to the number of chapter directories found in runs.
done, total = _count_translated_chapters(runs_dir) if runs_dir else (0, 0)
languages: list[dict[str, Any]] = []
# English source row is always present when the project exists.
languages.append(
{
"language_tag": "en",
"language_label": "English (source)",
"role": "source",
"source_present": True,
"translation": {"state": "complete", "chapters_done": total, "chapters_total": total},
"audio": {"state": "unknown"},
"outputs": {},
"review": {"state": "ok"},
}
)
target_langs = _TARGET_LANGUAGES.get(getattr(project, "slug", ""), {})
for tag in sorted(target_langs.keys()):
if tag == "en":
continue # source row already added above
label, role = target_langs[tag]
if prod_dir and prod_dir.is_dir():
# A language-specific production directory would be named
# deal-hunter-<tag> (e.g. deal-hunter-es for Spanish).
lang_prod_name = f"deal-hunter-{tag.split('-')[0]}"
lang_prod_dir = self.productions_root / lang_prod_name
lang_runs = lang_prod_dir / "runs" if lang_prod_dir.is_dir() else None
lang_out_progress = lang_prod_dir / "output-progress" if lang_prod_dir.is_dir() else None
lang_out_final = lang_prod_dir / "output" if lang_prod_dir.is_dir() else None
if lang_runs and lang_runs.is_dir():
lang_done, lang_total = _count_translated_chapters(lang_runs)
state = (
"complete"
if lang_total and lang_done >= lang_total
else ("in_progress" if lang_done else "not_started")
)
outputs = _read_outputs(
lang_out_progress or Path(), lang_out_final or Path()
)
updated_at = _max_mtime(
[lang_runs / f"ch-{i}" / "translation.json" for i in range(1, 40)]
if lang_runs.is_dir()
else []
)
else:
state = "not_started"
lang_done, lang_total = 0, 0
outputs = {}
updated_at = None
else:
state = "not_started"
lang_done, lang_total = 0, 0
outputs = {}
updated_at = None
languages.append(
{
"language_tag": tag,
"language_label": label,
"role": role,
"source_present": tag == "en",
"translation": {
"state": state,
"chapters_done": lang_done,
"chapters_total": lang_total,
},
"audio": {"state": "not_started"},
"outputs": outputs,
"review": {
"state": "needs_review" if state == "complete" else "pending"
},
"updated_at": updated_at,
}
)
return {"project_id": str(getattr(project, "id")), "languages": languages}

226
umbratrans_web/routes_workspace.py
View File

@@ -26,18 +26,28 @@ see it.
from __future__ import annotations from __future__ import annotations
from typing import Any
from fastapi import APIRouter, Depends, Request from fastapi import APIRouter, Depends, Request
from .app import get_service, require_principal from .app import get_service, require_principal
from .models import Principal from .models import Principal
from .schemas import ( from .schemas import (
AudioQueueResponse,
FrontMatterResponse,
FormatMetadataResponse,
OrganizationResponse, OrganizationResponse,
ProjectDetailResponse,
ProjectResponse, ProjectResponse,
ProjectStatusResponse,
TranslationQueueResponse,
UtteranceReviewResponse,
UserResponse, UserResponse,
WorkspaceProjectResponse, WorkspaceProjectResponse,
WorkspaceResponse, WorkspaceResponse,
) )
from .service import AccountService from .service import AccountService
from .project_status import ProjectStatusAdapter
router = APIRouter() router = APIRouter()
@@ -74,3 +84,219 @@ def workspace(
total=snapshot.total, total=snapshot.total,
assigned_to_me=render(snapshot.assigned_to_me), assigned_to_me=render(snapshot.assigned_to_me),
) )
@router.get("/{project_id}", response_model=ProjectDetailResponse)
def project_detail(
project_id,
request: Request,
principal: Principal = Depends(require_principal),
) -> ProjectDetailResponse:
"""One project and its real state, scoped to the caller."""
service: AccountService = get_service(request)
from uuid import UUID
project = service.get_project(principal, UUID(str(project_id)))
adapter = ProjectStatusAdapter(getattr(request.app.state.settings, "productions_root", None))
status = adapter.status_for(project)
return ProjectDetailResponse(
project=ProjectResponse.from_view(project),
languages=status.get("languages", []),
review={"needs_attention": any((lang.get("review") or {}).get("state") == "needs_review" for lang in status.get("languages", []))},
)
@router.get("/{project_id}/status", response_model=ProjectStatusResponse)
def project_status(
project_id,
request: Request,
principal: Principal = Depends(require_principal),
) -> ProjectStatusResponse:
"""Per-language translation/audio/format state, backed by the productions directory."""
service: AccountService = get_service(request)
from uuid import UUID
project = service.get_project(principal, UUID(str(project_id)))
adapter = ProjectStatusAdapter(getattr(request.app.state.settings, "productions_root", None))
status = adapter.status_for(project)
return ProjectStatusResponse(project_id=status["project_id"], languages=status.get("languages", []))
@router.get("/{project_id}/chapters/{language_tag}", response_model=list[dict[str, Any]])
def list_chapters(
project_id: str,
language_tag: str,
request: Request,
principal: Principal = Depends(require_principal),
) -> list[dict[str, Any]]:
"""Return chapter metadata for one language project."""
service: AccountService = get_service(request)
from uuid import UUID
return service.get_chapters(principal, UUID(str(project_id)), language_tag)
@router.get("/{project_id}/chapter-content/{language_tag}/{chapter_id}", response_model=dict[str, Any])
def get_chapter_content(
project_id: str,
language_tag: str,
chapter_id: str,
request: Request,
principal: Principal = Depends(require_principal),
) -> dict[str, Any]:
"""Return full paragraph content for one chapter (front matter editing)."""
service: AccountService = get_service(request)
from uuid import UUID
return service.get_chapter_content(principal, UUID(str(project_id)), language_tag, chapter_id)
@router.post("/{project_id}/translation/{language_tag}/queue", response_model=TranslationQueueResponse)
def queue_translation(
project_id: str,
language_tag: str,
request: Request,
principal: Principal = Depends(require_principal),
) -> TranslationQueueResponse:
"""Queue a translation job for one language project."""
service: AccountService = get_service(request)
from uuid import UUID
return service.queue_translation(principal, UUID(str(project_id)), language_tag)
@router.post("/{project_id}/audio/{language_tag}/queue", response_model=AudioQueueResponse)
def queue_audio(
project_id: str,
language_tag: str,
request: Request,
principal: Principal = Depends(require_principal),
voice_config: dict | None = None,
) -> AudioQueueResponse:
"""Queue an audio generation job for one language project.
Requires the language's translation to be complete first, matching the
workflow: translate, then audio.
"""
service: AccountService = get_service(request)
from uuid import UUID
return service.queue_audio(
principal, UUID(str(project_id)), language_tag, voice_config=voice_config
)
@router.get("/{project_id}/format/{language_tag}", response_model=FormatMetadataResponse)
def get_format_metadata(
project_id: str,
language_tag: str,
request: Request,
principal: Principal = Depends(require_principal),
) -> FormatMetadataResponse:
"""Read format metadata (EPUB/PDF) for one language project."""
service: AccountService = get_service(request)
from uuid import UUID
return service.get_format_metadata(principal, UUID(str(project_id)), language_tag)
@router.post("/{project_id}/format/{language_tag}", response_model=FormatMetadataResponse)
def set_format_metadata(
project_id: str,
language_tag: str,
metadata: FormatMetadataRequest,
request: Request,
principal: Principal = Depends(require_principal),
) -> FormatMetadataResponse:
"""Save format metadata (cover, ISBN, blurb, search terms, etc.) for one language project."""
service: AccountService = get_service(request)
from uuid import UUID
return service.set_format_metadata(
principal,
UUID(str(project_id)),
language_tag,
metadata=metadata.model_dump(exclude={"language_tag"}),
)
@router.post("/{project_id}/front-matter/{language_tag}", response_model=FrontMatterResponse)
def save_front_matter(
project_id: str,
language_tag: str,
payload: FrontMatterRequest,
request: Request,
principal: Principal = Depends(require_principal),
) -> FrontMatterResponse:
"""Save front-matter items for one language project."""
service: AccountService = get_service(request)
from uuid import UUID
items = [item.model_dump() for item in payload.items]
return service.save_front_matter(
principal, UUID(str(project_id)), language_tag, items=items
)
@router.get(
"/{project_id}/review/{language_tag}/chapters/{chapter_id}",
response_model=UtteranceReviewResponse,
)
def get_utterance_review(
project_id: str,
language_tag: str,
chapter_id: str,
request: Request,
principal: Principal = Depends(require_principal),
) -> UtteranceReviewResponse:
"""Return utterances for one chapter of one language, for speaker-alignment review."""
service: AccountService = get_service(request)
from uuid import UUID
return service.get_utterance_review(
principal, UUID(str(project_id)), language_tag, chapter_id
)
@router.get("/{project_id}/front-matter/{language_tag}", response_model=FrontMatterResponse)
def get_front_matter(
project_id: str,
language_tag: str,
request: Request,
principal: Principal = Depends(require_principal),
) -> FrontMatterResponse:
"""Read front-matter items for one language project."""
service: AccountService = get_service(request)
from uuid import UUID
return service.get_front_matter(principal, UUID(str(project_id)), language_tag)
@router.post("/{project_id}/front-matter/{language_tag}", response_model=FrontMatterResponse)
def save_front_matter(
project_id: str,
language_tag: str,
payload: FrontMatterRequest,
request: Request,
principal: Principal = Depends(require_principal),
) -> FrontMatterResponse:
"""Save front-matter items for one language project."""
service: AccountService = get_service(request)
from uuid import UUID
items = [item.model_dump() for item in payload.items]
return service.save_front_matter(
principal, UUID(str(project_id)), language_tag, items=items
)

879
umbratrans_web/schemas.py
View File

@@ -1,76 +1,83 @@
# Copyright 2026 Umbra Tools (umbratools.com) # Copyright 2026 Umbra Tools (umbratools.com)
# SPDX-License-Identifier: Apache-2.0 # SPDX-License-Identifier: Apache-2.0
"""Pydantic request and response models for the phase-1 API. """Public API schemas and their mapping from internal views.
Two rules shape every model here: The boundary keeps internal invariants out of the wire. ``from_view``
constructors are the only place an internal representation is trusted to become
* Request models forbid unknown fields. A ``password`` field sent to wire JSON; route handlers assemble responses rather than dumping domain objects.
``POST /api/uadmin/accounts`` is therefore rejected by the schema itself, so
a one-time password can never arrive through a normal JSON body.
* Response models contain no credential material. Accounts are serialized from
:class:`umbratrans_web.models.AccountView`, which exposes a
:class:`~umbratrans_web.models.CredentialSummary` and never a hash.
""" """
from __future__ import annotations from __future__ import annotations
from datetime import datetime from datetime import datetime, timezone
from enum import StrEnum
from typing import Any
from uuid import UUID from uuid import UUID
from pydantic import BaseModel, ConfigDict, Field from pydantic import BaseModel, ConfigDict, Field, field_validator
from .errors import InputError
from .models import ( from .models import (
AccountStatus, AccountStatus,
AccountView,
AuditEvent,
CredentialSummary,
CredentialType, CredentialType,
EnrollmentMode, EnrollmentMode,
OrganizationStatus, OrganizationStatus,
OrganizationView,
ProjectStatus, ProjectStatus,
ProjectView,
RoleName, RoleName,
) )
class RequestModel(BaseModel): class RequestModel(BaseModel):
"""Base for request bodies: strict, no silent field coercion.""" """Base for request bodies.
model_config = ConfigDict(extra="forbid", str_strip_whitespace=False) Unknown fields are rejected so a mistyped property cannot be silently ignored
or, worse, mask a client bug while the server does something unexpected.
"""
model_config = ConfigDict(extra="forbid")
class ErrorBody(BaseModel): class ResponseModel(BaseModel):
code: str """Base for responses.
message: str
Extra fields are not allowed, so the wire shape cannot drift without the
tests catching it.
"""
model_config = ConfigDict(extra="forbid")
class ErrorResponse(BaseModel): class UserResponse(ResponseModel):
error: ErrorBody id: UUID
email: str
username: str
display_name: str
role: RoleName
status: AccountStatus
organization_id: UUID | None
is_email_verified: bool
created_at: datetime | None = None
updated_at: datetime | None = None
@classmethod
def from_view(cls, view) -> UserResponse:
return cls(
id=view.id,
email=view.email,
username=view.username,
display_name=view.display_name,
role=view.role,
status=view.status,
organization_id=view.organization_id,
is_email_verified=getattr(view, "is_email_verified", False),
created_at=getattr(view, "created_at", None),
updated_at=getattr(view, "updated_at", None),
)
class HealthResponse(BaseModel):
status: str
service: str
version: str
environment: str
repository: str
cookie_secure: bool
csrf_required: bool
#: Whether verification mail can actually be delivered. False means new
#: accounts cannot be activated by email, so it is reported here rather than
#: discovered when someone waits for a link that never arrives.
mail_deliverable: bool = True
#: True only in staging, where the operator has explicitly accepted that mail
#: is not configured yet.
mail_delivery_waived: bool = False
#: Whether the deployment still needs its first platform administrator.
setup_required: bool = False
time: datetime
class CredentialSummaryResponse(ResponseModel):
class CredentialSummaryResponse(BaseModel):
credential_type: CredentialType credential_type: CredentialType
must_change_password: bool must_change_password: bool
failed_login_count: int failed_login_count: int
@@ -78,254 +85,196 @@ class CredentialSummaryResponse(BaseModel):
last_login_at: datetime | None = None last_login_at: datetime | None = None
password_changed_at: datetime | None = None password_changed_at: datetime | None = None
@classmethod
def from_summary(cls, summary: CredentialSummary) -> CredentialSummaryResponse:
return cls(
credential_type=summary.credential_type,
must_change_password=summary.must_change_password,
failed_login_count=summary.failed_login_count,
locked=summary.locked,
last_login_at=summary.last_login_at,
password_changed_at=summary.password_changed_at,
)
class AccountResponse(ResponseModel):
class UserResponse(BaseModel):
id: UUID id: UUID
email: str email: str
username: str | None = None username: str
display_name: str display_name: str
role: RoleName role: RoleName
status: AccountStatus status: AccountStatus
organization_id: UUID | None = None organization_id: UUID | None
email: str | None = None is_email_verified: bool
is_email_verified: bool = True
created_at: datetime | None = None created_at: datetime | None = None
updated_at: datetime | None = None updated_at: datetime | None = None
@classmethod
def from_view(cls, view: AccountView) -> UserResponse:
return cls(
id=view.id,
email=view.email,
username=view.username,
display_name=view.display_name,
role=view.role,
status=view.status,
organization_id=view.organization_id,
created_at=view.created_at,
updated_at=view.updated_at,
)
class AccountResponse(UserResponse):
credential: CredentialSummaryResponse credential: CredentialSummaryResponse
active_session_count: int active_session_count: int = 0
@classmethod
def from_view(cls, view: AccountView) -> AccountResponse:
return cls(
id=view.id,
username=view.username,
display_name=view.display_name,
role=view.role,
status=view.status,
organization_id=view.organization_id,
email=view.email,
created_at=view.created_at,
updated_at=view.updated_at,
# Passed explicitly. The field defaults to True, so omitting it made
# every account in the list report itself as confirmed, including the
# invited ones - and the control panel is about to offer to re-send a
# verification link, so it has to be able to tell them apart.
is_email_verified=view.is_email_verified,
credential=CredentialSummaryResponse.from_summary(view.credential),
active_session_count=view.active_session_count,
)
class AccountListResponse(BaseModel): class AccountListResponse(ResponseModel):
items: list[AccountResponse] accounts: list[AccountResponse]
total: int total: int
limit: int limit: int
offset: int offset: int
class SessionUserResponse(BaseModel):
"""``GET /api/auth/me``. Includes the CSRF token the browser must echo."""
user: UserResponse
role: RoleName
organization_id: UUID | None = None
must_change_password: bool
csrf_token: str
idle_expires_at: datetime | None = None
absolute_expires_at: datetime | None = None
class LoginResponse(BaseModel):
user: UserResponse
must_change_password: bool
csrf_token: str
idle_expires_at: datetime
absolute_expires_at: datetime
class LogoutResponse(BaseModel):
revoked_sessions: int
class RevokeSessionsResponse(BaseModel):
revoked_sessions: int
class AuditEventResponse(BaseModel):
# A store-owned identifier, so a string rather than the UUID the
# development repository happens to use.
id: str
occurred_at: datetime
action: str
outcome: str
actor_user_id: UUID | None = None
target_user_id: UUID | None = None
organization_id: UUID | None = None
request_id: str | None = None
detail: dict[str, str]
@classmethod
def from_event(cls, event: AuditEvent) -> AuditEventResponse:
return cls(
id=event.id,
occurred_at=event.occurred_at,
action=event.action,
outcome=event.outcome,
actor_user_id=event.actor_user_id,
target_user_id=event.target_user_id,
organization_id=event.organization_id,
request_id=event.request_id,
detail=dict(event.detail),
)
class AuditListResponse(BaseModel):
items: list[AuditEventResponse]
total: int
limit: int
offset: int
class LoginRequest(RequestModel):
"""Sign-in payload.
The identifier is the email address, not a username:
``users.email`` is ``citext NOT NULL UNIQUE`` in the control plane, so it is
already a case-insensitively unique login key and no second identifier is
needed.
"""
email: str = Field(min_length=3, max_length=254)
password: str = Field(min_length=1, max_length=1024)
class VerifyEmailRequest(RequestModel):
"""Consume a mailed verification link.
``extra="forbid"`` matters here: a link is a bearer capability, and quietly
ignoring unexpected fields would make a malformed link look like a valid
one.
"""
token: str = Field(min_length=1, max_length=512)
class ResendVerificationRequest(RequestModel):
"""Ask for another verification link.
Takes an address rather than a session because an unverified account cannot
hold one. The response is identical whether or not an account exists.
"""
email: str = Field(min_length=3, max_length=254)
class VerifyEmailResponse(BaseModel):
account: UserResponse
message: str
class ResendVerificationResponse(BaseModel):
"""Deliberately says nothing about whether an account exists."""
message: str
class ChangeFirstPasswordRequest(RequestModel):
current_password: str = Field(min_length=1, max_length=1024)
new_password: str = Field(min_length=1, max_length=1024)
class CreateAccountRequest(RequestModel): class CreateAccountRequest(RequestModel):
"""Administrator account creation for either enrollment mode. """Administrator account creation for either enrollment mode.
The request never carries a password: the schema forbids unknown fields, so The request never carries a password: the schema forbids unknown fields, so
a submitted ``password`` is rejected outright. For a submitted ``password`` is rejected outright. For
``enrollment_mode="temporary_password"`` the service generates the one-time ``enrollment_mode="temporary_password"`` the service generates the one-time
password itself and returns it once in :class:`CreateAccountResponse`. password and returns it exactly once. For ``enrollment_mode="oidc"`` no
password is stored at all.
``email`` is required because the control plane declares ``users.email`` as
``NOT NULL UNIQUE``. Making it mandatory here means a missing address is a
clear field-level rejection rather than a failure that only appears once
the authoritative store is in use.
""" """
username: str | None = Field( enrollment_mode: EnrollmentMode
default=None, username: str = Field(min_length=1, max_length=64)
max_length=64, display_name: str = Field(min_length=1, max_length=128)
description=( email: str = Field(min_length=3, max_length=320)
"Optional short handle for audit lines. Not a login path: the email "
"is the identifier, so this is never asked for by sign-in."
),
)
display_name: str = Field(min_length=1, max_length=120)
email: str = Field(min_length=3, max_length=254)
enrollment_mode: EnrollmentMode = EnrollmentMode.OIDC
role: RoleName = RoleName.MEMBER role: RoleName = RoleName.MEMBER
organization_id: UUID | None = None organization_id: UUID | None = None
class CreateAccountResponse(BaseModel): class CreateAccountResponse(ResponseModel):
"""Result of ``POST /api/uadmin/accounts``.
``temporary_password`` is populated only when the account was enrolled with
a one-time local password. It is the single response that ever contains
that value, it is served with ``Cache-Control: no-store``, and it is never
persisted, logged, or retrievable again: the account must replace it on its
first login. It is ``None`` for OIDC enrollment.
"""
account: AccountResponse account: AccountResponse
temporary_password: str | None = None temporary_password: str | None = None
@classmethod
def from_oidc(cls, view: AccountView) -> CreateAccountResponse:
return cls(account=AccountResponse.from_view(view))
@classmethod class StatusChangeRequest(RequestModel):
def from_temporary_password(cls, view: AccountView, temporary_password: str) -> CreateAccountResponse:
return cls(account=AccountResponse.from_view(view), temporary_password=temporary_password)
class SetAccountStatusRequest(RequestModel):
status: AccountStatus status: AccountStatus
reason: str | None = Field(default=None, max_length=200) reason: str | None = Field(default=None, max_length=500)
class StatusChangeResponse(BaseModel): SetAccountStatusRequest = StatusChangeRequest
class StatusChangeResponse(ResponseModel):
account: AccountResponse account: AccountResponse
revoked_sessions: int revoked_sessions: int = 0
class ProjectResponse(BaseModel): class RevokeSessionsResponse(ResponseModel):
account: AccountResponse
revoked_sessions: int = 0
class PasswordResetRequestedResponse(ResponseModel):
requested: bool
class SetupStatusResponse(ResponseModel):
setup_required: bool
class SetupInitializeRequest(RequestModel):
email: str = Field(min_length=3, max_length=320)
display_name: str = Field(min_length=1, max_length=128)
username: str | None = Field(default=None, max_length=64)
password: str = Field(min_length=1, max_length=1024)
class SetupInitializeResponse(ResponseModel):
account: AccountResponse
csrf_token: str | None = None
class BootstrapAdminRequest(RequestModel):
username: str | None = Field(default=None, max_length=64)
display_name: str = Field(min_length=1, max_length=128)
email: str = Field(min_length=3, max_length=320)
organization_id: UUID | None = None
role: RoleName = RoleName.PLATFORM_ADMIN
class LoginRequest(RequestModel):
email: str = Field(min_length=3, max_length=320)
password: str = Field(min_length=1, max_length=1024)
class ChangeFirstPasswordRequest(RequestModel):
current_password: str = Field(min_length=1, max_length=1024)
new_password: str = Field(min_length=1, max_length=1024)
class ChangeOwnPasswordRequest(RequestModel):
current_password: str = Field(min_length=1, max_length=1024)
new_password: str = Field(min_length=1, max_length=1024)
class VerifyEmailRequest(RequestModel):
token: str = Field(min_length=1, max_length=4096)
class ResendVerificationRequest(RequestModel):
email: str = Field(min_length=3, max_length=320)
class LoginResponse(ResponseModel):
user: UserResponse
must_change_password: bool
csrf_token: str | None = None
idle_expires_at: datetime | None = None
absolute_expires_at: datetime | None = None
class SessionResponse(ResponseModel):
user: UserResponse
must_change_password: bool
idle_expires_at: datetime | None = None
absolute_expires_at: datetime | None = None
class VerifyEmailResponse(ResponseModel):
account: AccountResponse
activated: bool
class ResendVerificationResponse(ResponseModel):
requested: bool
class OrganizationResponse(ResponseModel):
id: UUID
name: str
slug: str
status: OrganizationStatus
member_count: int
created_at: datetime | None = None
updated_at: datetime | None = None
@classmethod
def from_view(cls, view) -> OrganizationResponse:
return cls(
id=view.id,
name=view.name,
slug=view.slug,
status=view.status,
member_count=view.member_count,
created_at=getattr(view, "created_at", None),
updated_at=getattr(view, "updated_at", None),
)
class OrganizationCreateRequest(RequestModel):
name: str = Field(min_length=1, max_length=128)
slug: str | None = Field(default=None, max_length=64)
status: OrganizationStatus = OrganizationStatus.ACTIVE
CreateOrganizationRequest = OrganizationCreateRequest
class OrganizationUpdateRequest(RequestModel):
name: str | None = Field(default=None, min_length=1, max_length=128)
slug: str | None = Field(default=None, max_length=64)
status: OrganizationStatus | None = None
UpdateOrganizationRequest = OrganizationUpdateRequest
SetOrganizationStatusRequest = OrganizationUpdateRequest
class OrganizationListResponse(ResponseModel):
organizations: list[OrganizationResponse]
total: int
limit: int
offset: int
class ProjectResponse(ResponseModel):
"""A project as returned by the API. """A project as returned by the API.
``assigned_account_ids`` is who is *working* on the project. It is not a ``assigned_account_ids`` is who is *working* on the project. It is not a
@@ -346,207 +295,53 @@ class ProjectResponse(BaseModel):
updated_at: datetime | None = None updated_at: datetime | None = None
@classmethod @classmethod
def from_view(cls, view: ProjectView) -> ProjectResponse: def from_view(cls, view) -> ProjectResponse:
return cls( return cls(
id=view.id, id=view.id,
organization_id=view.organization_id, organization_id=view.organization_id,
name=view.name, name=view.name,
slug=view.slug, slug=view.slug,
status=view.status, status=view.status,
assigned_account_ids=list(view.assigned_account_ids), assigned_account_ids=list(getattr(view, "assigned_account_ids", []) or []),
assigned_count=view.assigned_count, assigned_count=getattr(view, "assigned_count", 0) or 0,
created_at=view.created_at, created_at=getattr(view, "created_at", None),
created_by=view.created_by, created_by=getattr(view, "created_by", None),
updated_at=view.updated_at, updated_at=getattr(view, "updated_at", None),
) )
class ProjectListResponse(BaseModel): class ProjectCreateRequest(RequestModel):
"""Projects in the caller's organization, and the count of the whole set. name: str = Field(min_length=1, max_length=128)
slug: str | None = Field(default=None, max_length=64)
``total`` counts every matching project, not the page, so a client can say
"showing 20 of 47" without a second request.
"""
projects: list[ProjectResponse]
total: int
class CreateProjectRequest(RequestModel):
"""Create a project inside the caller's organization.
``slug`` is optional and derived from ``name`` when omitted. A supplied slug
is canonicalised rather than required to be already well formed, so an
administrator does not have to learn the rules before naming a project.
"""
name: str = Field(min_length=1, max_length=120)
slug: str | None = Field(default=None, max_length=63)
# Honoured only by a platform administrator. For anyone else the service pins
# the project to their own organization, and refuses a body naming another.
organization_id: UUID | None = None organization_id: UUID | None = None
class SetProjectStatusRequest(RequestModel): CreateProjectRequest = ProjectCreateRequest
class ProjectStatusRequest(RequestModel):
status: ProjectStatus status: ProjectStatus
reason: str | None = Field(default=None, max_length=200)
class ProjectAssignmentResponse(BaseModel): SetProjectStatusRequest = ProjectStatusRequest
"""The project as it stands after an assignment change.
The assignment count moves even when the answer is 404-shaped by intent: an
unassign that removed nothing is reported, not treated as a failure, because
asking for a state that is already true is a reasonable thing to do.
"""
project: ProjectResponse
changed: bool
class ProjectMemberRequest(RequestModel): class ProjectListResponse(ResponseModel):
user_id: UUID projects: list[ProjectResponse]
class WorkspaceProjectResponse(ProjectResponse):
"""A project row on the dashboard, with the caller's own involvement.
``assigned_to_me`` is computed per caller, so the same endpoint can show
"your work" and "everything in the organization" without the client
reassembling who is who.
"""
assigned_to_me: bool = False
class WorkspaceResponse(BaseModel):
"""``GET /api/workspace``: the signed-in viewer's organization and its work.
Deliberately one call. A dashboard that fetches the organization, then its
projects, then each project's members is a dashboard that renders three
partial states and can render two organizations at once if a filter is
mistimed. Everything a member may see comes from here, scoped server-side.
"""
user: UserResponse
organization: OrganizationResponse | None
#: Projects in the organization, organization-wide. Empty is a normal state,
#: not an error, and the client renders it as "no projects yet".
projects: list[WorkspaceProjectResponse]
total: int
#: The subset assigned to the caller, for a "your work" view. Always a subset
#: of ``projects``; computed here so the two cannot disagree.
assigned_to_me: list[WorkspaceProjectResponse]
class OrganizationResponse(BaseModel):
"""An organization as returned by the administrator API."""
id: UUID
name: str
slug: str
status: OrganizationStatus
member_count: int
created_at: datetime | None = None
updated_at: datetime | None = None
@classmethod
def from_view(cls, view: OrganizationView) -> OrganizationResponse:
return cls(
id=view.id,
name=view.name,
slug=view.slug,
status=view.status,
member_count=view.member_count,
created_at=view.created_at,
updated_at=view.updated_at,
)
class OrganizationListResponse(BaseModel):
items: list[OrganizationResponse]
total: int total: int
limit: int limit: int
offset: int offset: int
class CreateOrganizationRequest(RequestModel): class ProjectAssignmentRequest(RequestModel):
"""Create a tenant. Platform administrators only. account_id: UUID
``slug`` is optional and derived from ``name`` when omitted. A supplied
slug is validated as written, so a rejected value reports the slug rule.
"""
name: str = Field(min_length=1, max_length=120)
slug: str | None = Field(default=None, max_length=63)
class UpdateOrganizationRequest(RequestModel): class ProjectAssignmentResponse(ResponseModel):
"""Rename or re-slug an organization. At least one field is required.""" project: ProjectResponse
name: str | None = Field(default=None, min_length=1, max_length=120)
slug: str | None = Field(default=None, max_length=63)
class PasswordResetAvailableResponse(BaseModel): class OrganizationSummaryResponse(ResponseModel):
"""Whether a reset link can still be used.
One answer for every failure reason, so it cannot be used to learn whether a
given link ever existed.
"""
available: bool
message: str
class ResetPasswordRequest(RequestModel):
"""A token from a reset link, and the password to set with it."""
token: str = Field(min_length=16, max_length=256)
new_password: str = Field(min_length=1, max_length=1024)
class ChangeOwnPasswordRequest(RequestModel):
"""A signed-in person changing their own password.
The current password is required, so a stolen session cookie alone cannot
take the account over. It is not required by the first-login flow, which
exists precisely because the holder has only the one-time password.
"""
current_password: str = Field(min_length=1, max_length=1024)
new_password: str = Field(min_length=1, max_length=1024)
class PasswordResetRequestedResponse(BaseModel):
"""The outcome of an administrator asking for a reset.
Says a link was sent and nothing more. The address is not echoed and no
detail about the account is returned, so the endpoint is safe to leave open
to an administrator who is entitled to see the account list anyway but not to
probe addresses outside their scope.
"""
message: str
class SetOrganizationStatusRequest(RequestModel):
status: OrganizationStatus
reason: str | None = Field(default=None, max_length=200)
class OrganizationStatusChangeResponse(BaseModel):
organization: OrganizationResponse
revoked_sessions: int
class OrganizationSummaryResponse(BaseModel):
"""Counts for the administrator dashboard.
Derived from the caller's visible scope on every request, so it can never
report a number the caller could not obtain from the list endpoints.
"""
total_organizations: int total_organizations: int
active_organizations: int active_organizations: int
total_accounts: int total_accounts: int
@@ -556,34 +351,200 @@ class OrganizationSummaryResponse(BaseModel):
organization_admin_count: int organization_admin_count: int
__all__ = [ class AuditEventResponse(ResponseModel):
"AccountListResponse", id: str
"AccountResponse", occurred_at: datetime
"AuditEventResponse", action: str
"AuditListResponse", outcome: str
"ChangeFirstPasswordRequest", actor_user_id: UUID | None = None
"CreateAccountRequest", target_user_id: UUID | None = None
"CreateAccountResponse", organization_id: UUID | None = None
"CredentialSummaryResponse", request_id: str | None = None
"ErrorResponse", detail: dict[str, Any] = Field(default_factory=dict)
"HealthResponse",
"LoginRequest",
"LoginResponse", class AuditListResponse(ResponseModel):
"LogoutResponse", items: list[AuditEventResponse]
"CreateOrganizationRequest", total: int
"OrganizationListResponse", limit: int
"OrganizationResponse", offset: int
"OrganizationStatusChangeResponse",
"OrganizationSummaryResponse",
"ResendVerificationRequest", class WorkspaceProjectResponse(ProjectResponse):
"ResendVerificationResponse", """A project row inside ``/api/workspace`` with per-caller assignment state."""
"RevokeSessionsResponse",
"SessionUserResponse", assigned_to_me: bool = False
"SetAccountStatusRequest",
"SetOrganizationStatusRequest",
"StatusChangeResponse", class WorkspaceResponse(ResponseModel):
"UpdateOrganizationRequest", """``GET /api/workspace``: the signed-in viewer's organization and its work."""
"UserResponse",
"VerifyEmailRequest", user: UserResponse
"VerifyEmailResponse", organization: OrganizationResponse | None
] projects: list[WorkspaceProjectResponse]
total: int
assigned_to_me: list[WorkspaceProjectResponse]
class ProjectDetailResponse(ResponseModel):
project: ProjectResponse
languages: list[dict[str, Any]]
review: dict[str, Any] = Field(default_factory=dict)
class ProjectStatusLanguageResponse(ResponseModel):
language_tag: str
language_label: str
role: str
source_present: bool
translation: dict[str, Any]
audio: dict[str, Any]
outputs: dict[str, Any]
review: dict[str, Any]
updated_at: str | None = None
class ProjectStatusResponse(ResponseModel):
project_id: str
languages: list[ProjectStatusLanguageResponse]
# ---------------------------------------------------------------------------
# Translation queue
class TranslationQueueRequest(RequestModel):
language_tag: str = Field(min_length=1, max_length=16)
class TranslationQueueResponse(ResponseModel):
project_id: UUID
language_tag: str
state: str
job_id: str | None = None
queued_at: datetime | None = None
# ---------------------------------------------------------------------------
# Audio queue
class AudioQueueRequest(RequestModel):
language_tag: str = Field(min_length=1, max_length=16)
voice_config: dict[str, Any] = Field(default_factory=dict)
class AudioQueueResponse(ResponseModel):
project_id: UUID
language_tag: str
state: str
job_id: str | None = None
queued_at: datetime | None = None
# ---------------------------------------------------------------------------
# Format metadata (EPUB / PDF)
class FormatMetadataRequest(RequestModel):
language_tag: str = Field(min_length=1, max_length=16)
cover: str | None = None
isbn: str | None = Field(default=None, max_length=64)
blurb: str | None = None
amazon_search_terms: str | None = None
print_profile: str | None = Field(default=None, max_length=64)
page_count: str | None = Field(default=None, max_length=32)
class FormatMetadataResponse(ResponseModel):
project_id: UUID
language_tag: str
saved: bool
updated_at: datetime | None = None
# ---------------------------------------------------------------------------
# Utterance review by chapter
class UtteranceReviewResponse(ResponseModel):
project_id: UUID
language_tag: str
chapter_id: str
utterances: list[dict[str, Any]]
speakers: list[dict[str, Any]]
# ---------------------------------------------------------------------------
# Front matter (per language)
class FrontMatterItemRequest(RequestModel):
key: str = Field(min_length=1, max_length=128)
label: str = Field(min_length=1, max_length=128)
value: str | None = None
output_page: str | None = Field(default=None, max_length=16)
class FrontMatterItemResponse(ResponseModel):
key: str
label: str
value: str | None = None
output_page: str | None = None
class FrontMatterRequest(RequestModel):
language_tag: str = Field(min_length=1, max_length=16)
items: list[FrontMatterItemRequest] = Field(default_factory=list)
class FrontMatterResponse(ResponseModel):
project_id: UUID
language_tag: str
items: list[FrontMatterItemResponse]
version: int = 1
class HealthResponse(ResponseModel):
status: str
service: str
version: str
environment: str
repository: str
cookie_secure: bool
csrf_required: bool
mail_deliverable: bool
mail_delivery_waived: bool
setup_required: bool
time: datetime | None = None
class LogoutResponse(ResponseModel):
logged_out: bool = True
class ResetPasswordRequest(RequestModel):
token: str = Field(min_length=1, max_length=4096)
new_password: str = Field(min_length=1, max_length=1024)
class PasswordResetAvailableResponse(ResponseModel):
available: bool
class SessionUserResponse(ResponseModel):
user: UserResponse
role: RoleName
organization_id: UUID | None = None
must_change_password: bool = False
csrf_token: str | None = None
class OrganizationStatusChangeResponse(ResponseModel):
organization: OrganizationResponse
revoked_sessions: int = 0

748
umbratrans_web/service.py
View File

@@ -34,9 +34,11 @@ Rules enforced here:
from __future__ import annotations from __future__ import annotations
import json
import logging import logging
from dataclasses import dataclass, field, replace from dataclasses import dataclass, field, replace
from datetime import datetime, timedelta from datetime import datetime, timedelta, timezone
from pathlib import Path
from types import MappingProxyType from types import MappingProxyType
from typing import Callable, Mapping, Sequence from typing import Callable, Mapping, Sequence
from uuid import UUID, uuid4 from uuid import UUID, uuid4
@@ -109,9 +111,22 @@ from .security import (
utc_now, utc_now,
) )
from .tenancy import TenantScope, resolve_target, scope_for from .tenancy import TenantScope, resolve_target, scope_for
from .project_status import ProjectStatusAdapter
from .schemas import (
AudioQueueResponse,
FrontMatterResponse,
FormatMetadataResponse,
TranslationQueueResponse,
UtteranceReviewResponse,
)
LOGGER = logging.getLogger("umbratrans_web.service") LOGGER = logging.getLogger("umbratrans_web.service")
#: In-memory job store for language-project workflow jobs (translation,
#: audio queue, etc.). The productions directory is read-only in some
#: deployments, so job state is held here and reported honestly as ephemeral.
_JOB_STORE: dict[str, dict] = {}
#: Sources that may request the operator-only local temporary-password path. #: Sources that may request the operator-only local temporary-password path.
#: Sources that may prove the operator path. An allowlist, not a free string, #: Sources that may prove the operator path. An allowlist, not a free string,
#: because holding one of these is what permits creating an account with a local #: because holding one of these is what permits creating an account with a local
@@ -165,6 +180,10 @@ ACTION_PROJECT_CREATED = "project.created"
ACTION_PROJECT_STATUS_CHANGED = "project.status_changed" ACTION_PROJECT_STATUS_CHANGED = "project.status_changed"
ACTION_PROJECT_ASSIGNMENT_ADDED = "project.assignment_added" ACTION_PROJECT_ASSIGNMENT_ADDED = "project.assignment_added"
ACTION_PROJECT_ASSIGNMENT_REMOVED = "project.assignment_removed" ACTION_PROJECT_ASSIGNMENT_REMOVED = "project.assignment_removed"
ACTION_TRANSLATION_QUEUED = "translation.queued"
ACTION_AUDIO_QUEUED = "audio.queued"
ACTION_FORMAT_METADATA_SAVED = "format.metadata_saved"
ACTION_FRONT_MATTER_SAVED = "front_matter.saved"
ACTION_PASSWORD_RESET_REQUESTED = "password.reset_requested" ACTION_PASSWORD_RESET_REQUESTED = "password.reset_requested"
ACTION_PASSWORD_RESET_COMPLETED = "password.reset_completed" ACTION_PASSWORD_RESET_COMPLETED = "password.reset_completed"
ACTION_SESSION_REVOKED = "session.revoked" ACTION_SESSION_REVOKED = "session.revoked"
@@ -2185,6 +2204,733 @@ class AccountService:
) )
return self._project_view(updated) return self._project_view(updated)
# -- language-project workflow ------------------------------------------
def _productions_dir_for(self, project: Project) -> Path | None:
"""Return the productions directory for a project, or None if unmapped."""
from .project_status import _SLUG_TO_PROD
prod_name = _SLUG_TO_PROD.get(getattr(project, "slug", ""))
if not prod_name:
return None
root = self._settings.productions_root
if root is None:
return None
return Path(root) / prod_name
def _job_key(self, project_id: UUID, language_tag: str, job_type: str) -> str:
return f"{project_id}:{language_tag}:{job_type}"
def _read_json(self, path: Path) -> dict | None:
if not path.is_file():
return None
try:
with path.open("r", encoding="utf-8") as fh:
data = json.load(fh)
return data if isinstance(data, dict) else None
except Exception:
return None
def _parse_utc(self, value: str | None) -> datetime | None:
if not value:
return None
try:
return datetime.fromisoformat(value)
except ValueError:
return None
def queue_translation(
self,
actor: Principal,
project_id: UUID,
language_tag: str,
*,
request_id: str | None = None,
) -> TranslationQueueResponse:
"""Queue a translation job for one language project."""
scope = scope_for(actor)
project = self._repository.get_project(project_id, organization_id=scope.organization_id)
if project is None:
if scope.unrestricted:
raise NotFoundError("project_not_found")
raise ForbiddenError("cross_organization_forbidden")
adapter = ProjectStatusAdapter(self._settings.productions_root)
status = adapter.status_for(project)
lang = next(
(l for l in status.get("languages", []) if l.get("language_tag") == language_tag),
None,
)
if lang is None:
raise NotFoundError("language_not_found")
tr_state = (lang.get("translation") or {}).get("state")
if tr_state in ("in_progress", "complete"):
raise ConflictError(
"translation_already_active",
f"Translation for {language_tag} is already {tr_state}.",
)
job_id = str(uuid4())
job_record = {
"job_id": job_id,
"project_id": str(project_id),
"language_tag": language_tag,
"type": "translation",
"state": "queued",
"created_at": datetime.now(timezone.utc).isoformat(),
}
_JOB_STORE[self._job_key(project_id, language_tag, "translation")] = job_record
self._audit(
action=ACTION_TRANSLATION_QUEUED,
outcome="success",
actor=actor,
organization_id=project.organization_id,
request_id=request_id,
detail={
"project_slug": project.slug,
"language_tag": language_tag,
"job_id": job_id,
},
)
LOGGER.info(
"translation queued project_id=%s language_tag=%s job_id=%s",
project_id,
language_tag,
job_id,
)
return TranslationQueueResponse(
project_id=project_id,
language_tag=language_tag,
state="queued",
job_id=job_id,
queued_at=datetime.now(timezone.utc),
)
def queue_audio(
self,
actor: Principal,
project_id: UUID,
language_tag: str,
voice_config: dict | None = None,
*,
request_id: str | None = None,
) -> AudioQueueResponse:
"""Queue an audio generation job for one language project.
Refuses unless the language's translation is already complete, matching
the workflow: translate first, then audio.
"""
scope = scope_for(actor)
project = self._repository.get_project(project_id, organization_id=scope.organization_id)
if project is None:
if scope.unrestricted:
raise NotFoundError("project_not_found")
raise ForbiddenError("cross_organization_forbidden")
adapter = ProjectStatusAdapter(self._settings.productions_root)
status = adapter.status_for(project)
lang = next(
(l for l in status.get("languages", []) if l.get("language_tag") == language_tag),
None,
)
if lang is None:
raise NotFoundError("language_not_found")
tr_state = (lang.get("translation") or {}).get("state")
if tr_state != "complete":
raise ConflictError(
"translation_not_complete",
f"Translation for {language_tag} must be complete before audio can be queued (current: {tr_state}).",
)
job_id = str(uuid4())
job_record = {
"job_id": job_id,
"project_id": str(project_id),
"language_tag": language_tag,
"type": "audio",
"state": "queued",
"voice_config": voice_config or {},
"created_at": datetime.now(timezone.utc).isoformat(),
}
_JOB_STORE[self._job_key(project_id, language_tag, "audio")] = job_record
self._audit(
action=ACTION_AUDIO_QUEUED,
outcome="success",
actor=actor,
organization_id=project.organization_id,
request_id=request_id,
detail={
"project_slug": project.slug,
"language_tag": language_tag,
"job_id": job_id,
},
)
LOGGER.info(
"audio queued project_id=%s language_tag=%s job_id=%s",
project_id,
language_tag,
job_id,
)
return AudioQueueResponse(
project_id=project_id,
language_tag=language_tag,
state="queued",
job_id=job_id,
queued_at=datetime.now(timezone.utc),
)
def set_format_metadata(
self,
actor: Principal,
project_id: UUID,
language_tag: str,
metadata: dict,
*,
request_id: str | None = None,
) -> FormatMetadataResponse:
"""Save format metadata (EPUB/PDF) for one language project."""
scope = scope_for(actor)
project = self._repository.get_project(project_id, organization_id=scope.organization_id)
if project is None:
if scope.unrestricted:
raise NotFoundError("project_not_found")
raise ForbiddenError("cross_organization_forbidden")
prod_dir = self._productions_dir_for(project)
if prod_dir is None:
# No productions directory: hold metadata in the in-memory job store.
key = self._job_key(project_id, language_tag, "format")
_JOB_STORE[key] = {
"language_tag": language_tag,
"updated_at": datetime.now(timezone.utc).isoformat(),
**{k: v for k, v in metadata.items() if v is not None},
}
self._audit(
action=ACTION_FORMAT_METADATA_SAVED,
outcome="success",
actor=actor,
organization_id=project.organization_id,
request_id=request_id,
detail={
"project_slug": project.slug,
"language_tag": language_tag,
},
)
return FormatMetadataResponse(
project_id=project_id,
language_tag=language_tag,
saved=True,
updated_at=datetime.now(timezone.utc),
)
meta_dir = prod_dir / "metadata" / language_tag
meta_dir.mkdir(parents=True, exist_ok=True)
existing = self._read_json(meta_dir / "format.json") or {}
existing.update(
{key: value for key, value in metadata.items() if value is not None}
)
existing["language_tag"] = language_tag
existing["updated_at"] = datetime.now(timezone.utc).isoformat()
self._write_json(meta_dir / "format.json", existing)
self._audit(
action=ACTION_FORMAT_METADATA_SAVED,
outcome="success",
actor=actor,
organization_id=project.organization_id,
request_id=request_id,
detail={
"project_slug": project.slug,
"language_tag": language_tag,
},
)
return FormatMetadataResponse(
project_id=project_id,
language_tag=language_tag,
saved=True,
updated_at=datetime.now(timezone.utc),
)
def get_format_metadata(
self,
actor: Principal,
project_id: UUID,
language_tag: str,
) -> FormatMetadataResponse:
"""Read format metadata for one language project."""
scope = scope_for(actor)
project = self._repository.get_project(project_id, organization_id=scope.organization_id)
if project is None:
if scope.unrestricted:
raise NotFoundError("project_not_found")
raise ForbiddenError("cross_organization_forbidden")
prod_dir = self._productions_dir_for(project)
if prod_dir is None:
stored = _JOB_STORE.get(self._job_key(project_id, language_tag, "format"))
return FormatMetadataResponse(
project_id=project_id,
language_tag=language_tag,
saved=stored is not None,
updated_at=self._parse_utc(stored.get("updated_at") if stored else None),
)
existing = self._read_json(prod_dir / "metadata" / language_tag / "format.json")
# Also check in-memory store (for deployments with read-only productions).
stored = _JOB_STORE.get(self._job_key(project_id, language_tag, "format"))
if stored and (not existing or stored.get("updated_at", "") >= existing.get("updated_at", "")):
existing = stored
return FormatMetadataResponse(
project_id=project_id,
language_tag=language_tag,
saved=existing is not None,
updated_at=self._parse_utc(existing.get("updated_at") if existing else None),
)
def get_chapters(
self,
actor: Principal,
project_id: UUID,
language_tag: str,
) -> list[dict]:
"""Return chapter metadata (id, title, matter, paragraph count) for one language project.
English chapters are read from source/book.json. Translated chapters
are read from runs/<chapter>/translation.json.
"""
scope = scope_for(actor)
project = self._repository.get_project(project_id, organization_id=scope.organization_id)
if project is None:
if scope.unrestricted:
raise NotFoundError("project_not_found")
raise ForbiddenError("cross_organization_forbidden")
prod_dir = self._productions_dir_for(project)
if prod_dir is None:
return []
is_source = language_tag == "en"
if is_source:
book_path = prod_dir / "source" / "book.json"
book = self._read_json(book_path)
if book is None:
return []
chapters = []
for ch in book.get("chapters", []):
chapters.append({
"id": ch.get("id", ""),
"title": ch.get("title", ""),
"matter": ch.get("matter", "body"),
"index": ch.get("index", 0),
"paragraph_count": len(ch.get("paragraphs", []) or []),
})
chapters.sort(key=lambda c: c["index"])
return chapters
# Translated language: scan runs directory for chapter folders.
runs_dir = prod_dir / "runs"
if not runs_dir.is_dir():
return []
chapters = []
for ch_dir in sorted(runs_dir.iterdir()):
if ch_dir.is_dir() and ch_dir.name.startswith("ch-"):
translation_path = ch_dir / "translation.json"
data = self._read_json(translation_path)
if data is None:
continue
chapters.append({
"id": data.get("chapter_id", ch_dir.name),
"title": data.get("title", ch_dir.name),
"matter": "body",
"index": data.get("chapter_index", 0),
"paragraph_count": len(data.get("paragraphs", {}) or {}),
})
chapters.sort(key=lambda c: c["index"])
return chapters
def get_chapter_content(
self,
actor: Principal,
project_id: UUID,
language_tag: str,
chapter_id: str,
) -> dict:
"""Return full paragraph content for one chapter (front matter editing).
English chapters are read from source/book.json. Translated chapters
read from runs/<chapter>/translation.json.
"""
scope = scope_for(actor)
project = self._repository.get_project(project_id, organization_id=scope.organization_id)
if project is None:
if scope.unrestricted:
raise NotFoundError("project_not_found")
raise ForbiddenError("cross_organization_forbidden")
prod_dir = self._productions_dir_for(project)
if prod_dir is None:
return {"chapter_id": chapter_id, "matter": "body", "paragraphs": []}
is_source = language_tag == "en"
if is_source:
book_path = prod_dir / "source" / "book.json"
book = self._read_json(book_path)
if book is None:
return {"chapter_id": chapter_id, "matter": "body", "paragraphs": []}
for ch in book.get("chapters", []):
if ch.get("id") == chapter_id:
return {
"chapter_id": ch.get("id", chapter_id),
"title": ch.get("title", ""),
"matter": ch.get("matter", "body"),
"paragraphs": ch.get("paragraphs", []) or [],
}
return {"chapter_id": chapter_id, "matter": "body", "paragraphs": []}
# Translated language: read from runs/<chapter>/translation.json
translation_path = prod_dir / "runs" / chapter_id / "translation.json"
data = self._read_json(translation_path)
if data is None:
return {"chapter_id": chapter_id, "matter": "body", "paragraphs": []}
paragraphs = data.get("paragraphs", {}) or {}
return {
"chapter_id": data.get("chapter_id", chapter_id),
"title": data.get("title", chapter_id),
"matter": "body",
"paragraphs": [
{"id": key, "speaker": None, "type": "dialogue", "text": value}
for key, value in sorted(paragraphs.items())
],
}
def get_utterance_review(
self,
actor: Principal,
project_id: UUID,
language_tag: str,
chapter_id: str,
) -> UtteranceReviewResponse:
"""Return utterances for one chapter of one language, for speaker-alignment review.
English source text is read from ``source/book.json``; translated text is
read from ``runs/<chapter>/translation.json``.
"""
scope = scope_for(actor)
project = self._repository.get_project(project_id, organization_id=scope.organization_id)
if project is None:
if scope.unrestricted:
raise NotFoundError("project_not_found")
raise ForbiddenError("cross_organization_forbidden")
prod_dir = self._productions_dir_for(project)
if prod_dir is None:
return UtteranceReviewResponse(
project_id=project_id,
language_tag=language_tag,
chapter_id=chapter_id,
utterances=[],
speakers=[],
)
is_source = language_tag == "en"
if is_source:
# English source: read from source/book.json
book_path = prod_dir / "source" / "book.json"
book = self._read_json(book_path)
if book is None:
return UtteranceReviewResponse(
project_id=project_id,
language_tag=language_tag,
chapter_id=chapter_id,
utterances=[],
speakers=[],
)
paragraphs = []
for ch in book.get("chapters", []):
if ch.get("id") == chapter_id:
paragraphs = ch.get("paragraphs", []) or []
break
utterances = [
{
"id": p.get("id", ""),
"text": p.get("text", ""),
"speaker": p.get("speaker"),
"speaker_confidence": None,
"needs_review": True,
}
for p in paragraphs
]
# Collect all unique speakers from this chapter for the dropdown.
speaker_ids = set()
for p in paragraphs:
sp = p.get("speaker")
if sp:
speaker_ids.add(sp)
speakers = [
{"id": sp, "name": sp, "role": "speaker"}
for sp in sorted(speaker_ids)
]
else:
# Translated language: read from runs/<chapter>/translation.json.
# Speaker info comes from the English source chapter, matched by
# paragraph id, since translated text doesn't carry speaker data.
translation_path = prod_dir / "runs" / chapter_id / "translation.json"
data = self._read_json(translation_path)
if data is None:
return UtteranceReviewResponse(
project_id=project_id,
language_tag=language_tag,
chapter_id=chapter_id,
utterances=[],
speakers=[],
)
trans_paragraphs = data.get("paragraphs", {}) or {}
# Load English source for speaker data, matched by paragraph id.
speaker_by_id = {}
book_path = prod_dir / "source" / "book.json"
book = self._read_json(book_path)
if book is not None:
for ch in book.get("chapters", []):
if ch.get("id") == chapter_id:
for p in ch.get("paragraphs", []) or []:
speaker_by_id[p.get("id", "")] = p.get("speaker")
break
utterances = [
{
"id": key,
"text": value,
"speaker": speaker_by_id.get(key),
"speaker_confidence": None,
"needs_review": True,
}
for key, value in sorted(trans_paragraphs.items())
]
# Collect all unique speakers from the English source for the dropdown.
speaker_ids = set(speaker_by_id.values())
speaker_ids.discard(None)
# Build speaker dropdown options.
if speaker_ids:
speakers = [
{"id": sp, "name": sp, "role": "speaker"}
for sp in sorted(speaker_ids)
]
else:
speakers = [
{"id": "narrator", "name": "Narrator", "role": "narrator"}
]
return UtteranceReviewResponse(
project_id=project_id,
language_tag=language_tag,
chapter_id=chapter_id,
utterances=utterances,
speakers=speakers,
)
def get_front_matter(
self,
actor: Principal,
project_id: UUID,
language_tag: str,
) -> FrontMatterResponse:
"""Read front-matter items for one language project."""
scope = scope_for(actor)
project = self._repository.get_project(project_id, organization_id=scope.organization_id)
if project is None:
if scope.unrestricted:
raise NotFoundError("project_not_found")
raise ForbiddenError("cross_organization_forbidden")
prod_dir = self._productions_dir_for(project)
if prod_dir is None:
stored = _JOB_STORE.get(self._job_key(project_id, language_tag, "front_matter"))
if stored:
items = [
FrontMatterItemResponse(
key=item.get("key", ""),
label=item.get("label", ""),
value=item.get("value"),
output_page=item.get("output_page"),
)
for item in stored.get("items", [])
]
return FrontMatterResponse(
project_id=project_id,
language_tag=language_tag,
items=items,
version=stored.get("version", 1),
)
return FrontMatterResponse(
project_id=project_id,
language_tag=language_tag,
items=[],
version=1,
)
data = self._read_json(prod_dir / "front-matter" / language_tag / "items.json")
# Also check in-memory store (for deployments with read-only productions).
stored = _JOB_STORE.get(self._job_key(project_id, language_tag, "front_matter"))
if stored and (not data or stored.get("version", 0) >= data.get("version", 0)):
data = stored
if data is None:
return FrontMatterResponse(
project_id=project_id,
language_tag=language_tag,
items=[],
version=1,
)
items = [
FrontMatterItemResponse(
key=item.get("key", ""),
label=item.get("label", ""),
value=item.get("value"),
output_page=item.get("output_page"),
)
for item in data.get("items", [])
]
return FrontMatterResponse(
project_id=project_id,
language_tag=language_tag,
items=items,
version=data.get("version", 1),
)
def save_front_matter(
self,
actor: Principal,
project_id: UUID,
language_tag: str,
items: list[dict],
*,
request_id: str | None = None,
) -> FrontMatterResponse:
"""Save front-matter items for one language project."""
scope = scope_for(actor)
project = self._repository.get_project(project_id, organization_id=scope.organization_id)
if project is None:
if scope.unrestricted:
raise NotFoundError("project_not_found")
raise ForbiddenError("cross_organization_forbidden")
prod_dir = self._productions_dir_for(project)
if prod_dir is None:
# No productions directory: hold front matter in the in-memory job store.
key = self._job_key(project_id, language_tag, "front_matter")
version = int((_JOB_STORE.get(key) or {}).get("version", 1)) + 1
item_records = [
{
"key": item.get("key", ""),
"label": item.get("label", ""),
"value": item.get("value"),
"output_page": item.get("output_page"),
}
for item in items
]
_JOB_STORE[key] = {
"language_tag": language_tag,
"items": item_records,
"version": version,
"updated_at": datetime.now(timezone.utc).isoformat(),
}
self._audit(
action=ACTION_FRONT_MATTER_SAVED,
outcome="success",
actor=actor,
organization_id=project.organization_id,
request_id=request_id,
detail={
"project_slug": project.slug,
"language_tag": language_tag,
"version": version,
},
)
return FrontMatterResponse(
project_id=project_id,
language_tag=language_tag,
items=[
FrontMatterItemResponse(
key=r["key"],
label=r["label"],
value=r["value"],
output_page=r["output_page"],
)
for r in item_records
],
version=version,
)
fm_dir = prod_dir / "front-matter" / language_tag
fm_dir.mkdir(parents=True, exist_ok=True)
existing = self._read_json(fm_dir / "items.json") or {}
version = int(existing.get("version", 1)) + 1
item_records = [
{
"key": item.get("key", ""),
"label": item.get("label", ""),
"value": item.get("value"),
"output_page": item.get("output_page"),
}
for item in items
]
self._write_json(
fm_dir / "items.json",
{
"language_tag": language_tag,
"items": item_records,
"version": version,
"updated_at": datetime.now(timezone.utc).isoformat(),
},
)
self._audit(
action=ACTION_FRONT_MATTER_SAVED,
outcome="success",
actor=actor,
organization_id=project.organization_id,
request_id=request_id,
detail={
"project_slug": project.slug,
"language_tag": language_tag,
"version": version,
},
)
return FrontMatterResponse(
project_id=project_id,
language_tag=language_tag,
items=[
FrontMatterItemResponse(
key=r["key"],
label=r["label"],
value=r["value"],
output_page=r["output_page"],
)
for r in item_records
],
version=version,
)
def workspace(self, actor: Principal) -> "WorkspaceSnapshot": def workspace(self, actor: Principal) -> "WorkspaceSnapshot":
"""The signed-in caller's organization and the work inside it. """The signed-in caller's organization and the work inside it.

357
web/assets/admin-view.js Normal file
View File

@@ -0,0 +1,357 @@
// Copyright 2026 Umbra Tools (umbratools.com)
// SPDX-License-Identifier: Apache-2.0
/*
* Embeddable administration views.
*
* `UmbraAdminView.mount(container, { section, onSectionChange })` renders one
* administration section into `container` and returns a handle with `show()` and
* `destroy()`. The Umbra Tools shell mounts this inline; the standalone `uadmin/`
* page keeps its own shell and script and does not use this module.
*
* Why this exists as a separate file: the control panel is served with
* `X-Frame-Options: DENY` and `frame-ancestors 'none'` on purpose, so it must never
* be framed. Embedding means rendering the same views in the same document, not
* loading the other page inside a frame.
*
* Only what the shell's rail exposes is implemented here: dashboard, accounts,
* organizations, projects, audit. Nodes stays on its standalone page (it has its
* own pairing flow) and is linked, not embedded.
*/
(function (global) {
'use strict';
var api = global.UmbraApi;
if (!api) {
return;
}
var el = api.el;
var q = api.query;
var SUMMARY_CARDS = [
{ key: 'total_organizations', label: 'Organizations', note: 'Tenants in scope' },
{ key: 'active_organizations', label: 'Active organizations', note: 'Accept new accounts' },
{ key: 'total_accounts', label: 'Accounts', note: 'Every account in scope' },
{ key: 'active_accounts', label: 'Active accounts', note: 'Can open a session' },
{ key: 'suspended_accounts', label: 'Suspended accounts', note: 'Sessions revoked' },
{ key: 'pending_first_login', label: 'Awaiting first sign-in', note: 'Must change a temporary password' },
{ key: 'organization_admin_count', label: 'Organization admins', note: 'Administer their own tenant' }
];
var STATUS_TEXT = {
active: 'Active', suspended: 'Suspended', deactivated: 'Deactivated',
on_hold: 'On hold', released: 'Released', archived: 'Archived'
};
function statusPill(status) {
var tone =
status === 'active' ? 'success'
: status === 'suspended' || status === 'on_hold' ? 'warning'
: status === 'released' ? 'info'
: 'muted';
return el('span', { class: 'status-pill status-pill--' + tone, text: STATUS_TEXT[status] || api.label(status) });
}
function emptyBlock(title, body, action) {
var children = [
el('p', { class: 'empty-state__title', text: title }),
el('p', { class: 'empty-state__body', text: body })
];
if (action) {
children.push(action);
}
return el('div', { class: 'empty-state' }, children);
}
function heading(eyebrow, title, body, action) {
var right = el('div', {});
if (action) {
right.appendChild(action);
}
return el('div', { class: 'admin-heading' }, [
el('div', {}, [
el('p', { class: 'eyebrow', text: eyebrow }),
el('h1', { text: title }),
el('p', { text: body })
]),
right
]);
}
function loadingNote(text) {
return el('p', { class: 'loading-note', text: text });
}
function statGrid() {
return el('div', { class: 'admin-stat-grid' }, [loadingNote('Loading summary…')]);
}
/* ---------------------------------------------------------- dashboard */
function renderDashboard(container) {
var grid = statGrid();
var activity = el('div', { class: 'audit-list' }, [loadingNote('Loading recent events…')]);
container.appendChild(heading('Administration', 'System overview', 'Counts cover only the organizations and accounts you are authorized to see.'));
container.appendChild(el('section', { class: 'admin-card' }, [
el('div', { class: 'admin-card-heading' }, [
el('div', {}, [el('p', { class: 'eyebrow', text: 'Totals' }), el('h2', { text: 'Account and organization summary' })])
]),
grid
]));
container.appendChild(el('section', { class: 'admin-card' }, [
el('div', { class: 'admin-card-heading' }, [
el('div', {}, [el('p', { class: 'eyebrow', text: 'Traceability' }), el('h2', { text: 'Recent activity' })])
]),
activity
]));
api.getSummary().then(function (data) {
api.clear(grid);
SUMMARY_CARDS.forEach(function (card) {
var value = data && typeof data[card.key] === 'number' ? data[card.key] : 0;
grid.appendChild(el('article', { class: 'admin-stat' }, [
el('span', { class: 'admin-stat-label', text: card.label }),
el('strong', { text: String(value) }),
el('small', { text: card.note })
]));
});
}, function (error) {
api.clear(grid);
grid.appendChild(emptyBlock('Summary unavailable', api.describeError(error)));
});
api.listAudit({ limit: 10 }).then(function (data) {
api.clear(activity);
var events = (data && data.items) || [];
if (!events.length) {
activity.appendChild(emptyBlock('No recent activity', 'Nothing has been recorded yet.'));
return;
}
events.forEach(function (event) {
activity.appendChild(el('div', { class: 'audit-row' }, [
el('span', { class: 'audit-row__when', text: api.formatDateTime(event.occurred_at) }),
el('span', { class: 'audit-row__action', text: event.action }),
statusPill(event.outcome === 'success' ? 'active' : 'suspended')
]));
});
}, function (error) {
api.clear(activity);
activity.appendChild(emptyBlock('Activity unavailable', api.describeError(error)));
});
}
/* ----------------------------------------------------------- accounts */
function renderAccounts(container) {
container.appendChild(heading('Identity', 'Accounts', 'Every account inside your administrative scope, with credential and session state.'));
var tbody = el('tbody', {});
container.appendChild(el('section', { class: 'admin-card' }, [
el('div', { class: 'admin-table-wrap' }, [
el('table', { class: 'admin-table admin-table--wide' }, [
el('thead', {}, [el('tr', {}, [
el('th', { scope: 'col', text: 'Account' }),
el('th', { scope: 'col', text: 'Email' }),
el('th', { scope: 'col', text: 'Role' }),
el('th', { scope: 'col', text: 'Status' })
])]),
tbody
])
])
]));
tbody.appendChild(el('tr', {}, [el('td', { colspan: '4' }, [loadingNote('Loading accounts…')])]));
api.listAccounts({ limit: 50 }).then(function (data) {
api.clear(tbody);
var items = (data && data.items) || [];
if (!items.length) {
tbody.appendChild(el('tr', {}, [el('td', { colspan: '4' }, [emptyBlock('No accounts', 'No accounts are in scope.')])]));
return;
}
items.forEach(function (account) {
tbody.appendChild(el('tr', {}, [
el('td', { text: account.display_name || account.username || '—' }),
el('td', { text: account.email || '—' }),
el('td', { text: api.label(account.role) }),
el('td', {}, [statusPill(account.status)])
]));
});
}, function (error) {
api.clear(tbody);
tbody.appendChild(el('tr', {}, [el('td', { colspan: '4' }, [emptyBlock('Accounts unavailable', api.describeError(error))])]));
});
}
/* ------------------------------------------------------- organizations */
function renderOrganizations(container) {
container.appendChild(heading('Tenancy', 'Organizations', 'Tenants that own accounts.'));
var tbody = el('tbody', {});
container.appendChild(el('section', { class: 'admin-card' }, [
el('div', { class: 'admin-table-wrap' }, [
el('table', { class: 'admin-table admin-table--wide' }, [
el('thead', {}, [el('tr', {}, [
el('th', { scope: 'col', text: 'Organization' }),
el('th', { scope: 'col', text: 'Slug' }),
el('th', { scope: 'col', text: 'Status' }),
el('th', { scope: 'col', text: 'Members' })
])]),
tbody
])
])
]));
tbody.appendChild(el('tr', {}, [el('td', { colspan: '4' }, [loadingNote('Loading organizations…')])]));
api.listOrganizations({ limit: 100 }).then(function (data) {
api.clear(tbody);
var items = (data && data.items) || [];
if (!items.length) {
tbody.appendChild(el('tr', {}, [el('td', { colspan: '4' }, [emptyBlock('No organizations', 'No organizations are in scope.')])]));
return;
}
items.forEach(function (organization) {
tbody.appendChild(el('tr', {}, [
el('td', { text: organization.name }),
el('td', { text: organization.slug }),
el('td', {}, [statusPill(organization.status)]),
el('td', { text: String(organization.member_count == null ? '—' : organization.member_count) })
]));
});
}, function (error) {
api.clear(tbody);
tbody.appendChild(el('tr', {}, [el('td', { colspan: '4' }, [emptyBlock('Organizations unavailable', api.describeError(error))])]));
});
}
/* ----------------------------------------------------------- projects */
function renderProjects(container) {
container.appendChild(heading('Work', 'Projects', 'Units of work inside your organization. Everyone in the organization sees every project.'));
var tbody = el('tbody', {});
container.appendChild(el('section', { class: 'admin-card' }, [
el('div', { class: 'admin-table-wrap' }, [
el('table', { class: 'admin-table' }, [
el('thead', {}, [el('tr', {}, [
el('th', { scope: 'col', text: 'Project' }),
el('th', { scope: 'col', text: 'Slug' }),
el('th', { scope: 'col', text: 'Status' }),
el('th', { scope: 'col', text: 'Working on it' })
])]),
tbody
])
])
]));
tbody.appendChild(el('tr', {}, [el('td', { colspan: '4' }, [loadingNote('Loading projects…')])]));
api.listProjects({ limit: 200 }).then(function (data) {
api.clear(tbody);
// /api/uadmin/projects answers with `projects`, unlike the other admin
// lists, which answer with `items`. Accept either rather than rendering an
// empty table against a response that is not empty.
var items = (data && (data.projects || data.items)) || [];
if (!items.length) {
tbody.appendChild(el('tr', {}, [el('td', { colspan: '4' }, [emptyBlock('No projects yet', 'Create the first project from the control panel.')])]));
return;
}
items.forEach(function (project) {
tbody.appendChild(el('tr', {}, [
el('td', { text: project.name }),
el('td', { text: project.slug }),
el('td', {}, [statusPill(project.status)]),
el('td', { text: (project.assigned_count || 0) === 1 ? '1 person' : (project.assigned_count || 0) + ' people' })
]));
});
}, function (error) {
api.clear(tbody);
tbody.appendChild(el('tr', {}, [el('td', { colspan: '4' }, [emptyBlock('Projects unavailable', api.describeError(error))])]));
});
}
/* -------------------------------------------------------------- audit */
function renderAudit(container) {
container.appendChild(heading('Traceability', 'Audit log', 'Sanitized identity events.'));
var tbody = el('tbody', {});
container.appendChild(el('section', { class: 'admin-card' }, [
el('div', { class: 'admin-table-wrap' }, [
el('table', { class: 'admin-table admin-table--audit' }, [
el('thead', {}, [el('tr', {}, [
el('th', { scope: 'col', text: 'Occurred' }),
el('th', { scope: 'col', text: 'Action' }),
el('th', { scope: 'col', text: 'Outcome' })
])]),
tbody
])
])
]));
tbody.appendChild(el('tr', {}, [el('td', { colspan: '3' }, [loadingNote('Loading audit events…')])]));
api.listAudit({ limit: 50 }).then(function (data) {
api.clear(tbody);
var events = (data && data.items) || [];
if (!events.length) {
tbody.appendChild(el('tr', {}, [el('td', { colspan: '3' }, [emptyBlock('No audit events', 'Nothing has been recorded yet.')])]));
return;
}
events.forEach(function (event) {
tbody.appendChild(el('tr', {}, [
el('td', { text: api.formatDateTime(event.occurred_at) }),
el('td', { text: event.action }),
el('td', { text: event.outcome })
]));
});
}, function (error) {
api.clear(tbody);
tbody.appendChild(el('tr', {}, [el('td', { colspan: '3' }, [emptyBlock('Audit log unavailable', api.describeError(error))])]));
});
}
/* ------------------------------------------------------------- mount */
var RENDERERS = {
dashboard: renderDashboard,
accounts: renderAccounts,
organizations: renderOrganizations,
projects: renderProjects,
audit: renderAudit
};
function mount(container, options) {
options = options || {};
var current = options.section || 'dashboard';
function draw() {
api.clear(container);
var render = RENDERERS[current] || renderDashboard;
try {
render(container);
} catch (error) {
container.appendChild(emptyBlock('Section failed to render', api.describeError(error)));
}
}
draw();
return {
show: function (section) {
if (section && section !== current) {
current = section;
draw();
}
},
destroy: function () { api.clear(container); },
section: function () { return current; }
};
}
global.UmbraAdminView = { mount: mount };
})(window);

120
web/assets/api.js
View File

@@ -443,6 +443,14 @@
return request('/api/uadmin/projects', { params: filters, signal: signal }); return request('/api/uadmin/projects', { params: filters, signal: signal });
} }
function projectStatus(projectId, signal) {
return request('/api/workspace/' + encodeURIComponent(projectId) + '/status', { signal: signal });
}
function projectDetail(projectId, signal) {
return request('/api/workspace/' + encodeURIComponent(projectId), { signal: signal });
}
function createProject(payload, signal) { function createProject(payload, signal) {
return request('/api/uadmin/projects', { method: 'POST', body: payload, signal: signal }); return request('/api/uadmin/projects', { method: 'POST', body: payload, signal: signal });
} }
@@ -478,6 +486,107 @@
); );
} }
/* ------------------------------------------------- language-project workflow */
/**
* Queue a translation job for one language project.
*/
function queueTranslation(projectId, languageTag, signal) {
return request(
'/api/workspace/' + encodeURIComponent(projectId) + '/translation/' + encodeURIComponent(languageTag) + '/queue',
{ method: 'POST', signal: signal }
);
}
/**
* Queue an audio generation job for one language project.
* Requires the language's translation to be complete first.
*/
function queueAudio(projectId, languageTag, voiceConfig, signal) {
return request(
'/api/workspace/' + encodeURIComponent(projectId) + '/audio/' + encodeURIComponent(languageTag) + '/queue',
{ method: 'POST', body: { voice_config: voiceConfig || {} }, signal: signal }
);
}
/**
* Read format metadata (EPUB/PDF) for one language project.
*/
function getFormatMetadata(projectId, languageTag, signal) {
return request(
'/api/workspace/' + encodeURIComponent(projectId) + '/format/' + encodeURIComponent(languageTag),
{ signal: signal }
);
}
/**
* Save format metadata (cover, ISBN, blurb, search terms, etc.) for one
* language project.
*/
function setFormatMetadata(projectId, languageTag, metadata, signal) {
return request(
'/api/workspace/' + encodeURIComponent(projectId) + '/format/' + encodeURIComponent(languageTag),
{ method: 'POST', body: metadata, signal: signal }
);
}
/**
* Return utterances for one chapter of one language, for speaker-alignment
* review.
*/
function getUtteranceReview(projectId, languageTag, chapterId, signal) {
return request(
'/api/workspace/' +
encodeURIComponent(projectId) +
'/review/' +
encodeURIComponent(languageTag) +
'/chapters/' +
encodeURIComponent(chapterId),
{ signal: signal }
);
}
/**
* Read front-matter items for one language project.
*/
function getFrontMatter(projectId, languageTag, signal) {
return request(
'/api/workspace/' + encodeURIComponent(projectId) + '/front-matter/' + encodeURIComponent(languageTag),
{ signal: signal }
);
}
/**
* Save front-matter items for one language project.
*/
function saveFrontMatter(projectId, languageTag, items, signal) {
return request(
'/api/workspace/' + encodeURIComponent(projectId) + '/front-matter/' + encodeURIComponent(languageTag),
{ method: 'POST', body: { language_tag: languageTag, items: items }, signal: signal }
);
}
/**
* Return chapter metadata for one language project.
*/
function getChapters(projectId, languageTag, signal) {
return request(
'/api/workspace/' + encodeURIComponent(projectId) + '/chapters/' + encodeURIComponent(languageTag),
{ signal: signal }
);
}
/**
* Return full paragraph content for one chapter (front matter editing).
*/
function getChapterContent(projectId, languageTag, chapterId, signal) {
return request(
'/api/workspace/' + encodeURIComponent(projectId) + '/chapter-content/' +
encodeURIComponent(languageTag) + '/' + encodeURIComponent(chapterId),
{ signal: signal }
);
}
/* --------------------------------------------------------- DOM utilities */ /* --------------------------------------------------------- DOM utilities */
function appendChild(node, child) { function appendChild(node, child) {
@@ -864,8 +973,19 @@
listProjects: listProjects, listProjects: listProjects,
createProject: createProject, createProject: createProject,
setProjectStatus: setProjectStatus, setProjectStatus: setProjectStatus,
projectStatus: projectStatus,
projectDetail: projectDetail,
assignToProject: assignToProject, assignToProject: assignToProject,
unassignFromProject: unassignFromProject, unassignFromProject: unassignFromProject,
queueTranslation: queueTranslation,
queueAudio: queueAudio,
getFormatMetadata: getFormatMetadata,
setFormatMetadata: setFormatMetadata,
getUtteranceReview: getUtteranceReview,
getFrontMatter: getFrontMatter,
saveFrontMatter: saveFrontMatter,
getChapters: getChapters,
getChapterContent: getChapterContent,
request: request, request: request,
el: el, el: el,
clear: clearNode, clear: clearNode,

29
web/assets/app.js
View File

@@ -61,7 +61,13 @@
} }
function destinationFor(role) { function destinationFor(role) {
return role === 'platform_admin' || role === 'organization_admin' ? 'uadmin/' : 'workspace.html'; // Every session lands in the Umbra Tools shell, which hosts the control
// panel, project work, the editor, and audio as activities. Previously this
// sent administrators straight to `uadmin/` and members to `workspace.html`,
// which meant the shell was never reached and the two old pages drifted
// apart. The shell now owns the top banner, left rail, and activity switch,
// and embeds the control panel rather than replacing the navigation with it.
return 'umbra-app.html';
} }
function showPasswordToggle(button, input) { function showPasswordToggle(button, input) {
@@ -599,13 +605,28 @@
side.push(api.el('span', { class: 'status-pill marker--unclaimed', text: 'Needs someone' })); side.push(api.el('span', { class: 'status-pill marker--unclaimed', text: 'Needs someone' }));
} }
return api.el('article', { class: 'project-card project-card--' + project.status }, [ var card = api.el('article', { class: 'project-card project-card--' + project.status, role: 'button', tabIndex: 0 }, [
api.el('div', { class: 'project-card__main' }, [ api.el('div', { class: 'project-card__main' }, [
api.el('span', { class: 'project-card__name', text: project.name }), api.el('span', { class: 'project-card__name', text: project.name }),
api.el('div', { class: 'project-card__meta' }, meta) api.el('div', { class: 'project-card__meta' }, meta)
]), ]),
api.el('div', { class: 'project-card__side' }, side) api.el('div', { class: 'project-card__side' }, side)
]); ]);
card.addEventListener('click', function () {
try {
var url = 'workspace-project.html?id=' + encodeURIComponent(project.id);
window.location.href = url;
} catch (e) {
// ignore
}
});
card.addEventListener('keydown', function (ev) {
if (ev.key === 'Enter' || ev.key === ' ') {
ev.preventDefault();
card.click();
}
});
return card;
} }
function setMetric(selector, value) { function setMetric(selector, value) {
@@ -647,7 +668,7 @@
var mine = snapshot.assigned_to_me || []; var mine = snapshot.assigned_to_me || [];
var visible = filter === 'mine' ? mine : all; var visible = filter === 'mine' ? mine : all;
api.clearNode(list); api.clear(list);
visible.forEach(function (project) { visible.forEach(function (project) {
list.appendChild(projectCard(project)); list.appendChild(projectCard(project));
}); });
@@ -814,3 +835,5 @@
setupWorkspace(); setupWorkspace();
offerSetupIfRequired(); offerSetupIfRequired();
})(window); })(window);
try { var p = new URL(window.location.href); if (p.searchParams.get('shell') === 'umbra') { window.location.href = 'umbra-app.html'; } } catch(e){}

198
web/assets/styles.css
View File

@@ -480,3 +480,201 @@ html.modal-open { overflow: hidden; }
font-size: .95em; font-size: .95em;
word-break: break-all; word-break: break-all;
} }
/* === Umbra Tools New Shell === */
:root {
--shell-bg: #0b1020;
--shell-panel: #0f1529;
--shell-border: #1f2937;
--shell-text: #e5e7eb;
--shell-muted: #94a3b8;
--shell-accent: #22d3ee;
--shell-accent-2: #38bdf8;
}
[data-theme="light"] {
--shell-bg: #f8fafc;
--shell-panel: #ffffff;
--shell-border: #e5e7eb;
--shell-text: #0f172a;
--shell-muted: #475569;
--shell-accent: #06b6d4;
--shell-accent-2: #2563eb;
}
.umbra-shell { display: grid; grid-template-rows: 4rem 1fr; grid-template-columns: 280px 1fr; grid-template-areas: "topbar topbar" "sidebar main"; min-height: 100vh; background: var(--shell-bg); color: var(--shell-text); }
.umbra-topbar { grid-area: topbar; display: grid; grid-template-columns: 280px 1fr auto; align-items: center; gap: 1rem; padding: 0 1rem; background: linear-gradient(180deg, rgba(30,41,59,.9), rgba(15,23,42,.9)); border-bottom: 1px solid var(--shell-border); }
.umbra-topbar__left { display: flex; align-items: center; gap: .75rem; }
.umbra-topbar__logo { width: 28px; height: 28px; border-radius: 6px; }
.umbra-topbar__brand { font-weight: 800; letter-spacing: .02em; }
.umbra-topbar__mode { background: var(--shell-panel); color: var(--shell-text); border: 1px solid var(--shell-border); border-radius: .5rem; padding: .4rem .6rem; }
.umbra-topbar__middle { color: var(--shell-muted); font-weight: 600; }
.umbra-topbar__right { display: flex; align-items: center; gap: .6rem; }
.umbra-user { display: flex; align-items: center; gap: .5rem; }
.umbra-user__meta { display: flex; flex-direction: column; line-height: 1.1; }
.umbra-user__name { font-size: .9rem; }
.umbra-user__role { font-size: .72rem; color: var(--shell-muted); }
.umbra-dropdown { background: var(--shell-panel); color: var(--shell-text); border: 1px solid var(--shell-border); border-radius: .5rem; padding: .35rem .5rem; }
.umbra-sidebar { grid-area: sidebar; background: var(--shell-panel); border-right: 1px solid var(--shell-border); padding: .8rem; overflow-y: auto; }
.umbra-sidebar__section { margin-bottom: 1rem; }
.umbra-sidebar__title { color: var(--shell-muted); text-transform: uppercase; font-size: .7rem; letter-spacing: .12em; margin: .4rem 0 .6rem; }
.umbra-nav__item { display: flex; align-items: center; gap: .5rem; width: 100%; text-align: left; background: transparent; color: var(--shell-text); border: 0; padding: .5rem .6rem; border-radius: .5rem; cursor: pointer; text-decoration: none; }
.umbra-nav__item:hover { background: rgba(56,189,248,.12); }
.umbra-nav__item--active { background: linear-gradient(90deg, rgba(34,211,238,.18), transparent); border-left: 2px solid var(--shell-accent); }
.umbra-main { grid-area: main; padding: 1.2rem 1.6rem; overflow-y: auto; }
.project-row { display: grid; grid-template-columns: 1.2fr 1fr auto; gap: 1rem; align-items: center; padding: .8rem 1rem; background: var(--shell-panel); border: 1px solid var(--shell-border); border-radius: .7rem; margin-bottom: .6rem; }
.project-row__lang { font-weight: 700; }
.project-row__status { color: var(--shell-muted); }
.project-row__actions { display: flex; gap: .4rem; flex-wrap: wrap; }
.btn { background: linear-gradient(180deg, rgba(56,189,248,.9), rgba(14,165,233,.9)); color: white; border: 0; border-radius: .5rem; padding: .4rem .7rem; cursor: pointer; }
.btn--ghost { background: transparent; color: var(--shell-text); border: 1px solid var(--shell-border); }
.btn--subtle { background: rgba(148,163,184,.18); color: var(--shell-text); border: 1px solid var(--shell-border); }
.format-tabs { display: flex; gap: .4rem; margin-bottom: 1rem; }
.format-tab { padding: .4rem .7rem; border-radius: .5rem; border: 1px solid var(--shell-border); background: var(--shell-panel); color: var(--shell-text); cursor: pointer; }
.format-tab--active { background: rgba(34,211,238,.18); border-color: rgba(34,211,238,.6); }
.form-grid { display: grid; grid-template-columns: repeat(2, minmax(0,1fr)); gap: .8rem; }
.form-grid__full { grid-column: 1 / -1; }
.input { width: 100%; background: var(--shell-panel); color: var(--shell-text); border: 1px solid var(--shell-border); border-radius: .5rem; padding: .5rem .6rem; }
.binder { display: grid; grid-template-columns: 260px 1fr; gap: 1rem; }
.binder__tree { background: var(--shell-panel); border: 1px solid var(--shell-border); border-radius: .7rem; padding: .6rem; }
.binder__editor { background: var(--shell-panel); border: 1px solid var(--shell-border); border-radius: .7rem; padding: 1rem; min-height: 70vh; }
.tree__item { padding: .35rem .5rem; border-radius: .4rem; cursor: pointer; }
.tree__item:hover { background: rgba(56,189,248,.12); }
.versions { display: flex; gap: .4rem; flex-wrap: wrap; }
.version-pill { padding: .3rem .5rem; border-radius: .4rem; border: 1px solid var(--shell-border); background: var(--shell-panel); color: var(--shell-text); cursor: pointer; }
.audio-layout { display: grid; grid-template-columns: 260px 1fr; gap: 1rem; }
.audio-sidebar { background: var(--shell-panel); border: 1px solid var(--shell-border); border-radius: .7rem; padding: .6rem; }
.audio-main { background: var(--shell-panel); border: 1px solid var(--shell-border); border-radius: .7rem; padding: 1rem; }
/* === Umbra Tools shell: additions === */
.umbra-topbar__brandlink { display: flex; align-items: center; gap: .6rem; color: inherit; text-decoration: none; }
.umbra-main--framed { padding: 0; }
.umbra-frame { display: block; width: 100%; height: 100%; min-height: calc(100vh - 4rem); border: 0; background: var(--shell-bg); }
.umbra-heading { margin-bottom: 1.2rem; }
.umbra-heading--row { display: flex; align-items: flex-start; justify-content: space-between; gap: 1rem; flex-wrap: wrap; }
.umbra-heading h1 { margin: .1rem 0 .3rem; font-size: 1.5rem; }
.umbra-heading .lede { margin: 0; color: var(--shell-muted); max-width: 60ch; }
.umbra-heading .eyebrow { margin: 0; color: var(--shell-accent); text-transform: uppercase; letter-spacing: .12em; font-size: .7rem; font-weight: 700; }
.umbra-inline-actions { display: flex; gap: .5rem; flex-wrap: wrap; margin-top: 1rem; }
.project-row__sub { display: block; color: var(--shell-muted); font-size: .72rem; font-weight: 500; margin-top: .1rem; }
.versions__label { color: var(--shell-muted); font-size: .78rem; margin-right: .2rem; }
/* Disabled controls must be legibly disabled, not just inert. */
.btn[disabled], .umbra-nav__item[disabled] { opacity: .5; cursor: not-allowed; }
.btn:disabled { cursor: not-allowed; }
/* === Umbra Tools shell: embedded admin views === */
.audit-row { display: grid; grid-template-columns: 12rem 1fr auto; gap: .8rem; align-items: center; padding: .5rem .2rem; border-bottom: 1px solid var(--shell-border); }
.audit-row__when { color: var(--shell-muted); font-size: .78rem; }
.audit-row__action { font-weight: 600; }
.admin-table td .empty-state { padding: 1rem 0; }
.umbra-main .admin-card { background: var(--shell-panel); border: 1px solid var(--shell-border); border-radius: .7rem; }
/* === Program / language selector visibility === */
.umbra-topbar__mode--hidden { display: none !important; }
/* Project rows list (program detail) */
.project-rows { display: flex; flex-direction: column; gap: .4rem; margin-top: 1rem; }
/* Language row (program detail view) */
.language-row { display: grid; grid-template-columns: 1fr auto; gap: 1rem; align-items: center; padding: .75rem 1rem; background: var(--shell-panel); border: 1px solid var(--shell-border); border-radius: .7rem; margin-bottom: .6rem; cursor: pointer; }
.language-row:hover { background: rgba(56,189,248,.06); }
.language-row--selected { border-color: rgba(34,211,238,.6); background: rgba(34,211,238,.08); }
.language-row__label { display: flex; flex-direction: column; gap: .15rem; }
.language-row__label strong { font-weight: 700; }
.language-row__role { font-size: .7rem; color: var(--shell-muted); }
.language-row__actions { display: flex; gap: .35rem; flex-wrap: wrap; }
/* Binder tree groups */
.tree__group { color: var(--shell-muted); text-transform: uppercase; font-size: .7rem; letter-spacing: .12em; margin: .5rem 0 .3rem; padding: 0 .5rem; }
/* Control panel: no topbar name when no program */
#topbar-language-name { color: var(--shell-text); font-weight: 700; }
/* Language detail panel */
.language-detail { margin-top: 1.5rem; }
.language-detail .umbra-heading { margin-bottom: 1rem; }
.language-detail table { margin-top: 0.5rem; }
/* Language row compact actions */
.language-row__actions .btn { font-size: 0.8rem; padding: 0.3rem 0.5rem; }
.language-row__actions a.btn--ghost { font-size: 0.8rem; padding: 0.3rem 0.5rem; }
/* Ensure program selector fits in topbar */
.umbra-topbar__middle { display: flex; align-items: center; gap: 0.5rem; }
#program-select { min-width: 200px; }
/* Audio layout */
.audio-layout { display: grid; grid-template-columns: 260px 1fr; gap: 1rem; }
.audio-sidebar { background: var(--shell-panel); border: 1px solid var(--shell-border); border-radius: .7rem; padding: .6rem; }
.audio-main { background: var(--shell-panel); border: 1px solid var(--shell-border); border-radius: .7rem; padding: 1rem; }
.audio-section-list { list-style: none; padding: 0; margin: 1rem 0 0 0; }
.audio-section-list li { padding: .4rem .6rem; border-radius: .4rem; cursor: pointer; }
.audio-section-list li:hover { background: rgba(56,189,248,.12); }
/* Chapter review placeholder */
.chapter-review-placeholder { margin-top: 1rem; }
/* Language detail panel spacing */
.language-detail { margin-top: 1.5rem; }
.language-detail .umbra-heading { margin-bottom: 1rem; }
.language-detail table { margin-top: 0.5rem; }
/* Program selector width */
#program-select { min-width: 220px; }
/* Language row compact actions */
.language-row__actions .btn { font-size: 0.8rem; padding: 0.3rem 0.5rem; }
.language-row__actions a.btn--ghost { font-size: 0.8rem; padding: 0.3rem 0.5rem; }
/* Ensure program selector fits in topbar */
.umbra-topbar__middle { display: flex; align-items: center; gap: 0.5rem; }
/* Utterance review */
.utterance-list { margin-top: 1rem; border: 1px solid var(--shell-border); border-radius: .6rem; background: var(--shell-panel); }
.utterance-row { display: grid; grid-template-columns: 130px 90px 1fr 140px; gap: .5rem; padding: .5rem .75rem; border-bottom: 1px solid var(--shell-border); font-size: .85rem; }
.utterance-row:last-child { border-bottom: none; }
.utterance__id { font-family: monospace; color: var(--shell-muted); }
.utterance__text { color: var(--shell-text); }
.utterance__speaker { display: flex; align-items: center; }
/* Front matter panel */
.front-matter-panel { margin-top: 1.5rem; border: 1px solid var(--shell-border); border-radius: .6rem; background: var(--shell-panel); padding: 1rem; }
.front-matter-panel h3 { margin: 0 0 .75rem 0; font-size: .9rem; color: var(--shell-muted); }
.fm-item { display: grid; grid-template-columns: 160px 1fr 120px 120px; gap: .5rem; align-items: center; margin-bottom: .5rem; }
.fm-item label { font-size: .85rem; color: var(--shell-text); }
.fm-item input, .fm-item select { font-size: .85rem; }
/* Utterance review panel */
.utterance-review-panel { margin-top: 1rem; border: 1px solid var(--shell-border); border-radius: .6rem; background: var(--shell-panel); padding: 1rem; }
.utterance-header { display: grid; grid-template-columns: 140px 1fr 180px; gap: .5rem; padding: .5rem .75rem; border-bottom: 2px solid var(--shell-border); font-size: .85rem; font-weight: bold; color: var(--shell-muted); }
.chapter-select { width: 160px; margin-bottom: 0.75rem; }
/* Chapter items in edit sidebar */
.tree__item--chapter { padding: 0.4rem 0.6rem; cursor: pointer; border-radius: 0.3rem; font-size: 0.85rem; }
.tree__item--chapter:hover { background: rgba(56,189,248,0.12); }
.tree__item--chapter.tree__item--active { background: #38bdf8; color: #0b1220; }
/* Format toolbar */
.format-toolbar { margin: 0.75rem 0; padding: 0.5rem; background: var(--shell-panel); border: 1px solid var(--shell-border); border-radius: 0.5rem; display: flex; gap: 0.4rem; }
.format-toolbar .btn { font-size: 0.8rem; padding: 0.3rem 0.6rem; }
/* Edit main content layout */
.edit-main-content { margin-top: 1rem; }
/* Utterance row speaker select */
.utterance-row select { font-size: 0.8rem; }

1570
web/assets/umbra-app.js Normal file
View File

File diff suppressed because it is too large Load Diff

166
web/assets/workspace-project.js Normal file
View File

@@ -0,0 +1,166 @@
(function () {
'use strict';
var api = window.UmbraAPI;
var nav = window.UmbraNav;
function query(sel) {
return document.querySelector(sel);
}
function el(tag, attrs, children) {
return api.el(tag, attrs || {}, children || []);
}
function getParam(name) {
try {
var url = new URL(window.location.href);
return url.searchParams.get(name);
} catch (e) {
return null;
}
}
function formatDate(s) {
if (!s) return '—';
try {
var d = new Date(s);
if (isNaN(d.getTime())) return s;
return d.toLocaleString();
} catch (e) {
return s;
}
}
function statusPill(status) {
var s = (status || '').toString();
var tone = s === 'active' ? 'success' : s === 'on_hold' ? 'warning' : s === 'released' ? 'info' : s === 'queued' || s === 'running' ? 'warning' : s === 'done' || s === 'complete' || s === 'available' ? 'success' : 'muted';
return api.el('span', { class: 'status-pill status-pill--' + tone, text: api.label(s) });
}
function load() {
if (!api) return;
api.getSession({ cache: 'no-store' })
.then(function (session) {
var user = session && session.user ? session.user : null;
var role = user && user.role ? user.role : '';
if (query('#rail-display-name')) query('#rail-display-name').textContent = user && user.display_name ? user.display_name : (user && user.username ? user.username : 'Account');
if (query('#rail-role')) query('#rail-role').textContent = api.label(role) || '—';
var avatar = query('#rail-avatar');
if (avatar) {
var initials = (user && user.display_name ? user.display_name : (user && user.username ? user.username : 'UT')).trim().split(/\s+/).slice(0, 2).map(function (p) { return p[0] ? p[0].toUpperCase() : ''; }).join('');
avatar.textContent = initials || 'UT';
}
var projectId = getParam('id');
if (!projectId) {
if (query('#project-title')) query('#project-title').textContent = 'No project selected';
if (query('#project-subtitle')) query('#project-subtitle').textContent = 'Return to workspace and select a project.';
return;
}
// Try to load project details from admin API if available, else from workspace snapshot
Promise.all([
api.fetch('/api/uadmin/projects/' + encodeURIComponent(projectId), { method: 'GET' }).catch(function(){ return null; }),
api.fetch('/api/workspace', { method: 'GET', cache: 'no-store' }).catch(function(){ return null; })
]).then(function (results) {
var projRes = results[0];
var wsRes = results[1];
var project = null;
if (projRes && projRes.ok) {
project = projRes.project || projRes.data || projRes;
}
if (!project && wsRes && wsRes.ok) {
var data = wsRes.data || wsRes;
var list = (data.projects || []).concat(data.assigned_to_me || []);
for (var i = 0; i < list.length; i++) {
if (String(list[i].id) === String(projectId)) { project = list[i]; break; }
}
}
if (!project) {
if (query('#project-title')) query('#project-title').textContent = 'Project';
if (query('#project-subtitle')) query('#project-subtitle').textContent = 'Project details unavailable from API.';
return;
}
if (query('#breadcrumb-identity')) query('#breadcrumb-identity').textContent = project.name;
if (query('#project-title')) query('#project-title').textContent = project.name;
if (query('#project-subtitle')) query('#project-subtitle').textContent = (project.slug ? '/' + project.slug : '') + (project.organization_id ? '' : '');
var pill = query('#project-status-pill');
if (pill) { pill.className = 'status-pill status-pill--' + (project.status === 'active' ? 'success' : project.status === 'on_hold' ? 'warning' : project.status === 'released' ? 'info' : 'muted'); pill.textContent = api.label(project.status); }
// English source table (placeholder based on known structure)
var tbody = query('#english-tbody');
if (tbody) {
api.clear(tbody);
var rows = [
{ch: 'Front', title: 'Front matter', words: '—', status: 'available'},
{ch: 'Body', title: 'Chapters (Deal Hunter)', words: '~66,884', status: 'available'},
{ch: 'Back', title: 'Back matter', words: '—', status: 'available'}
];
rows.forEach(function(r) {
var tr = document.createElement('tr');
tr.appendChild(el('td', {}, [document.createTextNode(r.ch)]));
tr.appendChild(el('td', {}, [document.createTextNode(r.title)]));
tr.appendChild(el('td', {}, [document.createTextNode(r.words)]));
tr.appendChild(el('td', {}, [statusPill(r.status)]));
tr.appendChild(el('td', {}, [api.el('button', { class: 'text-button', type: 'button', text: 'Edit/View' })]));
tbody.appendChild(tr);
});
}
// Translations
var tlist = query('#translations-list');
var tempty = query('#translations-empty');
if (tlist) api.clear(tlist);
var trans = [
{lang: 'English', code: 'en', status: 'source'},
{lang: 'Simplified Chinese (zh-CN)', code: 'zh-Hans', status: 'in_progress'}
];
if (tlist && trans.length) {
trans.forEach(function(t) {
var card = api.el('article', { class: 'project-card project-card--' + (t.status === 'in_progress' ? 'on_hold' : 'active') }, [
api.el('div', { class: 'project-card__main' }, [
api.el('span', { class: 'project-card__name', text: t.lang + ' (' + t.code + ')' }),
api.el('div', { class: 'project-card__meta' }, [api.el('span', { text: 'Status: ' + api.label(t.status) })])
]),
api.el('div', { class: 'project-card__side' }, [statusPill(t.status), api.el('button', { class: 'text-button', type: 'button', text: 'View / Tag for review' })])
]);
tlist.appendChild(card);
});
if (tempty) tempty.hidden = true;
} else if (tempty) {
tempty.hidden = false;
}
// Outputs
var oepub = query('#out-epub');
var opdf = query('#out-pdf');
var oaudio = query('#out-audio');
if (oepub) oepub.textContent = 'Not generated';
if (opdf) opdf.textContent = 'Not generated';
if (oaudio) oaudio.textContent = 'Not generated';
// Buttons
var bq = query('#btn-queue-translation');
if (bq) bq.addEventListener('click', function(){ api.showToast && api.showToast('Translation queued (placeholder)'); });
var bep = query('#btn-generate-epub');
if (bep) bep.addEventListener('click', function(){ api.showToast && api.showToast('EPUB generation queued (placeholder)'); });
var bpdf = query('#btn-generate-pdf');
if (bpdf) bpdf.addEventListener('click', function(){ api.showToast && api.showToast('PDF generation queued (placeholder)'); });
var ba = query('#btn-generate-audio');
if (ba) ba.addEventListener('click', function(){ api.showToast && api.showToast('Audio generation queued (placeholder)'); });
var bal = query('#btn-add-lang');
if (bal) bal.addEventListener('click', function(){ api.showToast && api.showToast('Add language (placeholder)'); });
});
})
.catch(function () {
// ignore
});
}
if (document.readyState === 'loading') {
document.addEventListener('DOMContentLoaded', load);
} else {
load();
}
})();

2
web/index.html
View File

@@ -67,7 +67,7 @@
<div id="setup-entry" class="auth-divider" hidden><span>First run</span></div> <div id="setup-entry" class="auth-divider" hidden><span>First run</span></div>
<a id="setup-link" class="admin-link" href="setup.html" hidden><span>Set up this deployment</span><span aria-hidden="true">↗</span></a> <a id="setup-link" class="admin-link" href="setup.html" hidden><span>Set up this deployment</span><span aria-hidden="true">↗</span></a>
<div id="admin-entry" class="auth-divider"><span>Administrator?</span></div> <div id="admin-entry" class="auth-divider"><span>Administrator?</span></div>
<a id="admin-link" class="admin-link" href="uadmin/"><span>Open control panel</span><span aria-hidden="true">↗</span></a> <a id="admin-link" class="admin-link" href="umbra-app.html"><span>Open control panel</span><span aria-hidden="true">↗</span></a>
<p class="auth-footnote">An account issued a temporary password must replace it at first sign-in before any workspace access is granted.</p> <p class="auth-footnote">An account issued a temporary password must replace it at first sign-in before any workspace access is granted.</p>
</div> </div>

80
web/umbra-app.html Normal file
View File

@@ -0,0 +1,80 @@
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta name="description" content="Umbra Tools workspace">
<title>Umbra Tools</title>
<link rel="icon" type="image/png" href="assets/umbratrans-emblem.png">
<link rel="stylesheet" href="assets/styles.css">
</head>
<body>
<!--
The Umbra Tools shell. Every signed-in session lands here.
Banner: brand and activity switch (left), the program selector (middle-left),
the selected language/project name (middle), the account and role (right).
Two-level navigation:
1. Activity: Control Panel, Program, Edit, Audio
2. Program: "Select Program" until one is chosen; then the dropdown lists
the program's languages as "IT - Deal Hunter", "SP - Deal Hunter", etc.
A Program is a book being translated and produced as audiobooks. Each
language under a program is a "project" with its own state: metadata/ISBN,
utterance review, audio generation, epub/pdf formatting, production download.
The control panel is mounted inline from `assets/admin-view.js`, which the
standalone `uadmin/` page also uses, so the two cannot drift.
-->
<div class="umbra-shell">
<header class="umbra-topbar">
<div class="umbra-topbar__left">
<a class="umbra-topbar__brandlink" href="umbra-app.html" aria-label="Umbra Tools home">
<img class="umbra-topbar__logo" src="assets/umbratrans-emblem.png" alt="" aria-hidden="true">
<span class="umbra-topbar__brand">Umbra Tools</span>
</a>
<select id="activity-select" class="umbra-topbar__mode" aria-label="Active activity">
<option value="control">Control Panel</option>
<option value="program">Program</option>
<option value="edit">Edit</option>
<option value="audio">Audio</option>
<option value="format">Format</option>
<option value="review">Review</option>
</select>
</div>
<div class="umbra-topbar__middle">
<select id="program-select" class="umbra-topbar__mode umbra-topbar__mode--hidden" aria-label="Program">
<option value="">Select Program</option>
</select>
<div id="topbar-language-name" class="umbra-topbar__name" aria-live="polite"></div>
</div>
<div class="umbra-topbar__right">
<div class="umbra-user">
<span class="avatar" id="topbar-avatar" aria-hidden="true">··</span>
<span class="umbra-user__meta">
<span class="umbra-user__name" id="topbar-username">Loading…</span>
<span class="umbra-user__role" id="topbar-role">Checking session</span>
</span>
</div>
<select id="account-menu" class="umbra-dropdown" aria-label="Account menu">
<option value="">Account ▾</option>
<option value="settings">Account settings</option>
<option value="signout">Sign out</option>
</select>
</div>
</header>
<aside class="umbra-sidebar" id="sidebar" aria-label="Activity navigation"></aside>
<main class="umbra-main" id="main">
<noscript>Umbra Tools needs JavaScript enabled.</noscript>
</main>
</div>
<div id="toast" class="toast" role="status" aria-live="polite"></div>
<script src="assets/api.js" defer></script>
<script src="assets/admin-view.js" defer></script>
<script src="assets/umbra-app.js" defer></script>
</body>
</html>

143
web/workspace-project.html Normal file
View File

@@ -0,0 +1,143 @@
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>Project · UmbraTrans</title>
<link rel="icon" type="image/png" href="assets/umbratrans-emblem.png">
<link rel="stylesheet" href="assets/styles.css">
</head>
<body class="workspace-page">
<div class="workspace-shell">
<aside class="workspace-rail" id="workspace-rail" aria-label="Workspace navigation">
<a class="brand-lockup brand-lockup--rail" href="workspace.html">
<img class="brand-image" src="assets/umbratrans-emblem.png" alt="" aria-hidden="true">
<span>UmbraTrans</span>
</a>
<div class="rail-section-label">Project</div>
<nav class="rail-nav">
<a class="rail-link rail-link--active" href="#overview"><span class="rail-icon" aria-hidden="true">⌂</span>Overview</a>
<a class="rail-link" href="#english"><span class="rail-icon" aria-hidden="true">文</span>English Source</a>
<a class="rail-link" href="#translations"><span class="rail-icon" aria-hidden="true">译</span>Translations</a>
<a class="rail-link" href="#outputs"><span class="rail-icon" aria-hidden="true">◇</span>Outputs</a>
<a class="rail-link" href="#audio"><span class="rail-icon" aria-hidden="true">▶</span>Audio</a>
</nav>
<div class="rail-footer">
<div class="rail-user">
<span class="avatar" id="rail-avatar" aria-hidden="true">··</span>
<span>
<strong id="rail-display-name">Loading…</strong>
<small id="rail-role">Checking session</small>
</span>
</div>
<a class="text-button" href="workspace.html">Back to workspace</a>
<button class="text-button" type="button" data-signout>Sign out</button>
</div>
</aside>
<main class="workspace-main" id="overview">
<header class="workspace-topbar">
<button class="text-button" type="button" data-nav-toggle aria-expanded="false" aria-controls="workspace-rail" aria-label="Open navigation" hidden>☰</button>
<div class="breadcrumb"><a href="workspace.html">Workspace</a><span aria-hidden="true">/</span><strong id="breadcrumb-identity">Loading…</strong></div>
<div class="topbar-actions">
<span class="status-pill status-pill--info" id="project-status-pill">—</span>
<a class="text-button" href="workspace.html">Workspace</a>
<button class="text-button" type="button" data-signout>Sign out</button>
</div>
</header>
<section class="workspace-hero">
<div>
<p class="eyebrow eyebrow--teal">Project workspace</p>
<h1 id="project-title">Loading…</h1>
<p class="hero-subtitle" id="project-subtitle">—</p>
</div>
<div class="hero-actions">
<button class="button button--secondary" type="button" id="btn-queue-translation">Queue Translation</button>
<button class="button button--primary" type="button" id="btn-generate-epub">Generate EPUB</button>
<button class="button button--primary" type="button" id="btn-generate-pdf">Generate PDF</button>
<button class="button button--primary" type="button" id="btn-generate-audio">Generate Audio</button>
</div>
</section>
<section class="pipeline-card" id="english">
<div class="section-heading">
<div>
<p class="eyebrow eyebrow--teal">Source</p>
<h2>English Source</h2>
</div>
</div>
<p class="list-summary">Original input book text (editable in line with Umbra Audio UI).</p>
<div class="table-wrap">
<table class="data-table">
<thead>
<tr><th>Chapter</th><th>Title</th><th>Words</th><th>Status</th><th></th></tr>
</thead>
<tbody id="english-tbody"></tbody>
</table>
</div>
</section>
<section class="pipeline-card" id="translations">
<div class="section-heading">
<div>
<p class="eyebrow eyebrow--teal">Translations</p>
<h2>Translated Versions</h2>
</div>
<div class="section-heading__actions">
<button class="chip" type="button" id="btn-add-lang">Add Language</button>
</div>
</div>
<p class="list-summary">View translations, tag for review/updates. English and Chinese shown when available.</p>
<div id="translations-list" class="project-list"></div>
<div class="empty-state" id="translations-empty">
<span class="empty-state__icon" aria-hidden="true">译</span>
<p class="empty-state__title">No translations yet</p>
<p class="empty-state__body">Queue a translation to start. Chinese translation may be in progress.</p>
</div>
</section>
<section class="pipeline-card" id="outputs">
<div class="section-heading">
<div>
<p class="eyebrow eyebrow--teal">Outputs</p>
<h2>Output Formats</h2>
</div>
</div>
<div class="metric-grid">
<article class="metric-card">
<span class="metric-label">EPUB</span>
<strong id="out-epub">—</strong>
<span class="metric-note">Status / availability</span>
</article>
<article class="metric-card">
<span class="metric-label">PDF</span>
<strong id="out-pdf">—</strong>
<span class="metric-note">Status / availability</span>
</article>
<article class="metric-card">
<span class="metric-label">Audio</span>
<strong id="out-audio">—</strong>
<span class="metric-note">Status / availability</span>
</article>
</div>
</section>
<section class="pipeline-card" id="audio">
<div class="section-heading">
<div>
<p class="eyebrow eyebrow--teal">Audio</p>
<h2>Audio Production</h2>
</div>
</div>
<p class="list-summary">Audio generation queued separately. Uses translated text when available.</p>
</section>
</main>
</div>
<div id="toast" class="toast" role="status" aria-live="polite"></div>
<script src="assets/nav.js" defer></script>
<script src="assets/api.js" defer></script>
<script src="assets/workspace-project.js" defer></script>
</body>
</html>