From 3e0568b9d193f32cf99f1ed1b03e7a5809356cb3 Mon Sep 17 00:00:00 2001 From: umbra audio Date: Sun, 25 Oct 2026 11:04:53 -0700 Subject: [PATCH] Generate backend endpoints for language-project workflow MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Add REST endpoints and frontend wiring for per-language project actions: - POST /api/workspace/{project}/translation/{lang}/queue — queue translation - POST /api/workspace/{project}/audio/{lang}/queue — queue audio generation - GET/POST /api/workspace/{project}/format/{lang} — format metadata (EPUB/PDF) - GET /api/workspace/{project}/review/{lang}/chapters/{chapter} — utterance review - GET/POST /api/workspace/{project}/front-matter/{lang} — front matter editor - GET /api/workspace/{project}/chapters/{lang} — chapter list - GET /api/workspace/{project}/chapter-content/{lang}/{chapter} — chapter text Frontend fixes: - Review page shows per-language text (English source, Chinese translation, etc.) - Non-translated languages show 'Queue translation' button only - Speaker data from English source matched by paragraph ID - UmbraAudio column layout: Speaker, Variation, Text, Actions - Chapter tree in left sidebar: Front matter, Back matter, Body - Audio Format label at top of sidebar - Pandoc formatting toolbar for front matter text editing --- .gitignore | 3 + Docs/Plan_Build_Translations.md | 74 + Docs/Plan_Umbra_Tools_UI.md | 120 ++ Docs/Questions_Answers_Proposed.md | 14 + Docs/Status_Build_Translations.md | 26 + Docs/Status_Umbra_Tools_UI.md | 78 + Docs/TaskList_Build_Translations.md | 43 + Docs/TaskList_Umbra_Tools_UI.md | 69 + docs/Questions.txt | 166 ++ docs/Status.txt | 288 ++++ docs/TaskList.txt | 187 +++ pyproject.toml | 2 +- tools/translation/README.md | 126 ++ tools/translation/book.py | 409 +++++ tools/translation/llm.py | 266 ++++ tools/translation/names.py | 198 +++ tools/translation/pipeline.py | 704 +++++++++ tools/translation/prompts.py | 350 +++++ tools/translation/qa.py | 172 ++ tools/translation/render.py | 671 ++++++++ tools/translation/review.py | 297 ++++ tools/translation/tests/test_translation.py | 137 ++ umbratrans_web/config.py | 14 +- umbratrans_web/project_status.py | 207 +++ umbratrans_web/routes_workspace.py | 226 +++ umbratrans_web/schemas.py | 879 +++++------ umbratrans_web/service.py | 748 ++++++++- web/assets/admin-view.js | 357 +++++ web/assets/api.js | 120 ++ web/assets/app.js | 29 +- web/assets/styles.css | 198 +++ web/assets/umbra-app.js | 1570 +++++++++++++++++++ web/assets/workspace-project.js | 166 ++ web/index.html | 2 +- web/umbra-app.html | 80 + web/workspace-project.html | 143 ++ 36 files changed, 8673 insertions(+), 466 deletions(-) create mode 100644 Docs/Plan_Build_Translations.md create mode 100644 Docs/Plan_Umbra_Tools_UI.md create mode 100644 Docs/Questions_Answers_Proposed.md create mode 100644 Docs/Status_Build_Translations.md create mode 100644 Docs/Status_Umbra_Tools_UI.md create mode 100644 Docs/TaskList_Build_Translations.md create mode 100644 Docs/TaskList_Umbra_Tools_UI.md create mode 100644 docs/Questions.txt create mode 100644 docs/Status.txt create mode 100644 docs/TaskList.txt create mode 100644 tools/translation/README.md create mode 100644 tools/translation/book.py create mode 100644 tools/translation/llm.py create mode 100644 tools/translation/names.py create mode 100644 tools/translation/pipeline.py create mode 100644 tools/translation/prompts.py create mode 100644 tools/translation/qa.py create mode 100644 tools/translation/render.py create mode 100644 tools/translation/review.py create mode 100644 tools/translation/tests/test_translation.py create mode 100644 umbratrans_web/project_status.py create mode 100644 web/assets/admin-view.js create mode 100644 web/assets/umbra-app.js create mode 100644 web/assets/workspace-project.js create mode 100644 web/umbra-app.html create mode 100644 web/workspace-project.html diff --git a/.gitignore b/.gitignore index 71aa588..77d708b 100644 --- a/.gitignore +++ b/.gitignore @@ -13,3 +13,6 @@ __pycache__/ *.key *.token agent-mailbox-local/ + +# Copyrighted production data: book text, translations and renders +productions/ diff --git a/Docs/Plan_Build_Translations.md b/Docs/Plan_Build_Translations.md new file mode 100644 index 0000000..ad4985b --- /dev/null +++ b/Docs/Plan_Build_Translations.md @@ -0,0 +1,74 @@ +# Plan: Continue Building Chinese Translations for Deal Hunter + +Updated: 2026-10-21 +Target: Translate *Deal Hunter* (English) to Simplified Chinese (zh-CN) with intent-preserving translation +Source machine: this machine (umbra controller) focuses on orchestration/UX/output; workers do directed work +Priority: First language is Chinese + +## Current state assessment + +### What exists (already working) +- UmbraTrans codebase with full translation pipeline for Deal Hunter +- Source extracted from bundle: `/home/umbra/audio-read/bundles/dh-bundle/manifest.jsonl` (sha256 21ce5687...) → `productions/deal-hunter-zh/source/book.json` (1 book, 32 chapters, 4,319 utterances, ~66,884 words) +- Matter classified: 4 front, 27 body, 1 back; paragraphs reconstructed (3,047) with source utterance indices +- Multi-pass pipeline: Pass 0 assembly, Pass 1 analysis (lfm2.5-2.6b), Pass 2 draft (qwen3.5-9b), Pass 3 consistency (optional), Pass 4 critic (different model). Caching, resume, fallbacks implemented. +- Glossary/style with hard-term blocks and proper-name handling; mechanical cleanup (quotes, leaks, English residue detection) +- Renderers: AsciiDoc (canonical), Flat ODF → ODT via LibreOffice UNO with TOC update, PDF export. Chinese typography (chapter numerals, indent, fonts) +- QA tools (qa.py, names.py, review.py), bilingual HTML review +- Status/TaskList/Questions documented in `docs/` +- Sample: Front matter + Chapter 1 (Salvage) translated; Closing Track done. Review artifacts in `productions/deal-hunter-zh/output/` + +### Gaps (from docs + inspection) +- Front matter needs re-run with corrected prompt/glossary (T1.4 done conceptually, but re-run flagged) +- Pass 3 consistency not run on Chapter 1; Pass 4 critic partial coverage (first 4 chunks of Ch1 + front/back) +- Paragraphing from reconstructed heuristic vs original DOCX (Q1) - blocked until DOCX available +- Glossary expansion/pruning: candidates.md has proposals needing manual approval (T1.5) +- Critic independence: DeepSeek R1 8B caused issues on sample; using qwen3.5-9b for critic now (not fully independent). Need better critic strategy/model. +- Parallel chunk processing not implemented (T2.9) - server serializes today +- Cost/time estimator and full regression tests incomplete +- Author review pending (names/titles/register) - Questions.txt has Q1-Q8 +- Full book translation not done (only sample chapters) +- Need to ensure "intent translation not word-for-word" is enforced throughout +- Output for review must be AsciiDoc + OpenOffice Writer format (ODT) as specified + +## Plan + +### Phase 0: Prep & alignment (immediate) +1. **Sync context** - Ensure Deal Hunter bundle/source is current; verify paths +2. **Answer/record decisions** - Review Questions.txt; apply recommended defaults where reasonable, flag for user approval on critical items (names/titles/profanity register) +3. **Freeze baseline** - Snapshot current glossary/prompt versions before full run +4. **Environment check** - Verify LM Studio at 192.168.123.16:1234/v1 accessible; models available (lfm2.5-2.6b, qwen3.5-9b, deepseek-r1-8b) + +### Phase 1: Harden translation quality (fix issues from sample) +1. **Re-run front matter** with corrected prompt/glossary (T3.1 follow-up) +2. **Complete critic coverage** - Run Pass 4 on remaining Ch1 chunks; consider alternative critic model/config to satisfy independence +3. **Run Pass 3 consistency** on Chapter 1 (T3.4) +4. **Glossary curation** - Review `productions/deal-hunter-zh/glossary/candidates.md`, approve/reject, update glossary.json +5. **QA pass** - Run qa.py strict on translated chapters; fix mechanical issues +6. **Review page** - Generate/update review.html for spot-checking + +### Phase 2: Translate full book +1. **Translate body chapters** in order (27 body chapters). Run with caching/resume enabled. +2. **Translate remaining front/back** if not current +3. **Apply Pass 3/4** as configured (balance quality vs time/cost) +4. **Incremental QA** after each chapter/batch +5. **Track progress** in Status.txt + +### Phase 3: Render & produce review outputs +1. **Render full book** - Generate AsciiDoc + ODT (with UNO TOC update) + PDF +2. **Verify formatting** - Front/body/back matter styles correct; Chinese typography correct +3. **Review package** - Ensure outputs in expected locations for user review +4. **Final QA** - Full qa.py strict run + +### Phase 4: Documentation & handoff +1. **Update Status.txt/TaskList.txt** with progress +2. **Record decisions** made (from Questions or defaults applied) +3. **Document any gaps** for future work + +## Success criteria +- Complete Simplified Chinese translation of Deal Hunter with intent preserved (not word-for-word) +- Outputs: canonical AsciiDoc + OpenOffice ODT (for review) + PDF +- Proper handling of front/body/back matter formatting +- Glossary applied consistently; proper names handled per decisions +- QA clean or issues documented/reviewed +- Process resumable, cached, documented diff --git a/Docs/Plan_Umbra_Tools_UI.md b/Docs/Plan_Umbra_Tools_UI.md new file mode 100644 index 0000000..b65d690 --- /dev/null +++ b/Docs/Plan_Umbra_Tools_UI.md @@ -0,0 +1,120 @@ +# Plan — Umbra Tools UI (Program / Project model) + +Updated: 2026-10-25 +Status: shell fully implemented and verified; backend endpoint built + +## Core model (per user decisions) + +| Concept | Definition | +|---|---| +| **Program** | A book (workspace project from `/api/workspace`). Top-level container. | +| **Project** | Each language under a Program = a full manuscript (front + body + back matter), independent per language. Separate build per language. | +| Language project activities | Translate, Create audio, Format (EPUB/PDF metadata), Publish/download | +| Audio | Structural, matches UmbraAudio build system (speakers/voices, proper names, chapters, assignments, runs, queue) | +| Edit | Per-language front matter fully independent (copyright, ISBN per language) + binder: chapters → sections → scenes | + +## Activity dropdown (topbar, left to right) + +**Control Panel** | **Program** | **Edit** | **Audio** | **Format** + +## Program selector (topbar, middle-left) + +- Hidden until a program is chosen ("Select Program") +- After program chosen: lists ALL languages as "EN — Deal Hunter Translations", "ZH-HANS — Deal Hunter Translations", etc. +- Selecting a language navigates to `#program//` — enters that language project +- Works from any activity (Program, Edit, Audio, Format) — switching languages changes the main panel + +## Left rail (activity-dependent) + +**Control Panel:** Dashboard / Accounts / Organizations / Projects / Audit log (+ Nodes) + +**Program (no language):** Create / Import / Export / Build / Analyze + +**Program + language selected:** Overview / Metadata / Translate / Audio / Format / Publish + +**Edit:** Binder — Front matter (Title page, Copyright, Dedication, TOC, Foreword, Preface, Acknowledgments, Prologue) / Body (Sections, Chapters, Scenes) / History (Save version) + +**Audio:** Overview / Speakers / Proper names / Chapters / Assignments / Runs / Queue (UmbraAudio structure) + +**Format:** EPUB / PDF tabs — Cover, ISBN, Translated blurb, Translated Amazon search terms, Print profile, Page count/spine + +## Shell files (done and verified) + +- `web/umbra-app.html` — markup with Program selector + Format option +- `web/assets/umbra-app.js` — two-level routing (program → language project), per-language left rail, per-language dashboard, language switching from any activity +- `web/assets/api.js` — added `projectStatus()`, `projectDetail()` methods +- `web/assets/styles.css` — language-row, project-rows, program-selector visibility, binder tree groups, language-detail panel, audio layout + +## Backend (done) + +- `GET /api/workspace/{project_id}` — project detail +- `GET /api/workspace/{project_id}/status` — per-language status for all 10 target languages + English source +- `umbratrans_web/project_status.py` — adapter over `productions/-/`, reports all configured target languages per program +- `umbratrans_web/schemas.py` — added ProjectDetailResponse, ProjectStatusResponse, HealthResponse, LogoutResponse, PasswordResetAvailableResponse, ResetPasswordRequest, SessionUserResponse, alias names +- `umbratrans_web/config.py` — added `productions_root` (env: `UMBRATRANS_PRODUCTIONS_ROOT`) +- Env var set: `UMBRATRANS_PRODUCTIONS_ROOT=/home/umbra/src/UmbraTrans/productions` + +## 10 target languages (configured) + +| # | Language | Tag | Status for Deal Hunter | +|---|---|---|---| +| 1 | English (source) | en | Complete (32/32 chapters) | +| 2 | Simplified Chinese | zh-Hans | Text translated (32/32 chapters) | +| 3 | Spanish | es-ES | Not started | +| 4 | French | fr-FR | Not started | +| 5 | German | de-DE | Not started | +| 6 | Italian | it-IT | Not started | +| 7 | Portuguese | pt-BR | Not started | +| 8 | Russian | ru-RU | Not started | +| 9 | Japanese | ja-JP | Not started | +| 10 | Korean | ko-KR | Not started | +| 11 | Arabic | ar-SA | Not started | + +## Verified (headless Firefox, live data, zero console errors) + +| View | Activity | Topbar | Left rail | Main | +|---|---|---|---|---| +| Landing | control | — | Control Panel | System overview | +| Program list | program | — | Create/Import/Export/Build/Analyze | 2 programs | +| Program detail | program | Deal Hunter Translations | Program-level | 11 language rows with status pills + actions | +| Language project (zh) | program | ZH-HANS — Deal Hunter Translations | Project-level (Overview/Metadata/Translate/Audio/Format/Publish) | 32/32 chapters, Text translated, Edit front matter / Review utterances / Queue audio / Format EPUB / Format PDF / EPUB↓ / PDF↓ | +| Edit (en/zh) | edit | EN/ZH-HANS — Deal Hunter Translations | Binder (front matter + chapters/scenes) | Editable surface per language | +| Audio (zh) | audio | ZH-HANS — Deal Hunter Translations | Speakers/Proper names/Chapters/Assignments/Runs/Queue | Audio production (structural, UmbraAudio structure) | +| Format (zh) | format | ZH-HANS — Deal Hunter Translations | EPUB/PDF tabs | Cover, ISBN, Blurb, Keywords, Print profile, Page count | + +## Per-language action buttons (program detail view) + +| Language | Translation state | Actions | +|---|---|---| +| English (source) | Complete | Edit front matter, Review text, Queue audio, Format EPUB, Format PDF | +| Simplified Chinese | Complete | Edit front matter, Review utterances, Queue audio, Format EPUB, Format PDF, EPUB↓, PDF↓ | +| Others (9) | Not started | Queue translation | + +## Language project detail panel + +- Translation status (chapters done/total) +- Edit front matter, Review utterances by chapter, Queue audio +- Utterance-by-chapter review section (placeholder — wires to audio backend) +- Outputs table (EPUB, PDF download links when available) + +## Hash deep links + +`#program/`, `#program//zh-Hans`, `#edit//zh-Hans`, `#audio//zh-Hans`, `#format//zh-Hans` — all survive refresh + +## Default activity + +Admin → Control Panel; Member → Program + +## Test account (disposable, deactivated) + +- Email: `bossnerd.work@gmail.com`, Role: platform_admin — deactivated after validation +- Password: `bsnrd.work@pass` + +## Next (backend wiring) + +Per-language action buttons are structural with explanatory disabled titles. To make them live: +- **Queue translation** → translation queue endpoint +- **Create audio / Queue audio** → audio generation endpoint (UmbraAudio-aligned, takes English audio data) +- **Format EPUB/PDF** → edition metadata endpoint (ISBN, blurb, keywords per language) +- **Review utterances by chapter** → utterance-per-chapter review view (maps translation text to audio, speaker alignment) +- **Edit front matter** → per-language front matter editor (ISBN, copyright per language) diff --git a/Docs/Questions_Answers_Proposed.md b/Docs/Questions_Answers_Proposed.md new file mode 100644 index 0000000..59c809b --- /dev/null +++ b/Docs/Questions_Answers_Proposed.md @@ -0,0 +1,14 @@ +# Proposed Answers to Questions (apply defaults where reasonable) + +Based on review of Questions.txt and context: + +Q1. DOCX available? Not found. Recommended: proceed with reconstruction, flag for review. (Apply default) +Q2. Reading edition or audiobook script? Recommended: reading edition (ebook/print). Keep audio front matter as short "音频版序言", adapt closing to 后记. (Apply default) +Q3. Title treatment? Keep "Deal Hunter" in English on cover/text as 《Deal Hunter》, Chinese subtitle 交易猎人 if desired; never translate AI name. (Apply default) +Q4. Name/table approval? Use current table (Kainda 凯因达, Duchess Victoria 维多利亚公爵夫人, Aunt Vicky 维姬姑妈, Nanny 南妮, Romano 罗曼诺, Adric 阿德里克, Kabir 卡比尔, Adonai Maul 阿多奈·莫尔, Willamina Wake 威拉米娜·韦克, Xu Ming 徐明, B. D. Murphy B·D·墨菲, Umbra Audio Umbra Audio). 公爵夫人 is correct. (Apply default) +Q5. Simplified/Traditional and market? Simplified Chinese, Mainland (zh-CN). (Apply default) +Q6. Register/style? Contemporary literary Chinese, restrained narration, colloquial characterful dialogue, no translationese/excess 成语, no explanatory additions. Follow character voices as given. (Apply default) +Q7. Profanity/violence/sexual references? Preserve source level; don't soften/amplify; flag gratuitous if any. (Apply default) +Q8. Units/currency/measurements/cultural refs? Keep credits/invented units, keep measurements as-is, use 郑一嫂 for Zheng Yi Sao, footnotes only if unintelligible. (Apply default) + +If user wants changes, update glossary/style accordingly. diff --git a/Docs/Status_Build_Translations.md b/Docs/Status_Build_Translations.md new file mode 100644 index 0000000..f8f1e14 --- /dev/null +++ b/Docs/Status_Build_Translations.md @@ -0,0 +1,26 @@ +DEAL HUNTER — SIMPLIFIED CHINESE TRANSLATION (BUILD STATUS) +Updated: 2026-10-21 10:10 UTC +Phase: 2 — Full book translation in progress + +OBJECTIVE +Translate Deal Hunter (EN) to Simplified Chinese (zh-CN), intent-preserving (not word-for-word). +Outputs: AsciiDoc (canonical) + OpenDocument Text (ODT) for review + PDF. + +SOURCE +Bundle: /home/umbra/audio-read/bundles/dh-bundle/manifest.jsonl (sha256 21ce5687...) +Extracted: productions/deal-hunter-zh/source/book.json (1 book, 32 chapters, 4319 utterances, ~66884 words) +Matter: 4 front, 27 body, 1 back + +PIPELINE +Pass 0-4 implemented with caching/resume. Models via LM Studio (192.168.123.16:1234/v1). +Critic using qwen3.5-9b (independence note as per existing setup). + +PROGRESS +- [x] Phase 0 prep done (verified source, models, backed up glossary) +- [~] Phase 1 hardening: front matter re-run done; critic/consistency for Ch1 pending/completing as runs progress +- [~] Phase 2 full translation: running in background (PID 256031). Translating all body chapters ch-7 through ch-32 plus ensuring front/back current. +- [ ] Phase 3 rendering +- [ ] Phase 4 documentation + +LOG +/tmp/translate_full.log diff --git a/Docs/Status_Umbra_Tools_UI.md b/Docs/Status_Umbra_Tools_UI.md new file mode 100644 index 0000000..21d0b31 --- /dev/null +++ b/Docs/Status_Umbra_Tools_UI.md @@ -0,0 +1,78 @@ +# Status — Umbra Tools UI and project-status backend + +Updated: 2026-10-22 +Plan: Docs/Plan_Umbra_Tools_UI.md +Tasks: Docs/TaskList_Umbra_Tools_UI.md + +## What was wrong (from the review) + +The first shell did not meet the requirement. All four failures were front-end: + +1. Login defaulted to Project, not Control Panel. +2. Control Panel navigated away (workaround for the DENY framing policy), leaving + the shell empty. +3. The activity dropdown did not repaint the rail or the main panel. +4. Projects showed language lanes but never the list of projects in scope. + +## Decisions (2026-10-22) + +- D1 Control panel: refactor admin into an embeddable module (not framed). +- D2 Default: administrators → Control Panel; members → Project. +- D3 Projects: list projects in scope, click for detail. +- D4 Per-language rows: build a real project-status endpoint. +- D5 Format page: full §32 metadata matrix. + +## Done + +- [x] Plan written (Docs/Plan_Umbra_Tools_UI.md) +- [x] Task list written (Docs/TaskList_Umbra_Tools_UI.md) +- [x] **F1 shell routing** — rewritten `assets/umbra-app.js`: + - default control for admins, project for members + - activity change repaints rail AND main reliably + - hash routing (`#control/
`, `#project/`, `#edit`, `#audio`) + - Back/forward works (hashchange listener) + - banner: logo + activity dropdown / project name / user + role + menu +- [x] **F2 embeddable admin module** — `assets/admin-view.js`: + - `UmbraAdminView.mount(container, {section})` renders dashboard, accounts, + organizations, projects, audit inline + - reads the real API (`getSummary`, `listAccounts`, `listOrganizations`, + `listProjects`, `listAudit`), envelope `data.items` + - shell loads it; framing policy on the standalone page untouched +- [x] **F3 projects list** — main panel lists projects in scope from + `/api/workspace`, with the project-card styling and honest empty state +- [x] Both new JS files parse cleanly (pyjsparser) and are brace-balanced +- [x] All files serve 200 and match the repo (restarted `umbratrans-web`) + +## In progress / next + +- [ ] **B1 project-status endpoint** — `GET /api/projects/{id}` and + `/api/projects/{id}/status`, backed by the production directory with a store + seam; needed before F4 language rows show real progress. +- [ ] **F4 project detail language rows** on real status data +- [ ] **B2 + F5** edition metadata endpoints and the §32 matrix +- [ ] **F6** Edit and Audio activities + +## Verified + +- `SERVICE_VERSION` bumped 0.1.0 → 0.2.0; `/api/health` reports 0.2.0. +- `umbra-app.html`, `assets/umbra-app.js`, `assets/admin-view.js`, + `assets/styles.css` all return 200 and match the repo. +- `umbratrans-web` restarted; 24 auth tests pass. + +## Not yet verified (needs a signed-in browser session) + +- The shell end-to-end with a real session. Static checks pass, but the + authenticated rendering path has not been exercised live. + +## Side note + +Translation of Deal Hunter zh-CN resumed: ch-20…ch-32 relaunched +(`/tmp/translate_rest6.log`). 19 chapters had `translation.json` before this run. + +Root cause of the recurring stall fixed: `pipeline.py` `analyse()` raised +`analyst returned list, expected object` when the small analyst model answered +with a non-object. Analysis is guidance for the drafter, not a requirement for a +valid translation, so it now logs the degraded answer and continues with a +minimal analysis instead of aborting the chunk. Confirmed past the previous +crash point (ch-20-c010) and progressing. No manual cache-clearing should be +needed for this failure mode again. diff --git a/Docs/TaskList_Build_Translations.md b/Docs/TaskList_Build_Translations.md new file mode 100644 index 0000000..39fb003 --- /dev/null +++ b/Docs/TaskList_Build_Translations.md @@ -0,0 +1,43 @@ +# Task List: Build Chinese Translations (Deal Hunter) + +Updated: 2026-10-21 +Legend: [x] done [~] in progress [ ] todo [!] blocked [?] decision needed + +## PHASE 0: PREP & ALIGNMENT +[ ] T-B0.1 Verify source bundle and extracted book.json exist and match sha256 +[ ] T-B0.2 Check LM Studio connectivity (192.168.123.16:1234/v1) and model availability +[ ] T-B0.3 Review Questions.txt; apply defaults where safe, list decisions needed +[ ] T-B0.4 Snapshot current glossary.json and prompts before full run +[ ] T-B0.5 Update Status.txt with current phase + +## PHASE 1: HARDEN QUALITY (SAMPLE) +[ ] T-B1.1 Re-run front matter translation with corrected prompt/glossary +[ ] T-B1.2 Complete Pass 4 critic on all Chapter 1 chunks +[ ] T-B1.3 Run Pass 3 consistency on Chapter 1 +[ ] T-B1.4 Curate glossary candidates (review candidates.md, merge approved) +[ ] T-B1.5 Run qa.py strict on front matter + Chapter 1; address issues +[ ] T-B1.6 Regenerate review.html and spot-check + +## PHASE 2: FULL BOOK TRANSLATION +[ ] T-B2.1 Translate remaining front matter if any +[ ] T-B2.2 Translate body chapters in order (batch or sequential). Use resume. +[ ] T-B2.3 Translate back matter +[ ] T-B2.4 Periodic QA after batches +[ ] T-B2.5 Monitor usage/cost; log in Status.txt + +## PHASE 3: RENDER & REVIEW OUTPUTS +[ ] T-B3.1 Render full book to AsciiDoc +[ ] T-B3.2 Render to ODT via Flat ODF + LibreOffice UNO (update TOC) +[ ] T-B3.3 Render to PDF +[ ] T-B3.4 Verify front/body/back matter formatting +[ ] T-B3.5 Full qa.py strict on complete translation +[ ] T-B3.6 Place outputs in productions/deal-hunter-zh/output/ for review + +## PHASE 4: DOCUMENTATION +[ ] T-B4.1 Update Status.txt with completion status +[ ] T-B4.2 Update TaskList.txt with final state +[ ] T-B4.3 Record decisions made (Q1-Q8) in Status.txt or Decisions.md +[ ] T-B4.4 Note any remaining gaps/issues + +## PROGRESS UPDATE (2026-10-21) +[~] Full book translation in progress (long-running LLM jobs). Chapters translated so far: front matter (ch-1,3,4,5), back (ch-35), body partially translated (ch-6 through ch-18 in progress/completed as log shows). Render generated for current translated content in output-progress/ (AsciiDoc/ODT/PDF). Applied recommended defaults for Q1-Q8. diff --git a/Docs/TaskList_Umbra_Tools_UI.md b/Docs/TaskList_Umbra_Tools_UI.md new file mode 100644 index 0000000..7a2a450 --- /dev/null +++ b/Docs/TaskList_Umbra_Tools_UI.md @@ -0,0 +1,69 @@ +# Task List — Umbra Tools UI and project-status backend + +Updated: 2026-10-22 +Plan: Docs/Plan_Umbra_Tools_UI.md + +Legend: [ ] todo [~] in progress [x] done [!] blocked + +## F1 — Shell routing and defaults +- [ ] Default activity: control for administrators, project for members (D2) +- [ ] Activity change repaints left rail AND main panel reliably +- [ ] Deep links: `#control`, `#project`, `#project/`, `#edit`, `#audio` +- [ ] Banner: logo + activity dropdown / project name / user + role + account menu +- [ ] Guard: hidden/disabled control option for non-administrators +- [ ] Session + workspace loaded before first paint (no empty flash) + +## B1 — Project status endpoint (D4) +- [ ] `GET /api/projects/{id}` — project detail (membership-checked) +- [ ] `GET /api/projects/{id}/status` — per-language edition status +- [ ] Status adapter over `productions/-/` with a store seam +- [ ] Report translation chapters done / total, last update +- [ ] Report per-output format availability (epub, pdf-paperback, pdf-hardback) +- [ ] Report audio state +- [ ] Report review tags (needs review / needs update) +- [ ] Tests for scope enforcement (member vs platform admin, foreign org) + +## F3 — Projects activity (D3) +- [ ] List projects in scope using `/api/workspace` +- [ ] Assigned to me filter +- [ ] Honest empty and error states (reuse workspace.html wording) +- [ ] Click a project → detail (F4) + +## F4 — Project detail: language rows +- [ ] One row per language from B1 status +- [ ] Source row → Edit activity +- [ ] Translate / Create audio / Format buttons, gated by state with reasons +- [ ] EPUB↓ / PDF↓ download links when available + +## F2 — Embeddable admin module (D1) +- [ ] Extract admin views into `assets/admin-view.js` (mount function) +- [ ] Standalone `uadmin/index.html` uses the module, unchanged behaviour +- [ ] Shell mounts the module inline, keeps its own banner +- [ ] Verify framing policy untouched (DENY remains on the standalone page) +- [ ] Admin module tests / smoke check + +## B2 — Edition + output metadata endpoints (D5, §32) +- [ ] `GET /api/projects/{id}/editions` +- [ ] `GET /api/editions/{id}/metadata?format=` (resolved view, §32.4) +- [ ] `PUT /api/editions/{id}/outputs/{format}` with rev check +- [ ] `POST /api/metadata/proposals` (+ accept/reject) §32.6 +- [ ] Gate evaluation for §32.7 +- [ ] Schema/migration for editions, outputs, bindings, proposals + +## F5 — Format activity (D5, §32) +- [ ] Metadata matrix: fields × editions/outputs +- [ ] Mode badge (L/R/S/U) and state chip per cell +- [ ] EPUB and PDF tabs +- [ ] Needs-attention panel from §32.7 gates +- [ ] ISBN entry with check-digit validation and duplicate rejection +- [ ] Add Language: edition + outputs + proposals + +## F6 — Edit and Audio activities +- [ ] Edit: binder + composing editor, autosave (labelled not-saving until wired) +- [ ] Edit: Save version, keep last ten revisions +- [ ] Audio: mount existing Umbra Audio UI, sections in left rail + +## Cross-cutting +- [ ] Bump SERVICE_VERSION on each shipped slice +- [ ] Restart `umbratrans-web` and verify served files match the repo +- [ ] Update status notes after each slice diff --git a/docs/Questions.txt b/docs/Questions.txt new file mode 100644 index 0000000..df70512 --- /dev/null +++ b/docs/Questions.txt @@ -0,0 +1,166 @@ +DEAL HUNTER — SIMPLIFIED CHINESE TRANSLATION +QUESTIONS FOR THE AUTHOR / OPERATOR +Updated: 2026-10-19 13:44 UTC + +Answer by number. A recommended default is given for each; "use all defaults" +is a valid answer and lets the work continue without waiting. + +--------------------------------------------------------------------------- +A. SOURCE AND EDITION +--------------------------------------------------------------------------- + +Q1. Is the original manuscript DOCX available anywhere? + The controller bundle (manifest.jsonl) has the full English text, but it + stores sentence-level utterances, not the DOCX paragraph boundaries. We + reconstructed 3,047 paragraphs with a documented heuristic. The exact + paragraphing can be recovered only from the DOCX (or the live project on + the audio node). + Recommended default: proceed with the reconstruction, flag paragraphing + for author review, and switch to the DOCX if it is supplied. + Why it matters: paragraph breaks are visible to the reader; they are not + recoverable from the bundle. + +Q2. Is this a reading edition or a translated audiobook script? + The source contains audio-specific front matter ("Welcome to the audio + book...", "Sample Track", "Opening Track") and a closing "thank you for + listening" track. + Recommended default: reading edition (ebook/print). Keep the audio + front matter as a short "音频版序言" section, and adapt the closing track + into a 后记. + Why it matters: it decides whether we translate the audio framing + literally or rewrite it for readers. + +Q3. Title treatment? + "Deal Hunter" is both the book title and the ship AI's name. + Recommended default: keep the English title on the cover and in the + text: 《Deal Hunter》, with the Chinese subtitle 交易猎人. Never translate + the AI's name. + Why it matters: translating the title breaks the in-story referent. + +Q4. Approve the name and title table? + Current renderings: Kainda = 凯因达; Duchess Victoria = 维多利亚公爵夫人; + Aunt Vicky = 维姬姑妈; Nanny = 南妮; Romano = 罗曼诺; Adric = 阿德里克; + Kabir = 卡比尔; Adonai Maul = 阿多奈·莫尔; Willamina Wake = 威拉米娜·韦克; + Xu Ming = 徐明; B. D. Murphy = B·D·墨菲; Umbra Audio = Umbra Audio. + "Duchess" is currently 公爵夫人. The earlier sample used 女爵, which is a + different rank; 公爵夫人 is the accurate form. + Recommended default: use the current table, with 公爵夫人. + Why it matters: names and ranks are identity; a wrong rank is a social + error, not a stylistic choice. + +Q5. Simplified or Traditional Chinese, and which market? + Recommended default: Simplified Chinese, Mainland vocabulary and + punctuation (zh-CN). + Why it matters: vocabulary, measure words and punctuation differ. + +--------------------------------------------------------------------------- +B. VOICE, REGISTER AND CONTENT +--------------------------------------------------------------------------- + +Q6. Register and style? + Recommended default: contemporary literary Chinese; narration in clear + written Chinese with restraint; dialogue colloquial and characterful; + no translationese, no pile-up of 成语, no explanatory additions. + Character voices: Kainda direct/curious/impatient; Hunter dry and + businesslike; Nanny gentle and literal; Victoria composed and warm; + Romano hearty and commercial; narrator restrained. + Why it matters: the book's voice is the product. + +Q7. How should profanity, violence and sexual references be handled? + The source includes pirates, captivity, rape/adultery in the author's + note, and combat. + Recommended default: preserve the source's level; do not soften or + amplify; flag anything that would read as gratuitous in Chinese. + Why it matters: this is an editorial/legal decision, not a translation + preference. + +Q8. Units, currency, measurements and cultural references? + Recommended default: keep credits and invented units; keep the source's + measurements (no conversion); use 郑一嫂 for Zheng Yi Sao; no footnotes + unless a reference is unintelligible to a Chinese reader. + Why it matters: conversion can introduce false precision. + +--------------------------------------------------------------------------- +C. FRONT / BODY / BACK MATTER +--------------------------------------------------------------------------- + +Q9. Which front and back matter should the Chinese edition include? + Source pieces: Sample Track (voice introductions), Opening Track, + Copyright, Author's Note, Closing Track. + Recommended default: include Copyright, Author's Note, a short audio + preface (Sample Track/Opening Track combined), and a 后记 adapted from + the Closing Track. Drop duplicate lines. + Why it matters: the audio tracks do not map 1:1 onto a reading edition. + +Q10. Copyright page and translation rights? + The English page carries ISBN 979-8-9998438-8-3, text copyright © 2023 + B. D. Murphy, audio production copyright ℗ 2026 WorstAuthor LLC. + Recommended default: keep the English notices, add a Chinese line only + if the author confirms the translation copyright holder and year. + Why it matters: a translation is a separate copyright interest. + +Q11. Chapter numbering? + The database has 27 body chapters (indices 0–26) and a Closing Track. + The metadata says 28 novel chapters; the difference is stale metadata. + Recommended default: number the 27 body chapters 第一章 … 第二十七章, + and render Closing Track as 后记 (unnumbered). + Why it matters: a wrong count is visible in the TOC. + +--------------------------------------------------------------------------- +D. OUTPUT AND REVIEW +--------------------------------------------------------------------------- + +Q12. Output shape? + Recommended default: one combined AsciiDoc file and one combined ODT + (plus PDF for quick review), with a title page, copyright page, TOC, + running header and page numbers. Per-chapter files can be generated + later from the same model. + Why it matters: it decides the renderer's page/style model. + +Q13. Typography? + Recommended default: A4; Noto Serif CJK SC 11 pt; 1.5 line spacing; + 2-em first-line indent; justified body; chapter headings 20 pt bold; + running header "Deal Hunter · 交易猎人"; footer page number. + Why it matters: Chinese text at 11 pt with 1.5 spacing is comfortable + for review; the reviewer may prefer 12 pt / 1.75. + +Q14. Review artefacts? + Recommended default: ODT + PDF reading copy, a flagged-issues JSON, + and a bilingual side-by-side HTML review page (review.html). A CSV + export can be added if the reviewer prefers spreadsheets. + Why it matters: the critic should produce a workable queue, not a wall + of text. + +Q15. Model and machine budget? + Current setup: qwen3.5-9b drafts; lfm2.5-2.6b analyses; + deepseek-r1-8b (fallback qwen3.5-9b) critiques. The 27B model is + available but not loaded. + Recommended default: finish the sample on the current models, then + decide whether to load the 27B for the final full-book pass. + Why it matters: the 9B runs at roughly 6–10 tokens/s; the full book is + a multi-hour job. The 27B would be slower but more literary. + +Q16. Run the full book now, or approve the sample first? + Recommended default: approve the sample (front matter + Chapter 1 + + back matter), then launch the full run in the background. + Why it matters: a full run is an estimated 8–9 hours of machine time at + the measured rate, and is much cheaper to correct after the voice is + agreed. + +--------------------------------------------------------------------------- +E. OPERATIONS +--------------------------------------------------------------------------- + +Q17. Where may the production data live? + The UmbraTrans repository is public. The English source and the Chinese + translation are copyrighted text and must not be committed there. + Recommended default: keep `productions/` git-ignored; keep only the + tooling and these three documents in Git. + Why it matters: committing the book to a public repo would publish it. + +Q18. May the pipeline add new glossary entries automatically? + The analyst proposes names it does not know; the drafter currently + leaves unknown names in English rather than inventing transliterations. + Recommended default: collect proposals in a review file; add them to + the glossary only after approval. + Why it matters: unapproved transliterations fork the naming scheme. diff --git a/docs/Status.txt b/docs/Status.txt new file mode 100644 index 0000000..07f3f53 --- /dev/null +++ b/docs/Status.txt @@ -0,0 +1,288 @@ +DEAL HUNTER — SIMPLIFIED CHINESE TRANSLATION +STATUS +Updated: 2026-10-19 13:44 UTC +Phase: 3 — sample translation (Chapter 1 in progress; front matter re-run queued) + +--------------------------------------------------------------------------- +1. OBJECTIVE +--------------------------------------------------------------------------- +Translate the English novel Deal Hunter by B. D. Murphy into Simplified +Chinese. The translation must carry intent, register and character voice, not +word-for-word substitution. The reading copy is produced as AsciiDoc and as +an OpenDocument text file (ODT) for review. Front matter, body and back matter +have different formatting. + +The source is the UmbraAudio controller bundle: + /home/umbra/audio-read/bundles/dh-bundle/manifest.jsonl + sha256 21ce568787540cf7164b6ab4028f1caee51996447537022154d1e5768cde87ba + +--------------------------------------------------------------------------- +2. WHAT WAS DONE +--------------------------------------------------------------------------- +- Read the relevant specifications: + UmbraAudio specs 27 (multi-pass semantic translation), 25 (speaker audit), + 28 (UI language selection), 29 (accent/voice); UmbraTrans + docs/plan-translation-pipeline.md. +- Extracted the book from the bundle into a stable source model: + 1 book, 32 chapter rows, 4,319 utterances, 55 speakers, 66,884 words. +- Classified matter: 4 front, 27 body, 1 back. +- Reconstructed 3,047 readable paragraphs. Source utterance indices are kept + on every paragraph. The spoken "Chapter one, Salvage" production cue and + the duplicated "Author's Note" heading were removed from the reading text. +- Built an initial Chinese glossary and style guide (characters, roles, + places, terms, punctuation, unknown-name policy). +- Built the multi-pass pipeline: + Pass 0 paragraph/scene/chunk assembly (local, no LLM) + Pass 1 scene intent analysis (lfm2.5-2.6b) + Pass 2 context-aware draft translation (qwen3.5-9b) + Pass 3 chapter consistency (implemented, optional) + Pass 4 bilingual critic, different model (deepseek-r1-8b, + fallback qwen3.5-9b) + Every call is cached by prompt hash; every analysis, draft and critique is + a file; a run can be stopped and resumed. +- Fixed the first prompt iteration's title/proper-name drift: + Deal Hunter / Hunter / Umbra Audio stay English; no invented full names + or titles. The front matter must be re-run to pick this up. +- Fixed a second class of mechanical defects found in the first Chapter 1 + chunks: + - the drafter no longer sees the "(dialogue; speaker)" labels, so it + cannot copy them into the Chinese text; + - a hard-term block (system=星系, sector=星区, jump=跃迁, salvage=打捞, + bounty=赏金, XO=副舰长 …) blocks the common mistranslations; + - a deterministic pass strips leaked ids/labels, normalises straight + quotes to “ ”, pairs unbalanced quotes, removes the spurious quote the + model wrapped around unquoted narration, and fixes known title forms; + - stray lowercase English words are detected and repaired in place; + - the remaining Latin words are recorded per paragraph as warnings + (proper names go to glossary review, not silent transliteration). +- Added the QA, names and review tools: + - qa.py missing paragraphs, metadata leaks, straight/unbalanced + quotes, double punctuation, title drift, English residue, + system->系统, proper-name review queue. + - names.py merges analyst proposed_terms with mid-sentence capitalised + source names into glossary/candidates.md (no auto-merge). + - review.py self-contained bilingual HTML review page, issues attached + to each paragraph, "only flagged" and "hide English" filters. +- Renderer fixes: Chinese chapter numerals (第一章　打捞), renamed the heading + styles to avoid LibreOffice built-in collisions, two-character first-line + indent, cleaner AsciiDoc document header, right-aligned TOC page numbers + with dot leaders (the tab-stop attributes were in the wrong namespace), + title-page author line B·D·墨菲, and a review note that points to + review.html. +- Critic hardening: model ids contain "/", which was being used literally in + the cache filename and crashed every critic call; the filename is now + sanitised. A failed critic (primary and fallback) is recorded as + "skipped" instead of aborting a long run. Confirmation-only issues, source- + grammar notes and empty minor suggestions are filtered out. +- Built the renderer: + - AsciiDoc book (title, TOC, chapter headings, dialogue roles, scene + breaks, source comments). + - Flat ODF -> ODT through LibreOffice, with title page, copyright page, + TOC, A4 layout, running header, page numbers, distinct front/body/back + styles. + - TOC page numbers are populated through UNO; a plain --convert-to + leaves the index empty. + - PDF export for quick review. +- Rendered a 9-page front-matter sample (title page, TOC, review note, + copyright, author's note, audio preface). + +--------------------------------------------------------------------------- +3. CURRENT METRICS +--------------------------------------------------------------------------- +Source: + Chapters 32 (4 front, 27 body, 1 back) + Utterances 4,319 + Reconstructed 3,047 paragraphs + Source words 66,884 + Speakers 55 + +Translated: + Front matter 4 chapters / 26 paragraphs (re-run with corrected prompt) + Chapter 1 24/24 chunks, 232 paragraphs + Back matter 1 chapter / 3 paragraphs + Sample total 261 paragraphs, 8,943 Chinese characters + +Glossary: + Core cast and roles, 9 places, ~60 terms. 204 source name candidates and 14 + analyst term proposals are queued in glossary/candidates.md for approval. + +Render: + deal-hunter-zh-sample.asciidoc + deal-hunter-zh-sample.odt + deal-hunter-zh-sample.pdf (22 pages: title, TOC, review note, copyright, + author's note, audio preface, Chapter 1, back + matter, sample end) + review.html (bilingual side-by-side review page) + run-manifest.json (hashes, models, counts) + +--------------------------------------------------------------------------- +4. SAMPLE SCOPE +--------------------------------------------------------------------------- +The sample is intended to prove the whole chain on a meaningful slice: + - front matter: Sample Track, Opening Track, Copyright, Author's Note + - body: Chapter 1, Salvage (232 paragraphs, all 24 chunks) + - back matter: Closing Track + +The sample is complete at the draft/QA/render level. The critic pass covers +the front/back matter and the first four Chapter 1 chunks; the remaining +Chapter 1 chunks are deliberately left for the full-book critic pass. + +--------------------------------------------------------------------------- +5. WHAT IS RUNNING +--------------------------------------------------------------------------- +A background process is running the bounded qwen critic on the sample and +then re-rendering: + log: productions/deal-hunter-zh/runs/qwen-critic.log + mode: critic on front/back matter + first four Chapter 1 chunks + note: the critic model is qwen3.5-9b for this sample. DeepSeek R1 8B was + tested and rejected: on Chapter 1 chunk c000 it produced 28 issues, + almost all duplicates of three contradictory term claims (it wanted + "236 systems" rendered as 系统 rather than 星系). Qwen is also the + drafter, so this pass is a consistency check, not an independent + critic; the full-run critic model is still a decision for the author. + +Do not start a second full-book run until this finishes; the LM Studio server +serialises requests and two runs would only slow each other down. + +--------------------------------------------------------------------------- +6. BLOCKERS AND DECISIONS +--------------------------------------------------------------------------- +Blocking a full-book run (answers in docs/Questions.txt): + Q1 original DOCX for exact paragraphing (default: proceed with heuristic) + Q3 title treatment (default: 《Deal Hunter》, subtitle 交易猎人) + Q4 name/rank table (default: current glossary, 公爵夫人) + Q5 Simplified vs Traditional (default: Simplified, zh-CN) + Q15 model budget (default: current 9B/2.6B mix for the sample) + Q16 sample approval before the full run (default: yes) + +Not blocking: + Q2 reading edition vs audiobook script (default: reading edition) + Q6-Q8 voice/register/content (defaults recorded) + Q9-Q11 front/back matter and numbering (defaults recorded) + Q12-Q14 output and review artefacts (defaults recorded) + Q17-Q18 data location and glossary additions (defaults recorded) + +Known limitations: + - Paragraph boundaries are reconstructed, not original. The DOCX would + settle them. + - The translation is machine output. It has not had native-speaker review. + - The 27B model is not loaded; the 9B is the quality/speed compromise. + - The sample critic is qwen, the same model as the drafter. DeepSeek R1 8B + was tested and its term judgments were too unreliable to use without a + human filter. + - The critic covers only the first four Chapter 1 chunks in the sample. + - Full-book estimate at the measured rate (~1.6 min/chunk, ~315 chunks) is + about 8-9 h for the draft, plus 3-5 h for the critic. This is higher than + the earlier 4-6 h estimate and should be planned for accordingly. + - The ODT has been checked in LibreOffice only; Word compatibility is + untested. + - The glossary candidate list is heuristic (204 entries) and needs manual + pruning; unapproved names stay English by policy. + +--------------------------------------------------------------------------- +7. NEXT ACTIONS +--------------------------------------------------------------------------- +1. Finish the bounded qwen critic and re-render (background). +2. Operator spot-check of the 22-page sample: layout, names, register, + paragraphing, punctuation, no English residue. +3. Decide the full-run critic model (qwen consistency check vs DeepSeek with + human filtering vs a larger model when available). +4. Answer Q1, Q3-Q5, Q15, Q16; prune glossary/candidates.md and freeze the + glossary. +5. If approved, launch the full-book run (~8-9 h at the measured rate), then + the critic pass, then native review. + +--------------------------------------------------------------------------- +8. FILES +--------------------------------------------------------------------------- +Docs: + docs/Questions.txt + docs/TaskList.txt + docs/Status.txt + +Tools (tracked): + tools/translation/book.py source model / paragraph reconstruction + tools/translation/llm.py LM Studio client, cache, JSON handling + tools/translation/prompts.py analyst/drafter/critic prompts + tools/translation/pipeline.py Pass 0-4 orchestration, cleanup, warnings + tools/translation/render.py AsciiDoc + FODT/ODT/PDF + tools/translation/qa.py mechanical QA and issue extraction + tools/translation/names.py glossary candidate collector + tools/translation/review.py bilingual HTML review page + tools/translation/README.md tool usage notes + +Production (git-ignored, copyrighted text): + productions/deal-hunter-zh/source/book.json + productions/deal-hunter-zh/glossary/glossary.json + productions/deal-hunter-zh/glossary/candidates.md + productions/deal-hunter-zh/runs//*.analysis.json + productions/deal-hunter-zh/runs//*.draft.json + productions/deal-hunter-zh/runs//*.critic.json + productions/deal-hunter-zh/runs//translation.json + productions/deal-hunter-zh/runs/usage.jsonl + productions/deal-hunter-zh/runs/qa.json + productions/deal-hunter-zh/output/deal-hunter-zh-sample.* + productions/deal-hunter-zh/output/review.html + +--------------------------------------------------------------------------- +9. COMMANDS +--------------------------------------------------------------------------- +Extract: + python3 tools/translation/book.py \ + --out productions/deal-hunter-zh/source/book.json + +Translate a chapter: + python3 tools/translation/pipeline.py \ + --book productions/deal-hunter-zh/source/book.json \ + --glossary productions/deal-hunter-zh/glossary/glossary.json \ + --run-dir productions/deal-hunter-zh/runs \ + --chapters ch-6 + +Translate the whole book (after approval): + python3 tools/translation/pipeline.py \ + --book productions/deal-hunter-zh/source/book.json \ + --glossary productions/deal-hunter-zh/glossary/glossary.json \ + --run-dir productions/deal-hunter-zh/runs \ + --chapters all --consistency + +Render: + python3 tools/translation/render.py \ + --book productions/deal-hunter-zh/source/book.json \ + --run-dir productions/deal-hunter-zh/runs \ + --out-dir productions/deal-hunter-zh/output \ + --sample + +Report: + python3 tools/translation/pipeline.py \ + --book productions/deal-hunter-zh/source/book.json \ + --glossary productions/deal-hunter-zh/glossary/glossary.json \ + --run-dir productions/deal-hunter-zh/runs --report + +QA, glossary candidates and review page: + python3 tools/translation/qa.py \ + --book productions/deal-hunter-zh/source/book.json \ + --run-dir productions/deal-hunter-zh/runs \ + --json-out productions/deal-hunter-zh/runs/qa.json + + python3 tools/translation/names.py \ + --book productions/deal-hunter-zh/source/book.json \ + --run-dir productions/deal-hunter-zh/runs \ + --glossary productions/deal-hunter-zh/glossary/glossary.json \ + --out productions/deal-hunter-zh/glossary/candidates.md + + python3 tools/translation/review.py \ + --book productions/deal-hunter-zh/source/book.json \ + --run-dir productions/deal-hunter-zh/runs \ + --out productions/deal-hunter-zh/output/review.html + +--------------------------------------------------------------------------- +10. DECISIONS RECORDED +--------------------------------------------------------------------------- +- Keep "Deal Hunter" (book title and ship AI), "Hunter" and "Umbra Audio" in + English. +- Character names are transliterated by sound, not translated by meaning. +- Narration is restrained written Chinese; dialogue is colloquial and + character-specific. +- Unknown names stay in English until the glossary is approved; no invented + transliterations. +- Production data stays out of the public repository. diff --git a/docs/TaskList.txt b/docs/TaskList.txt new file mode 100644 index 0000000..f34c4af --- /dev/null +++ b/docs/TaskList.txt @@ -0,0 +1,187 @@ +DEAL HUNTER — SIMPLIFIED CHINESE TRANSLATION +TASK LIST +Updated: 2026-10-19 13:44 UTC + +Legend: [x] done [~] in progress [ ] todo [!] blocked [?] decision needed +Effort is a rough machine/operator estimate, not a promise. + +--------------------------------------------------------------------------- +PHASE 0 — SOURCE AND STRUCTURE +--------------------------------------------------------------------------- +[x] T0.1 Locate the English source. + artifacts: audio-read/bundles/dh-bundle/manifest.jsonl + sha256: 21ce568787540cf7164b6ab4028f1caee51996447537022154d1e5768cde87ba +[x] T0.2 Extract the book (1 book, 32 chapter rows, 4,319 utterances). + artifacts: productions/deal-hunter-zh/source/book.json +[x] T0.3 Classify matter: 4 front, 27 body, 1 back. +[x] T0.4 Reconstruct readable paragraphs; keep source utterance ranges. + 3,047 paragraphs; 66,884 source words. +[x] T0.5 Detect and remove production-only lines (the spoken "Chapter one" + announcement in Salvage; the repeated "Author's Note" heading). +[!] T0.6 Recover exact DOCX paragraphing. Blocked on Q1. + Default: proceed with the reconstruction and flag it for review. + +--------------------------------------------------------------------------- +PHASE 1 — GLOSSARY AND STYLE +--------------------------------------------------------------------------- +[x] T1.1 Initial glossary: characters, roles, places, terms, style rules. + artifacts: productions/deal-hunter-zh/glossary/glossary.json +[x] T1.2 Style guide: narration, dialogue, punctuation, numbers, unknown + names, no additions. +[?] T1.3 Author approval of names, ranks and title treatment. Q3–Q5. +[x] T1.4 Enforce the title/proper-name rules in the drafter prompt + (Deal Hunter/Hunter/Umbra Audio stay English; no invented full + names or titles). +[~] T1.5 Expand the glossary for minor characters and ship/place names + after the Chapter 1 analysis pass. + artifacts: glossary/candidates.md (14 analyst proposals, 204 source + candidates; needs manual pruning before merge). +[~] T1.6 Add a glossary validation check (every approved name appears + consistently; no unapproved transliteration). + note: qa.py flags proper-name residue and system->系统 today; a full + approved-name consistency check is still todo. + +--------------------------------------------------------------------------- +PHASE 2 — PIPELINE +--------------------------------------------------------------------------- +[x] T2.1 Local OpenAI-compatible client with reasoning disabled, retries, + JSON extraction, prompt-hash cache and usage log. + artifacts: tools/translation/llm.py +[x] T2.2 Pass 0: scene/chunk assembly, source-index preservation. + artifacts: tools/translation/book.py, pipeline.py +[x] T2.3 Pass 1: scene intent analysis (summary, register, tone, + participants, risks, proposed terms). +[x] T2.4 Pass 2: context-aware draft translation with glossary and + analysis; paragraph-id validation and per-paragraph fallback. + note: source lines no longer carry speaker labels; deterministic + cleanup removes leaks, normalises quotes and repairs English + residue; remaining Latin is recorded as a per-paragraph warning. +[x] T2.5 Pass 3: chapter voice/term consistency check (implemented, + optional; not yet run). +[~] T2.6 Pass 4: bilingual critic with a different model, severity and + minimal-fix output. + note: sample uses qwen3.5-9b, the drafter model, because DeepSeek + R1 8B produced 28 duplicate/contradictory term issues on one + chunk. The independence requirement is not met by the sample + critic; the full-run model is a decision for the author. +[x] T2.7 Resume/checkpoint: every analysis, draft and critique is a file. +[x] T2.8 JSON repair and per-paragraph fallback; no silent omissions. +[ ] T2.9 Parallelise independent chunks (the server serialises today; + measure before changing). +[~] T2.10 Add a cost/time estimator and a run manifest. + note: measured ~1.6 min/chunk gives a ~8-9 h full-book estimate; + a first-class estimator is still todo. +[~] T2.11 Add automated regression tests for the prompt/cache/validation + paths. + note: 17 unittest cases cover sanitise, residue detection, JSON + extraction and the mechanical QA checks; prompt/cache and live LLM + paths are still todo. + +--------------------------------------------------------------------------- +PHASE 3 — SAMPLE TRANSLATION +--------------------------------------------------------------------------- +[x] T3.1 Translate the front matter (Sample Track, Opening Track, + Copyright, Author's Note). + note: re-run with the corrected prompt and glossary. +[x] T3.2 Translate Chapter 1, Salvage (232 paragraphs, 24 chunks). + note: complete; QA clean apart from one ship-name review note. +[~] T3.3 Run Pass 4 critic on the front matter and Chapter 1. + note: front/back matter and the first four Chapter 1 chunks are + covered by the bounded qwen critic; the rest waits for the + full-book critic pass. +[ ] T3.4 Run Pass 3 consistency on Chapter 1. +[x] T3.5 Translate the Closing Track (3 paragraphs). +[~] T3.6 Operator spot-check: names, title, register, paragraphing, + Chinese punctuation, no English residue. + note: a 22-page render has been checked for layout and TOC; the + operator's language review is still pending. +[ ] T3.7 Fix the sample issues and freeze the prompt/glossary version. + +--------------------------------------------------------------------------- +PHASE 4 — RENDERING +--------------------------------------------------------------------------- +[x] T4.1 AsciiDoc writer (book doctype, TOC, chapter headings, dialogue + roles, scene breaks, source comments). + artifacts: tools/translation/render.py +[x] T4.2 Flat-ODF writer with title page, copyright, TOC, headings, + body/dialogue/quote/front-matter styles, A4 page layout, + running header and page numbers. +[x] T4.3 ODT conversion with UNO TOC update (page numbers populated). +[x] T4.4 PDF export for review. +[x] T4.5 Different formatting for front matter, body and back matter. +[x] T4.6 Sample render: title page, TOC, copyright, author's note, + audio preface. + note: 22-page render includes the full Chapter 1 and the Closing + Track. +[~] T4.7 Chinese typography check on rendered pages (font size, indent, + line spacing, punctuation, orphan/widow control). + note: font size (20pt headings, 11pt body), 2-em indent, 1.5 line + spacing, Chinese numerals and right-aligned TOC page numbers are + verified in LibreOffice; orphan/widow and full punctuation review + are still pending. +[~] T4.8 Verify the ODT opens cleanly in LibreOffice and Microsoft Word. + note: LibreOffice verified; Word is not available on this machine. +[ ] T4.9 Per-chapter ODT/AsciiDoc outputs (optional). +[ ] T4.10 Add a cover image / half-title if supplied. + +--------------------------------------------------------------------------- +PHASE 5 — FULL-BOOK RUN +--------------------------------------------------------------------------- +[ ] T5.1 Translate all 27 body chapters (~8–9 h at the measured rate). +[ ] T5.2 Run the critic pass on every chapter (3–5 h). +[ ] T5.3 Run the consistency pass and apply approved fixes. +[ ] T5.4 Normalise the glossary and re-run affected paragraphs. +[ ] T5.5 Render the full AsciiDoc, ODT and PDF. +[ ] T5.6 Record model names, prompt versions, glossary hash and run date + in the output metadata. + +--------------------------------------------------------------------------- +PHASE 6 — QA AND REVIEW +--------------------------------------------------------------------------- +[~] T6.1 Automated checks: + - every source paragraph has a translation (or an explicit flag) + - no paragraph-id drift or duplicate ids + - no unintended English residue + - Chinese punctuation and quote pairing + - approved names/terms used consistently + - no added or dropped sentences in flagged passages + note: qa.py covers missing paragraphs, id drift, metadata leaks, + straight/unbalanced quotes, double punctuation, title drift, + English residue and system->系统. Full approved-name consistency + and added/dropped-sentence detection are still todo. +[x] T6.2 Produce a bilingual side-by-side review table. + artifacts: tools/translation/review.py -> output/review.html +[~] T6.3 Produce a severity-sorted issues queue (block/major/minor). + note: qa.py --json-out and the critic issues are merged in + review.html; a standalone queue is still todo. +[ ] T6.4 Native-speaker review of the full book. +[ ] T6.5 Apply the review fixes and re-render. +[ ] T6.6 Final read-through of the ODT/PDF (TOC, page breaks, headers). + +--------------------------------------------------------------------------- +PHASE 7 — DELIVERY +--------------------------------------------------------------------------- +[ ] T7.1 Final deliverables: AsciiDoc, ODT, PDF, glossary, issue log. +[ ] T7.2 Version and date the output; keep the source hash. +[ ] T7.3 Archive the production outside the public Git repository. +[ ] T7.4 Optional: EPUB and print-ready PDF. + +--------------------------------------------------------------------------- +PHASE 8 — AUDIO ALIGNMENT (FUTURE) +--------------------------------------------------------------------------- +[x] T8.1 Preserve source utterance indices on every translated paragraph. +[ ] T8.2 1:1 utterance alignment for TTS (split or align the paragraph + translation back to the 4,319 source utterances). +[ ] T8.3 Update the UmbraAudio import format to accept paragraph-level + translations with source ranges. +[ ] T8.4 Re-render the audiobook in Chinese and compare timing. + +--------------------------------------------------------------------------- +CRITICAL PATH +--------------------------------------------------------------------------- +Q1/Q3/Q4/Q5 (source, title, names, script) + -> T3.6 sample approval + -> T5.1 full-book draft + -> T5.2 critic + -> T6.4 native review + -> T7.1 delivery diff --git a/pyproject.toml b/pyproject.toml index bdef949..0b3e48a 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -32,7 +32,7 @@ include = ["agent_comm*", "umbratrans_web*"] umbratrans_web = ["migrations/*.sql", "migrations/*.md"] [tool.pytest.ini_options] -testpaths = ["umbratrans_web/tests", "agent_comm/tests"] +testpaths = ["umbratrans_web/tests", "agent_comm/tests", "tools/translation/tests"] python_files = ["test_*.py"] [tool.ruff] diff --git a/tools/translation/README.md b/tools/translation/README.md new file mode 100644 index 0000000..c385ae7 --- /dev/null +++ b/tools/translation/README.md @@ -0,0 +1,126 @@ +# Deal Hunter Chinese translation tools + +This directory contains the translation and rendering pipeline used to produce +the Simplified Chinese edition of *Deal Hunter*. It is deliberately separate +from the web application: the book text and the translations are copyrighted +and are **not** committed to this repository. + +## Where the data lives + +``` +productions/deal-hunter-zh/ # git-ignored + source/book.json # extracted English source model + glossary/glossary.json # names, places, terms, style rules + runs//*.json # analyses, drafts, critiques, final text + runs/usage.jsonl # per-call token/latency log + output/*.asciidoc, *.odt, *.pdf # review artefacts +``` + +The source bundle is the UmbraAudio controller bundle: + +``` +/home/umbra/audio-read/bundles/dh-bundle/manifest.jsonl +``` + +Its sha256 is recorded in `Status.txt` and in `source/book.json`. + +## Passes + +| Pass | What it does | Model | +|---|---|---| +| 0 | Paragraph/scene/chunk assembly; keeps source utterance indices | none | +| 1 | Scene intent: summary, register, tone, participants, risks | `lfm2.5-2.6b` | +| 2 | Context-aware draft translation with glossary and analysis | `qwen3.5-9b` | +| 3 | Chapter voice/term consistency (optional) | critic model | +| 4 | Bilingual critic: meaning, pronoun, irony, term, omission | `deepseek-r1-8b`, fallback `qwen3.5-9b` | + +Every analysis, draft and critique is a file under `runs//`. A run can +be stopped and resumed; existing artefacts are reused unless `--force` is +given. LLM responses are cached by prompt hash, so editing a prompt +invalidates the old answer. + +The drafter is told to translate intent, not words, and is given a glossary, +character voices, the scene analysis, and the previous chunk's summary. The +critic uses a different model and returns a severity-sorted issue list; it does +not rewrite the text. + +## Commands + +Extract the source: + +```bash +python3 tools/translation/book.py \ + --out productions/deal-hunter-zh/source/book.json +``` + +Translate a chapter: + +```bash +python3 tools/translation/pipeline.py \ + --book productions/deal-hunter-zh/source/book.json \ + --glossary productions/deal-hunter-zh/glossary/glossary.json \ + --run-dir productions/deal-hunter-zh/runs \ + --chapters ch-6 +``` + +Selectors: `all`, `front`, `body`, `back`, or a comma-separated list of +chapter indices / ids / titles (`-4,-3,-2,-1` must be quoted or passed as +`--chapters=-4,-3`). + +Render: + +```bash +python3 tools/translation/render.py \ + --book productions/deal-hunter-zh/source/book.json \ + --run-dir productions/deal-hunter-zh/runs \ + --out-dir productions/deal-hunter-zh/output \ + --sample +``` + +Mechanical QA: + +```bash +python3 tools/translation/qa.py \ + --book productions/deal-hunter-zh/source/book.json \ + --run-dir productions/deal-hunter-zh/runs \ + --json-out productions/deal-hunter-zh/runs/qa.json \ + --strict +``` + +## Models + +The default server is the LAN LM Studio instance at +`http://192.168.123.16:1234/v1`. Override with: + +``` +UMBRA_TRANSLATION_BASE_URL +UMBRA_TRANSLATION_MODEL +UMBRA_TRANSLATION_ANALYST_MODEL +UMBRA_TRANSLATION_CRITIC_MODEL +``` + +Qwen models emit reasoning tokens by default; the client sends +`reasoning_effort: "none"` so the budget is spent on the translation. The small +analyst model rejects LM Studio's JSON-schema grammar, so its answer is parsed +from text and repaired if necessary. + +## Rendering + +The AsciiDoc file is the canonical text output. The ODT is produced from a Flat +ODF document (`.fodt`) so that page layout, styles and the table of contents are +under our control. LibreOffice converts it and updates the TOC through UNO; a +plain `--convert-to` leaves the index empty. A PDF is exported for quick review. + +Front matter, body and back matter use different styles. Chapter headings use +Chinese numerals (`第一章　打捞`). Body text is 11 pt Noto Serif CJK SC with +1.5 line spacing and a two-character first-line indent. + +## What this is not + +- It is not a native-speaker-reviewed translation. The critic and the QA script + catch mechanical defects; a human editor is still required. +- It does not recover the original DOCX paragraphing. The bundle is + sentence-level, so paragraphs are reconstructed and can be reviewed. +- It does not produce a 1:1 utterance alignment for TTS. Source ranges are + preserved on every paragraph; splitting or aligning back to the 4,319 + utterances is a separate pass. diff --git a/tools/translation/book.py b/tools/translation/book.py new file mode 100644 index 0000000..3b234d6 --- /dev/null +++ b/tools/translation/book.py @@ -0,0 +1,409 @@ +# Copyright 2026 Umbra Tools (umbratools.com) +# SPDX-License-Identifier: Apache-2.0 + +"""Load the Deal Hunter controller bundle into a book model. + +The bundle is a JSONL manifest produced by UmbraAudio. One utterance is a +sentence or a short paragraph; the original DOCX paragraph boundaries are not +in the bundle. This module therefore reconstructs readable paragraphs with a +documented heuristic and keeps the source utterance indices on every +paragraph so a later audio-alignment step can still map back. + +Nothing here calls an LLM. +""" + +from __future__ import annotations + +import argparse +import hashlib +import json +import re +from dataclasses import asdict, dataclass, field +from pathlib import Path + +DEFAULT_BUNDLE = Path("/home/umbra/audio-read/bundles/dh-bundle/manifest.jsonl") + +# Chinese chapter titles. Kept here, not inferred, so every render is stable. +# Reviewable in one place; the English title stays on the paragraph. +TITLE_ZH = { + "Sample Track": "音频版序言：声音介绍", + "Opening Track": "开篇说明", + "Copyright": "版权信息", + "Author's Note": "作者的话", + "Salvage": "打捞", + "Family Reunion": "家庭团聚", + "Selling Salvage": "出售打捞物", + "Frustration": "挫败", + "Next Bounty": "下一个赏金", + "Time for a Change": "改变的时刻", + "Trade Hub Vince-4": "贸易枢纽文斯-4", + "Can I Join the Crew": "我能加入船员吗", + "Salvage from Crashed Fighters": "坠毁战机的残骸", + "Where Is the Captain?": "船长在哪里？", + "Turn in Fugitives": "上交逃犯", + "Her New Ship": "她的新船", + "Introducing the new XO": "新任副舰长", + "A New Ship Salvage": "新船打捞", + "The Poker Game Distraction": "扑克牌局掩护", + "Deal Hunter Bounty": "Deal Hunter 的赏金", + "Selecting Bounties": "挑选赏金", + "Shuttle Scans": "穿梭机扫描", + "The Pirate Trap": "海盗陷阱", + "The Vid Interview": "视频采访", + "Hunting Pirates": "追猎海盗", + "The Friendly Pirate": "友好的海盗", + "Aunt Vicky": "维姬姑妈", + "Preparation": "准备", + "The Coronation": "加冕典礼", + "Long Live the King": "国王万岁", + "Let’s Make It Better": "让它变得更好", + "Let's Make It Better": "让它变得更好", + "Closing Track": "后记：感谢收听", +} + +MAX_NARRATION_SENTENCES = 3 +MAX_NARRATION_WORDS = 58 + + +@dataclass +class Utterance: + index: int + speaker: str + type: str + text: str + + +@dataclass +class Paragraph: + id: str + speaker: str + type: str + text: str + source_indices: list[int] = field(default_factory=list) + source_ids: list[str] = field(default_factory=list) + + @property + def words(self) -> int: + return len(self.text.split()) + + +@dataclass +class Scene: + id: str + index: int + paragraphs: list[Paragraph] = field(default_factory=list) + + +@dataclass +class Chapter: + id: str + index: int + row: int + title: str + title_zh: str + matter: str + paragraphs: list[Paragraph] = field(default_factory=list) + scenes: list[Scene] = field(default_factory=list) + + def scenes_json(self) -> list[dict]: + return [asdict(s) for s in self.scenes] + + +@dataclass +class Book: + title: str + title_zh: str + author: str + source_language: str + target_language: str + bundle: str + bundle_sha256: str + chapters: list[Chapter] = field(default_factory=list) + + def chapter(self, ref: str | int) -> Chapter: + for c in self.chapters: + if str(c.index) == str(ref) or c.id == str(ref): + return c + raise KeyError(f"chapter not found: {ref}") + + def to_json(self) -> dict: + return { + "title": self.title, + "title_zh": self.title_zh, + "author": self.author, + "source_language": self.source_language, + "target_language": self.target_language, + "bundle": self.bundle, + "bundle_sha256": self.bundle_sha256, + "chapters": [ + { + **{k: v for k, v in asdict(c).items() if k != "scenes"}, + "scenes": [asdict(s) for s in c.scenes], + } + for c in self.chapters + ], + } + + +def _sentences(text: str) -> int: + # A conservative sentence count for grouping only. Quotes and ellipses are + # the places where a naive split goes wrong, so count terminators instead. + n = len(re.findall(r"[.!?]+(?:[”’\"']|$|\s)", text)) + return max(1, n) + + +def _classify(index: int, title: str) -> str: + if index < 0: + if title in {"Sample Track", "Opening Track"}: + return "front_audio" + if title == "Copyright": + return "front_copyright" + if title == "Author's Note": + return "front_note" + return "front" + if title == "Closing Track" or index > 26: + return "back" + return "body" + + +def _clean_title(title: str) -> str: + return title.replace("’", "'").strip() + + +def _title_zh(title: str) -> str: + key = _clean_title(title) + if key in TITLE_ZH: + return TITLE_ZH[key] + # Unknown title: leave a visible marker rather than invent a translation. + return f"[待译] {title}" + + +def load_manifest(path: Path = DEFAULT_BUNDLE) -> tuple[list[dict], dict]: + raw = path.read_bytes() + digest = hashlib.sha256(raw).hexdigest() + records = [json.loads(line) for line in raw.decode("utf-8").splitlines() if line.strip()] + return records, {"path": str(path), "sha256": digest} + + +def build_book(path: Path = DEFAULT_BUNDLE) -> Book: + records, meta = load_manifest(path) + book_rec = next(r for r in records if r.get("kind") == "book") + chapter_recs = sorted( + (r for r in records if r.get("kind") == "chapter"), + key=lambda r: r["chapter_index"], + ) + utterances: dict[str, list[dict]] = {} + for r in records: + if r.get("kind") == "utterance": + utterances.setdefault(r["chapter_id"], []).append(r) + for rows in utterances.values(): + rows.sort(key=lambda r: r["utterance_index"]) + + book = Book( + title="Deal Hunter", + title_zh="交易猎人", + author="B. D. Murphy", + source_language="en", + target_language="zh-CN", + bundle=meta["path"], + bundle_sha256=meta["sha256"], + ) + + for cr in chapter_recs: + index = int(cr["chapter_index"]) + title = cr["title"] + matter = _classify(index, title) + rows = utterances.get(cr["id"], []) + paragraphs = _paragraphs(cr["id"], title, matter, rows) + chapter = Chapter( + id=cr["id"], + index=index, + row=int(cr["database_row_id"]), + title=title, + title_zh=_title_zh(title), + matter=matter, + paragraphs=paragraphs, + ) + chapter.scenes = _scenes(cr["id"], paragraphs) + book.chapters.append(chapter) + + return book + + +def _paragraphs(chapter_id: str, title: str, matter: str, rows: list[dict]) -> list[Paragraph]: + if not rows: + return [] + + # The parser leaves the spoken chapter announcement in the first utterance + # of Salvage ("Deal Hunter. Chapter one, Salvage."). It is a production + # cue, not prose, so it must not be translated into the reading edition. + if matter == "body" and rows[0]["text"].strip().startswith("Deal Hunter. Chapter"): + rows = rows[1:] + + # The author's-note heading is an utterance, not a paragraph. + if matter == "front_note" and rows and rows[0]["text"].strip().rstrip(".").lower() in { + "author's note", + "authors note", + "author’s note", + }: + rows = rows[1:] + + paragraphs: list[Paragraph] = [] + scene_no = 0 + buf: list[dict] = [] + buf_words = 0 + buf_sentences = 0 + + def flush() -> None: + nonlocal buf, buf_words, buf_sentences + if not buf: + return + paragraphs.append( + Paragraph( + id=f"{chapter_id}-p{len(paragraphs):04d}", + speaker=buf[0].get("speaker_name_resolved") or "narrator", + type=buf[0].get("utterance_type") or "narration", + text=" ".join(r["text"].strip() for r in buf), + source_indices=[int(r["utterance_index"]) for r in buf], + source_ids=[r["id"] for r in buf], + ) + ) + buf = [] + buf_words = 0 + buf_sentences = 0 + + for row in rows: + utype = row.get("utterance_type") or "narration" + speaker = row.get("speaker_name_resolved") or "narrator" + text = (row.get("text") or "").strip() + if utype == "scene_break": + flush() + scene_no += 1 + paragraphs.append( + Paragraph( + id=f"{chapter_id}-break{scene_no}", + speaker="", + type="scene_break", + text="* * *", + source_indices=[int(row["utterance_index"])], + source_ids=[row["id"]], + ) + ) + continue + + if matter.startswith("front") or matter == "back": + # Front and back matter are line-oriented: a copyright line and an + # author's-note bullet are not the same thing as a prose paragraph. + flush() + paragraphs.append( + Paragraph( + id=f"{chapter_id}-p{len(paragraphs):04d}", + speaker=speaker, + type=utype, + text=text, + source_indices=[int(row["utterance_index"])], + source_ids=[row["id"]], + ) + ) + continue + + if utype == "dialogue": + # Each speech turn is its own paragraph. + flush() + paragraphs.append( + Paragraph( + id=f"{chapter_id}-p{len(paragraphs):04d}", + speaker=speaker, + type=utype, + text=text, + source_indices=[int(row["utterance_index"])], + source_ids=[row["id"]], + ) + ) + continue + + # Narration: group consecutive sentences into a readable paragraph. + if buf and ( + buf[0].get("speaker_name_resolved") != speaker + or buf[0].get("utterance_type") != utype + ): + flush() + sentences = _sentences(text) + if buf and (buf_sentences + sentences > MAX_NARRATION_SENTENCES or buf_words + len(text.split()) > MAX_NARRATION_WORDS): + flush() + buf.append(row) + buf_words += len(text.split()) + buf_sentences += sentences + if buf_sentences >= MAX_NARRATION_SENTENCES or buf_words >= MAX_NARRATION_WORDS: + flush() + + flush() + # Drop a trailing scene-break marker. + while paragraphs and paragraphs[-1].type == "scene_break": + paragraphs.pop() + return paragraphs + + +def _scenes(chapter_id: str, paragraphs: list[Paragraph]) -> list[Scene]: + scenes: list[Scene] = [] + current = Scene(id=f"{chapter_id}-s{len(scenes)}", index=len(scenes)) + for p in paragraphs: + if p.type == "scene_break": + if current.paragraphs: + scenes.append(current) + current = Scene(id=f"{chapter_id}-s{len(scenes)}", index=len(scenes)) + continue + current.paragraphs.append(p) + if current.paragraphs: + scenes.append(current) + return scenes + + +def save(book: Book, out: Path) -> None: + out.parent.mkdir(parents=True, exist_ok=True) + out.write_text(json.dumps(book.to_json(), ensure_ascii=False, indent=2) + "\n", encoding="utf-8") + + +def load(path: Path) -> dict: + return json.loads(path.read_text(encoding="utf-8")) + + +def summary(book: Book) -> str: + lines = [ + f"Book: {book.title} / {book.title_zh}", + f"Author: {book.author}", + f"Bundle: {book.bundle}", + f"Bundle sha256: {book.bundle_sha256}", + "", + f"{'idx':>4} {'matter':16} {'paras':>6} {'words':>7} title", + ] + for c in book.chapters: + words = sum(p.words for p in c.paragraphs if p.type != "scene_break") + lines.append( + f"{c.index:>4} {c.matter:16} {len(c.paragraphs):>6} {words:>7} {c.title} / {c.title_zh}" + ) + body = [c for c in book.chapters if c.matter == "body"] + lines += [ + "", + f"Front matter chapters: {sum(1 for c in book.chapters if c.matter.startswith('front'))}", + f"Body chapters: {len(body)}", + f"Back matter chapters: {sum(1 for c in book.chapters if c.matter == 'back')}", + f"Paragraphs: {sum(len(c.paragraphs) for c in book.chapters)}", + f"Words: {sum(p.words for c in book.chapters for p in c.paragraphs if p.type != 'scene_break')}", + ] + return "\n".join(lines) + + +def main() -> None: + ap = argparse.ArgumentParser(description="Extract the Deal Hunter bundle into a book model.") + ap.add_argument("--bundle", type=Path, default=DEFAULT_BUNDLE) + ap.add_argument("--out", type=Path, required=True) + args = ap.parse_args() + book = build_book(args.bundle) + save(book, args.out) + print(summary(book)) + print(f"\nwrote {args.out}") + + +if __name__ == "__main__": + main() diff --git a/tools/translation/llm.py b/tools/translation/llm.py new file mode 100644 index 0000000..3f7689a --- /dev/null +++ b/tools/translation/llm.py @@ -0,0 +1,266 @@ +# Copyright 2026 Umbra Tools (umbratools.com) +# SPDX-License-Identifier: Apache-2.0 + +"""Minimal OpenAI-compatible client for the local LM Studio server. + +Only the standard library is used. The Qwen models on BigBox emit reasoning +tokens unless told not to; ``reasoning_effort="none"`` is what makes a +translation call return prose instead of a thinking trace. +""" + +from __future__ import annotations + +import hashlib +import json +import os +import re +import time +import urllib.error +import urllib.request +from dataclasses import dataclass +from pathlib import Path +from typing import Any + +DEFAULT_BASE_URL = os.environ.get("UMBRA_TRANSLATION_BASE_URL", "http://192.168.123.16:1234/v1") +DEFAULT_MODEL = os.environ.get("UMBRA_TRANSLATION_MODEL", "qwen/qwen3.5-9b") +DEFAULT_ANALYST = os.environ.get( + "UMBRA_TRANSLATION_ANALYST_MODEL", + "lfm2.5-2.6b-qwen3.8-turbo-brilliance-power-x12-neo-max", +) +DEFAULT_CRITIC = os.environ.get("UMBRA_TRANSLATION_CRITIC_MODEL", "deepseek/deepseek-r1-0528-qwen3-8b") + + +@dataclass +class Usage: + model: str + prompt_tokens: int + completion_tokens: int + elapsed_s: float + + +class LLMError(RuntimeError): + pass + + +class LLM: + def __init__( + self, + base_url: str = DEFAULT_BASE_URL, + cache_dir: Path | None = None, + usage_log: Path | None = None, + timeout: int = 600, + ) -> None: + self.base_url = base_url.rstrip("/") + self.cache_dir = cache_dir + self.usage_log = usage_log + self.timeout = timeout + if cache_dir: + cache_dir.mkdir(parents=True, exist_ok=True) + + # ------------------------------------------------------------------ HTTP + def _post(self, payload: dict[str, Any]) -> dict[str, Any]: + body = json.dumps(payload, ensure_ascii=False).encode("utf-8") + req = urllib.request.Request( + self.base_url + "/chat/completions", + data=body, + headers={"Content-Type": "application/json"}, + ) + try: + with urllib.request.urlopen(req, timeout=self.timeout) as resp: + return json.loads(resp.read().decode("utf-8")) + except urllib.error.HTTPError as exc: + detail = exc.read().decode("utf-8", "replace")[:800] + raise LLMError(f"HTTP {exc.code}: {detail}") from exc + except urllib.error.URLError as exc: + raise LLMError(f"cannot reach {self.base_url}: {exc.reason}") from exc + + def chat( + self, + system: str, + user: str, + *, + model: str = DEFAULT_MODEL, + temperature: float = 0.3, + max_tokens: int = 4096, + reasoning: bool = False, + json_mode: bool = False, + cache_key: str | None = None, + retries: int = 3, + ) -> tuple[str, Usage]: + digest = hashlib.sha256( + json.dumps( + { + "model": model, + "temperature": temperature, + "system": system, + "user": user, + "json_mode": json_mode, + }, + ensure_ascii=False, + sort_keys=True, + ).encode("utf-8") + ).hexdigest() + cache_path = None + if self.cache_dir: + # Tie the cache file to the prompt hash as well as the logical key. + # Otherwise a prompt edit silently reuses the old answer. + safe_key = re.sub(r"[^A-Za-z0-9_.-]+", "_", cache_key or "chat") + cache_path = self.cache_dir / f"{safe_key}.{digest[:12]}.json" + if cache_path.exists(): + cached = json.loads(cache_path.read_text(encoding="utf-8")) + return cached["content"], Usage( + model=model, + prompt_tokens=cached.get("prompt_tokens", 0), + completion_tokens=cached.get("completion_tokens", 0), + elapsed_s=0.0, + ) + + payload: dict[str, Any] = { + "model": model, + "messages": [ + {"role": "system", "content": system}, + {"role": "user", "content": user}, + ], + "temperature": temperature, + "max_tokens": max_tokens, + } + if not reasoning: + payload["reasoning_effort"] = "none" + if json_mode: + # LM Studio rejects the OpenAI "json_object" shorthand but accepts a + # permissive json_schema. This keeps the model from wrapping the + # answer in prose, which used to cost a full repair call. + payload["response_format"] = { + "type": "json_schema", + "json_schema": { + "name": "translation_payload", + "schema": {"type": "object", "additionalProperties": True}, + }, + } + + last: Exception | None = None + for attempt in range(retries): + started = time.time() + try: + data = self._post(payload) + message = data["choices"][0]["message"] + content = (message.get("content") or "").strip() + if not content: + # Reasoning models can spend the whole budget thinking. + raise LLMError( + "empty content " + f"(reasoning_tokens={data.get('usage', {}).get('completion_tokens_details', {}).get('reasoning_tokens')})" + ) + usage = data.get("usage", {}) + elapsed = time.time() - started + u = Usage( + model=model, + prompt_tokens=int(usage.get("prompt_tokens", 0)), + completion_tokens=int(usage.get("completion_tokens", 0)), + elapsed_s=elapsed, + ) + if cache_path: + cache_path.write_text( + json.dumps( + { + "content": content, + "prompt_tokens": u.prompt_tokens, + "completion_tokens": u.completion_tokens, + "model": model, + }, + ensure_ascii=False, + indent=2, + ) + + "\n", + encoding="utf-8", + ) + if self.usage_log: + self.usage_log.parent.mkdir(parents=True, exist_ok=True) + with self.usage_log.open("a", encoding="utf-8") as fh: + fh.write( + json.dumps( + { + "ts": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime()), + "model": model, + "prompt_tokens": u.prompt_tokens, + "completion_tokens": u.completion_tokens, + "elapsed_s": round(elapsed, 2), + "cache_key": cache_key, + } + ) + + "\n" + ) + return content, u + except Exception as exc: # noqa: BLE001 - retry boundary + last = exc + if attempt < retries - 1: + time.sleep(2**attempt) + raise LLMError(f"chat failed after {retries} attempts: {last}") + + # ------------------------------------------------------------------ JSON + def chat_json( + self, + system: str, + user: str, + *, + model: str = DEFAULT_MODEL, + temperature: float = 0.2, + max_tokens: int = 4096, + cache_key: str | None = None, + json_mode: bool = True, + ) -> tuple[Any, Usage]: + content, usage = self.chat( + system, + user, + model=model, + temperature=temperature, + max_tokens=max_tokens, + reasoning=False, + json_mode=json_mode, + cache_key=cache_key, + ) + try: + return parse_json(content), usage + except ValueError: + # One repair attempt, with the malformed output in front of the model. + repair = ( + "The previous answer was not valid JSON. Return only valid JSON, " + "no code fence, no commentary. The required shape was:\n" + f"{user}\n\nPrevious answer:\n{content[:6000]}" + ) + fixed, usage2 = self.chat( + system, + repair, + model=model, + temperature=0.0, + max_tokens=max_tokens, + reasoning=False, + json_mode=json_mode, + cache_key=(cache_key + "-repair") if cache_key else None, + ) + return parse_json(fixed), usage2 + + +def parse_json(text: str) -> Any: + """Parse a JSON object/array from a model answer. + + Accepts a code fence, leading prose, or a trailing sentence. Raises + ValueError when no complete JSON value can be decoded. + """ + text = text.strip() + if text.startswith("```"): + text = re.sub(r"^```[a-zA-Z0-9_-]*\s*", "", text) + text = re.sub(r"\s*```$", "", text).strip() + try: + return json.loads(text) + except json.JSONDecodeError: + pass + decoder = json.JSONDecoder() + for i, ch in enumerate(text): + if ch in "{[": + try: + value, _ = decoder.raw_decode(text[i:]) + return value + except json.JSONDecodeError: + continue + raise ValueError(f"no JSON value found in answer: {text[:300]!r}") diff --git a/tools/translation/names.py b/tools/translation/names.py new file mode 100644 index 0000000..e105723 --- /dev/null +++ b/tools/translation/names.py @@ -0,0 +1,198 @@ +# Copyright 2026 Umbra Tools (umbratools.com) +# SPDX-License-Identifier: Apache-2.0 + +"""Collect proper-name and term candidates for the glossary. + +Two sources are merged: + +1. Terms proposed by the scene analyst (``proposed_terms`` in every + ``*.analysis.json``). +2. Capitalised words in the English source that appear mid-sentence, which is + a decent signal for a name but filters out sentence-initial function words. + +Nothing is written into the glossary automatically. The output is a proposal +list for the operator to accept, edit or reject, because an unapproved +transliteration is worse than an English name left in place. +""" + +from __future__ import annotations + +import argparse +import json +import re +import sys +from collections import defaultdict +from pathlib import Path + +sys.path.insert(0, str(Path(__file__).resolve().parent)) + +from pipeline import LATIN_ALLOW # noqa: E402 + +STOPWORDS = { + "a", "about", "after", "again", "against", "all", "also", "am", "an", "and", + "any", "are", "aren", "as", "at", "back", "be", "because", "been", "before", + "being", "below", "between", "both", "but", "by", "can", "could", "did", + "didn", "do", "does", "doesn", "doing", "don", "down", "during", "each", + "even", "ever", "every", "few", "for", "from", "get", "gets", "getting", + "give", "go", "going", "gone", "good", "got", "had", "has", "have", + "having", "he", "her", "here", "hers", "herself", "him", "himself", "his", + "how", "however", "i", "if", "in", "into", "is", "isn", "it", "its", + "itself", "just", "know", "let", "like", "look", "looking", "made", "make", + "many", "may", "maybe", "me", "might", "mine", "more", "most", "much", + "must", "my", "myself", "need", "no", "nor", "not", "now", "of", "off", + "on", "once", "one", "only", "or", "other", "ought", "our", "ours", + "ourselves", "out", "over", "own", "perhaps", "please", "put", "quite", + "rather", "really", "right", "said", "same", "say", "saying", "see", + "seen", "she", "should", "since", "so", "some", "something", "still", + "such", "sure", "take", "than", "thank", "thanks", "that", "the", "their", + "theirs", "them", "themselves", "then", "there", "these", "they", "thing", + "things", "think", "this", "those", "though", "through", "to", "too", + "under", "until", "up", "upon", "us", "very", "was", "wasn", "we", "well", + "were", "weren", "what", "when", "where", "whether", "which", "while", + "who", "whom", "why", "will", "with", "within", "without", "would", + "yes", "yet", "you", "your", "yours", "yourself", "yourselves", + "okay", "ok", "hi", "hello", "hey", "oh", "ah", "um", "uh", "hmm", + "mr", "mrs", "ms", "dr", "sir", "ma", "am", "captain", "princess", + "doctor", "aunt", "uncle", "nanny", "hunter", + "everyone", "everything", "someone", "something", "nothing", "anyone", + "anything", "people", "one", "two", "three", "four", "five", "six", + "seven", "eight", "nine", "ten", "eleven", "twelve", "first", "second", + "third", "fourth", "fifth", "next", "last", "inside", "outside", "based", + "turning", "checking", "looking", "going", "trying", "waiting", + "standing", "sitting", "walking", "talking", "several", "tell", + "additionally", "besides", "however", "therefore", "meanwhile", + "suddenly", "finally", "actually", "apparently", "obviously", + "unfortunately", "luckily", "instead", "as", "at", "if", "is", "it", + "its", "be", "by", "do", "does", "did", "for", "from", "had", "has", + "he", "her", "him", "his", "how", "into", "me", "my", "of", "on", "or", + "our", "she", "so", "than", "that", "the", "their", "them", "then", + "there", "these", "they", "this", "those", "to", "up", "us", "was", + "we", "were", "what", "when", "where", "which", "who", "why", "will", + "with", "would", "you", "your", +} + +SENTENCE_BOUNDARY = set('.!?…"“”\n') + + +def _load(path: Path) -> dict: + return json.loads(path.read_text(encoding="utf-8")) + + +def _glossary_keys(glossary: dict) -> set[str]: + keys: set[str] = set() + for section in ("characters", "roles", "places", "terms"): + for key in glossary.get(section, {}) or {}: + keys.add(key.lower()) + for extra in (glossary.get("title") or {}).values(): + if isinstance(extra, str): + keys.add(extra.lower()) + return keys + + +def _analyst_terms(run_dir: Path) -> dict[str, dict]: + out: dict[str, dict] = {} + for path in sorted(run_dir.glob("*/*.analysis.json")): + try: + data = _load(path) + except (OSError, json.JSONDecodeError): + continue + for term in data.get("proposed_terms", []) or []: + if not isinstance(term, dict): + continue + en = str(term.get("en") or "").strip().lower() + if not en: + continue + entry = out.setdefault(en, {"en": term.get("en"), "zh": term.get("zh"), "why": [], "count": 0}) + entry["count"] += 1 + why = term.get("why") + if why and why not in entry["why"]: + entry["why"].append(why) + if term.get("zh") and not entry.get("zh"): + entry["zh"] = term.get("zh") + return out + + +def _source_names(book: dict, only: set[str] | None) -> dict[str, dict]: + counts: dict[str, dict] = defaultdict(lambda: {"count": 0, "contexts": []}) + pattern = re.compile(r"\b([A-Z][a-zA-Z'’-]+(?:\s+[A-Z][a-zA-Z'’-]+){0,2})\b") + for chapter in book["chapters"]: + if only and chapter["id"] not in only: + continue + for p in chapter["paragraphs"]: + text = p.get("text") or "" + for m in pattern.finditer(text): + name = m.group(1).strip() + name = re.sub(r"[’']s$", "", name).strip("-'") + if not name or "’" in name or "'" in name: + continue + words = name.split() + if all(w.lower() in STOPWORDS for w in words): + continue + if len(words) == 1 and words[0].lower().endswith("ly"): + continue + first = words[0].lower() + if first in STOPWORDS and len(words) == 1: + continue + start = m.start() + prev = (text[:start].rstrip() or " ")[-1] + mid_sentence = prev not in SENTENCE_BOUNDARY + if not mid_sentence and len(words) == 1: + # A single capitalised word at the start of a sentence is + # too weak a signal; require at least one mid-sentence use. + counts[name]["_initial"] = counts[name].get("_initial", 0) + 1 + continue + counts[name]["count"] += 1 + if len(counts[name]["contexts"]) < 3: + lo = max(0, start - 40) + hi = min(len(text), m.end() + 40) + counts[name]["contexts"].append("…" + text[lo:hi].replace("\n", " ") + "…") + return {k: v for k, v in counts.items() if v["count"] > 0} + + +def main() -> None: + ap = argparse.ArgumentParser(description="Propose glossary entries from the source and analyses.") + ap.add_argument("--book", type=Path, required=True) + ap.add_argument("--run-dir", type=Path, required=True) + ap.add_argument("--glossary", type=Path, required=True) + ap.add_argument("--out", type=Path, required=True) + ap.add_argument("--chapters", help="comma-separated chapter ids (default: all)") + args = ap.parse_args() + + book = _load(args.book) + glossary = _load(args.glossary) + only = {c.strip() for c in args.chapters.split(",")} if args.chapters else None + known = _glossary_keys(glossary) + analyst = _analyst_terms(args.run_dir) + names = _source_names(book, only) + + lines = ["# Deal Hunter glossary candidates", "", "未自动写入 glossary.json；请人工批准。", ""] + lines.append("## 分析器提议的术语 (proposed_terms)") + lines.append("") + lines.append("| en | zh | 次数 | 说明 |") + lines.append("|---|---|---|---|") + for en, entry in sorted(analyst.items()): + if en in known: + continue + why = " / ".join(entry["why"])[:120] + lines.append(f"| {entry['en']} | {entry.get('zh') or ''} | {entry['count']} | {why} |") + lines.append("") + lines.append("## 源码中的专名候选 (mid-sentence capitalised)") + lines.append("") + lines.append("| name | 次数 | 上下文 |") + lines.append("|---|---|---|") + for name, entry in sorted(names.items(), key=lambda kv: (-kv[1]["count"], kv[0])): + low = name.lower() + if low in known or low in LATIN_ALLOW: + continue + ctx = "
".join(entry["contexts"][:2]) + lines.append(f"| {name} | {entry['count']} | {ctx} |") + lines.append("") + + args.out.parent.mkdir(parents=True, exist_ok=True) + args.out.write_text("\n".join(lines) + "\n", encoding="utf-8") + print(f"wrote {args.out}") + print(f" analyst proposals: {len(analyst)} source candidates: {len(names)}") + + +if __name__ == "__main__": + main() diff --git a/tools/translation/pipeline.py b/tools/translation/pipeline.py new file mode 100644 index 0000000..e74cbc1 --- /dev/null +++ b/tools/translation/pipeline.py @@ -0,0 +1,704 @@ +# Copyright 2026 Umbra Tools (umbratools.com) +# SPDX-License-Identifier: Apache-2.0 + +"""Multi-pass translation orchestration. + +Pass 0 is local: paragraphs are grouped and chunked with source indices kept. +Pass 1 analyses intent, Pass 2 drafts, Pass 4 checks meaning with a different +model. Pass 3 (chapter voice consistency) is optional and runs after the +chapter's chunks are drafted. + +Every artefact is written to disk and reused on the next run, so a multi-hour +book run can be stopped and resumed. +""" + +from __future__ import annotations + +import argparse +import json +import re +import sys +import time +from dataclasses import dataclass +from pathlib import Path + +sys.path.insert(0, str(Path(__file__).resolve().parent)) + +from llm import DEFAULT_ANALYST, DEFAULT_CRITIC, DEFAULT_MODEL, LLM, LLMError, Usage # noqa: E402 +from prompts import ( # noqa: E402 + SYSTEM_ANALYST, + SYSTEM_CRITIC, + SYSTEM_DRAFTER, + analysis_prompt, + consistency_prompt, + critic_prompt, + draft_prompt, + load_glossary, +) + +CHARACTER_NOTES = """- 凯因达：聪明、直率、好奇，带一点叛逆；对权威不耐烦，对朋友温暖。不要把她写成娇滴滴的公主。 +- Hunter / Deal Hunter：干练、简洁、有冷幽默；是船，也是生意人；不用敬语堆砌。 +- 南妮：温和、认真、字面化，偶尔因为太认真而好笑；不要写成卖萌。 +- 维多利亚公爵夫人：端庄、克制、关切；对凯因达既有长辈的威严，也有家人的柔软。 +- 罗曼诺：热情、商人腔、爱谈价格，但不油滑。 +- 旁白：克制、清晰、有画面感；不替人物抒情。""" + +# Proper nouns that may legitimately stay in Latin script. Anything else that +# survives in the Chinese text is reported, not silently accepted. +LATIN_ALLOW = { + "ai", "audio", "authorbdmurphy", "b", "blazes", "brutus", "com", "d", + "deal", "dial", "eva", "felucca", "gisfrid", "goodreads", "hunter", + "http", "https", "isbn", "kabir", "kainda", "konrad", "llc", "l2", + "maul", "murphy", "nanny", "niñera", "romano", "sao", "umbra", "vince", + "wakam", "wake", "weston", "willamina", "witold", "worstauthors", + "worstauthor", "www", "zheng", "yi", "adonai", "adric", "albert", +} + + +def _normalise_quotes(text: str) -> str: + text = re.sub(r'"([^"\n]*)"', r"“\1”", text) + if '"' in text: + parts = text.split('"') + out = parts[0] + for i, part in enumerate(parts[1:]): + out += ("“" if i % 2 == 0 else "”") + part + text = out + if "“" not in text and text.endswith("”"): + text = text[:-1] + opens, closes = text.count("“"), text.count("”") + if opens > closes: + text += "”" * (opens - closes) + elif closes > opens: + for _ in range(closes - opens): + idx = text.rfind("”") + if idx > text.rfind("“"): + text = text[:idx] + text[idx + 1 :] + else: + break + return text + + +def sanitize(text: str) -> str: + text = text.strip() + text = re.sub(r"^\[[^\]]{1,40}\]\s*", "", text) + text = re.sub(r"^（(?:对话|旁白|叙述)[^）]{0,60}）\s*", "", text) + text = re.sub(r"^\((?:dialogue|narration)[^)]{0,60}\)\s*", "", text, flags=re.I) + text = text.replace("《交易猎人》", "《Deal Hunter》") + text = text.replace("交易猎人", "Deal Hunter") + text = text.replace("“猎人”", "Hunter") + text = text.replace("猎人号", "Deal Hunter") + text = text.replace("亨特", "Hunter") + text = text.replace("B.D. 墨菲", "B·D·墨菲") + text = text.replace("B. D. 墨菲", "B·D·墨菲") + text = text.replace("B.D.墨菲", "B·D·墨菲") + text = text.replace("郑义少", "郑一嫂") + text = text.replace("乌姆布拉音频系统", "Umbra Audio") + text = text.replace("乌姆布拉音频", "Umbra Audio") + text = text.replace("Umbra Audio 星系系统", "Umbra Audio 系统") + text = text.replace("Umbra Audio星系系统", "Umbra Audio 系统") + text = text.replace("音频星系系统", "音频系统") + text = text.replace("在小时前", "一小时前") + text = text.replace("...", "……").replace("--", "——") + return _normalise_quotes(text).strip() + + +def latin_residue(text: str) -> tuple[list[str], list[str]]: + words = re.findall(r"[A-Za-zÀ-ÿ][A-Za-zÀ-ÿ'’\-]{1,}", text) + common: set[str] = set() + proper: set[str] = set() + for word in words: + base = word.strip("'’-") + if not base or base.lower() in LATIN_ALLOW: + continue + if base.isupper() and len(base) <= 5: + continue + if word[0].isupper(): + proper.add(word) + else: + common.add(word) + return sorted(common), sorted(proper) + + +@dataclass +class Chunk: + id: str + scene_index: int + chunks_in_scene: int + paragraphs: list[dict] + + @property + def words(self) -> int: + return sum(len(p.get("text", "").split()) for p in self.paragraphs) + + +def load_book(path: Path) -> dict: + return json.loads(path.read_text(encoding="utf-8")) + + +def select_chapters(book: dict, selector: str) -> list[dict]: + chapters = book["chapters"] + if selector in {"all", "*"}: + return chapters + if selector == "front": + return [c for c in chapters if c["matter"].startswith("front")] + if selector == "body": + return [c for c in chapters if c["matter"] == "body"] + if selector == "back": + return [c for c in chapters if c["matter"] == "back"] + wanted = {s.strip() for s in selector.split(",") if s.strip()} + out = [] + for c in chapters: + if str(c["index"]) in wanted or c["id"] in wanted or c["title"] in wanted: + out.append(c) + missing = wanted - {str(c["index"]) for c in out} - {c["id"] for c in out} - {c["title"] for c in out} + if missing: + raise SystemExit(f"chapter selector did not match: {sorted(missing)}") + return out + + +def chunk_chapter(chapter: dict, max_words: int = 260, max_paras: int = 14) -> list[Chunk]: + paras = chapter["paragraphs"] + # Assign scene numbers. + scene_of: dict[str, int] = {} + scene = 0 + for p in paras: + if p.get("type") == "scene_break": + scene += 1 + continue + scene_of[p["id"]] = scene + + raw: list[list[dict]] = [] + current: list[dict] = [] + current_words = 0 + current_scene = None + for p in paras: + if p.get("type") == "scene_break": + if current: + raw.append(current) + current, current_words, current_scene = [], 0, None + continue + p_scene = scene_of.get(p["id"], 0) + if current and (p_scene != current_scene or current_words + len(p["text"].split()) > max_words or len(current) >= max_paras): + raw.append(current) + current, current_words = [], 0 + current.append(p) + current_words += len(p["text"].split()) + current_scene = p_scene + if current: + raw.append(current) + + # Count chunks per scene. + per_scene: dict[int, int] = {} + for group in raw: + s = scene_of.get(group[0]["id"], 0) + per_scene[s] = per_scene.get(s, 0) + 1 + + chunks: list[Chunk] = [] + seen: dict[int, int] = {} + for i, group in enumerate(raw): + s = scene_of.get(group[0]["id"], 0) + seen[s] = seen.get(s, 0) + 1 + chunks.append( + Chunk( + id=f"{chapter['id']}-c{i:03d}", + scene_index=s, + chunks_in_scene=per_scene[s], + paragraphs=group, + ) + ) + return chunks + + +class Runner: + def __init__( + self, + *, + book_path: Path, + glossary_path: Path, + run_dir: Path, + model: str = DEFAULT_MODEL, + analyst_model: str = DEFAULT_ANALYST, + critic_model: str = DEFAULT_CRITIC, + base_url: str | None = None, + force: bool = False, + critic: bool = True, + consistency: bool = False, + max_words: int = 260, + max_paras: int = 14, + critic_limit: int = 0, + ) -> None: + self.book = load_book(book_path) + self.glossary = load_glossary(glossary_path) + self.run_dir = run_dir + self.run_dir.mkdir(parents=True, exist_ok=True) + self.model = model + self.analyst_model = analyst_model + self.critic_model = critic_model + self.force = force + self.do_critic = critic + self.do_consistency = consistency + self.max_words = max_words + self.max_paras = max_paras + self.critic_limit = critic_limit + kwargs = {} + if base_url: + kwargs["base_url"] = base_url + self.llm = LLM( + cache_dir=run_dir / "cache", + usage_log=run_dir / "usage.jsonl", + **kwargs, + ) + + # ----------------------------------------------------------------- files + def chapter_dir(self, chapter: dict) -> Path: + d = self.run_dir / chapter["id"] + d.mkdir(parents=True, exist_ok=True) + return d + + def _read(self, path: Path) -> dict | None: + if path.exists() and not self.force: + return json.loads(path.read_text(encoding="utf-8")) + return None + + def _write(self, path: Path, value: object) -> None: + path.write_text(json.dumps(value, ensure_ascii=False, indent=2) + "\n", encoding="utf-8") + + # ----------------------------------------------------------------- passes + def analyse(self, chapter: dict, chunk: Chunk, previous_summary: str) -> dict: + path = self.chapter_dir(chapter) / f"{chunk.id}.analysis.json" + cached = self._read(path) + if cached is not None: + return cached + prompt = analysis_prompt( + chapter_title=f"{chapter['title']} / {chapter['title_zh']}", + scene_index=chunk.scene_index, + chunk_index=int(chunk.id.rsplit("c", 1)[-1]), + chunks_in_scene=chunk.chunks_in_scene, + paragraphs=chunk.paragraphs, + previous_summary=previous_summary, + glossary=self.glossary, + ) + value, usage = self.llm.chat_json( + SYSTEM_ANALYST, + prompt, + model=self.analyst_model, + temperature=0.2, + max_tokens=1400, + cache_key=f"{chunk.id}.analysis.{self.analyst_model}", + # The small analyst model rejects LM Studio's JSON-schema grammar; + # its answer is parsed from the text instead. + json_mode=False, + ) + if isinstance(value, list) and value and isinstance(value[0], dict): + value = value[0] + # Sometimes wrapped as {"analysis": {...}} or similar + if isinstance(value, dict): + for k in ("analysis", "result", "data"): + if isinstance(value.get(k), dict): + value = value[k] + break + if not isinstance(value, dict): + # The analyst is a small model asked for a JSON object, and it + # sometimes answers with a list or a bare string instead. Analysis is + # guidance for the drafter, not a requirement for a valid translation, + # so a malformed answer must not abort a chunk that could otherwise be + # translated. Record that the analysis is degraded and continue with a + # minimal one; the drafter falls back to the source and glossary. + print( + f" {chunk.id}: analyst returned {type(value).__name__}, " + "using a minimal analysis", + flush=True, + ) + value = {"degraded": True, "note": "analyst returned non-object JSON"} + value["_usage"] = usage.__dict__ + self._write(path, value) + return value + + def draft(self, chapter: dict, chunk: Chunk, analysis: dict, previous_summary: str) -> dict: + path = self.chapter_dir(chapter) / f"{chunk.id}.draft.json" + cached = self._read(path) + if cached is not None: + return cached + expected = [p["id"] for p in chunk.paragraphs if p.get("type") != "scene_break"] + prompt = draft_prompt( + chapter_title=f"{chapter['title']} / {chapter['title_zh']}", + scene_index=chunk.scene_index, + chunk_index=int(chunk.id.rsplit("c", 1)[-1]), + chunks_in_scene=chunk.chunks_in_scene, + paragraphs=chunk.paragraphs, + analysis={k: v for k, v in analysis.items() if k != "_usage"}, + previous_summary=previous_summary, + glossary=self.glossary, + ) + value, usage = self.llm.chat_json( + SYSTEM_DRAFTER, + prompt, + model=self.model, + temperature=0.45, + max_tokens=6000, + cache_key=f"{chunk.id}.draft", + ) + out = self._normalise_draft(value, expected) + missing = [i for i in expected if not out.get(i)] + if missing: + out.update(self._draft_missing(chapter, chunk, analysis, missing)) + out = self._repair_residue(chapter, chunk, out) + out = self._fix_paragraph_quotes(chunk, out) + warnings: dict[str, dict[str, list[str]]] = {} + for pid, zh in out.items(): + common, proper = latin_residue(zh) + if common or proper: + warnings[pid] = {"common_english": common, "proper_names": proper} + result = { + "paragraphs": out, + "analysis": analysis, + "warnings": warnings, + "_usage": usage.__dict__, + } + self._write(path, result) + return result + + def _normalise_draft(self, value: object, expected: list[str]) -> dict[str, str]: + out: dict[str, str] = {} + if isinstance(value, dict) and isinstance(value.get("paragraphs"), list): + items = value["paragraphs"] + elif isinstance(value, list): + items = value + elif isinstance(value, dict) and value.get("id") and value.get("zh"): + items = [value] + elif isinstance(value, dict): + # maybe mapping id->zh + for k, v in value.items(): + if isinstance(v, str) and k.startswith("ch-"): + out[k] = sanitize(v) + return {k: v for k, v in out.items() if k in expected and v} + else: + raise LLMError(f"drafter returned unexpected JSON: {str(value)[:200]}") + for item in items: + if isinstance(item, dict) and item.get("id") and item.get("zh"): + out[str(item["id"])] = sanitize(str(item["zh"])) + return {k: v for k, v in out.items() if k in expected and v} + + def _fix_paragraph_quotes(self, chunk: Chunk, drafts: dict[str, str]) -> dict[str, str]: + """Remove quotes the model wrapped around unquoted narration.""" + for p in chunk.paragraphs: + zh = drafts.get(p["id"]) + source = (p.get("text") or "").strip() + if ( + zh + and p.get("type") == "narration" + and not source.startswith("“") + and zh.startswith("“") + and zh.endswith("”") + and zh.count("“") == 1 + and zh.count("”") == 1 + ): + drafts[p["id"]] = zh[1:-1].strip() + return drafts + + def _repair_residue(self, chapter: dict, chunk: Chunk, drafts: dict[str, str]) -> dict[str, str]: + """Translate stray lowercase English words in place, without touching names.""" + by_id = {p["id"]: p for p in chunk.paragraphs} + for pid, zh in list(drafts.items()): + common, _proper = latin_residue(zh) + if not common: + continue + source = by_id.get(pid, {}).get("text", "") + prompt = f"""下面这段中文译文里混入了英文单词：{", ".join(common)}。 +请只把这些英文单词译成中文，其余内容尽量保持不变；不要解释，不要改写剧情。 +输出 JSON：{{"zh": "修正后的中文译文"}} + +英文原文： +{source} + +当前译文： +{zh}""" + try: + value, _ = self.llm.chat_json( + SYSTEM_DRAFTER, + prompt, + model=self.model, + temperature=0.2, + max_tokens=1500, + cache_key=f"{chunk.id}.{pid}.residue", + ) + fixed = value.get("zh") if isinstance(value, dict) else None + if fixed: + drafts[pid] = sanitize(str(fixed)) + except LLMError as exc: + print(f" {chunk.id}/{pid}: residue repair failed: {exc}", flush=True) + return drafts + + def _draft_missing(self, chapter: dict, chunk: Chunk, analysis: dict, missing: list[str]) -> dict[str, str]: + """Last-resort one-paragraph calls. Keeps a chunk failure from losing the run.""" + result: dict[str, str] = {} + by_id = {p["id"]: p for p in chunk.paragraphs} + for pid in missing: + p = by_id[pid] + try: + prompt = draft_prompt( + chapter_title=f"{chapter['title']} / {chapter['title_zh']}", + scene_index=chunk.scene_index, + chunk_index=0, + chunks_in_scene=1, + paragraphs=[p], + analysis=analysis, + previous_summary="", + glossary=self.glossary, + ) + value, _ = self.llm.chat_json( + SYSTEM_DRAFTER, + prompt, + model=self.model, + temperature=0.4, + max_tokens=1500, + cache_key=f"{chunk.id}.{pid}.draft", + ) + one = self._normalise_draft(value, [pid]) + if one.get(pid): + result[pid] = one[pid] + continue + except Exception: + pass + # Fallback to source text or empty to avoid blocking + result[pid] = sanitize(str(p.get("text", ""))) + return result + + def criticise(self, chapter: dict, chunk: Chunk, draft: dict[str, str]) -> dict: + path = self.chapter_dir(chapter) / f"{chunk.id}.critic.json" + cached = self._read(path) + if cached is not None: + return cached + prompt = critic_prompt( + chapter_title=f"{chapter['title']} / {chapter['title_zh']}", + paragraphs=chunk.paragraphs, + draft=draft, + glossary=self.glossary, + ) + try: + value, usage = self.llm.chat_json( + SYSTEM_CRITIC, + prompt, + model=self.critic_model, + temperature=0.15, + max_tokens=6000, + cache_key=f"{chunk.id}.critic.{self.critic_model}", + ) + except LLMError as exc: + # A reasoning-only critic produces empty content. Fall back to the + # drafter model rather than silently skipping the check. + print(f" critic {self.critic_model} failed ({exc}); falling back to {self.model}", flush=True) + try: + value, usage = self.llm.chat_json( + SYSTEM_CRITIC, + prompt, + model=self.model, + temperature=0.15, + max_tokens=6000, + cache_key=f"{chunk.id}.critic.{self.model}", + ) + except LLMError as exc2: + # Do not abort a multi-hour run because one critique failed. + print(f" critic fallback failed ({exc2}); recording a skipped critique", flush=True) + value = {"issues": [], "verdict": "skipped", "_error": str(exc2)} + usage = Usage(model=self.model, prompt_tokens=0, completion_tokens=0, elapsed_s=0.0) + if isinstance(value, list): + value = {"issues": value, "verdict": "revise" if value else "pass"} + if not isinstance(value, dict): + raise LLMError(f"{chunk.id}: critic returned {type(value).__name__}") + # DeepSeek sometimes invents its own key names. Normalise the common + # aliases so the report and the review queue stay machine-readable. + normalised = [] + confirmations = ( + "被正确翻译", + "翻译正确", + "已正确", + "无需修改", + "没有问题", + "不存在问题", + "未改变理解", + "英文原文存在", + "原文存在语法错误", + "建议英文", + "英文原文表达", + "原文错误", + "无需修改中文", + ) + for raw in value.get("issues", []) or []: + if not isinstance(raw, dict): + continue + problem_text = str(raw.get("problem") or raw.get("explanation") or "") + if any(marker in problem_text for marker in confirmations): + continue + fix_text = str(raw.get("fix") or raw.get("suggestion") or "").strip() + if not fix_text and str(raw.get("severity") or "major").lower() == "minor": + continue + normalised.append( + { + "id": raw.get("id") or raw.get("paragraph_id") or raw.get("sentence_id") or "", + "severity": raw.get("severity") or "major", + "type": raw.get("type") or "meaning", + "problem": raw.get("problem") or raw.get("explanation") or "", + "source_excerpt": raw.get("source_excerpt") or raw.get("english_sentence_part") or "", + "target_excerpt": raw.get("target_excerpt") or raw.get("chinese_sentence_part") or "", + "fix": raw.get("fix") or raw.get("suggestion") or "", + } + ) + value["issues"] = normalised + verdict = str(value.get("verdict", "")).lower() + if verdict == "skipped": + value["verdict"] = "skipped" + elif normalised or verdict in {"fail", "revise"}: + value["verdict"] = "revise" + else: + value["verdict"] = "pass" + value["_usage"] = usage.__dict__ + self._write(path, value) + return value + + def consistency(self, chapter: dict, drafts: dict[str, str], paragraphs: list[dict]) -> dict: + path = self.chapter_dir(chapter) / "consistency.json" + cached = self._read(path) + if cached is not None: + return cached + prompt = consistency_prompt( + chapter_title=f"{chapter['title']} / {chapter['title_zh']}", + paragraphs=paragraphs, + draft=drafts, + glossary=self.glossary, + character_notes=CHARACTER_NOTES, + ) + value, usage = self.llm.chat_json( + SYSTEM_CRITIC, + prompt, + model=self.critic_model, + temperature=0.15, + max_tokens=6000, + cache_key=f"{chapter['id']}.consistency", + ) + if isinstance(value, list): + value = {"issues": value} + if not isinstance(value, dict): + raise LLMError(f"{chapter['id']}: consistency returned {type(value).__name__}") + value["_usage"] = usage.__dict__ + self._write(path, value) + return value + + # ----------------------------------------------------------------- driver + def run(self, chapters: list[dict]) -> None: + started = time.time() + for ci, chapter in enumerate(chapters, 1): + chunks = chunk_chapter(chapter, self.max_words, self.max_paras) + print( + f"[{ci}/{len(chapters)}] {chapter['id']} {chapter['title']} / {chapter['title_zh']}: " + f"{len(chapter['paragraphs'])} paragraphs, {len(chunks)} chunks", + flush=True, + ) + previous_summary = "" + drafts: dict[str, str] = {} + issues: list[dict] = [] + for chunk in chunks: + analysis = self.analyse(chapter, chunk, previous_summary) + previous_summary = str(analysis.get("summary_zh") or previous_summary) + drafted = self.draft(chapter, chunk, analysis, previous_summary) + drafts.update(drafted["paragraphs"]) + if self.do_critic and ( + self.critic_limit <= 0 or int(chunk.id.rsplit("c", 1)[-1]) < self.critic_limit + ): + crit = self.criticise(chapter, chunk, drafted["paragraphs"]) + for issue in crit.get("issues", []) or []: + issue = dict(issue) + issue["chunk"] = chunk.id + issues.append(issue) + print( + f" {chunk.id}: {len(drafted['paragraphs'])}/{len([p for p in chunk.paragraphs if p.get('type') != 'scene_break'])} " + f"paras, {len(issues)} issue(s) so far", + flush=True, + ) + if self.do_consistency: + cons = self.consistency(chapter, drafts, chapter["paragraphs"]) + for issue in cons.get("issues", []) or []: + issue = dict(issue) + issue["type"] = "consistency:" + str(issue.get("type")) + issues.append(issue) + summary = { + "chapter_id": chapter["id"], + "chapter_index": chapter["index"], + "title": chapter["title"], + "title_zh": chapter["title_zh"], + "paragraphs": drafts, + "issues": issues, + "chunks": [c.id for c in chunks], + } + self._write(self.chapter_dir(chapter) / "translation.json", summary) + print(f" wrote {self.chapter_dir(chapter) / 'translation.json'}", flush=True) + print(f"done in {time.time() - started:.0f}s", flush=True) + + +def report(run_dir: Path) -> str: + lines = [] + total_paras = 0 + total_issues = 0 + for d in sorted(p for p in run_dir.iterdir() if p.is_dir() and p.name.startswith("ch-")): + t = d / "translation.json" + if not t.exists(): + lines.append(f"{d.name}: not started") + continue + data = json.loads(t.read_text(encoding="utf-8")) + n = len(data.get("paragraphs", {})) + issues = data.get("issues", []) + total_paras += n + total_issues += len(issues) + by_sev: dict[str, int] = {} + for i in issues: + by_sev[i.get("severity", "?")] = by_sev.get(i.get("severity", "?"), 0) + 1 + lines.append(f"{d.name}: {n} paragraphs, {len(issues)} issues {by_sev}") + lines.append(f"TOTAL: {total_paras} paragraphs, {total_issues} issues") + return "\n".join(lines) + + +def main() -> None: + ap = argparse.ArgumentParser(description="Run the Deal Hunter Chinese translation pipeline.") + ap.add_argument("--book", type=Path, required=True) + ap.add_argument("--glossary", type=Path, required=True) + ap.add_argument("--run-dir", type=Path, required=True) + ap.add_argument("--chapters", default="all", help="all, front, body, back, or comma-separated indices/ids") + ap.add_argument("--model", default=DEFAULT_MODEL) + ap.add_argument("--analyst-model", default=DEFAULT_ANALYST) + ap.add_argument("--critic-model", default=DEFAULT_CRITIC) + ap.add_argument("--base-url", default=None) + ap.add_argument("--force", action="store_true") + ap.add_argument("--no-critic", action="store_true") + ap.add_argument("--consistency", action="store_true") + ap.add_argument("--max-words", type=int, default=260) + ap.add_argument("--max-paras", type=int, default=14) + ap.add_argument("--critic-limit", type=int, default=0, help="criticise only the first N chunks per chapter (0 = all)") + ap.add_argument("--report", action="store_true") + args = ap.parse_args() + + if args.report: + print(report(args.run_dir)) + return + + book = load_book(args.book) + chapters = select_chapters(book, args.chapters) + runner = Runner( + book_path=args.book, + glossary_path=args.glossary, + run_dir=args.run_dir, + model=args.model, + analyst_model=args.analyst_model, + critic_model=args.critic_model, + base_url=args.base_url, + force=args.force, + critic=not args.no_critic, + consistency=args.consistency, + max_words=args.max_words, + max_paras=args.max_paras, + critic_limit=args.critic_limit, + ) + runner.run(chapters) + + +if __name__ == "__main__": + main() diff --git a/tools/translation/prompts.py b/tools/translation/prompts.py new file mode 100644 index 0000000..93b35b6 --- /dev/null +++ b/tools/translation/prompts.py @@ -0,0 +1,350 @@ +# Copyright 2026 Umbra Tools (umbratools.com) +# SPDX-License-Identifier: Apache-2.0 + +"""Prompt construction for the multi-pass translation pipeline. + +The prompts follow specs/27: analyse the scene before rendering it, give the +drafter the intent and the glossary, then check meaning with a different +model. They deliberately do not ask for word-for-word equivalence. +""" + +from __future__ import annotations + +import json +from pathlib import Path + +CORE_CHARACTERS = [ + "Kainda Walters Foster Murphy", + "Princess Kainda", + "Duchess Victoria", + "Aunt Vicky", + "Deal Hunter", + "Hunter", + "Nanny", + "Trader Romano", + "Romano", + "B. D. Murphy", + "Umbra Audio", +] + +CORE_TERMS = [ + "star system", + "sector", + "space station", + "salvage", + "bounty", + "bounty hunter", + "jump", + "wake cycle", + "frigate", + "shuttle", + "bridge", + "engineering", + "cargo bay", + "dock", + "XO", + "captain", + "pirate", + "fugitive", + "coronation", +] + + +def load_glossary(path: Path) -> dict: + return json.loads(path.read_text(encoding="utf-8")) + + +def _pick(mapping: dict, keys: list[str]) -> list[tuple[str, str]]: + out: list[tuple[str, str]] = [] + for key in keys: + if key in mapping: + value = mapping[key] + if isinstance(value, dict): + zh = value.get("zh") or value.get("short") or "" + note = value.get("note") + if note: + zh = f"{zh}（{note}）" + else: + zh = str(value) + if zh: + out.append((key, zh)) + return out + + +def glossary_block(glossary: dict, speakers: set[str], text: str) -> str: + chars = dict(glossary.get("characters", {})) + roles = dict(glossary.get("roles", {})) + places = dict(glossary.get("places", {})) + terms = dict(glossary.get("terms", {})) + + keys = list(CORE_CHARACTERS) + for s in sorted(speakers): + if s and s not in keys: + keys.append(s) + char_pairs = _pick({**chars, **roles}, keys) + + term_keys = list(CORE_TERMS) + low = text.lower() + for key in terms: + if key.lower() in low and key not in term_keys: + term_keys.append(key) + term_pairs = _pick(terms, term_keys) + place_pairs = _pick(places, list(places)) + + title = glossary.get("title", {}) + lines = ["【书名与专名规则】"] + if title: + en = title.get("en", "Deal Hunter") + sub = title.get("subtitle_zh", "") + lines.append(f"- 书名：{en}；正文书名写作《{en}》" + (f"，副标题可写“{sub}”" if sub else "")) + lines += [ + "- Deal Hunter 作为船名、AI 名或书名时保留英文，不译成中文；不要写成《交易猎人》", + "- Hunter 作为 AI 的简称保留英文，不译成“猎人”", + "- Umbra Audio 保留英文，不音译", + "- 只翻译原文出现的称谓；不要补全人物全名、头衔或身份", + "", + "【硬性术语：违反即为错译】", + "- 恒星系 system / star system = 星系；但 ship systems / audio system / computer system = 系统", + "- sector = 星区", + "- space station = 空间站；trade station = 贸易站", + "- jump = 跃迁（不译作“跳跃”）", + "- salvage = 打捞物 / 打捞（不译作“救援”）", + "- bounty = 赏金；bounty hunter = 赏金猎人", + "- XO = 副舰长；captain = 舰长", + "", + "【人物与称谓】", + ] + lines += [f"- {en} = {zh}" for en, zh in char_pairs] + if place_pairs: + lines.append("【地名与舰名】") + lines += [f"- {en} = {zh}" for en, zh in place_pairs] + if term_pairs: + lines.append("【术语】") + lines += [f"- {en} = {zh}" for en, zh in term_pairs] + return "\n".join(lines) + + +def style_block(glossary: dict) -> str: + style = glossary.get("style", {}) + return "\n".join(f"- {v}" for v in style.values() if isinstance(v, str)) + + +SYSTEM_ANALYST = """你是一位双语文学编辑，负责在翻译前分析英文小说的场景。 +你的任务不是翻译，而是读出这一段文字的意图、语气、人物关系和可能被直译毁掉的地方。 +只输出 JSON，不要解释，不要 Markdown 代码围栏。""" + +SYSTEM_DRAFTER = """你是一位资深文学译者，把英文太空冒险小说译成简体中文。 +你的目标不是逐字对应，而是让中文读者读到和英文读者相同的场景、语气、人物关系和潜台词。 +可以重组句子、合并或拆分分句，但不得增删事实、不得替作者解释、不得加译注。 +对话要像中文人物在说话；叙述要流畅、克制、有画面感。 +严格使用给出的术语表。未列入术语表的人名、地名、舰名保留英文原文，不要自行音译。 +不要补全原文没有出现的全名、头衔或身份；原文说“Princess Kainda”，不要写成“凯因达·沃尔特斯·福斯特·墨菲公主”。 +Deal Hunter 作为船名、AI 名或书名保留英文；Hunter 作为简称保留英文；Umbra Audio 保留英文。 +只输出 JSON，不要解释，不要 Markdown 代码围栏。""" + +SYSTEM_CRITIC = """你是一位严格的双语审校。对照英文原文和中文译文，只找意义、指代、语气、术语和遗漏方面的问题。 +不要因为“风格不同”而报错；只有当读者会因此理解错、或人物声音被抹平、或事实被改变时才报。 +不要改写全文，只报告问题并给出最小修正建议。 +只输出 JSON，不要解释，不要 Markdown 代码围栏。""" + + +def _paragraph_lines(paragraphs: list[dict]) -> str: + lines = [] + for p in paragraphs: + if p.get("type") == "scene_break": + lines.append(f'[{p["id"]}] (场景分隔)') + continue + speaker = p.get("speaker") or "narrator" + ptype = p.get("type") or "narration" + lines.append(f'[{p["id"]}] ({ptype}; {speaker}) {p["text"]}') + return "\n".join(lines) + + +def _draft_paragraph_lines(paragraphs: list[dict]) -> str: + """Source lines for the drafter: id + text only. + + Speaker/type labels are useful to the analyst but the drafter used to copy + them into the Chinese output ("（对话；凯因达）"). The analysis already + carries that information, so the draft prompt keeps the source clean. + """ + lines = [] + for p in paragraphs: + if p.get("type") == "scene_break": + continue + lines.append(f'[{p["id"]}] {p["text"]}') + return "\n".join(lines) + + +def analysis_prompt( + *, + chapter_title: str, + scene_index: int, + chunk_index: int, + chunks_in_scene: int, + paragraphs: list[dict], + previous_summary: str, + glossary: dict, +) -> str: + speakers = {p.get("speaker") or "narrator" for p in paragraphs} + text = " ".join(p.get("text", "") for p in paragraphs) + return f"""书名：Deal Hunter（交易猎人） +本章：{chapter_title} +场景 {scene_index + 1}，第 {chunk_index + 1}/{chunks_in_scene} 段 +上一段场景摘要：{previous_summary or "（无，本章开头）"} + +{glossary_block(glossary, speakers, text)} + +{style_block(glossary)} + +请分析下面这段英文。输出 JSON： +{{ + "summary_zh": "1-2句中文（80字以内），概括本段发生了什么、人物想要什么、语气如何；供下一段翻译参考", + "register": "正式/中性/口语（15字以内）", + "tone": "例如：紧张、戏谑、疲惫、克制（15字以内）", + "participants": ["本段实际说话或被点名的人物"], + "proposed_terms": [{{"en": "术语表里没有的专名", "zh": "建议译名", "why": "简短理由"}}], + "risks": [{{"id": "段落id", "issue": "直译会错在哪里", "guidance": "译者应注意什么"}}] +}} + +要求：risks 最多 4 条，只报真正会被直译毁掉的地方；没有就返回空数组。 +proposed_terms 最多 5 条，不要重复术语表已有的词条。 + +英文原文： +{_paragraph_lines(paragraphs)}""" + + +def draft_prompt( + *, + chapter_title: str, + scene_index: int, + chunk_index: int, + chunks_in_scene: int, + paragraphs: list[dict], + analysis: dict, + previous_summary: str, + glossary: dict, +) -> str: + speakers = {p.get("speaker") or "narrator" for p in paragraphs} + text = " ".join(p.get("text", "") for p in paragraphs) + analysis_text = json.dumps(analysis, ensure_ascii=False, indent=2) + return f"""书名：Deal Hunter（交易猎人） +本章：{chapter_title} +场景 {scene_index + 1}，第 {chunk_index + 1}/{chunks_in_scene} 段 +上一段场景摘要：{previous_summary or "（无，本章开头）"} + +{glossary_block(glossary, speakers, text)} + +{style_block(glossary)} + +翻译前的场景分析（必须遵守其中的语气、人物关系和风险提示）： +{analysis_text} + +请把下面英文译为简体中文。输出 JSON： +{{"paragraphs": [{{"id": "段落id", "zh": "中文译文"}}]}} + +规则： +- 段落 id 必须与原文一一对应，数量和顺序不变。 +- 每个 id 只输出一段中文；不要把两段合并成一个 id。 +- 只输出译文。绝对不要把段落 id、说话人、类型、括号注释写进译文。 +- 不要输出英文原文，不要输出解释。 +- 不要把英文单词留在译文里（术语表允许保留的专名和书名除外）；每个英文实词都必须译成中文。 +- 对话用中文“”，并保证每个引号成对；不要把英文直引号 " 原样搬进译文。叙述中不要滥用引号。 +- 如果原文有省略号、破折号、打断，保留其节奏；省略号用……，破折号用——。 +- 专名严格按术语表；术语表没有的名字保留英文。 +- 指恒星系时，system / star system 译作“星系”（如 Albert system = 阿尔伯特星系；236 systems = 236 个星系）。 +- 指船载、音频、计算机等设备时，system 译作“系统”（如 ship systems = 飞船系统；Umbra Audio system = Umbra Audio 系统）。不要一律译作“星系”。 + +英文原文： +{_draft_paragraph_lines(paragraphs)}""" + + +def critic_prompt( + *, + chapter_title: str, + paragraphs: list[dict], + draft: dict[str, str], + glossary: dict, +) -> str: + speakers = {p.get("speaker") or "narrator" for p in paragraphs} + text = " ".join(p.get("text", "") for p in paragraphs) + pairs = [] + for p in paragraphs: + if p.get("type") == "scene_break": + continue + pairs.append( + { + "id": p["id"], + "speaker": p.get("speaker"), + "type": p.get("type"), + "en": p["text"], + "zh": draft.get(p["id"], ""), + } + ) + return f"""书名：Deal Hunter（交易猎人） +本章：{chapter_title} + +{glossary_block(glossary, speakers, text)} + +请逐段对照英文与中文，只报告会让读者理解错、或改变人物声音、或丢失/增加事实的问题。 + +注意：原文 “236 systems in the Dial space sector” 中的 systems 指恒星系，译作“星系”是正确的，不要改成“系统”。ship systems / audio system 才译作“系统”。 + +输出 JSON： +{{ + "verdict": "pass" 或 "revise", + "issues": [ + {{ + "id": "段落id", + "severity": "block|major|minor", + "type": "meaning|pronoun|irony|term|omission|addition|register", + "problem": "中文说明问题", + "source_excerpt": "英文片段（不超过30词）", + "target_excerpt": "中文片段（不超过40字）", + "fix": "最小修正建议，不要重译整段" + }} + ] +}} + +严重度标准： +- block：事实被改变、整句遗漏或增加、说话人/指代错、关键术语错。 +- major：读者会明显理解错、反讽或人物声音被抹平、重要细节丢失。 +- minor：细微语气或可选润色；不要为了凑数报风格偏好。 + +id 必须使用输入中的段落 id（例如 ch-6-p0000），不要用句子序号。 +键名必须与上面的 JSON 完全一致，不要自创 english_sentence_part 之类的字段。 +没有问题时 issues 为空数组，verdict 为 pass。 + +对照材料： +{json.dumps(pairs, ensure_ascii=False, indent=2)}""" + + +def consistency_prompt( + *, + chapter_title: str, + paragraphs: list[dict], + draft: dict[str, str], + glossary: dict, + character_notes: str, +) -> str: + speakers = {p.get("speaker") or "narrator" for p in paragraphs} + text = " ".join(p.get("text", "") for p in paragraphs) + pairs = [ + {"id": p["id"], "speaker": p.get("speaker"), "zh": draft.get(p["id"], "")} + for p in paragraphs + if p.get("type") != "scene_break" + ] + return f"""书名：Deal Hunter（交易猎人） +本章：{chapter_title} + +{glossary_block(glossary, speakers, text)} + +人物声音约束： +{character_notes} + +请检查整章的译名、称谓、术语和人物声音是否一致。只报告需要统一的问题，不要重译。 + +输出 JSON： +{{"issues": [{{"ids": ["段落id"], "type": "name|term|register|voice", "problem": "...", "fix": "..."}}]}} + +译文： +{json.dumps(pairs, ensure_ascii=False, indent=2)}""" diff --git a/tools/translation/qa.py b/tools/translation/qa.py new file mode 100644 index 0000000..fa3b5a7 --- /dev/null +++ b/tools/translation/qa.py @@ -0,0 +1,172 @@ +# Copyright 2026 Umbra Tools (umbratools.com) +# SPDX-License-Identifier: Apache-2.0 + +"""Mechanical QA for the Deal Hunter Chinese translation. + +This does not judge literary quality. It catches the defects that can be +detected without a human: missing paragraphs, leaked prompt metadata, stray +English, unbalanced quotes, duplicated punctuation and forbidden title forms. +""" + +from __future__ import annotations + +import argparse +import json +import re +import sys +from pathlib import Path + +sys.path.insert(0, str(Path(__file__).resolve().parent)) + +from pipeline import latin_residue # noqa: E402 + + +def _load(path: Path) -> dict: + return json.loads(path.read_text(encoding="utf-8")) + + +def check_chapter(chapter: dict, data: dict) -> tuple[list[dict], dict]: + drafts: dict[str, str] = data.get("paragraphs", {}) + by_id = {p["id"]: p for p in chapter["paragraphs"] if p.get("type") != "scene_break"} + expected = list(by_id) + issues: list[dict] = [] + + for pid in expected: + if pid not in drafts or not str(drafts[pid]).strip(): + issues.append({"id": pid, "level": "block", "kind": "missing", "detail": "no translation"}) + for pid in drafts: + if pid not in by_id: + issues.append({"id": pid, "level": "major", "kind": "unknown_id", "detail": "id not in source"}) + + chars = 0 + for pid, zh in drafts.items(): + zh = str(zh) + chars += len(re.findall(r"[\u4e00-\u9fff]", zh)) + if re.search(r"^\[ch-[^\]]+\]", zh) or re.search(r"（(?:对话|旁白|叙述)(?:；[^）]*)?）", zh): + issues.append({"id": pid, "level": "block", "kind": "metadata_leak", "detail": zh[:40]}) + if '"' in zh: + issues.append({"id": pid, "level": "major", "kind": "straight_quote", "detail": zh[:60]}) + if zh.count("“") != zh.count("”"): + issues.append( + { + "id": pid, + "level": "major", + "kind": "quote_balance", + "detail": f"{zh.count('“')} open / {zh.count('”')} close", + } + ) + if re.search(r"[。，！？；：]{2,}", zh): + issues.append({"id": pid, "level": "minor", "kind": "double_punctuation", "detail": zh[:60]}) + if "《交易猎人》" in zh: + issues.append({"id": pid, "level": "major", "kind": "title_translated", "detail": zh[:60]}) + common, proper = latin_residue(zh) + if common: + issues.append( + {"id": pid, "level": "major", "kind": "english_residue", "detail": ", ".join(common)} + ) + if proper: + issues.append( + {"id": pid, "level": "info", "kind": "proper_name_review", "detail": ", ".join(proper)} + ) + source = by_id.get(pid, {}).get("text", "") + star_system = re.search( + r"\bstar system\b|\b(?!The\b)[A-Z][a-z]+\s+system\b|\b\d+\s+systems\b", source + ) + if star_system: + phrase = star_system.group(0).lower() + if any( + x in phrase + for x in ( + "audio system", + "ship system", + "computer system", + "operating system", + "weapon system", + "security system", + "navigation system", + "power system", + "life support system", + ) + ): + star_system = None + if star_system and "系统" in zh: + issues.append( + {"id": pid, "level": "major", "kind": "system_mistranslated", "detail": "star system -> 系统"} + ) + + metrics = { + "chapter_id": chapter["id"], + "chapter_index": chapter["index"], + "title": chapter["title"], + "expected": len(expected), + "translated": len(drafts), + "chinese_chars": chars, + "issues": len(issues), + } + return issues, metrics + + +def main() -> None: + ap = argparse.ArgumentParser(description="Mechanical QA for the Chinese translation.") + ap.add_argument("--book", type=Path, required=True) + ap.add_argument("--run-dir", type=Path, required=True) + ap.add_argument("--json-out", type=Path) + ap.add_argument("--strict", action="store_true", help="exit 1 on any block/major issue") + args = ap.parse_args() + + book = _load(args.book) + all_issues: list[dict] = [] + metrics: list[dict] = [] + + for chapter in book["chapters"]: + path = args.run_dir / chapter["id"] / "translation.json" + if not path.exists(): + continue + data = _load(path) + issues, m = check_chapter(chapter, data) + for issue in issues: + issue["chapter_id"] = chapter["id"] + issue["chapter_title"] = chapter["title"] + all_issues.extend(issues) + metrics.append(m) + + by_level: dict[str, int] = {} + by_kind: dict[str, int] = {} + for issue in all_issues: + by_level[issue["level"]] = by_level.get(issue["level"], 0) + 1 + by_kind[issue["kind"]] = by_kind.get(issue["kind"], 0) + 1 + + print("CHAPTERS") + for m in metrics: + print( + f" {m['chapter_id']:6} {m['title'][:28]:28} " + f"{m['translated']:>4}/{m['expected']:<4} paras {m['chinese_chars']:>6} chars {m['issues']:>3} issues" + ) + print() + print("ISSUES BY LEVEL:", dict(sorted(by_level.items()))) + print("ISSUES BY KIND: ", dict(sorted(by_kind.items()))) + print() + for issue in all_issues: + if issue["level"] in {"block", "major"}: + print( + f" [{issue['level']:5}] {issue['kind']:20} {issue['chapter_id']}/{issue['id']}: {issue['detail']}" + ) + elif issue["level"] == "minor": + print(f" [minor] {issue['kind']:20} {issue['chapter_id']}/{issue['id']}: {issue['detail']}") + else: + print(f" [info ] {issue['kind']:20} {issue['chapter_id']}/{issue['id']}: {issue['detail']}") + + if args.json_out: + args.json_out.parent.mkdir(parents=True, exist_ok=True) + args.json_out.write_text( + json.dumps({"metrics": metrics, "issues": all_issues}, ensure_ascii=False, indent=2) + "\n", + encoding="utf-8", + ) + print(f"\nwrote {args.json_out}") + + if args.strict and (by_level.get("block", 0) or by_level.get("major", 0)): + raise SystemExit(1) + + +if __name__ == "__main__": + main() diff --git a/tools/translation/render.py b/tools/translation/render.py new file mode 100644 index 0000000..2926203 --- /dev/null +++ b/tools/translation/render.py @@ -0,0 +1,671 @@ +# Copyright 2026 Umbra Tools (umbratools.com) +# SPDX-License-Identifier: Apache-2.0 + +"""Render the translated book as AsciiDoc and as an OpenDocument text file. + +The AsciiDoc file is the canonical text output. The ODT is produced from a +Flat ODF (.fodt) document so that styles, page breaks and the table of +contents are under our control, then converted with LibreOffice. The TOC is +updated through UNO, because a plain ``--convert-to`` leaves an empty index. +""" + +from __future__ import annotations + +import argparse +import hashlib +import json +import re +import shutil +import subprocess +import sys +import tempfile +import time +from pathlib import Path +from xml.sax.saxutils import escape + +from llm import DEFAULT_ANALYST, DEFAULT_CRITIC, DEFAULT_MODEL + +FONT_SERIF = "Noto Serif CJK SC" +FONT_SANS = "Noto Sans CJK SC" + + +def _esc(text: str) -> str: + return escape(text, {'"': """}) + + +LATIN_RUN = re.compile(r"[A-Za-z][A-Za-z0-9'’.\-·]*(?:\s+[A-Za-z][A-Za-z0-9'’.\-·]*)*") + + +def _rich_esc(text: str) -> str: + """Escape text and mark Latin runs as English. + + A Chinese paragraph tells LibreOffice to use Chinese line-breaking for the + whole run, which can split an English name in the middle of a word + ("Deal Hu / nter"). A character style with fo:language="en" keeps the + Latin run on Western line-breaking rules without changing its font. + """ + out: list[str] = [] + last = 0 + for m in LATIN_RUN.finditer(text): + out.append(_esc(text[last : m.start()])) + out.append(f'{_esc(m.group(0))}') + last = m.end() + out.append(_esc(text[last:])) + return "".join(out) + + +def load_run(run_dir: Path, chapter_id: str) -> dict | None: + path = run_dir / chapter_id / "translation.json" + if not path.exists(): + return None + return json.loads(path.read_text(encoding="utf-8")) + + +def chapter_drafts(run_dir: Path | None, chapter_id: str) -> dict[str, str]: + if not run_dir: + return {} + data = load_run(run_dir, chapter_id) + return dict(data.get("paragraphs", {})) if data else {} + + +def _ordered_chapters(book: dict, run_dir: Path | None, include_missing: bool = False) -> list[dict]: + chapters = book["chapters"] + out = [] + for c in chapters: + if c["matter"].startswith("front") and c["matter"] != "front_audio": + out.append(c) + out += [c for c in chapters if c["matter"] == "front_audio"] + out += [c for c in chapters if c["matter"] == "body"] + out += [c for c in chapters if c["matter"] == "back"] + if include_missing: + return out + return [c for c in out if chapter_drafts(run_dir, c["id"])] + + + +def _cn_num(n: int) -> str: + digits = "零一二三四五六七八九" + if n <= 0: + return str(n) + if n <= 10: + return "十" if n == 10 else digits[n] + if n < 20: + return "十" + digits[n - 10] + if n < 100: + return digits[n // 10] + "十" + (digits[n % 10] if n % 10 else "") + return str(n) + + +def _body_chapter_number(book: dict, chapter: dict) -> int: + body = [c for c in book["chapters"] if c["matter"] == "body"] + body.sort(key=lambda c: c["index"]) + return body.index(chapter) + 1 + + +def _paragraph_style(paragraph: dict, matter: str) -> str: + if matter == "front_copyright": + return "Copyright" + if matter == "front_note": + return "Quote" if _is_author_note_code(paragraph.get("text", "")) else "FrontText" + if matter.startswith("front"): + return "FrontText" + if paragraph.get("type") == "dialogue": + return "Dialogue" + return "BodyText" + + +def _is_author_note_code(text: str) -> bool: + keys = ( + "One of those codes was", + "All women captives", + "Those who rape", + ) + return any(text.startswith(k) for k in keys) + + +# --------------------------------------------------------------------- AsciiDoc +def build_asciidoc(book: dict, run_dir: Path, *, sample: bool = False, date: str = "") -> str: + chapters = _ordered_chapters(book, run_dir) + lines: list[str] = [] + lines.append("= Deal Hunter: 交易猎人") + lines.append("B. D. Murphy") + lines.append(":doctype: book") + lines.append(":lang: zh-CN") + lines.append(":toc: left") + lines.append(":toclevels: 2") + lines.append(f":revdate: {date or time.strftime('%Y-%m-%d')}") + if sample: + lines.append(":revremark: 审阅样本") + lines.append(":icons: font") + lines.append("") + lines.append("// 译者：Umbra Tools") + if sample: + lines.append("") + lines.append("NOTE: 本文件是简体中文翻译的审阅样本。已译部分见目录；未列出的章节将在全文运行中补齐。") + lines.append("") + + seen_audio = False + for c in chapters: + drafts = chapter_drafts(run_dir, c["id"]) + if not drafts: + continue + if c["matter"] == "front_audio": + if seen_audio: + continue + seen_audio = True + lines.append(f"// source: {c['id']} (audio front matter)") + lines.append("== 音频版序言") + lines.append("") + lines.append("以下是《Deal Hunter》有声版的开场白。") + lines.append("") + for ac in [x for x in book["chapters"] if x["matter"] == "front_audio"]: + adrafts = chapter_drafts(run_dir, ac["id"]) + if not adrafts: + continue + lines.append(f"=== {ac['title_zh']}") + lines.append("") + for p in ac["paragraphs"]: + zh = adrafts.get(p["id"]) + if not zh: + continue + if p.get("type") == "scene_break": + lines.append("'''") + lines.append("") + continue + lines.append(zh) + lines.append("") + continue + + if c["matter"] == "body": + n = _body_chapter_number(book, c) + heading = f"第{_cn_num(n)}章　{c['title_zh']}" + else: + heading = c["title_zh"] + lines.append(f"// source: {c['id']} (database row {c['row']}), {len(c['paragraphs'])} paragraphs") + lines.append(f"== {heading}") + lines.append("") + for p in c["paragraphs"]: + zh = drafts.get(p["id"]) + if not zh: + continue + if p.get("type") == "scene_break": + lines.append("'''") + lines.append("") + continue + style = _paragraph_style(p, c["matter"]) + if style in {"Quote", "Dialogue"}: + lines.append(f"[.{style.lower()}]" if style == "Dialogue" else "[quote]") + if style == "Quote": + lines.append("____") + lines.append(zh) + lines.append("____") + else: + lines.append(zh) + lines.append("") + return "\n".join(lines).rstrip() + "\n" + + +# --------------------------------------------------------------------- Flat ODF +STYLES = f""" + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +""" + + +def _odt_paragraph(style: str, text: str, outline: int | None = None) -> str: + if outline: + return f'{_rich_esc(text)}' + return f'{_rich_esc(text)}' + + +def build_fodt(book: dict, run_dir: Path, *, sample: bool = False, date: str = "") -> str: + chapters = _ordered_chapters(book, run_dir) + body: list[str] = [] + + # Title page + body.append(_odt_paragraph("Title", "Deal Hunter")) + body.append(_odt_paragraph("Subtitle", "交易猎人")) + body.append(_odt_paragraph("Author", "B·D·墨菲 著")) + body.append(_odt_paragraph("Small", "Umbra Tools 译")) + if sample: + body.append(_odt_paragraph("Small", "简体中文译本 · 审阅样本")) + body.append(_odt_paragraph("Small", date or time.strftime("%Y-%m-%d"))) + body.append('') + + # Table of contents + body.append( + '' + '' + '目录' + '' + "" + '' + "" + "" + "" + "" + ) + + if sample: + body.append(_odt_paragraph("ChapterHeading", "审阅样本说明", 1)) + body.append( + _odt_paragraph( + "FrontText", + "本文件是《Deal Hunter》简体中文翻译的审阅样本。已译部分见目录；未列出的章节将在全文运行中补齐。", + ) + ) + body.append( + _odt_paragraph( + "FrontText", + "翻译流程：段落重建 → 意图分析 → 初译 → 双语审校。术语见 glossary.json，审校意见见 review.html。", + ) + ) + + seen_audio = False + for c in chapters: + drafts = chapter_drafts(run_dir, c["id"]) + if not drafts: + continue + if c["matter"] == "front_audio": + if seen_audio: + continue + seen_audio = True + body.append(_odt_paragraph("ChapterHeading", "音频版序言", 1)) + body.append(_odt_paragraph("FrontText", "以下是《Deal Hunter》有声版的开场白。")) + for ac in [x for x in book["chapters"] if x["matter"] == "front_audio"]: + adrafts = chapter_drafts(run_dir, ac["id"]) + if not adrafts: + continue + body.append(_odt_paragraph("SectionHeading", ac["title_zh"], 2)) + for p in ac["paragraphs"]: + zh = adrafts.get(p["id"]) + if not zh: + continue + if p.get("type") == "scene_break": + body.append(_odt_paragraph("SceneBreak", "* * *")) + else: + body.append(_odt_paragraph("FrontText", zh)) + continue + + if c["matter"] == "body": + n = _body_chapter_number(book, c) + heading = f"第{_cn_num(n)}章　{c['title_zh']}" + else: + heading = c["title_zh"] + body.append(_odt_paragraph("ChapterHeading", heading, 1)) + for p in c["paragraphs"]: + zh = drafts.get(p["id"]) + if not zh: + continue + if p.get("type") == "scene_break": + body.append(_odt_paragraph("SceneBreak", "* * *")) + continue + body.append(_odt_paragraph(_paragraph_style(p, c["matter"]), zh)) + + if sample: + body.append(_odt_paragraph("ChapterHeading", "样本结束", 1)) + body.append( + _odt_paragraph( + "FrontText", + "以上为审阅样本。全文运行将覆盖其余章节；审校问题清单会随各章翻译记录一并提供。", + ) + ) + + return f""" + +{STYLES} + + + + + + + + + + Deal Hunter · 交易猎人 + 1 + + + + + + + + + + + +{chr(10).join(body)} + + + +""" + + +# --------------------------------------------------------------------- LibreOffice +def _soffice_binary() -> str: + for name in ("soffice", "libreoffice"): + path = shutil.which(name) + if path: + return path + raise SystemExit("LibreOffice not found; cannot produce ODT/PDF") + + +def convert_with_uno(fodt: Path, odt: Path, pdf: Path | None, port: int = 2002) -> None: + """Convert FODT -> ODT (and PDF) with an updated TOC. + + ``--convert-to`` leaves a text:table-of-content empty. Opening the document + through UNO lets us call update() on the index before storing. + """ + fodt = fodt.resolve() + odt = odt.resolve() + if pdf: + pdf = pdf.resolve() + soffice = _soffice_binary() + profile = Path(tempfile.mkdtemp(prefix="lo-profile-")) + proc = subprocess.Popen( + [ + soffice, + "--headless", + "--invisible", + "--norestore", + "--nofirststartwizard", + f"-env:UserInstallation={profile.as_uri()}", + f"--accept=socket,host=127.0.0.1,port={port};urp;", + ], + stdout=subprocess.DEVNULL, + stderr=subprocess.DEVNULL, + ) + try: + import uno # type: ignore + from com.sun.star.beans import PropertyValue # type: ignore + + local = uno.getComponentContext() + resolver = local.ServiceManager.createInstanceWithContext( + "com.sun.star.bridge.UnoUrlResolver", local + ) + ctx = None + for _ in range(80): + try: + ctx = resolver.resolve( + f"uno:socket,host=127.0.0.1,port={port};urp;StarOffice.ComponentContext" + ) + break + except Exception: + time.sleep(0.25) + if ctx is None: + raise RuntimeError("could not connect to LibreOffice") + + desktop = ctx.ServiceManager.createInstanceWithContext("com.sun.star.frame.Desktop", ctx) + doc = desktop.loadComponentFromURL(uno.systemPathToFileUrl(str(fodt)), "_blank", 0, ()) + indexes = doc.getDocumentIndexes() + for i in range(indexes.getCount()): + indexes.getByIndex(i).update() + + def prop(name: str, value: object) -> object: + p = PropertyValue() + p.Name = name + p.Value = value + return p + + doc.storeToURL(uno.systemPathToFileUrl(str(odt)), (prop("FilterName", "writer8"),)) + if pdf: + doc.storeToURL(uno.systemPathToFileUrl(str(pdf)), (prop("FilterName", "writer_pdf_Export"),)) + doc.close(True) + finally: + proc.terminate() + try: + proc.wait(timeout=10) + except subprocess.TimeoutExpired: + proc.kill() + shutil.rmtree(profile, ignore_errors=True) + + +def render( + *, + book_path: Path, + run_dir: Path, + out_dir: Path, + sample: bool = False, + date: str = "", + pdf: bool = True, +) -> dict[str, Path]: + book = json.loads(book_path.read_text(encoding="utf-8")) + out_dir.mkdir(parents=True, exist_ok=True) + stem = "deal-hunter-zh-sample" if sample else "deal-hunter-zh" + asciidoc = out_dir / f"{stem}.asciidoc" + fodt = out_dir / f"{stem}.fodt" + odt = out_dir / f"{stem}.odt" + pdf_path = out_dir / f"{stem}.pdf" if pdf else None + + asciidoc.write_text(build_asciidoc(book, run_dir, sample=sample, date=date), encoding="utf-8") + fodt.write_text(build_fodt(book, run_dir, sample=sample, date=date), encoding="utf-8") + convert_with_uno(fodt, odt, pdf_path) + out = {"asciidoc": asciidoc, "fodt": fodt, "odt": odt} + if pdf_path: + out["pdf"] = pdf_path + return out + + +def _sha256(path: Path) -> str: + h = hashlib.sha256() + with path.open("rb") as f: + for block in iter(lambda: f.read(1024 * 1024), b""): + h.update(block) + return h.hexdigest() + + +def _repo_commit(start: Path) -> str | None: + for p in [start, *start.parents]: + if (p / ".git").exists(): + try: + return subprocess.check_output( + ["git", "-C", str(p), "rev-parse", "HEAD"], + text=True, + stderr=subprocess.DEVNULL, + ).strip() + except (OSError, subprocess.CalledProcessError): + return None + return None + + +def write_manifest( + *, + book_path: Path, + run_dir: Path, + out_dir: Path, + sample: bool, + files: dict[str, Path], +) -> Path: + """Write the reproducibility manifest next to the rendered files. + + It contains hashes, counts and model names, never the book text. + """ + book = json.loads(book_path.read_text(encoding="utf-8")) + chapters: list[dict] = [] + total = translated = issues = scene_breaks = 0 + for c in book["chapters"]: + data = load_run(run_dir, c["id"]) + expected = len([p for p in c["paragraphs"] if p.get("type") != "scene_break"]) + scene_breaks += len([p for p in c["paragraphs"] if p.get("type") == "scene_break"]) + n = len(data.get("paragraphs", {})) if data else 0 + total += expected + translated += n + if not data: + continue + chapter_issues = len(data.get("issues", []) or []) + issues += chapter_issues + chapters.append( + { + "id": c["id"], + "index": c["index"], + "matter": c["matter"], + "title": c["title"], + "title_zh": c["title_zh"], + "paragraphs_expected": expected, + "paragraphs_translated": n, + "critic_issues": chapter_issues, + } + ) + glossary = run_dir.parent / "glossary" / "glossary.json" + manifest = { + "title": book.get("title"), + "title_zh": book.get("title_zh"), + "author": book.get("author"), + "generated_at": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime()), + "sample": sample, + "source": { + "bundle": book.get("bundle"), + "bundle_sha256": book.get("bundle_sha256"), + "book_json_sha256": _sha256(book_path), + }, + "glossary_sha256": _sha256(glossary) if glossary.exists() else None, + "prompts_sha256": _sha256(Path(__file__).with_name("prompts.py")), + "git_commit": _repo_commit(book_path), + "models": { + "drafter": DEFAULT_MODEL, + "analyst": DEFAULT_ANALYST, + "critic": DEFAULT_CRITIC, + }, + "counts": { + "chapters": len(chapters), + "paragraphs_expected": total, + "paragraphs_translated": translated, + "scene_breaks": scene_breaks, + "critic_issues": issues, + }, + "chapters": chapters, + "outputs": {k: str(v) for k, v in files.items()}, + } + path = out_dir / "run-manifest.json" + path.write_text(json.dumps(manifest, ensure_ascii=False, indent=2) + "\n", encoding="utf-8") + return path + + +def main() -> None: + ap = argparse.ArgumentParser(description="Render Deal Hunter Chinese AsciiDoc + ODT.") + ap.add_argument("--book", type=Path, required=True) + ap.add_argument("--run-dir", type=Path, required=True) + ap.add_argument("--out-dir", type=Path, required=True) + ap.add_argument("--sample", action="store_true") + ap.add_argument("--date", default="") + ap.add_argument("--no-pdf", action="store_true") + args = ap.parse_args() + out = render( + book_path=args.book, + run_dir=args.run_dir, + out_dir=args.out_dir, + sample=args.sample, + date=args.date, + pdf=not args.no_pdf, + ) + out["manifest"] = write_manifest( + book_path=args.book, + run_dir=args.run_dir, + out_dir=args.out_dir, + sample=args.sample, + files=out, + ) + for k, v in out.items(): + print(f"{k:9} {v}") + + +if __name__ == "__main__": + main() diff --git a/tools/translation/review.py b/tools/translation/review.py new file mode 100644 index 0000000..78b7a4b --- /dev/null +++ b/tools/translation/review.py @@ -0,0 +1,297 @@ +# Copyright 2026 Umbra Tools (umbratools.com) +# SPDX-License-Identifier: Apache-2.0 + +"""Build a bilingual side-by-side review page for the translation. + +The output is a single self-contained HTML file. It contains the English source +and the Chinese draft, so it must stay inside the git-ignored production +directory. It is meant for a human editor: every paragraph is on one row, the +critic and mechanical issues are attached to the row, and a filter hides the +paragraphs that have no issues. +""" + +from __future__ import annotations + +import argparse +import html +import json +import re +import sys +import time +from pathlib import Path + +sys.path.insert(0, str(Path(__file__).resolve().parent)) + +from qa import check_chapter # noqa: E402 + + +def _load(path: Path) -> dict: + return json.loads(path.read_text(encoding="utf-8")) + + +def _severity_rank(sev: str) -> int: + return {"block": 0, "major": 1, "minor": 2, "info": 3}.get(sev, 4) + + +def _critic_issues(data: dict) -> list[dict]: + out = [] + for raw in data.get("issues", []) or []: + if not isinstance(raw, dict): + continue + out.append( + { + "id": str(raw.get("id") or ""), + "severity": str(raw.get("severity") or "major"), + "type": str(raw.get("type") or "meaning"), + "problem": str(raw.get("problem") or ""), + "source_excerpt": str(raw.get("source_excerpt") or ""), + "target_excerpt": str(raw.get("target_excerpt") or ""), + "fix": str(raw.get("fix") or ""), + "origin": "critic", + } + ) + return out + + +def _mechanical_issues(chapter: dict, data: dict) -> list[dict]: + issues, _metrics = check_chapter(chapter, data) + out = [] + for raw in issues: + out.append( + { + "id": str(raw.get("id") or ""), + "severity": str(raw.get("level") or "major"), + "type": str(raw.get("kind") or "mechanical"), + "problem": str(raw.get("detail") or ""), + "source_excerpt": "", + "target_excerpt": "", + "fix": "", + "origin": "qa", + } + ) + return out + + +def _badge(sev: str) -> str: + label = {"block": "BLOCK", "major": "MAJOR", "minor": "minor", "info": "info"}.get(sev, sev) + return f'{html.escape(label)}' + + +def _issue_html(issue: dict) -> str: + parts = [ + f'
', + _badge(issue["severity"]), + f' {html.escape(issue["type"])} ', + f'[{html.escape(issue["origin"])}]', + ] + if issue["problem"]: + parts.append(f'
{html.escape(issue["problem"])}
') + if issue["source_excerpt"]: + parts.append(f'
EN: {html.escape(issue["source_excerpt"])}
') + if issue["target_excerpt"]: + parts.append(f'
ZH: {html.escape(issue["target_excerpt"])}
') + if issue["fix"]: + parts.append(f'
建议：{html.escape(issue["fix"])}
') + parts.append("
") + return "".join(parts) + + +def build_html(book: dict, run_dir: Path, out_path: Path, only: set[str] | None = None) -> None: + sections: list[str] = [] + stats = {"chapters": 0, "paragraphs": 0, "flagged": 0, "chars": 0} + total_issues = {"block": 0, "major": 0, "minor": 0, "info": 0} + + for chapter in book["chapters"]: + if only and chapter["id"] not in only: + continue + path = run_dir / chapter["id"] / "translation.json" + if not path.exists(): + continue + data = _load(path) + drafts: dict[str, str] = data.get("paragraphs", {}) + if not drafts: + continue + issues = _critic_issues(data) + _mechanical_issues(chapter, data) + by_id: dict[str, list[dict]] = {} + unmatched: list[dict] = [] + for issue in issues: + total_issues[issue["severity"]] = total_issues.get(issue["severity"], 0) + 1 + if issue["id"] and issue["id"] in drafts: + by_id.setdefault(issue["id"], []).append(issue) + else: + unmatched.append(issue) + + rows: list[str] = [] + chapter_chars = 0 + flagged = 0 + heading = chapter.get("heading_zh") or chapter["title_zh"] + for p in chapter["paragraphs"]: + pid = p["id"] + if p.get("type") == "scene_break": + rows.append('* * *') + continue + zh = drafts.get(pid) + if zh is None: + rows.append( + f'' + f'{pid}' + f'{html.escape(p.get("text") or "")}' + f'（未译）' + ) + stats["flagged"] += 1 + continue + zh = str(zh) + chapter_chars += len(re.findall(r"[\u4e00-\u9fff]", zh)) + row_issues = sorted(by_id.get(pid, []), key=lambda i: _severity_rank(i["severity"])) + if row_issues: + flagged += 1 + row_class = "row flagged" if row_issues else "row" + detail = "".join(_issue_html(i) for i in row_issues) + rows.append( + f'' + f'{html.escape(pid)}' + f'{html.escape(p.get("text") or "")}' + f'
{html.escape(zh)}
{detail}' + f"" + ) + + stats["chapters"] += 1 + stats["paragraphs"] += len(drafts) + stats["chars"] += chapter_chars + stats["flagged"] += flagged + + unmatched_html = "" + if unmatched: + unmatched_html = ( + '

未能定位到段落的审校意见

' + + "".join(_issue_html(i) for i in unmatched) + + "
" + ) + + sections.append( + f""" +
+

{html.escape(heading)} {html.escape(chapter['title'])} · {chapter['id']}

+

{len(drafts)} 段 · {chapter_chars} 汉字 · {flagged} 段有意见

+ {unmatched_html} + + + + {''.join(rows)} + +
IDEnglish source中文译文 / 意见
+
""" + ) + + doc = f""" + + + + +Deal Hunter 中译审校表 + + + +
+

Deal Hunter · 交易猎人 — 中译审校表

+
生成 {time.strftime('%Y-%m-%d %H:%M')} · {stats['chapters']} 章 · {stats['paragraphs']} 段 · {stats['chars']} 汉字 · {stats['flagged']} 段有意见 · block {total_issues['block']} / major {total_issues['major']} / minor {total_issues['minor']}
+
+ + + +
+
+
+{''.join(sections)} +
+ + + +""" + out_path.parent.mkdir(parents=True, exist_ok=True) + out_path.write_text(doc, encoding="utf-8") + print(f"html {out_path}") + print( + f" {stats['chapters']} chapters, {stats['paragraphs']} paragraphs, " + f"{stats['chars']} chars, {stats['flagged']} flagged" + ) + + +def main() -> None: + ap = argparse.ArgumentParser(description="Build a bilingual HTML review page.") + ap.add_argument("--book", type=Path, required=True) + ap.add_argument("--run-dir", type=Path, required=True) + ap.add_argument("--out", type=Path, required=True) + ap.add_argument("--chapters", help="comma-separated chapter ids (default: all translated)") + args = ap.parse_args() + book = _load(args.book) + only = {c.strip() for c in args.chapters.split(",")} if args.chapters else None + build_html(book, args.run_dir, args.out, only) + + +if __name__ == "__main__": + main() diff --git a/tools/translation/tests/test_translation.py b/tools/translation/tests/test_translation.py new file mode 100644 index 0000000..641c6d1 --- /dev/null +++ b/tools/translation/tests/test_translation.py @@ -0,0 +1,137 @@ +# Copyright 2026 Umbra Tools (umbratools.com) +# SPDX-License-Identifier: Apache-2.0 + +"""Unit tests for the translation cleanup and QA helpers. + +The pipeline calls an LLM, so these tests deliberately cover only the +deterministic parts: prompt-leak removal, quote normalisation, English +residue detection, JSON extraction and the mechanical QA checks. +""" + +from __future__ import annotations + +import sys +import unittest +from pathlib import Path + +HERE = Path(__file__).resolve().parent +sys.path.insert(0, str(HERE.parent)) + +from llm import parse_json # noqa: E402 +from pipeline import latin_residue, sanitize # noqa: E402 +from qa import check_chapter # noqa: E402 + + +class SanitizeTests(unittest.TestCase): + def test_strips_dialogue_label(self) -> None: + self.assertEqual(sanitize("（对话；凯因达）“我在游艇上。”"), "“我在游艇上。”") + + def test_strips_narration_label(self) -> None: + self.assertEqual(sanitize("（旁白）凯因达登上了游艇。"), "凯因达登上了游艇。") + + def test_strips_paragraph_id(self) -> None: + self.assertEqual(sanitize("[ch-6-p0000] “你好。”"), "“你好。”") + + def test_title_and_brand(self) -> None: + self.assertEqual(sanitize("《交易猎人》"), "《Deal Hunter》") + self.assertEqual(sanitize("由乌姆布拉音频系统制作"), "由Umbra Audio制作") + + def test_missing_hour(self) -> None: + self.assertEqual(sanitize("她在小时前登船。"), "她一小时前登船。") + + def test_straight_quotes_become_chinese(self) -> None: + self.assertEqual(sanitize('"你好"，她说。'), "“你好”，她说。") + + def test_unmatched_closing_quote_removed(self) -> None: + self.assertEqual(sanitize("凯因达登上了游艇。”"), "凯因达登上了游艇。") + + def test_unmatched_opening_quote_closed(self) -> None: + self.assertEqual(sanitize("“你好"), "“你好”") + + def test_ellipsis_and_dash(self) -> None: + self.assertEqual(sanitize("等等... 然后--"), "等等…… 然后——") + + +class ResidueTests(unittest.TestCase): + def test_lowercase_english_is_flagged(self) -> None: + common, proper = latin_residue("她被 escorted 到了舱室。") + self.assertEqual(common, ["escorted"]) + self.assertEqual(proper, []) + + def test_allowed_names_are_ignored(self) -> None: + common, proper = latin_residue("Deal Hunter 和 Hunter 在 Umbra Audio。") + self.assertEqual(common, []) + self.assertEqual(proper, []) + + def test_unknown_capitalised_name_is_reported(self) -> None: + common, proper = latin_residue("这是 Foo 的船。") + self.assertEqual(common, []) + self.assertEqual(proper, ["Foo"]) + + +class JsonTests(unittest.TestCase): + def test_plain_json(self) -> None: + self.assertEqual(parse_json('{"a": 1}'), {"a": 1}) + + def test_fenced_json(self) -> None: + self.assertEqual(parse_json('```json\n{"a": 1}\n```'), {"a": 1}) + + def test_json_with_prose(self) -> None: + self.assertEqual(parse_json('结果如下：\n{"a": 1}\n完毕。'), {"a": 1}) + + +class QaTests(unittest.TestCase): + def test_missing_paragraph_is_block(self) -> None: + chapter = { + "id": "ch-x", + "index": 1, + "title": "Test", + "paragraphs": [ + {"id": "ch-x-p0000", "type": "narration", "text": "One."}, + {"id": "ch-x-p0001", "type": "narration", "text": "Two."}, + ], + } + data = {"paragraphs": {"ch-x-p0000": "第一段。"}} + issues, metrics = check_chapter(chapter, data) + kinds = {i["kind"] for i in issues} + self.assertIn("missing", kinds) + self.assertEqual(metrics["translated"], 1) + + def test_metadata_leak_and_residue(self) -> None: + chapter = { + "id": "ch-x", + "index": 1, + "title": "Test", + "paragraphs": [ + {"id": "ch-x-p0000", "type": "narration", "text": "One."}, + {"id": "ch-x-p0001", "type": "narration", "text": "The Albert system is far."}, + ], + } + data = { + "paragraphs": { + "ch-x-p0000": "（旁白）第一段 escorted。", + "ch-x-p0001": "阿尔伯特系统很远。", + } + } + issues, _ = check_chapter(chapter, data) + kinds = {i["kind"] for i in issues} + self.assertIn("metadata_leak", kinds) + self.assertIn("english_residue", kinds) + self.assertIn("system_mistranslated", kinds) + + def test_ship_systems_are_not_star_systems(self) -> None: + chapter = { + "id": "ch-x", + "index": 1, + "title": "Test", + "paragraphs": [ + {"id": "ch-x-p0000", "type": "narration", "text": "Keep away from the ship systems."}, + ], + } + data = {"paragraphs": {"ch-x-p0000": "远离飞船系统。"}} + issues, _ = check_chapter(chapter, data) + self.assertNotIn("system_mistranslated", {i["kind"] for i in issues}) + + +if __name__ == "__main__": + unittest.main() diff --git a/umbratrans_web/config.py b/umbratrans_web/config.py index 2c79ac4..147c036 100644 --- a/umbratrans_web/config.py +++ b/umbratrans_web/config.py @@ -34,7 +34,7 @@ from .security import ( ) SERVICE_NAME = "umbratrans-web" -SERVICE_VERSION = "0.1.0" +SERVICE_VERSION = "0.2.0" DEFAULT_SESSION_COOKIE_NAME = "umbratrans_session" DEFAULT_CSRF_COOKIE_NAME = "umbratrans_csrf" @@ -155,6 +155,12 @@ class Settings: database_pool_min: int = 1 database_pool_max: int = 10 static_root: Path | None = None + #: Where production directories live (``productions/-/``). Read + #: by the project-status adapter so the workspace can report real translation + #: and output state before the master store holds those rows. Unset means + #: "no production data is reachable", which the adapter reports honestly + #: rather than inventing progress. + productions_root: Path | None = None mailer_transport: str = "log" smtp_host: str = "" #: Set only to bring a staging deployment up before SMTP exists. It is @@ -346,6 +352,11 @@ def load_settings(environ: Mapping[str, str] | None = None) -> Settings: if raw_static: static_root = Path(raw_static).expanduser() + productions_root: Path | None = None + raw_productions = env.get("UMBRATRANS_PRODUCTIONS_ROOT") + if raw_productions: + productions_root = Path(raw_productions).expanduser() + # The DSN is read from the environment and is never logged, echoed by an # endpoint, or written to a development data file. repr/compare are off on # the field so it cannot leak through a settings dump. @@ -400,6 +411,7 @@ def load_settings(environ: Mapping[str, str] | None = None) -> Settings: database_pool_min=pool_min, database_pool_max=pool_max, static_root=static_root, + productions_root=productions_root, mailer_transport=env.get("UMBRATRANS_MAILER", "log").strip().lower(), smtp_host=env.get("UMBRATRANS_SMTP_HOST", "").strip(), smtp_port=_int_from(env.get("UMBRATRANS_SMTP_PORT", "587"), name="UMBRATRANS_SMTP_PORT", minimum=1), diff --git a/umbratrans_web/project_status.py b/umbratrans_web/project_status.py new file mode 100644 index 0000000..e58766c --- /dev/null +++ b/umbratrans_web/project_status.py @@ -0,0 +1,207 @@ +# Copyright 2026 Umbra Tools (umbratools.com) +# SPDX-License-Identifier: Apache-2.0 + +"""Project-status adapter backed by the ``productions/`` directory. + +The master store does not yet hold translation/audio/format state, so this +adapter reads ``productions/-/`` directly. It is a seam: when the +store gains those rows, this module can be replaced without changing the API. + +For each program, the adapter returns every configured target language plus the +English source row. Languages without a production directory are reported as +``not_started``; languages with a production directory report real chapter +progress from the runs directory. +""" + +from __future__ import annotations + +import json +from dataclasses import dataclass +from datetime import datetime +from pathlib import Path +from typing import Any + +# slug -> productions directory name stem +_SLUG_TO_PROD = { + "dealhunter-trans": "deal-hunter-zh", +} + +# Configured target languages per program slug. +# Each entry: language_tag -> (label, role) +_TARGET_LANGUAGES = { + "dealhunter-trans": { + "en": ("English (source)", "source"), + "zh-Hans": ("Simplified Chinese (zh-CN)", "translation"), + "es-ES": ("Spanish (es-ES)", "translation"), + "fr-FR": ("French (fr-FR)", "translation"), + "de-DE": ("German (de-DE)", "translation"), + "it-IT": ("Italian (it-IT)", "translation"), + "pt-BR": ("Portuguese (pt-BR)", "translation"), + "ru-RU": ("Russian (ru-RU)", "translation"), + "ja-JP": ("Japanese (ja-JP)", "translation"), + "ko-KR": ("Korean (ko-KR)", "translation"), + "ar-SA": ("Arabic (ar-SA)", "translation"), + }, +} + + +@dataclass(slots=True) +class LanguageStatus: + language_tag: str + language_label: str + source_present: bool + translation: dict[str, Any] + audio: dict[str, Any] + outputs: dict[str, Any] + review: dict[str, Any] + updated_at: str | None + + +def _load_json(path: Path) -> dict[str, Any] | None: + if not path.is_file(): + return None + try: + with path.open("r", encoding="utf-8") as fh: + data = json.load(fh) + return data if isinstance(data, dict) else None + except Exception: + return None + + +def _max_mtime(paths: list[Path]) -> str | None: + mtimes = [p.stat().st_mtime for p in paths if p.exists()] + if not mtimes: + return None + return datetime.fromtimestamp(max(mtimes)).isoformat() + + +def _count_translated_chapters(runs_dir: Path) -> tuple[int, int]: + """Return (done, total) chapter counts from the runs directory.""" + done = 0 + chapter_dirs: list[Path] = [] + if runs_dir.is_dir(): + for ch in sorted(runs_dir.iterdir()): + if ch.is_dir() and ch.name.startswith("ch-"): + chapter_dirs.append(ch) + if (ch / "translation.json").is_file(): + done += 1 + return done, len(chapter_dirs) + + +def _read_outputs(out_progress: Path, out_final: Path) -> dict[str, Any]: + """Read available output files from progress and final directories.""" + outputs: dict[str, Any] = {} + candidates: list[tuple[Path, str]] = [] + if out_progress.is_dir(): + candidates.append((out_progress, "output-progress")) + if out_final.is_dir(): + candidates.append((out_final, "output")) + for subdir, key in candidates: + for p in subdir.iterdir(): + if not p.is_file(): + continue + name = p.name + if name.endswith(".pdf"): + outputs.setdefault("pdf-paperback", {"available": True, "path": str(p), "source": key}) + elif name.endswith(".odt") or name.endswith(".asciidoc"): + outputs.setdefault("epub", {"available": True, "path": str(p), "source": key}) + return outputs + + +class ProjectStatusAdapter: + def __init__(self, productions_root: Path | None = None) -> None: + if productions_root is not None: + self.productions_root = Path(productions_root) + else: + self.productions_root = Path(__file__).resolve().parents[1] / "productions" + + def status_for(self, project) -> dict[str, Any]: + prod_name = _SLUG_TO_PROD.get(getattr(project, "slug", "")) + prod_dir = self.productions_root / prod_name if prod_name else None + runs_dir = prod_dir / "runs" if prod_dir else None + out_progress = prod_dir / "output-progress" if prod_dir else None + out_final = prod_dir / "output" if prod_dir else None + + # Chapter totals come from the source book when available; fall back + # to the number of chapter directories found in runs. + done, total = _count_translated_chapters(runs_dir) if runs_dir else (0, 0) + + languages: list[dict[str, Any]] = [] + + # English source row is always present when the project exists. + languages.append( + { + "language_tag": "en", + "language_label": "English (source)", + "role": "source", + "source_present": True, + "translation": {"state": "complete", "chapters_done": total, "chapters_total": total}, + "audio": {"state": "unknown"}, + "outputs": {}, + "review": {"state": "ok"}, + } + ) + + target_langs = _TARGET_LANGUAGES.get(getattr(project, "slug", ""), {}) + + for tag in sorted(target_langs.keys()): + if tag == "en": + continue # source row already added above + label, role = target_langs[tag] + + if prod_dir and prod_dir.is_dir(): + # A language-specific production directory would be named + # deal-hunter- (e.g. deal-hunter-es for Spanish). + lang_prod_name = f"deal-hunter-{tag.split('-')[0]}" + lang_prod_dir = self.productions_root / lang_prod_name + lang_runs = lang_prod_dir / "runs" if lang_prod_dir.is_dir() else None + lang_out_progress = lang_prod_dir / "output-progress" if lang_prod_dir.is_dir() else None + lang_out_final = lang_prod_dir / "output" if lang_prod_dir.is_dir() else None + + if lang_runs and lang_runs.is_dir(): + lang_done, lang_total = _count_translated_chapters(lang_runs) + state = ( + "complete" + if lang_total and lang_done >= lang_total + else ("in_progress" if lang_done else "not_started") + ) + outputs = _read_outputs( + lang_out_progress or Path(), lang_out_final or Path() + ) + updated_at = _max_mtime( + [lang_runs / f"ch-{i}" / "translation.json" for i in range(1, 40)] + if lang_runs.is_dir() + else [] + ) + else: + state = "not_started" + lang_done, lang_total = 0, 0 + outputs = {} + updated_at = None + else: + state = "not_started" + lang_done, lang_total = 0, 0 + outputs = {} + updated_at = None + + languages.append( + { + "language_tag": tag, + "language_label": label, + "role": role, + "source_present": tag == "en", + "translation": { + "state": state, + "chapters_done": lang_done, + "chapters_total": lang_total, + }, + "audio": {"state": "not_started"}, + "outputs": outputs, + "review": { + "state": "needs_review" if state == "complete" else "pending" + }, + "updated_at": updated_at, + } + ) + + return {"project_id": str(getattr(project, "id")), "languages": languages} diff --git a/umbratrans_web/routes_workspace.py b/umbratrans_web/routes_workspace.py index 49e40bd..983a140 100644 --- a/umbratrans_web/routes_workspace.py +++ b/umbratrans_web/routes_workspace.py @@ -26,18 +26,28 @@ see it. from __future__ import annotations +from typing import Any + from fastapi import APIRouter, Depends, Request from .app import get_service, require_principal from .models import Principal from .schemas import ( + AudioQueueResponse, + FrontMatterResponse, + FormatMetadataResponse, OrganizationResponse, + ProjectDetailResponse, ProjectResponse, + ProjectStatusResponse, + TranslationQueueResponse, + UtteranceReviewResponse, UserResponse, WorkspaceProjectResponse, WorkspaceResponse, ) from .service import AccountService +from .project_status import ProjectStatusAdapter router = APIRouter() @@ -74,3 +84,219 @@ def workspace( total=snapshot.total, assigned_to_me=render(snapshot.assigned_to_me), ) + + +@router.get("/{project_id}", response_model=ProjectDetailResponse) +def project_detail( + project_id, + request: Request, + principal: Principal = Depends(require_principal), +) -> ProjectDetailResponse: + """One project and its real state, scoped to the caller.""" + + service: AccountService = get_service(request) + from uuid import UUID + + project = service.get_project(principal, UUID(str(project_id))) + adapter = ProjectStatusAdapter(getattr(request.app.state.settings, "productions_root", None)) + status = adapter.status_for(project) + return ProjectDetailResponse( + project=ProjectResponse.from_view(project), + languages=status.get("languages", []), + review={"needs_attention": any((lang.get("review") or {}).get("state") == "needs_review" for lang in status.get("languages", []))}, + ) + + +@router.get("/{project_id}/status", response_model=ProjectStatusResponse) +def project_status( + project_id, + request: Request, + principal: Principal = Depends(require_principal), +) -> ProjectStatusResponse: + """Per-language translation/audio/format state, backed by the productions directory.""" + + service: AccountService = get_service(request) + from uuid import UUID + + project = service.get_project(principal, UUID(str(project_id))) + adapter = ProjectStatusAdapter(getattr(request.app.state.settings, "productions_root", None)) + status = adapter.status_for(project) + return ProjectStatusResponse(project_id=status["project_id"], languages=status.get("languages", [])) + + +@router.get("/{project_id}/chapters/{language_tag}", response_model=list[dict[str, Any]]) +def list_chapters( + project_id: str, + language_tag: str, + request: Request, + principal: Principal = Depends(require_principal), +) -> list[dict[str, Any]]: + """Return chapter metadata for one language project.""" + + service: AccountService = get_service(request) + from uuid import UUID + + return service.get_chapters(principal, UUID(str(project_id)), language_tag) + + +@router.get("/{project_id}/chapter-content/{language_tag}/{chapter_id}", response_model=dict[str, Any]) +def get_chapter_content( + project_id: str, + language_tag: str, + chapter_id: str, + request: Request, + principal: Principal = Depends(require_principal), +) -> dict[str, Any]: + """Return full paragraph content for one chapter (front matter editing).""" + + service: AccountService = get_service(request) + from uuid import UUID + + return service.get_chapter_content(principal, UUID(str(project_id)), language_tag, chapter_id) + + +@router.post("/{project_id}/translation/{language_tag}/queue", response_model=TranslationQueueResponse) +def queue_translation( + project_id: str, + language_tag: str, + request: Request, + principal: Principal = Depends(require_principal), +) -> TranslationQueueResponse: + """Queue a translation job for one language project.""" + + service: AccountService = get_service(request) + from uuid import UUID + + return service.queue_translation(principal, UUID(str(project_id)), language_tag) + + +@router.post("/{project_id}/audio/{language_tag}/queue", response_model=AudioQueueResponse) +def queue_audio( + project_id: str, + language_tag: str, + request: Request, + principal: Principal = Depends(require_principal), + voice_config: dict | None = None, +) -> AudioQueueResponse: + """Queue an audio generation job for one language project. + + Requires the language's translation to be complete first, matching the + workflow: translate, then audio. + """ + + service: AccountService = get_service(request) + from uuid import UUID + + return service.queue_audio( + principal, UUID(str(project_id)), language_tag, voice_config=voice_config + ) + + +@router.get("/{project_id}/format/{language_tag}", response_model=FormatMetadataResponse) +def get_format_metadata( + project_id: str, + language_tag: str, + request: Request, + principal: Principal = Depends(require_principal), +) -> FormatMetadataResponse: + """Read format metadata (EPUB/PDF) for one language project.""" + + service: AccountService = get_service(request) + from uuid import UUID + + return service.get_format_metadata(principal, UUID(str(project_id)), language_tag) + + +@router.post("/{project_id}/format/{language_tag}", response_model=FormatMetadataResponse) +def set_format_metadata( + project_id: str, + language_tag: str, + metadata: FormatMetadataRequest, + request: Request, + principal: Principal = Depends(require_principal), +) -> FormatMetadataResponse: + """Save format metadata (cover, ISBN, blurb, search terms, etc.) for one language project.""" + + service: AccountService = get_service(request) + from uuid import UUID + + return service.set_format_metadata( + principal, + UUID(str(project_id)), + language_tag, + metadata=metadata.model_dump(exclude={"language_tag"}), + ) + + +@router.post("/{project_id}/front-matter/{language_tag}", response_model=FrontMatterResponse) +def save_front_matter( + project_id: str, + language_tag: str, + payload: FrontMatterRequest, + request: Request, + principal: Principal = Depends(require_principal), +) -> FrontMatterResponse: + """Save front-matter items for one language project.""" + + service: AccountService = get_service(request) + from uuid import UUID + + items = [item.model_dump() for item in payload.items] + return service.save_front_matter( + principal, UUID(str(project_id)), language_tag, items=items + ) + + +@router.get( + "/{project_id}/review/{language_tag}/chapters/{chapter_id}", + response_model=UtteranceReviewResponse, +) +def get_utterance_review( + project_id: str, + language_tag: str, + chapter_id: str, + request: Request, + principal: Principal = Depends(require_principal), +) -> UtteranceReviewResponse: + """Return utterances for one chapter of one language, for speaker-alignment review.""" + + service: AccountService = get_service(request) + from uuid import UUID + + return service.get_utterance_review( + principal, UUID(str(project_id)), language_tag, chapter_id + ) + + +@router.get("/{project_id}/front-matter/{language_tag}", response_model=FrontMatterResponse) +def get_front_matter( + project_id: str, + language_tag: str, + request: Request, + principal: Principal = Depends(require_principal), +) -> FrontMatterResponse: + """Read front-matter items for one language project.""" + + service: AccountService = get_service(request) + from uuid import UUID + + return service.get_front_matter(principal, UUID(str(project_id)), language_tag) + + +@router.post("/{project_id}/front-matter/{language_tag}", response_model=FrontMatterResponse) +def save_front_matter( + project_id: str, + language_tag: str, + payload: FrontMatterRequest, + request: Request, + principal: Principal = Depends(require_principal), +) -> FrontMatterResponse: + """Save front-matter items for one language project.""" + + service: AccountService = get_service(request) + from uuid import UUID + + items = [item.model_dump() for item in payload.items] + return service.save_front_matter( + principal, UUID(str(project_id)), language_tag, items=items + ) diff --git a/umbratrans_web/schemas.py b/umbratrans_web/schemas.py index a8684de..a45cb4b 100644 --- a/umbratrans_web/schemas.py +++ b/umbratrans_web/schemas.py @@ -1,76 +1,83 @@ # Copyright 2026 Umbra Tools (umbratools.com) # SPDX-License-Identifier: Apache-2.0 -"""Pydantic request and response models for the phase-1 API. +"""Public API schemas and their mapping from internal views. -Two rules shape every model here: - -* Request models forbid unknown fields. A ``password`` field sent to - ``POST /api/uadmin/accounts`` is therefore rejected by the schema itself, so - a one-time password can never arrive through a normal JSON body. -* Response models contain no credential material. Accounts are serialized from - :class:`umbratrans_web.models.AccountView`, which exposes a - :class:`~umbratrans_web.models.CredentialSummary` and never a hash. +The boundary keeps internal invariants out of the wire. ``from_view`` +constructors are the only place an internal representation is trusted to become +wire JSON; route handlers assemble responses rather than dumping domain objects. """ from __future__ import annotations -from datetime import datetime +from datetime import datetime, timezone +from enum import StrEnum +from typing import Any from uuid import UUID -from pydantic import BaseModel, ConfigDict, Field +from pydantic import BaseModel, ConfigDict, Field, field_validator +from .errors import InputError from .models import ( AccountStatus, - AccountView, - AuditEvent, - CredentialSummary, CredentialType, EnrollmentMode, OrganizationStatus, - OrganizationView, ProjectStatus, - ProjectView, RoleName, ) class RequestModel(BaseModel): - """Base for request bodies: strict, no silent field coercion.""" + """Base for request bodies. - model_config = ConfigDict(extra="forbid", str_strip_whitespace=False) + Unknown fields are rejected so a mistyped property cannot be silently ignored + or, worse, mask a client bug while the server does something unexpected. + """ + + model_config = ConfigDict(extra="forbid") -class ErrorBody(BaseModel): - code: str - message: str +class ResponseModel(BaseModel): + """Base for responses. + + Extra fields are not allowed, so the wire shape cannot drift without the + tests catching it. + """ + + model_config = ConfigDict(extra="forbid") -class ErrorResponse(BaseModel): - error: ErrorBody +class UserResponse(ResponseModel): + id: UUID + email: str + username: str + display_name: str + role: RoleName + status: AccountStatus + organization_id: UUID | None + is_email_verified: bool + created_at: datetime | None = None + updated_at: datetime | None = None + + @classmethod + def from_view(cls, view) -> UserResponse: + return cls( + id=view.id, + email=view.email, + username=view.username, + display_name=view.display_name, + role=view.role, + status=view.status, + organization_id=view.organization_id, + is_email_verified=getattr(view, "is_email_verified", False), + created_at=getattr(view, "created_at", None), + updated_at=getattr(view, "updated_at", None), + ) -class HealthResponse(BaseModel): - status: str - service: str - version: str - environment: str - repository: str - cookie_secure: bool - csrf_required: bool - #: Whether verification mail can actually be delivered. False means new - #: accounts cannot be activated by email, so it is reported here rather than - #: discovered when someone waits for a link that never arrives. - mail_deliverable: bool = True - #: True only in staging, where the operator has explicitly accepted that mail - #: is not configured yet. - mail_delivery_waived: bool = False - #: Whether the deployment still needs its first platform administrator. - setup_required: bool = False - time: datetime - -class CredentialSummaryResponse(BaseModel): +class CredentialSummaryResponse(ResponseModel): credential_type: CredentialType must_change_password: bool failed_login_count: int @@ -78,254 +85,196 @@ class CredentialSummaryResponse(BaseModel): last_login_at: datetime | None = None password_changed_at: datetime | None = None - @classmethod - def from_summary(cls, summary: CredentialSummary) -> CredentialSummaryResponse: - return cls( - credential_type=summary.credential_type, - must_change_password=summary.must_change_password, - failed_login_count=summary.failed_login_count, - locked=summary.locked, - last_login_at=summary.last_login_at, - password_changed_at=summary.password_changed_at, - ) - -class UserResponse(BaseModel): +class AccountResponse(ResponseModel): id: UUID email: str - username: str | None = None + username: str display_name: str role: RoleName status: AccountStatus - organization_id: UUID | None = None - email: str | None = None - is_email_verified: bool = True + organization_id: UUID | None + is_email_verified: bool created_at: datetime | None = None updated_at: datetime | None = None - - @classmethod - def from_view(cls, view: AccountView) -> UserResponse: - return cls( - id=view.id, - email=view.email, - username=view.username, - display_name=view.display_name, - role=view.role, - status=view.status, - organization_id=view.organization_id, - created_at=view.created_at, - updated_at=view.updated_at, - ) - - -class AccountResponse(UserResponse): credential: CredentialSummaryResponse - active_session_count: int - - @classmethod - def from_view(cls, view: AccountView) -> AccountResponse: - return cls( - id=view.id, - username=view.username, - display_name=view.display_name, - role=view.role, - status=view.status, - organization_id=view.organization_id, - email=view.email, - created_at=view.created_at, - updated_at=view.updated_at, - # Passed explicitly. The field defaults to True, so omitting it made - # every account in the list report itself as confirmed, including the - # invited ones - and the control panel is about to offer to re-send a - # verification link, so it has to be able to tell them apart. - is_email_verified=view.is_email_verified, - credential=CredentialSummaryResponse.from_summary(view.credential), - active_session_count=view.active_session_count, - ) + active_session_count: int = 0 -class AccountListResponse(BaseModel): - items: list[AccountResponse] +class AccountListResponse(ResponseModel): + accounts: list[AccountResponse] total: int limit: int offset: int -class SessionUserResponse(BaseModel): - """``GET /api/auth/me``. Includes the CSRF token the browser must echo.""" - - user: UserResponse - role: RoleName - organization_id: UUID | None = None - must_change_password: bool - csrf_token: str - idle_expires_at: datetime | None = None - absolute_expires_at: datetime | None = None - - -class LoginResponse(BaseModel): - user: UserResponse - must_change_password: bool - csrf_token: str - idle_expires_at: datetime - absolute_expires_at: datetime - - -class LogoutResponse(BaseModel): - revoked_sessions: int - - -class RevokeSessionsResponse(BaseModel): - revoked_sessions: int - - -class AuditEventResponse(BaseModel): - # A store-owned identifier, so a string rather than the UUID the - # development repository happens to use. - id: str - occurred_at: datetime - action: str - outcome: str - actor_user_id: UUID | None = None - target_user_id: UUID | None = None - organization_id: UUID | None = None - request_id: str | None = None - detail: dict[str, str] - - @classmethod - def from_event(cls, event: AuditEvent) -> AuditEventResponse: - return cls( - id=event.id, - occurred_at=event.occurred_at, - action=event.action, - outcome=event.outcome, - actor_user_id=event.actor_user_id, - target_user_id=event.target_user_id, - organization_id=event.organization_id, - request_id=event.request_id, - detail=dict(event.detail), - ) - - -class AuditListResponse(BaseModel): - items: list[AuditEventResponse] - total: int - limit: int - offset: int - - -class LoginRequest(RequestModel): - """Sign-in payload. - - The identifier is the email address, not a username: - ``users.email`` is ``citext NOT NULL UNIQUE`` in the control plane, so it is - already a case-insensitively unique login key and no second identifier is - needed. - """ - - email: str = Field(min_length=3, max_length=254) - password: str = Field(min_length=1, max_length=1024) - - -class VerifyEmailRequest(RequestModel): - """Consume a mailed verification link. - - ``extra="forbid"`` matters here: a link is a bearer capability, and quietly - ignoring unexpected fields would make a malformed link look like a valid - one. - """ - - token: str = Field(min_length=1, max_length=512) - - -class ResendVerificationRequest(RequestModel): - """Ask for another verification link. - - Takes an address rather than a session because an unverified account cannot - hold one. The response is identical whether or not an account exists. - """ - - email: str = Field(min_length=3, max_length=254) - - -class VerifyEmailResponse(BaseModel): - account: UserResponse - message: str - - -class ResendVerificationResponse(BaseModel): - """Deliberately says nothing about whether an account exists.""" - - message: str - - -class ChangeFirstPasswordRequest(RequestModel): - current_password: str = Field(min_length=1, max_length=1024) - new_password: str = Field(min_length=1, max_length=1024) - - class CreateAccountRequest(RequestModel): """Administrator account creation for either enrollment mode. The request never carries a password: the schema forbids unknown fields, so a submitted ``password`` is rejected outright. For ``enrollment_mode="temporary_password"`` the service generates the one-time - password itself and returns it once in :class:`CreateAccountResponse`. - - ``email`` is required because the control plane declares ``users.email`` as - ``NOT NULL UNIQUE``. Making it mandatory here means a missing address is a - clear field-level rejection rather than a failure that only appears once - the authoritative store is in use. + password and returns it exactly once. For ``enrollment_mode="oidc"`` no + password is stored at all. """ - username: str | None = Field( - default=None, - max_length=64, - description=( - "Optional short handle for audit lines. Not a login path: the email " - "is the identifier, so this is never asked for by sign-in." - ), - ) - display_name: str = Field(min_length=1, max_length=120) - email: str = Field(min_length=3, max_length=254) - enrollment_mode: EnrollmentMode = EnrollmentMode.OIDC + enrollment_mode: EnrollmentMode + username: str = Field(min_length=1, max_length=64) + display_name: str = Field(min_length=1, max_length=128) + email: str = Field(min_length=3, max_length=320) role: RoleName = RoleName.MEMBER organization_id: UUID | None = None -class CreateAccountResponse(BaseModel): - """Result of ``POST /api/uadmin/accounts``. - - ``temporary_password`` is populated only when the account was enrolled with - a one-time local password. It is the single response that ever contains - that value, it is served with ``Cache-Control: no-store``, and it is never - persisted, logged, or retrievable again: the account must replace it on its - first login. It is ``None`` for OIDC enrollment. - """ - +class CreateAccountResponse(ResponseModel): account: AccountResponse temporary_password: str | None = None - @classmethod - def from_oidc(cls, view: AccountView) -> CreateAccountResponse: - return cls(account=AccountResponse.from_view(view)) - @classmethod - def from_temporary_password(cls, view: AccountView, temporary_password: str) -> CreateAccountResponse: - return cls(account=AccountResponse.from_view(view), temporary_password=temporary_password) - - -class SetAccountStatusRequest(RequestModel): +class StatusChangeRequest(RequestModel): status: AccountStatus - reason: str | None = Field(default=None, max_length=200) + reason: str | None = Field(default=None, max_length=500) -class StatusChangeResponse(BaseModel): +SetAccountStatusRequest = StatusChangeRequest + + +class StatusChangeResponse(ResponseModel): account: AccountResponse - revoked_sessions: int + revoked_sessions: int = 0 -class ProjectResponse(BaseModel): +class RevokeSessionsResponse(ResponseModel): + account: AccountResponse + revoked_sessions: int = 0 + + +class PasswordResetRequestedResponse(ResponseModel): + requested: bool + + +class SetupStatusResponse(ResponseModel): + setup_required: bool + + +class SetupInitializeRequest(RequestModel): + email: str = Field(min_length=3, max_length=320) + display_name: str = Field(min_length=1, max_length=128) + username: str | None = Field(default=None, max_length=64) + password: str = Field(min_length=1, max_length=1024) + + +class SetupInitializeResponse(ResponseModel): + account: AccountResponse + csrf_token: str | None = None + + +class BootstrapAdminRequest(RequestModel): + username: str | None = Field(default=None, max_length=64) + display_name: str = Field(min_length=1, max_length=128) + email: str = Field(min_length=3, max_length=320) + organization_id: UUID | None = None + role: RoleName = RoleName.PLATFORM_ADMIN + + +class LoginRequest(RequestModel): + email: str = Field(min_length=3, max_length=320) + password: str = Field(min_length=1, max_length=1024) + + +class ChangeFirstPasswordRequest(RequestModel): + current_password: str = Field(min_length=1, max_length=1024) + new_password: str = Field(min_length=1, max_length=1024) + + +class ChangeOwnPasswordRequest(RequestModel): + current_password: str = Field(min_length=1, max_length=1024) + new_password: str = Field(min_length=1, max_length=1024) + + +class VerifyEmailRequest(RequestModel): + token: str = Field(min_length=1, max_length=4096) + + +class ResendVerificationRequest(RequestModel): + email: str = Field(min_length=3, max_length=320) + + +class LoginResponse(ResponseModel): + user: UserResponse + must_change_password: bool + csrf_token: str | None = None + idle_expires_at: datetime | None = None + absolute_expires_at: datetime | None = None + + +class SessionResponse(ResponseModel): + user: UserResponse + must_change_password: bool + idle_expires_at: datetime | None = None + absolute_expires_at: datetime | None = None + + +class VerifyEmailResponse(ResponseModel): + account: AccountResponse + activated: bool + + +class ResendVerificationResponse(ResponseModel): + requested: bool + + +class OrganizationResponse(ResponseModel): + id: UUID + name: str + slug: str + status: OrganizationStatus + member_count: int + created_at: datetime | None = None + updated_at: datetime | None = None + + @classmethod + def from_view(cls, view) -> OrganizationResponse: + return cls( + id=view.id, + name=view.name, + slug=view.slug, + status=view.status, + member_count=view.member_count, + created_at=getattr(view, "created_at", None), + updated_at=getattr(view, "updated_at", None), + ) + + +class OrganizationCreateRequest(RequestModel): + name: str = Field(min_length=1, max_length=128) + slug: str | None = Field(default=None, max_length=64) + status: OrganizationStatus = OrganizationStatus.ACTIVE + + +CreateOrganizationRequest = OrganizationCreateRequest + + +class OrganizationUpdateRequest(RequestModel): + name: str | None = Field(default=None, min_length=1, max_length=128) + slug: str | None = Field(default=None, max_length=64) + status: OrganizationStatus | None = None + + +UpdateOrganizationRequest = OrganizationUpdateRequest +SetOrganizationStatusRequest = OrganizationUpdateRequest + + + + +class OrganizationListResponse(ResponseModel): + organizations: list[OrganizationResponse] + total: int + limit: int + offset: int + + +class ProjectResponse(ResponseModel): """A project as returned by the API. ``assigned_account_ids`` is who is *working* on the project. It is not a @@ -346,207 +295,53 @@ class ProjectResponse(BaseModel): updated_at: datetime | None = None @classmethod - def from_view(cls, view: ProjectView) -> ProjectResponse: + def from_view(cls, view) -> ProjectResponse: return cls( id=view.id, organization_id=view.organization_id, name=view.name, slug=view.slug, status=view.status, - assigned_account_ids=list(view.assigned_account_ids), - assigned_count=view.assigned_count, - created_at=view.created_at, - created_by=view.created_by, - updated_at=view.updated_at, + assigned_account_ids=list(getattr(view, "assigned_account_ids", []) or []), + assigned_count=getattr(view, "assigned_count", 0) or 0, + created_at=getattr(view, "created_at", None), + created_by=getattr(view, "created_by", None), + updated_at=getattr(view, "updated_at", None), ) -class ProjectListResponse(BaseModel): - """Projects in the caller's organization, and the count of the whole set. - - ``total`` counts every matching project, not the page, so a client can say - "showing 20 of 47" without a second request. - """ - - projects: list[ProjectResponse] - total: int - - -class CreateProjectRequest(RequestModel): - """Create a project inside the caller's organization. - - ``slug`` is optional and derived from ``name`` when omitted. A supplied slug - is canonicalised rather than required to be already well formed, so an - administrator does not have to learn the rules before naming a project. - """ - - name: str = Field(min_length=1, max_length=120) - slug: str | None = Field(default=None, max_length=63) - # Honoured only by a platform administrator. For anyone else the service pins - # the project to their own organization, and refuses a body naming another. +class ProjectCreateRequest(RequestModel): + name: str = Field(min_length=1, max_length=128) + slug: str | None = Field(default=None, max_length=64) organization_id: UUID | None = None -class SetProjectStatusRequest(RequestModel): +CreateProjectRequest = ProjectCreateRequest + + +class ProjectStatusRequest(RequestModel): status: ProjectStatus - reason: str | None = Field(default=None, max_length=200) -class ProjectAssignmentResponse(BaseModel): - """The project as it stands after an assignment change. - - The assignment count moves even when the answer is 404-shaped by intent: an - unassign that removed nothing is reported, not treated as a failure, because - asking for a state that is already true is a reasonable thing to do. - """ - - project: ProjectResponse - changed: bool +SetProjectStatusRequest = ProjectStatusRequest -class ProjectMemberRequest(RequestModel): - user_id: UUID - - -class WorkspaceProjectResponse(ProjectResponse): - """A project row on the dashboard, with the caller's own involvement. - - ``assigned_to_me`` is computed per caller, so the same endpoint can show - "your work" and "everything in the organization" without the client - reassembling who is who. - """ - - assigned_to_me: bool = False - - -class WorkspaceResponse(BaseModel): - """``GET /api/workspace``: the signed-in viewer's organization and its work. - - Deliberately one call. A dashboard that fetches the organization, then its - projects, then each project's members is a dashboard that renders three - partial states and can render two organizations at once if a filter is - mistimed. Everything a member may see comes from here, scoped server-side. - """ - - user: UserResponse - organization: OrganizationResponse | None - #: Projects in the organization, organization-wide. Empty is a normal state, - #: not an error, and the client renders it as "no projects yet". - projects: list[WorkspaceProjectResponse] - total: int - #: The subset assigned to the caller, for a "your work" view. Always a subset - #: of ``projects``; computed here so the two cannot disagree. - assigned_to_me: list[WorkspaceProjectResponse] - - -class OrganizationResponse(BaseModel): - """An organization as returned by the administrator API.""" - - id: UUID - name: str - slug: str - status: OrganizationStatus - member_count: int - created_at: datetime | None = None - updated_at: datetime | None = None - - @classmethod - def from_view(cls, view: OrganizationView) -> OrganizationResponse: - return cls( - id=view.id, - name=view.name, - slug=view.slug, - status=view.status, - member_count=view.member_count, - created_at=view.created_at, - updated_at=view.updated_at, - ) - - -class OrganizationListResponse(BaseModel): - items: list[OrganizationResponse] +class ProjectListResponse(ResponseModel): + projects: list[ProjectResponse] total: int limit: int offset: int -class CreateOrganizationRequest(RequestModel): - """Create a tenant. Platform administrators only. - - ``slug`` is optional and derived from ``name`` when omitted. A supplied - slug is validated as written, so a rejected value reports the slug rule. - """ - - name: str = Field(min_length=1, max_length=120) - slug: str | None = Field(default=None, max_length=63) +class ProjectAssignmentRequest(RequestModel): + account_id: UUID -class UpdateOrganizationRequest(RequestModel): - """Rename or re-slug an organization. At least one field is required.""" - - name: str | None = Field(default=None, min_length=1, max_length=120) - slug: str | None = Field(default=None, max_length=63) +class ProjectAssignmentResponse(ResponseModel): + project: ProjectResponse -class PasswordResetAvailableResponse(BaseModel): - """Whether a reset link can still be used. - - One answer for every failure reason, so it cannot be used to learn whether a - given link ever existed. - """ - - available: bool - message: str - - -class ResetPasswordRequest(RequestModel): - """A token from a reset link, and the password to set with it.""" - - token: str = Field(min_length=16, max_length=256) - new_password: str = Field(min_length=1, max_length=1024) - - -class ChangeOwnPasswordRequest(RequestModel): - """A signed-in person changing their own password. - - The current password is required, so a stolen session cookie alone cannot - take the account over. It is not required by the first-login flow, which - exists precisely because the holder has only the one-time password. - """ - - current_password: str = Field(min_length=1, max_length=1024) - new_password: str = Field(min_length=1, max_length=1024) - - -class PasswordResetRequestedResponse(BaseModel): - """The outcome of an administrator asking for a reset. - - Says a link was sent and nothing more. The address is not echoed and no - detail about the account is returned, so the endpoint is safe to leave open - to an administrator who is entitled to see the account list anyway but not to - probe addresses outside their scope. - """ - - message: str - - -class SetOrganizationStatusRequest(RequestModel): - status: OrganizationStatus - reason: str | None = Field(default=None, max_length=200) - - -class OrganizationStatusChangeResponse(BaseModel): - organization: OrganizationResponse - revoked_sessions: int - - -class OrganizationSummaryResponse(BaseModel): - """Counts for the administrator dashboard. - - Derived from the caller's visible scope on every request, so it can never - report a number the caller could not obtain from the list endpoints. - """ - +class OrganizationSummaryResponse(ResponseModel): total_organizations: int active_organizations: int total_accounts: int @@ -556,34 +351,200 @@ class OrganizationSummaryResponse(BaseModel): organization_admin_count: int -__all__ = [ - "AccountListResponse", - "AccountResponse", - "AuditEventResponse", - "AuditListResponse", - "ChangeFirstPasswordRequest", - "CreateAccountRequest", - "CreateAccountResponse", - "CredentialSummaryResponse", - "ErrorResponse", - "HealthResponse", - "LoginRequest", - "LoginResponse", - "LogoutResponse", - "CreateOrganizationRequest", - "OrganizationListResponse", - "OrganizationResponse", - "OrganizationStatusChangeResponse", - "OrganizationSummaryResponse", - "ResendVerificationRequest", - "ResendVerificationResponse", - "RevokeSessionsResponse", - "SessionUserResponse", - "SetAccountStatusRequest", - "SetOrganizationStatusRequest", - "StatusChangeResponse", - "UpdateOrganizationRequest", - "UserResponse", - "VerifyEmailRequest", - "VerifyEmailResponse", -] +class AuditEventResponse(ResponseModel): + id: str + occurred_at: datetime + action: str + outcome: str + actor_user_id: UUID | None = None + target_user_id: UUID | None = None + organization_id: UUID | None = None + request_id: str | None = None + detail: dict[str, Any] = Field(default_factory=dict) + + +class AuditListResponse(ResponseModel): + items: list[AuditEventResponse] + total: int + limit: int + offset: int + + +class WorkspaceProjectResponse(ProjectResponse): + """A project row inside ``/api/workspace`` with per-caller assignment state.""" + + assigned_to_me: bool = False + + +class WorkspaceResponse(ResponseModel): + """``GET /api/workspace``: the signed-in viewer's organization and its work.""" + + user: UserResponse + organization: OrganizationResponse | None + projects: list[WorkspaceProjectResponse] + total: int + assigned_to_me: list[WorkspaceProjectResponse] + + +class ProjectDetailResponse(ResponseModel): + project: ProjectResponse + languages: list[dict[str, Any]] + review: dict[str, Any] = Field(default_factory=dict) + + +class ProjectStatusLanguageResponse(ResponseModel): + language_tag: str + language_label: str + role: str + source_present: bool + translation: dict[str, Any] + audio: dict[str, Any] + outputs: dict[str, Any] + review: dict[str, Any] + updated_at: str | None = None + + +class ProjectStatusResponse(ResponseModel): + project_id: str + languages: list[ProjectStatusLanguageResponse] + + +# --------------------------------------------------------------------------- +# Translation queue + + +class TranslationQueueRequest(RequestModel): + language_tag: str = Field(min_length=1, max_length=16) + + +class TranslationQueueResponse(ResponseModel): + project_id: UUID + language_tag: str + state: str + job_id: str | None = None + queued_at: datetime | None = None + + +# --------------------------------------------------------------------------- +# Audio queue + + +class AudioQueueRequest(RequestModel): + language_tag: str = Field(min_length=1, max_length=16) + voice_config: dict[str, Any] = Field(default_factory=dict) + + +class AudioQueueResponse(ResponseModel): + project_id: UUID + language_tag: str + state: str + job_id: str | None = None + queued_at: datetime | None = None + + +# --------------------------------------------------------------------------- +# Format metadata (EPUB / PDF) + + +class FormatMetadataRequest(RequestModel): + language_tag: str = Field(min_length=1, max_length=16) + cover: str | None = None + isbn: str | None = Field(default=None, max_length=64) + blurb: str | None = None + amazon_search_terms: str | None = None + print_profile: str | None = Field(default=None, max_length=64) + page_count: str | None = Field(default=None, max_length=32) + + +class FormatMetadataResponse(ResponseModel): + project_id: UUID + language_tag: str + saved: bool + updated_at: datetime | None = None + + +# --------------------------------------------------------------------------- +# Utterance review by chapter + + +class UtteranceReviewResponse(ResponseModel): + project_id: UUID + language_tag: str + chapter_id: str + utterances: list[dict[str, Any]] + speakers: list[dict[str, Any]] + + +# --------------------------------------------------------------------------- +# Front matter (per language) + + +class FrontMatterItemRequest(RequestModel): + key: str = Field(min_length=1, max_length=128) + label: str = Field(min_length=1, max_length=128) + value: str | None = None + output_page: str | None = Field(default=None, max_length=16) + + +class FrontMatterItemResponse(ResponseModel): + key: str + label: str + value: str | None = None + output_page: str | None = None + + +class FrontMatterRequest(RequestModel): + language_tag: str = Field(min_length=1, max_length=16) + items: list[FrontMatterItemRequest] = Field(default_factory=list) + + +class FrontMatterResponse(ResponseModel): + project_id: UUID + language_tag: str + items: list[FrontMatterItemResponse] + version: int = 1 + + +class HealthResponse(ResponseModel): + status: str + service: str + version: str + environment: str + repository: str + cookie_secure: bool + csrf_required: bool + mail_deliverable: bool + mail_delivery_waived: bool + setup_required: bool + time: datetime | None = None + + +class LogoutResponse(ResponseModel): + logged_out: bool = True + + +class ResetPasswordRequest(RequestModel): + token: str = Field(min_length=1, max_length=4096) + new_password: str = Field(min_length=1, max_length=1024) + + +class PasswordResetAvailableResponse(ResponseModel): + available: bool + + +class SessionUserResponse(ResponseModel): + user: UserResponse + role: RoleName + organization_id: UUID | None = None + must_change_password: bool = False + csrf_token: str | None = None + + + +class OrganizationStatusChangeResponse(ResponseModel): + organization: OrganizationResponse + revoked_sessions: int = 0 + + + + diff --git a/umbratrans_web/service.py b/umbratrans_web/service.py index 79f02fb..94bd2ba 100644 --- a/umbratrans_web/service.py +++ b/umbratrans_web/service.py @@ -34,9 +34,11 @@ Rules enforced here: from __future__ import annotations +import json import logging from dataclasses import dataclass, field, replace -from datetime import datetime, timedelta +from datetime import datetime, timedelta, timezone +from pathlib import Path from types import MappingProxyType from typing import Callable, Mapping, Sequence from uuid import UUID, uuid4 @@ -109,9 +111,22 @@ from .security import ( utc_now, ) from .tenancy import TenantScope, resolve_target, scope_for +from .project_status import ProjectStatusAdapter +from .schemas import ( + AudioQueueResponse, + FrontMatterResponse, + FormatMetadataResponse, + TranslationQueueResponse, + UtteranceReviewResponse, +) LOGGER = logging.getLogger("umbratrans_web.service") +#: In-memory job store for language-project workflow jobs (translation, +#: audio queue, etc.). The productions directory is read-only in some +#: deployments, so job state is held here and reported honestly as ephemeral. +_JOB_STORE: dict[str, dict] = {} + #: Sources that may request the operator-only local temporary-password path. #: Sources that may prove the operator path. An allowlist, not a free string, #: because holding one of these is what permits creating an account with a local @@ -165,6 +180,10 @@ ACTION_PROJECT_CREATED = "project.created" ACTION_PROJECT_STATUS_CHANGED = "project.status_changed" ACTION_PROJECT_ASSIGNMENT_ADDED = "project.assignment_added" ACTION_PROJECT_ASSIGNMENT_REMOVED = "project.assignment_removed" +ACTION_TRANSLATION_QUEUED = "translation.queued" +ACTION_AUDIO_QUEUED = "audio.queued" +ACTION_FORMAT_METADATA_SAVED = "format.metadata_saved" +ACTION_FRONT_MATTER_SAVED = "front_matter.saved" ACTION_PASSWORD_RESET_REQUESTED = "password.reset_requested" ACTION_PASSWORD_RESET_COMPLETED = "password.reset_completed" ACTION_SESSION_REVOKED = "session.revoked" @@ -2185,6 +2204,733 @@ class AccountService: ) return self._project_view(updated) + # -- language-project workflow ------------------------------------------ + + def _productions_dir_for(self, project: Project) -> Path | None: + """Return the productions directory for a project, or None if unmapped.""" + from .project_status import _SLUG_TO_PROD + + prod_name = _SLUG_TO_PROD.get(getattr(project, "slug", "")) + if not prod_name: + return None + root = self._settings.productions_root + if root is None: + return None + return Path(root) / prod_name + + def _job_key(self, project_id: UUID, language_tag: str, job_type: str) -> str: + return f"{project_id}:{language_tag}:{job_type}" + + def _read_json(self, path: Path) -> dict | None: + if not path.is_file(): + return None + try: + with path.open("r", encoding="utf-8") as fh: + data = json.load(fh) + return data if isinstance(data, dict) else None + except Exception: + return None + + def _parse_utc(self, value: str | None) -> datetime | None: + if not value: + return None + try: + return datetime.fromisoformat(value) + except ValueError: + return None + + def queue_translation( + self, + actor: Principal, + project_id: UUID, + language_tag: str, + *, + request_id: str | None = None, + ) -> TranslationQueueResponse: + """Queue a translation job for one language project.""" + + scope = scope_for(actor) + project = self._repository.get_project(project_id, organization_id=scope.organization_id) + if project is None: + if scope.unrestricted: + raise NotFoundError("project_not_found") + raise ForbiddenError("cross_organization_forbidden") + + adapter = ProjectStatusAdapter(self._settings.productions_root) + status = adapter.status_for(project) + lang = next( + (l for l in status.get("languages", []) if l.get("language_tag") == language_tag), + None, + ) + if lang is None: + raise NotFoundError("language_not_found") + tr_state = (lang.get("translation") or {}).get("state") + if tr_state in ("in_progress", "complete"): + raise ConflictError( + "translation_already_active", + f"Translation for {language_tag} is already {tr_state}.", + ) + + job_id = str(uuid4()) + job_record = { + "job_id": job_id, + "project_id": str(project_id), + "language_tag": language_tag, + "type": "translation", + "state": "queued", + "created_at": datetime.now(timezone.utc).isoformat(), + } + _JOB_STORE[self._job_key(project_id, language_tag, "translation")] = job_record + + self._audit( + action=ACTION_TRANSLATION_QUEUED, + outcome="success", + actor=actor, + organization_id=project.organization_id, + request_id=request_id, + detail={ + "project_slug": project.slug, + "language_tag": language_tag, + "job_id": job_id, + }, + ) + LOGGER.info( + "translation queued project_id=%s language_tag=%s job_id=%s", + project_id, + language_tag, + job_id, + ) + return TranslationQueueResponse( + project_id=project_id, + language_tag=language_tag, + state="queued", + job_id=job_id, + queued_at=datetime.now(timezone.utc), + ) + + def queue_audio( + self, + actor: Principal, + project_id: UUID, + language_tag: str, + voice_config: dict | None = None, + *, + request_id: str | None = None, + ) -> AudioQueueResponse: + """Queue an audio generation job for one language project. + + Refuses unless the language's translation is already complete, matching + the workflow: translate first, then audio. + """ + + scope = scope_for(actor) + project = self._repository.get_project(project_id, organization_id=scope.organization_id) + if project is None: + if scope.unrestricted: + raise NotFoundError("project_not_found") + raise ForbiddenError("cross_organization_forbidden") + + adapter = ProjectStatusAdapter(self._settings.productions_root) + status = adapter.status_for(project) + lang = next( + (l for l in status.get("languages", []) if l.get("language_tag") == language_tag), + None, + ) + if lang is None: + raise NotFoundError("language_not_found") + tr_state = (lang.get("translation") or {}).get("state") + if tr_state != "complete": + raise ConflictError( + "translation_not_complete", + f"Translation for {language_tag} must be complete before audio can be queued (current: {tr_state}).", + ) + + job_id = str(uuid4()) + job_record = { + "job_id": job_id, + "project_id": str(project_id), + "language_tag": language_tag, + "type": "audio", + "state": "queued", + "voice_config": voice_config or {}, + "created_at": datetime.now(timezone.utc).isoformat(), + } + _JOB_STORE[self._job_key(project_id, language_tag, "audio")] = job_record + + self._audit( + action=ACTION_AUDIO_QUEUED, + outcome="success", + actor=actor, + organization_id=project.organization_id, + request_id=request_id, + detail={ + "project_slug": project.slug, + "language_tag": language_tag, + "job_id": job_id, + }, + ) + LOGGER.info( + "audio queued project_id=%s language_tag=%s job_id=%s", + project_id, + language_tag, + job_id, + ) + return AudioQueueResponse( + project_id=project_id, + language_tag=language_tag, + state="queued", + job_id=job_id, + queued_at=datetime.now(timezone.utc), + ) + + def set_format_metadata( + self, + actor: Principal, + project_id: UUID, + language_tag: str, + metadata: dict, + *, + request_id: str | None = None, + ) -> FormatMetadataResponse: + """Save format metadata (EPUB/PDF) for one language project.""" + + scope = scope_for(actor) + project = self._repository.get_project(project_id, organization_id=scope.organization_id) + if project is None: + if scope.unrestricted: + raise NotFoundError("project_not_found") + raise ForbiddenError("cross_organization_forbidden") + + prod_dir = self._productions_dir_for(project) + if prod_dir is None: + # No productions directory: hold metadata in the in-memory job store. + key = self._job_key(project_id, language_tag, "format") + _JOB_STORE[key] = { + "language_tag": language_tag, + "updated_at": datetime.now(timezone.utc).isoformat(), + **{k: v for k, v in metadata.items() if v is not None}, + } + self._audit( + action=ACTION_FORMAT_METADATA_SAVED, + outcome="success", + actor=actor, + organization_id=project.organization_id, + request_id=request_id, + detail={ + "project_slug": project.slug, + "language_tag": language_tag, + }, + ) + return FormatMetadataResponse( + project_id=project_id, + language_tag=language_tag, + saved=True, + updated_at=datetime.now(timezone.utc), + ) + + meta_dir = prod_dir / "metadata" / language_tag + meta_dir.mkdir(parents=True, exist_ok=True) + + existing = self._read_json(meta_dir / "format.json") or {} + existing.update( + {key: value for key, value in metadata.items() if value is not None} + ) + existing["language_tag"] = language_tag + existing["updated_at"] = datetime.now(timezone.utc).isoformat() + self._write_json(meta_dir / "format.json", existing) + + self._audit( + action=ACTION_FORMAT_METADATA_SAVED, + outcome="success", + actor=actor, + organization_id=project.organization_id, + request_id=request_id, + detail={ + "project_slug": project.slug, + "language_tag": language_tag, + }, + ) + return FormatMetadataResponse( + project_id=project_id, + language_tag=language_tag, + saved=True, + updated_at=datetime.now(timezone.utc), + ) + + def get_format_metadata( + self, + actor: Principal, + project_id: UUID, + language_tag: str, + ) -> FormatMetadataResponse: + """Read format metadata for one language project.""" + + scope = scope_for(actor) + project = self._repository.get_project(project_id, organization_id=scope.organization_id) + if project is None: + if scope.unrestricted: + raise NotFoundError("project_not_found") + raise ForbiddenError("cross_organization_forbidden") + + prod_dir = self._productions_dir_for(project) + if prod_dir is None: + stored = _JOB_STORE.get(self._job_key(project_id, language_tag, "format")) + return FormatMetadataResponse( + project_id=project_id, + language_tag=language_tag, + saved=stored is not None, + updated_at=self._parse_utc(stored.get("updated_at") if stored else None), + ) + + existing = self._read_json(prod_dir / "metadata" / language_tag / "format.json") + # Also check in-memory store (for deployments with read-only productions). + stored = _JOB_STORE.get(self._job_key(project_id, language_tag, "format")) + if stored and (not existing or stored.get("updated_at", "") >= existing.get("updated_at", "")): + existing = stored + return FormatMetadataResponse( + project_id=project_id, + language_tag=language_tag, + saved=existing is not None, + updated_at=self._parse_utc(existing.get("updated_at") if existing else None), + ) + + def get_chapters( + self, + actor: Principal, + project_id: UUID, + language_tag: str, + ) -> list[dict]: + """Return chapter metadata (id, title, matter, paragraph count) for one language project. + + English chapters are read from source/book.json. Translated chapters + are read from runs//translation.json. + """ + + scope = scope_for(actor) + project = self._repository.get_project(project_id, organization_id=scope.organization_id) + if project is None: + if scope.unrestricted: + raise NotFoundError("project_not_found") + raise ForbiddenError("cross_organization_forbidden") + + prod_dir = self._productions_dir_for(project) + if prod_dir is None: + return [] + + is_source = language_tag == "en" + + if is_source: + book_path = prod_dir / "source" / "book.json" + book = self._read_json(book_path) + if book is None: + return [] + chapters = [] + for ch in book.get("chapters", []): + chapters.append({ + "id": ch.get("id", ""), + "title": ch.get("title", ""), + "matter": ch.get("matter", "body"), + "index": ch.get("index", 0), + "paragraph_count": len(ch.get("paragraphs", []) or []), + }) + chapters.sort(key=lambda c: c["index"]) + return chapters + + # Translated language: scan runs directory for chapter folders. + runs_dir = prod_dir / "runs" + if not runs_dir.is_dir(): + return [] + chapters = [] + for ch_dir in sorted(runs_dir.iterdir()): + if ch_dir.is_dir() and ch_dir.name.startswith("ch-"): + translation_path = ch_dir / "translation.json" + data = self._read_json(translation_path) + if data is None: + continue + chapters.append({ + "id": data.get("chapter_id", ch_dir.name), + "title": data.get("title", ch_dir.name), + "matter": "body", + "index": data.get("chapter_index", 0), + "paragraph_count": len(data.get("paragraphs", {}) or {}), + }) + chapters.sort(key=lambda c: c["index"]) + return chapters + + def get_chapter_content( + self, + actor: Principal, + project_id: UUID, + language_tag: str, + chapter_id: str, + ) -> dict: + """Return full paragraph content for one chapter (front matter editing). + + English chapters are read from source/book.json. Translated chapters + read from runs//translation.json. + """ + + scope = scope_for(actor) + project = self._repository.get_project(project_id, organization_id=scope.organization_id) + if project is None: + if scope.unrestricted: + raise NotFoundError("project_not_found") + raise ForbiddenError("cross_organization_forbidden") + + prod_dir = self._productions_dir_for(project) + if prod_dir is None: + return {"chapter_id": chapter_id, "matter": "body", "paragraphs": []} + + is_source = language_tag == "en" + + if is_source: + book_path = prod_dir / "source" / "book.json" + book = self._read_json(book_path) + if book is None: + return {"chapter_id": chapter_id, "matter": "body", "paragraphs": []} + for ch in book.get("chapters", []): + if ch.get("id") == chapter_id: + return { + "chapter_id": ch.get("id", chapter_id), + "title": ch.get("title", ""), + "matter": ch.get("matter", "body"), + "paragraphs": ch.get("paragraphs", []) or [], + } + return {"chapter_id": chapter_id, "matter": "body", "paragraphs": []} + + # Translated language: read from runs//translation.json + translation_path = prod_dir / "runs" / chapter_id / "translation.json" + data = self._read_json(translation_path) + if data is None: + return {"chapter_id": chapter_id, "matter": "body", "paragraphs": []} + paragraphs = data.get("paragraphs", {}) or {} + return { + "chapter_id": data.get("chapter_id", chapter_id), + "title": data.get("title", chapter_id), + "matter": "body", + "paragraphs": [ + {"id": key, "speaker": None, "type": "dialogue", "text": value} + for key, value in sorted(paragraphs.items()) + ], + } + + def get_utterance_review( + self, + actor: Principal, + project_id: UUID, + language_tag: str, + chapter_id: str, + ) -> UtteranceReviewResponse: + """Return utterances for one chapter of one language, for speaker-alignment review. + + English source text is read from ``source/book.json``; translated text is + read from ``runs//translation.json``. + """ + + scope = scope_for(actor) + project = self._repository.get_project(project_id, organization_id=scope.organization_id) + if project is None: + if scope.unrestricted: + raise NotFoundError("project_not_found") + raise ForbiddenError("cross_organization_forbidden") + + prod_dir = self._productions_dir_for(project) + if prod_dir is None: + return UtteranceReviewResponse( + project_id=project_id, + language_tag=language_tag, + chapter_id=chapter_id, + utterances=[], + speakers=[], + ) + + is_source = language_tag == "en" + + if is_source: + # English source: read from source/book.json + book_path = prod_dir / "source" / "book.json" + book = self._read_json(book_path) + if book is None: + return UtteranceReviewResponse( + project_id=project_id, + language_tag=language_tag, + chapter_id=chapter_id, + utterances=[], + speakers=[], + ) + paragraphs = [] + for ch in book.get("chapters", []): + if ch.get("id") == chapter_id: + paragraphs = ch.get("paragraphs", []) or [] + break + utterances = [ + { + "id": p.get("id", ""), + "text": p.get("text", ""), + "speaker": p.get("speaker"), + "speaker_confidence": None, + "needs_review": True, + } + for p in paragraphs + ] + # Collect all unique speakers from this chapter for the dropdown. + speaker_ids = set() + for p in paragraphs: + sp = p.get("speaker") + if sp: + speaker_ids.add(sp) + speakers = [ + {"id": sp, "name": sp, "role": "speaker"} + for sp in sorted(speaker_ids) + ] + else: + # Translated language: read from runs//translation.json. + # Speaker info comes from the English source chapter, matched by + # paragraph id, since translated text doesn't carry speaker data. + translation_path = prod_dir / "runs" / chapter_id / "translation.json" + data = self._read_json(translation_path) + if data is None: + return UtteranceReviewResponse( + project_id=project_id, + language_tag=language_tag, + chapter_id=chapter_id, + utterances=[], + speakers=[], + ) + trans_paragraphs = data.get("paragraphs", {}) or {} + + # Load English source for speaker data, matched by paragraph id. + speaker_by_id = {} + book_path = prod_dir / "source" / "book.json" + book = self._read_json(book_path) + if book is not None: + for ch in book.get("chapters", []): + if ch.get("id") == chapter_id: + for p in ch.get("paragraphs", []) or []: + speaker_by_id[p.get("id", "")] = p.get("speaker") + break + + utterances = [ + { + "id": key, + "text": value, + "speaker": speaker_by_id.get(key), + "speaker_confidence": None, + "needs_review": True, + } + for key, value in sorted(trans_paragraphs.items()) + ] + # Collect all unique speakers from the English source for the dropdown. + speaker_ids = set(speaker_by_id.values()) + speaker_ids.discard(None) + + # Build speaker dropdown options. + if speaker_ids: + speakers = [ + {"id": sp, "name": sp, "role": "speaker"} + for sp in sorted(speaker_ids) + ] + else: + speakers = [ + {"id": "narrator", "name": "Narrator", "role": "narrator"} + ] + + return UtteranceReviewResponse( + project_id=project_id, + language_tag=language_tag, + chapter_id=chapter_id, + utterances=utterances, + speakers=speakers, + ) + + def get_front_matter( + self, + actor: Principal, + project_id: UUID, + language_tag: str, + ) -> FrontMatterResponse: + """Read front-matter items for one language project.""" + + scope = scope_for(actor) + project = self._repository.get_project(project_id, organization_id=scope.organization_id) + if project is None: + if scope.unrestricted: + raise NotFoundError("project_not_found") + raise ForbiddenError("cross_organization_forbidden") + + prod_dir = self._productions_dir_for(project) + if prod_dir is None: + stored = _JOB_STORE.get(self._job_key(project_id, language_tag, "front_matter")) + if stored: + items = [ + FrontMatterItemResponse( + key=item.get("key", ""), + label=item.get("label", ""), + value=item.get("value"), + output_page=item.get("output_page"), + ) + for item in stored.get("items", []) + ] + return FrontMatterResponse( + project_id=project_id, + language_tag=language_tag, + items=items, + version=stored.get("version", 1), + ) + return FrontMatterResponse( + project_id=project_id, + language_tag=language_tag, + items=[], + version=1, + ) + + data = self._read_json(prod_dir / "front-matter" / language_tag / "items.json") + # Also check in-memory store (for deployments with read-only productions). + stored = _JOB_STORE.get(self._job_key(project_id, language_tag, "front_matter")) + if stored and (not data or stored.get("version", 0) >= data.get("version", 0)): + data = stored + if data is None: + return FrontMatterResponse( + project_id=project_id, + language_tag=language_tag, + items=[], + version=1, + ) + + items = [ + FrontMatterItemResponse( + key=item.get("key", ""), + label=item.get("label", ""), + value=item.get("value"), + output_page=item.get("output_page"), + ) + for item in data.get("items", []) + ] + return FrontMatterResponse( + project_id=project_id, + language_tag=language_tag, + items=items, + version=data.get("version", 1), + ) + + def save_front_matter( + self, + actor: Principal, + project_id: UUID, + language_tag: str, + items: list[dict], + *, + request_id: str | None = None, + ) -> FrontMatterResponse: + """Save front-matter items for one language project.""" + + scope = scope_for(actor) + project = self._repository.get_project(project_id, organization_id=scope.organization_id) + if project is None: + if scope.unrestricted: + raise NotFoundError("project_not_found") + raise ForbiddenError("cross_organization_forbidden") + + prod_dir = self._productions_dir_for(project) + if prod_dir is None: + # No productions directory: hold front matter in the in-memory job store. + key = self._job_key(project_id, language_tag, "front_matter") + version = int((_JOB_STORE.get(key) or {}).get("version", 1)) + 1 + item_records = [ + { + "key": item.get("key", ""), + "label": item.get("label", ""), + "value": item.get("value"), + "output_page": item.get("output_page"), + } + for item in items + ] + _JOB_STORE[key] = { + "language_tag": language_tag, + "items": item_records, + "version": version, + "updated_at": datetime.now(timezone.utc).isoformat(), + } + + self._audit( + action=ACTION_FRONT_MATTER_SAVED, + outcome="success", + actor=actor, + organization_id=project.organization_id, + request_id=request_id, + detail={ + "project_slug": project.slug, + "language_tag": language_tag, + "version": version, + }, + ) + return FrontMatterResponse( + project_id=project_id, + language_tag=language_tag, + items=[ + FrontMatterItemResponse( + key=r["key"], + label=r["label"], + value=r["value"], + output_page=r["output_page"], + ) + for r in item_records + ], + version=version, + ) + + fm_dir = prod_dir / "front-matter" / language_tag + fm_dir.mkdir(parents=True, exist_ok=True) + + existing = self._read_json(fm_dir / "items.json") or {} + version = int(existing.get("version", 1)) + 1 + + item_records = [ + { + "key": item.get("key", ""), + "label": item.get("label", ""), + "value": item.get("value"), + "output_page": item.get("output_page"), + } + for item in items + ] + self._write_json( + fm_dir / "items.json", + { + "language_tag": language_tag, + "items": item_records, + "version": version, + "updated_at": datetime.now(timezone.utc).isoformat(), + }, + ) + + self._audit( + action=ACTION_FRONT_MATTER_SAVED, + outcome="success", + actor=actor, + organization_id=project.organization_id, + request_id=request_id, + detail={ + "project_slug": project.slug, + "language_tag": language_tag, + "version": version, + }, + ) + return FrontMatterResponse( + project_id=project_id, + language_tag=language_tag, + items=[ + FrontMatterItemResponse( + key=r["key"], + label=r["label"], + value=r["value"], + output_page=r["output_page"], + ) + for r in item_records + ], + version=version, + ) + def workspace(self, actor: Principal) -> "WorkspaceSnapshot": """The signed-in caller's organization and the work inside it. diff --git a/web/assets/admin-view.js b/web/assets/admin-view.js new file mode 100644 index 0000000..1f54683 --- /dev/null +++ b/web/assets/admin-view.js @@ -0,0 +1,357 @@ +// Copyright 2026 Umbra Tools (umbratools.com) +// SPDX-License-Identifier: Apache-2.0 + +/* + * Embeddable administration views. + * + * `UmbraAdminView.mount(container, { section, onSectionChange })` renders one + * administration section into `container` and returns a handle with `show()` and + * `destroy()`. The Umbra Tools shell mounts this inline; the standalone `uadmin/` + * page keeps its own shell and script and does not use this module. + * + * Why this exists as a separate file: the control panel is served with + * `X-Frame-Options: DENY` and `frame-ancestors 'none'` on purpose, so it must never + * be framed. Embedding means rendering the same views in the same document, not + * loading the other page inside a frame. + * + * Only what the shell's rail exposes is implemented here: dashboard, accounts, + * organizations, projects, audit. Nodes stays on its standalone page (it has its + * own pairing flow) and is linked, not embedded. + */ +(function (global) { + 'use strict'; + + var api = global.UmbraApi; + if (!api) { + return; + } + + var el = api.el; + var q = api.query; + + var SUMMARY_CARDS = [ + { key: 'total_organizations', label: 'Organizations', note: 'Tenants in scope' }, + { key: 'active_organizations', label: 'Active organizations', note: 'Accept new accounts' }, + { key: 'total_accounts', label: 'Accounts', note: 'Every account in scope' }, + { key: 'active_accounts', label: 'Active accounts', note: 'Can open a session' }, + { key: 'suspended_accounts', label: 'Suspended accounts', note: 'Sessions revoked' }, + { key: 'pending_first_login', label: 'Awaiting first sign-in', note: 'Must change a temporary password' }, + { key: 'organization_admin_count', label: 'Organization admins', note: 'Administer their own tenant' } + ]; + + var STATUS_TEXT = { + active: 'Active', suspended: 'Suspended', deactivated: 'Deactivated', + on_hold: 'On hold', released: 'Released', archived: 'Archived' + }; + + function statusPill(status) { + var tone = + status === 'active' ? 'success' + : status === 'suspended' || status === 'on_hold' ? 'warning' + : status === 'released' ? 'info' + : 'muted'; + return el('span', { class: 'status-pill status-pill--' + tone, text: STATUS_TEXT[status] || api.label(status) }); + } + + function emptyBlock(title, body, action) { + var children = [ + el('p', { class: 'empty-state__title', text: title }), + el('p', { class: 'empty-state__body', text: body }) + ]; + if (action) { + children.push(action); + } + return el('div', { class: 'empty-state' }, children); + } + + function heading(eyebrow, title, body, action) { + var right = el('div', {}); + if (action) { + right.appendChild(action); + } + return el('div', { class: 'admin-heading' }, [ + el('div', {}, [ + el('p', { class: 'eyebrow', text: eyebrow }), + el('h1', { text: title }), + el('p', { text: body }) + ]), + right + ]); + } + + function loadingNote(text) { + return el('p', { class: 'loading-note', text: text }); + } + + function statGrid() { + return el('div', { class: 'admin-stat-grid' }, [loadingNote('Loading summary…')]); + } + + /* ---------------------------------------------------------- dashboard */ + + function renderDashboard(container) { + var grid = statGrid(); + var activity = el('div', { class: 'audit-list' }, [loadingNote('Loading recent events…')]); + + container.appendChild(heading('Administration', 'System overview', 'Counts cover only the organizations and accounts you are authorized to see.')); + container.appendChild(el('section', { class: 'admin-card' }, [ + el('div', { class: 'admin-card-heading' }, [ + el('div', {}, [el('p', { class: 'eyebrow', text: 'Totals' }), el('h2', { text: 'Account and organization summary' })]) + ]), + grid + ])); + container.appendChild(el('section', { class: 'admin-card' }, [ + el('div', { class: 'admin-card-heading' }, [ + el('div', {}, [el('p', { class: 'eyebrow', text: 'Traceability' }), el('h2', { text: 'Recent activity' })]) + ]), + activity + ])); + + api.getSummary().then(function (data) { + api.clear(grid); + SUMMARY_CARDS.forEach(function (card) { + var value = data && typeof data[card.key] === 'number' ? data[card.key] : 0; + grid.appendChild(el('article', { class: 'admin-stat' }, [ + el('span', { class: 'admin-stat-label', text: card.label }), + el('strong', { text: String(value) }), + el('small', { text: card.note }) + ])); + }); + }, function (error) { + api.clear(grid); + grid.appendChild(emptyBlock('Summary unavailable', api.describeError(error))); + }); + + api.listAudit({ limit: 10 }).then(function (data) { + api.clear(activity); + var events = (data && data.items) || []; + if (!events.length) { + activity.appendChild(emptyBlock('No recent activity', 'Nothing has been recorded yet.')); + return; + } + events.forEach(function (event) { + activity.appendChild(el('div', { class: 'audit-row' }, [ + el('span', { class: 'audit-row__when', text: api.formatDateTime(event.occurred_at) }), + el('span', { class: 'audit-row__action', text: event.action }), + statusPill(event.outcome === 'success' ? 'active' : 'suspended') + ])); + }); + }, function (error) { + api.clear(activity); + activity.appendChild(emptyBlock('Activity unavailable', api.describeError(error))); + }); + } + + /* ----------------------------------------------------------- accounts */ + + function renderAccounts(container) { + container.appendChild(heading('Identity', 'Accounts', 'Every account inside your administrative scope, with credential and session state.')); + + var tbody = el('tbody', {}); + container.appendChild(el('section', { class: 'admin-card' }, [ + el('div', { class: 'admin-table-wrap' }, [ + el('table', { class: 'admin-table admin-table--wide' }, [ + el('thead', {}, [el('tr', {}, [ + el('th', { scope: 'col', text: 'Account' }), + el('th', { scope: 'col', text: 'Email' }), + el('th', { scope: 'col', text: 'Role' }), + el('th', { scope: 'col', text: 'Status' }) + ])]), + tbody + ]) + ]) + ])); + + tbody.appendChild(el('tr', {}, [el('td', { colspan: '4' }, [loadingNote('Loading accounts…')])])); + + api.listAccounts({ limit: 50 }).then(function (data) { + api.clear(tbody); + var items = (data && data.items) || []; + if (!items.length) { + tbody.appendChild(el('tr', {}, [el('td', { colspan: '4' }, [emptyBlock('No accounts', 'No accounts are in scope.')])])); + return; + } + items.forEach(function (account) { + tbody.appendChild(el('tr', {}, [ + el('td', { text: account.display_name || account.username || '—' }), + el('td', { text: account.email || '—' }), + el('td', { text: api.label(account.role) }), + el('td', {}, [statusPill(account.status)]) + ])); + }); + }, function (error) { + api.clear(tbody); + tbody.appendChild(el('tr', {}, [el('td', { colspan: '4' }, [emptyBlock('Accounts unavailable', api.describeError(error))])])); + }); + } + + /* ------------------------------------------------------- organizations */ + + function renderOrganizations(container) { + container.appendChild(heading('Tenancy', 'Organizations', 'Tenants that own accounts.')); + + var tbody = el('tbody', {}); + container.appendChild(el('section', { class: 'admin-card' }, [ + el('div', { class: 'admin-table-wrap' }, [ + el('table', { class: 'admin-table admin-table--wide' }, [ + el('thead', {}, [el('tr', {}, [ + el('th', { scope: 'col', text: 'Organization' }), + el('th', { scope: 'col', text: 'Slug' }), + el('th', { scope: 'col', text: 'Status' }), + el('th', { scope: 'col', text: 'Members' }) + ])]), + tbody + ]) + ]) + ])); + + tbody.appendChild(el('tr', {}, [el('td', { colspan: '4' }, [loadingNote('Loading organizations…')])])); + + api.listOrganizations({ limit: 100 }).then(function (data) { + api.clear(tbody); + var items = (data && data.items) || []; + if (!items.length) { + tbody.appendChild(el('tr', {}, [el('td', { colspan: '4' }, [emptyBlock('No organizations', 'No organizations are in scope.')])])); + return; + } + items.forEach(function (organization) { + tbody.appendChild(el('tr', {}, [ + el('td', { text: organization.name }), + el('td', { text: organization.slug }), + el('td', {}, [statusPill(organization.status)]), + el('td', { text: String(organization.member_count == null ? '—' : organization.member_count) }) + ])); + }); + }, function (error) { + api.clear(tbody); + tbody.appendChild(el('tr', {}, [el('td', { colspan: '4' }, [emptyBlock('Organizations unavailable', api.describeError(error))])])); + }); + } + + /* ----------------------------------------------------------- projects */ + + function renderProjects(container) { + container.appendChild(heading('Work', 'Projects', 'Units of work inside your organization. Everyone in the organization sees every project.')); + + var tbody = el('tbody', {}); + container.appendChild(el('section', { class: 'admin-card' }, [ + el('div', { class: 'admin-table-wrap' }, [ + el('table', { class: 'admin-table' }, [ + el('thead', {}, [el('tr', {}, [ + el('th', { scope: 'col', text: 'Project' }), + el('th', { scope: 'col', text: 'Slug' }), + el('th', { scope: 'col', text: 'Status' }), + el('th', { scope: 'col', text: 'Working on it' }) + ])]), + tbody + ]) + ]) + ])); + + tbody.appendChild(el('tr', {}, [el('td', { colspan: '4' }, [loadingNote('Loading projects…')])])); + + api.listProjects({ limit: 200 }).then(function (data) { + api.clear(tbody); + // /api/uadmin/projects answers with `projects`, unlike the other admin + // lists, which answer with `items`. Accept either rather than rendering an + // empty table against a response that is not empty. + var items = (data && (data.projects || data.items)) || []; + if (!items.length) { + tbody.appendChild(el('tr', {}, [el('td', { colspan: '4' }, [emptyBlock('No projects yet', 'Create the first project from the control panel.')])])); + return; + } + items.forEach(function (project) { + tbody.appendChild(el('tr', {}, [ + el('td', { text: project.name }), + el('td', { text: project.slug }), + el('td', {}, [statusPill(project.status)]), + el('td', { text: (project.assigned_count || 0) === 1 ? '1 person' : (project.assigned_count || 0) + ' people' }) + ])); + }); + }, function (error) { + api.clear(tbody); + tbody.appendChild(el('tr', {}, [el('td', { colspan: '4' }, [emptyBlock('Projects unavailable', api.describeError(error))])])); + }); + } + + /* -------------------------------------------------------------- audit */ + + function renderAudit(container) { + container.appendChild(heading('Traceability', 'Audit log', 'Sanitized identity events.')); + + var tbody = el('tbody', {}); + container.appendChild(el('section', { class: 'admin-card' }, [ + el('div', { class: 'admin-table-wrap' }, [ + el('table', { class: 'admin-table admin-table--audit' }, [ + el('thead', {}, [el('tr', {}, [ + el('th', { scope: 'col', text: 'Occurred' }), + el('th', { scope: 'col', text: 'Action' }), + el('th', { scope: 'col', text: 'Outcome' }) + ])]), + tbody + ]) + ]) + ])); + + tbody.appendChild(el('tr', {}, [el('td', { colspan: '3' }, [loadingNote('Loading audit events…')])])); + + api.listAudit({ limit: 50 }).then(function (data) { + api.clear(tbody); + var events = (data && data.items) || []; + if (!events.length) { + tbody.appendChild(el('tr', {}, [el('td', { colspan: '3' }, [emptyBlock('No audit events', 'Nothing has been recorded yet.')])])); + return; + } + events.forEach(function (event) { + tbody.appendChild(el('tr', {}, [ + el('td', { text: api.formatDateTime(event.occurred_at) }), + el('td', { text: event.action }), + el('td', { text: event.outcome }) + ])); + }); + }, function (error) { + api.clear(tbody); + tbody.appendChild(el('tr', {}, [el('td', { colspan: '3' }, [emptyBlock('Audit log unavailable', api.describeError(error))])])); + }); + } + + /* ------------------------------------------------------------- mount */ + + var RENDERERS = { + dashboard: renderDashboard, + accounts: renderAccounts, + organizations: renderOrganizations, + projects: renderProjects, + audit: renderAudit + }; + + function mount(container, options) { + options = options || {}; + var current = options.section || 'dashboard'; + + function draw() { + api.clear(container); + var render = RENDERERS[current] || renderDashboard; + try { + render(container); + } catch (error) { + container.appendChild(emptyBlock('Section failed to render', api.describeError(error))); + } + } + + draw(); + + return { + show: function (section) { + if (section && section !== current) { + current = section; + draw(); + } + }, + destroy: function () { api.clear(container); }, + section: function () { return current; } + }; + } + + global.UmbraAdminView = { mount: mount }; +})(window); diff --git a/web/assets/api.js b/web/assets/api.js index 91300a7..aa6a14b 100644 --- a/web/assets/api.js +++ b/web/assets/api.js @@ -443,6 +443,14 @@ return request('/api/uadmin/projects', { params: filters, signal: signal }); } + function projectStatus(projectId, signal) { + return request('/api/workspace/' + encodeURIComponent(projectId) + '/status', { signal: signal }); + } + + function projectDetail(projectId, signal) { + return request('/api/workspace/' + encodeURIComponent(projectId), { signal: signal }); + } + function createProject(payload, signal) { return request('/api/uadmin/projects', { method: 'POST', body: payload, signal: signal }); } @@ -478,6 +486,107 @@ ); } + /* ------------------------------------------------- language-project workflow */ + + /** + * Queue a translation job for one language project. + */ + function queueTranslation(projectId, languageTag, signal) { + return request( + '/api/workspace/' + encodeURIComponent(projectId) + '/translation/' + encodeURIComponent(languageTag) + '/queue', + { method: 'POST', signal: signal } + ); + } + + /** + * Queue an audio generation job for one language project. + * Requires the language's translation to be complete first. + */ + function queueAudio(projectId, languageTag, voiceConfig, signal) { + return request( + '/api/workspace/' + encodeURIComponent(projectId) + '/audio/' + encodeURIComponent(languageTag) + '/queue', + { method: 'POST', body: { voice_config: voiceConfig || {} }, signal: signal } + ); + } + + /** + * Read format metadata (EPUB/PDF) for one language project. + */ + function getFormatMetadata(projectId, languageTag, signal) { + return request( + '/api/workspace/' + encodeURIComponent(projectId) + '/format/' + encodeURIComponent(languageTag), + { signal: signal } + ); + } + + /** + * Save format metadata (cover, ISBN, blurb, search terms, etc.) for one + * language project. + */ + function setFormatMetadata(projectId, languageTag, metadata, signal) { + return request( + '/api/workspace/' + encodeURIComponent(projectId) + '/format/' + encodeURIComponent(languageTag), + { method: 'POST', body: metadata, signal: signal } + ); + } + + /** + * Return utterances for one chapter of one language, for speaker-alignment + * review. + */ + function getUtteranceReview(projectId, languageTag, chapterId, signal) { + return request( + '/api/workspace/' + + encodeURIComponent(projectId) + + '/review/' + + encodeURIComponent(languageTag) + + '/chapters/' + + encodeURIComponent(chapterId), + { signal: signal } + ); + } + + /** + * Read front-matter items for one language project. + */ + function getFrontMatter(projectId, languageTag, signal) { + return request( + '/api/workspace/' + encodeURIComponent(projectId) + '/front-matter/' + encodeURIComponent(languageTag), + { signal: signal } + ); + } + + /** + * Save front-matter items for one language project. + */ + function saveFrontMatter(projectId, languageTag, items, signal) { + return request( + '/api/workspace/' + encodeURIComponent(projectId) + '/front-matter/' + encodeURIComponent(languageTag), + { method: 'POST', body: { language_tag: languageTag, items: items }, signal: signal } + ); + } + + /** + * Return chapter metadata for one language project. + */ + function getChapters(projectId, languageTag, signal) { + return request( + '/api/workspace/' + encodeURIComponent(projectId) + '/chapters/' + encodeURIComponent(languageTag), + { signal: signal } + ); + } + + /** + * Return full paragraph content for one chapter (front matter editing). + */ + function getChapterContent(projectId, languageTag, chapterId, signal) { + return request( + '/api/workspace/' + encodeURIComponent(projectId) + '/chapter-content/' + + encodeURIComponent(languageTag) + '/' + encodeURIComponent(chapterId), + { signal: signal } + ); + } + /* --------------------------------------------------------- DOM utilities */ function appendChild(node, child) { @@ -864,8 +973,19 @@ listProjects: listProjects, createProject: createProject, setProjectStatus: setProjectStatus, + projectStatus: projectStatus, + projectDetail: projectDetail, assignToProject: assignToProject, unassignFromProject: unassignFromProject, + queueTranslation: queueTranslation, + queueAudio: queueAudio, + getFormatMetadata: getFormatMetadata, + setFormatMetadata: setFormatMetadata, + getUtteranceReview: getUtteranceReview, + getFrontMatter: getFrontMatter, + saveFrontMatter: saveFrontMatter, + getChapters: getChapters, + getChapterContent: getChapterContent, request: request, el: el, clear: clearNode, diff --git a/web/assets/app.js b/web/assets/app.js index 8829ff6..06c3a15 100644 --- a/web/assets/app.js +++ b/web/assets/app.js @@ -61,7 +61,13 @@ } function destinationFor(role) { - return role === 'platform_admin' || role === 'organization_admin' ? 'uadmin/' : 'workspace.html'; + // Every session lands in the Umbra Tools shell, which hosts the control + // panel, project work, the editor, and audio as activities. Previously this + // sent administrators straight to `uadmin/` and members to `workspace.html`, + // which meant the shell was never reached and the two old pages drifted + // apart. The shell now owns the top banner, left rail, and activity switch, + // and embeds the control panel rather than replacing the navigation with it. + return 'umbra-app.html'; } function showPasswordToggle(button, input) { @@ -599,13 +605,28 @@ side.push(api.el('span', { class: 'status-pill marker--unclaimed', text: 'Needs someone' })); } - return api.el('article', { class: 'project-card project-card--' + project.status }, [ + var card = api.el('article', { class: 'project-card project-card--' + project.status, role: 'button', tabIndex: 0 }, [ api.el('div', { class: 'project-card__main' }, [ api.el('span', { class: 'project-card__name', text: project.name }), api.el('div', { class: 'project-card__meta' }, meta) ]), api.el('div', { class: 'project-card__side' }, side) ]); + card.addEventListener('click', function () { + try { + var url = 'workspace-project.html?id=' + encodeURIComponent(project.id); + window.location.href = url; + } catch (e) { + // ignore + } + }); + card.addEventListener('keydown', function (ev) { + if (ev.key === 'Enter' || ev.key === ' ') { + ev.preventDefault(); + card.click(); + } + }); + return card; } function setMetric(selector, value) { @@ -647,7 +668,7 @@ var mine = snapshot.assigned_to_me || []; var visible = filter === 'mine' ? mine : all; - api.clearNode(list); + api.clear(list); visible.forEach(function (project) { list.appendChild(projectCard(project)); }); @@ -814,3 +835,5 @@ setupWorkspace(); offerSetupIfRequired(); })(window); + +try { var p = new URL(window.location.href); if (p.searchParams.get('shell') === 'umbra') { window.location.href = 'umbra-app.html'; } } catch(e){} diff --git a/web/assets/styles.css b/web/assets/styles.css index 7b0fee4..a03da88 100644 --- a/web/assets/styles.css +++ b/web/assets/styles.css @@ -480,3 +480,201 @@ html.modal-open { overflow: hidden; } font-size: .95em; word-break: break-all; } + +/* === Umbra Tools New Shell === */ +:root { + --shell-bg: #0b1020; + --shell-panel: #0f1529; + --shell-border: #1f2937; + --shell-text: #e5e7eb; + --shell-muted: #94a3b8; + --shell-accent: #22d3ee; + --shell-accent-2: #38bdf8; +} + +[data-theme="light"] { + --shell-bg: #f8fafc; + --shell-panel: #ffffff; + --shell-border: #e5e7eb; + --shell-text: #0f172a; + --shell-muted: #475569; + --shell-accent: #06b6d4; + --shell-accent-2: #2563eb; +} + +.umbra-shell { display: grid; grid-template-rows: 4rem 1fr; grid-template-columns: 280px 1fr; grid-template-areas: "topbar topbar" "sidebar main"; min-height: 100vh; background: var(--shell-bg); color: var(--shell-text); } +.umbra-topbar { grid-area: topbar; display: grid; grid-template-columns: 280px 1fr auto; align-items: center; gap: 1rem; padding: 0 1rem; background: linear-gradient(180deg, rgba(30,41,59,.9), rgba(15,23,42,.9)); border-bottom: 1px solid var(--shell-border); } +.umbra-topbar__left { display: flex; align-items: center; gap: .75rem; } +.umbra-topbar__logo { width: 28px; height: 28px; border-radius: 6px; } +.umbra-topbar__brand { font-weight: 800; letter-spacing: .02em; } +.umbra-topbar__mode { background: var(--shell-panel); color: var(--shell-text); border: 1px solid var(--shell-border); border-radius: .5rem; padding: .4rem .6rem; } +.umbra-topbar__middle { color: var(--shell-muted); font-weight: 600; } +.umbra-topbar__right { display: flex; align-items: center; gap: .6rem; } +.umbra-user { display: flex; align-items: center; gap: .5rem; } +.umbra-user__meta { display: flex; flex-direction: column; line-height: 1.1; } +.umbra-user__name { font-size: .9rem; } +.umbra-user__role { font-size: .72rem; color: var(--shell-muted); } +.umbra-dropdown { background: var(--shell-panel); color: var(--shell-text); border: 1px solid var(--shell-border); border-radius: .5rem; padding: .35rem .5rem; } + +.umbra-sidebar { grid-area: sidebar; background: var(--shell-panel); border-right: 1px solid var(--shell-border); padding: .8rem; overflow-y: auto; } +.umbra-sidebar__section { margin-bottom: 1rem; } +.umbra-sidebar__title { color: var(--shell-muted); text-transform: uppercase; font-size: .7rem; letter-spacing: .12em; margin: .4rem 0 .6rem; } +.umbra-nav__item { display: flex; align-items: center; gap: .5rem; width: 100%; text-align: left; background: transparent; color: var(--shell-text); border: 0; padding: .5rem .6rem; border-radius: .5rem; cursor: pointer; text-decoration: none; } +.umbra-nav__item:hover { background: rgba(56,189,248,.12); } +.umbra-nav__item--active { background: linear-gradient(90deg, rgba(34,211,238,.18), transparent); border-left: 2px solid var(--shell-accent); } + +.umbra-main { grid-area: main; padding: 1.2rem 1.6rem; overflow-y: auto; } + +.project-row { display: grid; grid-template-columns: 1.2fr 1fr auto; gap: 1rem; align-items: center; padding: .8rem 1rem; background: var(--shell-panel); border: 1px solid var(--shell-border); border-radius: .7rem; margin-bottom: .6rem; } +.project-row__lang { font-weight: 700; } +.project-row__status { color: var(--shell-muted); } +.project-row__actions { display: flex; gap: .4rem; flex-wrap: wrap; } +.btn { background: linear-gradient(180deg, rgba(56,189,248,.9), rgba(14,165,233,.9)); color: white; border: 0; border-radius: .5rem; padding: .4rem .7rem; cursor: pointer; } +.btn--ghost { background: transparent; color: var(--shell-text); border: 1px solid var(--shell-border); } +.btn--subtle { background: rgba(148,163,184,.18); color: var(--shell-text); border: 1px solid var(--shell-border); } + +.format-tabs { display: flex; gap: .4rem; margin-bottom: 1rem; } +.format-tab { padding: .4rem .7rem; border-radius: .5rem; border: 1px solid var(--shell-border); background: var(--shell-panel); color: var(--shell-text); cursor: pointer; } +.format-tab--active { background: rgba(34,211,238,.18); border-color: rgba(34,211,238,.6); } + +.form-grid { display: grid; grid-template-columns: repeat(2, minmax(0,1fr)); gap: .8rem; } +.form-grid__full { grid-column: 1 / -1; } +.input { width: 100%; background: var(--shell-panel); color: var(--shell-text); border: 1px solid var(--shell-border); border-radius: .5rem; padding: .5rem .6rem; } + +.binder { display: grid; grid-template-columns: 260px 1fr; gap: 1rem; } +.binder__tree { background: var(--shell-panel); border: 1px solid var(--shell-border); border-radius: .7rem; padding: .6rem; } +.binder__editor { background: var(--shell-panel); border: 1px solid var(--shell-border); border-radius: .7rem; padding: 1rem; min-height: 70vh; } +.tree__item { padding: .35rem .5rem; border-radius: .4rem; cursor: pointer; } +.tree__item:hover { background: rgba(56,189,248,.12); } + +.versions { display: flex; gap: .4rem; flex-wrap: wrap; } +.version-pill { padding: .3rem .5rem; border-radius: .4rem; border: 1px solid var(--shell-border); background: var(--shell-panel); color: var(--shell-text); cursor: pointer; } + +.audio-layout { display: grid; grid-template-columns: 260px 1fr; gap: 1rem; } +.audio-sidebar { background: var(--shell-panel); border: 1px solid var(--shell-border); border-radius: .7rem; padding: .6rem; } +.audio-main { background: var(--shell-panel); border: 1px solid var(--shell-border); border-radius: .7rem; padding: 1rem; } + +/* === Umbra Tools shell: additions === */ +.umbra-topbar__brandlink { display: flex; align-items: center; gap: .6rem; color: inherit; text-decoration: none; } + +.umbra-main--framed { padding: 0; } +.umbra-frame { display: block; width: 100%; height: 100%; min-height: calc(100vh - 4rem); border: 0; background: var(--shell-bg); } + +.umbra-heading { margin-bottom: 1.2rem; } +.umbra-heading--row { display: flex; align-items: flex-start; justify-content: space-between; gap: 1rem; flex-wrap: wrap; } +.umbra-heading h1 { margin: .1rem 0 .3rem; font-size: 1.5rem; } +.umbra-heading .lede { margin: 0; color: var(--shell-muted); max-width: 60ch; } +.umbra-heading .eyebrow { margin: 0; color: var(--shell-accent); text-transform: uppercase; letter-spacing: .12em; font-size: .7rem; font-weight: 700; } + +.umbra-inline-actions { display: flex; gap: .5rem; flex-wrap: wrap; margin-top: 1rem; } + +.project-row__sub { display: block; color: var(--shell-muted); font-size: .72rem; font-weight: 500; margin-top: .1rem; } + +.versions__label { color: var(--shell-muted); font-size: .78rem; margin-right: .2rem; } + +/* Disabled controls must be legibly disabled, not just inert. */ +.btn[disabled], .umbra-nav__item[disabled] { opacity: .5; cursor: not-allowed; } +.btn:disabled { cursor: not-allowed; } + +/* === Umbra Tools shell: embedded admin views === */ +.audit-row { display: grid; grid-template-columns: 12rem 1fr auto; gap: .8rem; align-items: center; padding: .5rem .2rem; border-bottom: 1px solid var(--shell-border); } +.audit-row__when { color: var(--shell-muted); font-size: .78rem; } +.audit-row__action { font-weight: 600; } +.admin-table td .empty-state { padding: 1rem 0; } +.umbra-main .admin-card { background: var(--shell-panel); border: 1px solid var(--shell-border); border-radius: .7rem; } + +/* === Program / language selector visibility === */ +.umbra-topbar__mode--hidden { display: none !important; } + +/* Project rows list (program detail) */ +.project-rows { display: flex; flex-direction: column; gap: .4rem; margin-top: 1rem; } + +/* Language row (program detail view) */ +.language-row { display: grid; grid-template-columns: 1fr auto; gap: 1rem; align-items: center; padding: .75rem 1rem; background: var(--shell-panel); border: 1px solid var(--shell-border); border-radius: .7rem; margin-bottom: .6rem; cursor: pointer; } +.language-row:hover { background: rgba(56,189,248,.06); } +.language-row--selected { border-color: rgba(34,211,238,.6); background: rgba(34,211,238,.08); } +.language-row__label { display: flex; flex-direction: column; gap: .15rem; } +.language-row__label strong { font-weight: 700; } +.language-row__role { font-size: .7rem; color: var(--shell-muted); } +.language-row__actions { display: flex; gap: .35rem; flex-wrap: wrap; } + +/* Binder tree groups */ +.tree__group { color: var(--shell-muted); text-transform: uppercase; font-size: .7rem; letter-spacing: .12em; margin: .5rem 0 .3rem; padding: 0 .5rem; } + +/* Control panel: no topbar name when no program */ +#topbar-language-name { color: var(--shell-text); font-weight: 700; } + +/* Language detail panel */ +.language-detail { margin-top: 1.5rem; } +.language-detail .umbra-heading { margin-bottom: 1rem; } +.language-detail table { margin-top: 0.5rem; } + +/* Language row compact actions */ +.language-row__actions .btn { font-size: 0.8rem; padding: 0.3rem 0.5rem; } +.language-row__actions a.btn--ghost { font-size: 0.8rem; padding: 0.3rem 0.5rem; } + +/* Ensure program selector fits in topbar */ +.umbra-topbar__middle { display: flex; align-items: center; gap: 0.5rem; } +#program-select { min-width: 200px; } + +/* Audio layout */ +.audio-layout { display: grid; grid-template-columns: 260px 1fr; gap: 1rem; } +.audio-sidebar { background: var(--shell-panel); border: 1px solid var(--shell-border); border-radius: .7rem; padding: .6rem; } +.audio-main { background: var(--shell-panel); border: 1px solid var(--shell-border); border-radius: .7rem; padding: 1rem; } +.audio-section-list { list-style: none; padding: 0; margin: 1rem 0 0 0; } +.audio-section-list li { padding: .4rem .6rem; border-radius: .4rem; cursor: pointer; } +.audio-section-list li:hover { background: rgba(56,189,248,.12); } + +/* Chapter review placeholder */ +.chapter-review-placeholder { margin-top: 1rem; } + +/* Language detail panel spacing */ +.language-detail { margin-top: 1.5rem; } +.language-detail .umbra-heading { margin-bottom: 1rem; } +.language-detail table { margin-top: 0.5rem; } + +/* Program selector width */ +#program-select { min-width: 220px; } + +/* Language row compact actions */ +.language-row__actions .btn { font-size: 0.8rem; padding: 0.3rem 0.5rem; } +.language-row__actions a.btn--ghost { font-size: 0.8rem; padding: 0.3rem 0.5rem; } + +/* Ensure program selector fits in topbar */ +.umbra-topbar__middle { display: flex; align-items: center; gap: 0.5rem; } + +/* Utterance review */ +.utterance-list { margin-top: 1rem; border: 1px solid var(--shell-border); border-radius: .6rem; background: var(--shell-panel); } +.utterance-row { display: grid; grid-template-columns: 130px 90px 1fr 140px; gap: .5rem; padding: .5rem .75rem; border-bottom: 1px solid var(--shell-border); font-size: .85rem; } +.utterance-row:last-child { border-bottom: none; } +.utterance__id { font-family: monospace; color: var(--shell-muted); } +.utterance__text { color: var(--shell-text); } +.utterance__speaker { display: flex; align-items: center; } + +/* Front matter panel */ +.front-matter-panel { margin-top: 1.5rem; border: 1px solid var(--shell-border); border-radius: .6rem; background: var(--shell-panel); padding: 1rem; } +.front-matter-panel h3 { margin: 0 0 .75rem 0; font-size: .9rem; color: var(--shell-muted); } +.fm-item { display: grid; grid-template-columns: 160px 1fr 120px 120px; gap: .5rem; align-items: center; margin-bottom: .5rem; } +.fm-item label { font-size: .85rem; color: var(--shell-text); } +.fm-item input, .fm-item select { font-size: .85rem; } + +/* Utterance review panel */ +.utterance-review-panel { margin-top: 1rem; border: 1px solid var(--shell-border); border-radius: .6rem; background: var(--shell-panel); padding: 1rem; } +.utterance-header { display: grid; grid-template-columns: 140px 1fr 180px; gap: .5rem; padding: .5rem .75rem; border-bottom: 2px solid var(--shell-border); font-size: .85rem; font-weight: bold; color: var(--shell-muted); } + +.chapter-select { width: 160px; margin-bottom: 0.75rem; } + +/* Chapter items in edit sidebar */ +.tree__item--chapter { padding: 0.4rem 0.6rem; cursor: pointer; border-radius: 0.3rem; font-size: 0.85rem; } +.tree__item--chapter:hover { background: rgba(56,189,248,0.12); } +.tree__item--chapter.tree__item--active { background: #38bdf8; color: #0b1220; } + +/* Format toolbar */ +.format-toolbar { margin: 0.75rem 0; padding: 0.5rem; background: var(--shell-panel); border: 1px solid var(--shell-border); border-radius: 0.5rem; display: flex; gap: 0.4rem; } +.format-toolbar .btn { font-size: 0.8rem; padding: 0.3rem 0.6rem; } + +/* Edit main content layout */ +.edit-main-content { margin-top: 1rem; } + +/* Utterance row speaker select */ +.utterance-row select { font-size: 0.8rem; } diff --git a/web/assets/umbra-app.js b/web/assets/umbra-app.js new file mode 100644 index 0000000..a3477af --- /dev/null +++ b/web/assets/umbra-app.js @@ -0,0 +1,1570 @@ +// Copyright 2026 Umbra Tools (umbratools.com) +// SPDX-License-Identifier: Apache-2.0 + +/*\ + * Umbra Tools shell behaviour — Program / Project model. + * + * Two-level navigation: + * 1. Activity: Control Panel, Program, Edit, Audio, Format + * 2. Program / language: topbar selector lists all languages; switching + * changes the main panel without leaving the program. + * + * A Program is a book being translated and produced as audiobooks. Each + * language under a program is a "project" (full manuscript: front + body + + * back matter, independent per language). + * + * Workflow (per language, English first): + * 1. Review / validate English text + * 2. Queue audio generation for English + * 3. After audio done + final approval, start text translation per language + * 4. After translation, review utterances by chapter (speaker alignment) + * 5. Queue audio generation for the language + * 6. Edit front matter (ISBN, copyright, etc.) in parallel with audio + * 7. Format EPUB / PDF, generate output, download + * + * The control panel is mounted inline from assets/admin-view.js. + */ +(function (global) { + 'use strict'; + + var api = global.UmbraApi; + if (!api) { + return; + } + + var el = api.el; + var q = api.query; + + var ACTIVITIES = ['control', 'program', 'edit', 'audio', 'format', 'review']; + + var state = { + activity: null, + section: null, + programId: null, + languageCode: null, + chapterId: null, + session: null, + workspace: null, + programs: [], + programById: {} + }; + + /* ---------------------------------------------------------------- routing */ + + function readHash() { + var raw = (global.location.hash || '').replace(/^#/, ''); + if (!raw) { + return { activity: null, section: null, programId: null, languageCode: null }; + } + var parts = raw.split('/'); + var activity = parts[0]; + if (ACTIVITIES.indexOf(activity) === -1) { + return { activity: null, section: null, programId: null, languageCode: null }; + } + var langActivities = ['program', 'edit', 'audio', 'format', 'review']; + return { + activity: activity, + section: activity === 'control' ? (parts[1] || null) : null, + programId: langActivities.indexOf(activity) !== -1 ? (parts[1] || null) : null, + languageCode: langActivities.indexOf(activity) !== -1 ? (parts[2] || null) : null, + chapterId: activity === 'review' ? (parts[3] || null) : null + }; + } + + function writeHash(activity, extra1, extra2, extra3) { + var hash = '#' + activity; + if (extra1) { + hash += '/' + extra1; + if (extra2) { + hash += '/' + extra2; + if (extra3) { + hash += '/' + extra3; + } + } + } + if (global.location.hash !== hash) { + global.location.hash = hash; + } + } + + /* ----------------------------------------------------- banner / identity */ + + function isAdmin(user) { + return user && (user.role === 'platform_admin' || user.role === 'organization_admin'); + } + + function paintIdentity(session) { + var user = (session && session.user) || session || {}; + var name = user.display_name || user.username || 'User'; + + api.setText(q('#topbar-username'), name); + api.setText(q('#topbar-role'), api.label(user.role) || '—'); + api.setText(q('#topbar-avatar'), api.initials(user.display_name, user.username) || '··'); + + var option = q('#activity-select option[value="control"]'); + if (option) { + option.hidden = !isAdmin(user); + option.disabled = !isAdmin(user); + } + } + + /* -------------------------------------------------------------- left rail */ + + var ADMIN_SECTIONS = [ + { id: 'dashboard', label: 'Dashboard', glyph: '▦' }, + { id: 'accounts', label: 'Accounts', glyph: '♙' }, + { id: 'organizations', label: 'Organizations', glyph: '◈' }, + { id: 'projects', label: 'Projects', glyph: '◇' }, + { id: 'audit', label: 'Audit log', glyph: '≡' } + ]; + + function railSection(title, nodes) { + var section = el('div', { class: 'umbra-sidebar__section' }); + if (title) { + section.appendChild(el('div', { class: 'umbra-sidebar__title', text: title })); + } + nodes.forEach(function (node) { section.appendChild(node); }); + return section; + } + + function railButton(text, options) { + options = options || {}; + var props = { + class: 'umbra-nav__item' + (options.active ? ' umbra-nav__item--active' : ''), + type: 'button', + text: text + }; + if (options.onClick) { + props.on = { click: options.onClick }; + } + return el('button', props); + } + + function renderSidebar() { + var sidebar = q('#sidebar'); + if (!sidebar) { + return; + } + api.clear(sidebar); + + if (state.activity === 'control') { + sidebar.appendChild(railSection('Administration', ADMIN_SECTIONS.map(function (item) { + return railButton(item.glyph + ' ' + item.label, { + active: (state.section || 'dashboard') === item.id, + onClick: function () { writeHash('control', item.id); } + }); + }))); + sidebar.appendChild(railSection('Nodes', [ + railButton('◉ Nodes', { onClick: function () { global.location.href = 'uadmin/nodes.html'; } }) + ])); + return; + } + + if (state.activity === 'program') { + if (state.languageCode) { + // Language project level. + var langNodes = [ + railButton('Overview', { active: true }), + railButton('Metadata', {}), + railButton('Translate', {}), + railButton('Audio', {}), + railButton('Format', {}), + railButton('Publish', {}) + ]; + sidebar.appendChild(railSection('Project: ' + languageTagLabel(state.languageCode), langNodes)); + return; + } + // Program level. + var programNodes = state.programId + ? [ + railButton('Overview', { active: true }), + railButton('Languages', {}), + railButton('Import', {}), + railButton('Export', {}), + railButton('Build', {}), + railButton('Analyze', {}) + ] + : [ + railButton('Create', {}), + railButton('Import', {}), + railButton('Export', {}), + railButton('Build', {}), + railButton('Analyze', {}) + ]; + sidebar.appendChild(railSection('Program', programNodes)); + return; + } + + if (state.activity === 'edit') { + sidebar.appendChild(railSection('Binder', [ + railButton('Front matter', { active: true }), + railButton('Chapters', {}), + railButton('Scenes', {}), + railButton('Sections', {}), + railButton('Characters', {}), + railButton('Research', {}), + railButton('Trash', {}) + ])); + sidebar.appendChild(railSection('History', [ + railButton('Save version', {}) + ])); + return; + } + + if (state.activity === 'audio') { + sidebar.appendChild(railSection('Audio', [ + railButton('Overview', { active: true }), + railButton('Speakers', {}), + railButton('Proper names', {}), + railButton('Chapters', {}), + railButton('Assignments', {}), + railButton('Runs', {}), + railButton('Queue', {}) + ])); + return; + } + + if (state.activity === 'format') { + sidebar.appendChild(railSection('Format', [ + railButton('EPUB', { active: true }), + railButton('PDF', {}), + railButton('Metadata', {}), + railButton('ISBN', {}), + railButton('Cover', {}), + railButton('Publish', {}) + ])); + return; + } + } + + /* ------------------------------------------------------ control panel */ + + function renderControlPanel() { + var main = q('#main'); + api.clear(main); + + if (!global.UmbraAdminView || typeof global.UmbraAdminView.mount !== 'function') { + main.appendChild(el('div', { class: 'empty-state' }, [ + el('span', { class: 'empty-state__icon', text: '⚙' }), + el('p', { class: 'empty-state__title', text: 'Control panel module unavailable' }), + el('p', { class: 'empty-state__body', text: 'The administration views could not be loaded. Open the standalone control panel at uadmin/.' }) + ])); + return; + } + global.UmbraAdminView.mount(main, { + section: state.section || 'dashboard', + onSectionChange: function (id) { writeHash('control', id); } + }); + } + + /* ------------------------------------------------------- program activity */ + + function statusPill(status) { + var tone = + status === 'active' ? 'success' + : status === 'on_hold' ? 'warning' + : status === 'released' ? 'info' + : 'muted'; + var text = { + active: 'Active', on_hold: 'On hold', released: 'Released', archived: 'Archived' + }[status] || api.label(status); + return el('span', { class: 'status-pill status-pill--' + tone, text: text }); + } + + function languageTagLabel(code) { + var map = { + 'en': 'English (source)', + 'zh-Hans': 'Simplified Chinese (zh-CN)', + 'es-ES': 'Spanish (es-ES)', + 'fr-FR': 'French (fr-FR)', + 'de-DE': 'German (de-DE)', + 'it-IT': 'Italian (it-IT)', + 'pt-BR': 'Portuguese (pt-BR)', + 'ru-RU': 'Russian (ru-RU)', + 'ja-JP': 'Japanese (ja-JP)', + 'ko-KR': 'Korean (ko-KR)', + 'ar-SA': 'Arabic (ar-SA)' + }; + return map[code] || api.label(code) || code; + } + + // Target language order for program detail view (English source first). + var TARGET_LANGUAGE_ORDER = ['en', 'zh-Hans', 'es-ES', 'fr-FR', 'de-DE', 'it-IT', 'pt-BR', 'ru-RU', 'ja-JP', 'ko-KR', 'ar-SA']; + + function translationStatusLabel(state) { + if (state === 'complete') return { text: 'Text translated', tone: 'success' }; + if (state === 'in_progress') return { text: 'In progress', tone: 'info' }; + if (state === 'needs_review') return { text: 'Needs review', tone: 'warning' }; + return { text: 'Not started', tone: 'muted' }; + } + + function programCard(program) { + var card = el('article', { + class: 'project-card project-card--' + program.status, + role: 'button', tabIndex: '0' + }, [ + el('div', { class: 'project-card__main' }, [ + el('span', { class: 'project-card__name', text: program.name }), + el('div', { class: 'project-card__meta' }, [ + el('span', { class: 'project-card__slug', text: '/' + program.slug }), + el('span', { text: (program.assigned_count || 0) === 1 ? '1 person assigned' : (program.assigned_count || 0) + ' assigned' }) + ]) + ]), + el('div', { class: 'project-card__side' }, [ + statusPill(program.status), + program.assigned_to_me ? el('span', { class: 'status-pill marker--mine', text: 'On your programs' }) + : (!program.assigned_count ? el('span', { class: 'status-pill marker--unclaimed', text: 'Needs someone' }) : null) + ].filter(Boolean)) + ]); + card.addEventListener('click', function () { writeHash('program', program.id); }); + card.addEventListener('keydown', function (ev) { + if (ev.key === 'Enter' || ev.key === ' ') { + ev.preventDefault(); + writeHash('program', program.id); + } + }); + return card; + } + + function renderProgramList() { + var main = q('#main'); + api.clear(main); + + main.appendChild(el('div', { class: 'umbra-heading' }, [ + el('div', {}, [ + el('p', { class: 'eyebrow', text: 'Programs' }), + el('h1', { text: 'Select a program' }), + el('p', { class: 'lede', text: 'A program is a book being translated and produced as audiobooks. Choose one, then select a language project.' }) + ]) + ])); + + var programs = state.programs || []; + var list = el('div', { class: 'project-list' }); + if (!programs.length) { + list.appendChild(el('div', { class: 'empty-state' }, [ + el('span', { class: 'empty-state__icon', text: '◇' }), + el('p', { class: 'empty-state__title', text: 'No programs in scope' }), + el('p', { class: 'empty-state__body', text: 'No programs are visible to this account, or none exist yet.' }) + ])); + } else { + programs.forEach(function (program) { list.appendChild(programCard(program)); }); + } + main.appendChild(el('p', { class: 'list-summary', text: programs.length === 1 ? 'Showing 1 program.' : 'Showing ' + programs.length + ' programs.' })); + main.appendChild(list); + } + + function languageDropdownOptions(program) { + // All configured languages for this program, in fixed order. + return TARGET_LANGUAGE_ORDER.map(function (code) { + return el('option', { value: code, text: languageTagLabel(code) + ' — ' + program.name }); + }); + } + + function languageActions(program, code, status) { + var isSource = code === 'en'; + var tr = status && status.translation || {}; + var out = status && status.outputs || {}; + var actions = []; + + function onQueueOk(result) { + api.showToast('Queued: ' + languageTagLabel(code), 'success'); + // Refresh status to reflect new state + if (typeof api.projectStatus === 'function') { + api.projectStatus(program.id).then(function () { applyRoute(); }); + } + } + function onQueueErr(err) { + api.showToast(api.describeError(err), 'error'); + } + + if (isSource) { + // English source: full workflow options. + actions.push(el('button', { + class: 'btn btn--ghost', type: 'button', text: 'Edit front matter', + on: { click: function () { writeHash('edit', program.id, code); } } + })); + actions.push(el('button', { + class: 'btn', type: 'button', text: 'Review text', + on: { click: function () { writeHash('review', program.id, code, 'ch-6'); } } + })); + actions.push(el('button', { + class: 'btn', type: 'button', text: 'Queue audio', + on: { click: function () { + api.queueAudio(program.id, code, {}).then(onQueueOk, onQueueErr); + } } + })); + actions.push(el('button', { + class: 'btn btn--subtle', type: 'button', text: 'Format EPUB', + on: { click: function () { writeHash('format', program.id, code); } } + })); + actions.push(el('button', { + class: 'btn btn--subtle', type: 'button', text: 'Format PDF', + on: { click: function () { writeHash('format', program.id, code); } } + })); + } else if (tr.state === 'complete') { + // Translation complete: edit front matter, review utterances, queue audio, format. + actions.push(el('button', { + class: 'btn btn--ghost', type: 'button', text: 'Edit front matter', + on: { click: function () { writeHash('edit', program.id, code); } } + })); + actions.push(el('button', { + class: 'btn btn--subtle', type: 'button', text: 'Review utterances', + on: { click: function () { writeHash('review', program.id, code, 'ch-6'); } } + })); + actions.push(el('button', { + class: 'btn', type: 'button', text: 'Queue audio', + on: { click: function () { + api.queueAudio(program.id, code, {}).then(onQueueOk, onQueueErr); + } } + })); + actions.push(el('button', { + class: 'btn btn--subtle', type: 'button', text: 'Format EPUB', + on: { click: function () { writeHash('format', program.id, code); } } + })); + actions.push(el('button', { + class: 'btn btn--subtle', type: 'button', text: 'Format PDF', + on: { click: function () { writeHash('format', program.id, code); } } + })); + if (out.epub) { + actions.push(el('a', { class: 'btn btn--ghost', href: out.epub.path, text: 'EPUB ↓', download: '' })); + } + if (out['pdf-paperback']) { + actions.push(el('a', { class: 'btn btn--ghost', href: out['pdf-paperback'].path, text: 'PDF ↓', download: '' })); + } + } else { + // Translation not started or in progress: queue translation. + actions.push(el('button', { + class: 'btn', type: 'button', text: 'Queue translation', + disabled: tr.state === 'in_progress', + title: tr.state === 'in_progress' ? 'Translation already queued.' : undefined, + on: { click: function () { + api.queueTranslation(program.id, code).then(onQueueOk, onQueueErr); + } } + })); + } + + return actions; + } + + function languageProjectCard(program, code, status) { + var label = languageTagLabel(code); + var isSource = code === 'en'; + var tr = status && status.translation || {}; + var out = status && status.outputs || {}; + + var badge = translationStatusLabel(tr.state || 'not_started'); + + var chapters = ''; + if (tr.chapters_done != null && tr.chapters_total != null) { + chapters = tr.chapters_done + '/' + tr.chapters_total + ' chapters'; + } else if (tr.state === 'complete') { + chapters = 'Complete'; + } + + return el('div', { class: 'language-row' + (code === state.languageCode ? ' language-row--selected' : ''), role: 'button', tabIndex: '0' }, [ + el('div', { class: 'language-row__label' }, [ + el('strong', { text: label }), + el('span', { class: 'language-row__role', text: isSource ? 'Source' : chapters }) + ]), + el('div', { class: 'language-row__status' }, [ + el('span', { class: 'status-pill status-pill--' + badge.tone, text: badge.text }) + ]), + el('div', { class: 'language-row__actions' }, languageActions(program, code, status)) + ]); + } + + function renderProgramDetail() { + var main = q('#main'); + api.clear(main); + + var program = state.programById[state.programId] || null; + if (!program) { + main.appendChild(el('div', { class: 'empty-state' }, [ + el('span', { class: 'empty-state__icon', text: '◇' }), + el('p', { class: 'empty-state__title', text: 'Program not found' }), + el('p', { class: 'empty-state__body', text: 'This program is not in your scope, or it has been removed.' }) + ])); + return; + } + + // Program selector: all languages, always visible. + var programSelect = q('#program-select'); + programSelect.classList.remove('umbra-topbar__mode--hidden'); + api.clear(programSelect); + programSelect.appendChild(el('option', { value: '', text: program.name + ' — select a language' })); + languageDropdownOptions(program).forEach(function (opt) { programSelect.appendChild(opt); }); + programSelect.value = state.languageCode || ''; + + if (state.languageCode) { + // -------- Language project view -------- + var langLabel = languageTagLabel(state.languageCode); + + api.setText(q('#topbar-language-name'), state.languageCode.toUpperCase() + ' — ' + program.name); + + main.appendChild(el('div', { class: 'umbra-heading' }, [ + el('div', {}, [ + el('p', { class: 'eyebrow', text: 'Language project' }), + el('h1', { text: langLabel }), + el('p', { class: 'lede', text: program.name + ' · ' + langLabel + '. Full manuscript: front, body, and back matter, independent per language.' }) + ]) + ])); + + main.appendChild(el('div', { class: 'umbra-inline-actions' }, [ + el('button', { class: 'btn btn--ghost', type: 'button', text: '← Programs', on: { click: function () { writeHash('program', program.id); } } }) + ])); + + if (typeof api.projectStatus === 'function') { + api.projectStatus(state.programId).then(function (data) { + var langs = (data && data.languages) || []; + var match = null; + langs.forEach(function (l) { if (l.language_tag === state.languageCode) { match = l; } }); + renderLanguageProject(main, program, state.languageCode, match); + }, function (err) { + main.appendChild(el('p', { class: 'list-summary list-summary--error', text: 'Status could not be loaded: ' + api.describeError(err) })); + main.appendChild(languageProjectCard(program, state.languageCode, null)); + }); + } else { + main.appendChild(languageProjectCard(program, state.languageCode, null)); + } + return; + } + + // -------- Program view (no language selected) -------- + api.setText(q('#topbar-language-name'), program.name); + + main.appendChild(el('div', { class: 'umbra-heading' }, [ + el('div', {}, [ + el('p', { class: 'eyebrow', text: 'Program' }), + el('h1', { text: program.name }), + el('p', { class: 'lede', text: 'Select a language to open its language project. English is reviewed first, then translations per language.' }) + ]) + ])); + + main.appendChild(el('div', { class: 'umbra-inline-actions' }, [ + el('button', { class: 'btn btn--ghost', type: 'button', text: '← All programs', on: { click: function () { writeHash('program'); } } }) + ])); + + var rows = el('div', { class: 'project-rows' }); + main.appendChild(rows); + + if (typeof api.projectStatus === 'function') { + api.projectStatus(state.programId).then(function (data) { + var langs = (data && data.languages) || []; + var pending = {}; + langs.forEach(function (l) { pending[l.language_tag] = l; }); + TARGET_LANGUAGE_ORDER.filter(function (code) { return pending[code] != null; }).forEach(function (code) { + rows.appendChild(languageProjectCard(program, code, pending[code] || null)); + }); + if (rows.children.length === 0) { + rows.appendChild(el('p', { class: 'list-summary', text: 'No languages defined for this program yet.' })); + } + }, function (err) { + api.clear(rows); + rows.appendChild(el('p', { class: 'list-summary list-summary--error', text: 'Status could not be loaded: ' + api.describeError(err) })); + TARGET_LANGUAGE_ORDER.forEach(function (code) { + rows.appendChild(languageProjectCard(program, code, null)); + }); + }); + } else { + TARGET_LANGUAGE_ORDER.forEach(function (code) { + rows.appendChild(languageProjectCard(program, code, null)); + }); + if (rows.children.length === 0) { + rows.appendChild(el('p', { class: 'list-summary', text: 'No languages defined for this program yet.' })); + } + } + } + + /* ----------------------------------------------- language project dashboard */ + + function renderLanguageProject(main, program, code, status) { + var tr = status && status.translation || {}; + var out = status && status.outputs || {}; + var badge = translationStatusLabel(tr.state || 'not_started'); + + main.appendChild(languageProjectCard(program, code, status)); + + // Translation status panel. + var detail = el('div', { class: 'language-detail' }); + + detail.appendChild(el('div', { class: 'umbra-heading' }, [ + el('div', {}, [ + el('p', { class: 'eyebrow', text: 'Translation status' }), + el('h2', { text: badge.text }), + el('p', { class: 'lede', text: tr.chapters_done != null && tr.chapters_total != null + ? tr.chapters_done + ' of ' + tr.chapters_total + ' chapters translated.' + : 'Translation status reported by the pipeline.' }) + ]) + ])); + + detail.appendChild(el('div', { class: 'umbra-inline-actions' }, [ + el('button', { class: 'btn btn--ghost', type: 'button', text: 'Edit front matter', on: { click: function () { writeHash('edit', program.id, code); } } }), + (tr.state === 'complete') ? el('button', { + class: 'btn btn--subtle', type: 'button', text: 'Review utterances by chapter', + on: { click: function () { writeHash('review', program.id, code, 'ch-6'); } } + }) : null, + (tr.state === 'complete') ? el('button', { + class: 'btn', type: 'button', text: 'Queue audio', + on: { click: function () { + api.queueAudio(program.id, code, {}).then( + function () { api.showToast('Audio queued for ' + languageTagLabel(code), 'success'); applyRoute(); }, + function (err) { api.showToast(api.describeError(err), 'error'); } + ); + } } + }) : null + ].filter(Boolean))); + + // Utterance-by-chapter review section (shown when translation complete). + if (tr.state === 'complete') { + detail.appendChild(el('div', { class: 'umbra-heading' }, [ + el('div', {}, [ + el('p', { class: 'eyebrow', text: 'Utterance review by chapter' }), + el('h2', { text: 'Speaker alignment & audio chapters' }), + el('p', { class: 'lede', text: 'Review utterances per chapter to confirm speaker assignments before audio generation. English utterance boundaries may differ from translation paragraph boundaries.' }) + ]) + ])); + detail.appendChild(el('div', { class: 'chapter-review-placeholder' }, [ + el('div', { class: 'empty-state' }, [ + el('span', { class: 'empty-state__icon', text: '⋮' }), + el('p', { class: 'empty-state__title', text: 'Utterance review view' }), + el('p', { class: 'empty-state__body', text: 'Per-chapter utterance listing with speaker assignments comes from the audio backend (UmbraAudio structure). Select a chapter to review its utterances and speaker alignment.' }) + ]) + ])); + } + + // Outputs panel. + detail.appendChild(el('div', { class: 'umbra-heading' }, [ + el('div', {}, [ + el('p', { class: 'eyebrow', text: 'Outputs' }), + el('h2', { text: 'EPUB / PDF / Audio' }), + el('p', { class: 'lede', text: 'Generated outputs appear here once the language project is formatted and produced.' }) + ]) + ])); + + var outRows = []; + if (out.epub) outRows.push(['EPUB', out.epub.path]); + if (out['pdf-paperback']) outRows.push(['PDF (paperback)', out['pdf-paperback'].path]); + if (!outRows.length) { + outRows.push(['EPUB', null]); + outRows.push(['PDF (paperback)', null]); + } + var outTable = el('table', { class: 'admin-table' }, [ + el('thead', {}, [el('tr', {}, [ + el('th', { scope: 'col', text: 'Format' }), + el('th', { scope: 'col', text: 'Download' }) + ])]), + el('tbody', {}, outRows.map(function (row) { + return el('tr', {}, [ + el('td', { text: row[0] }), + el('td', {}, row[1] ? el('a', { class: 'btn btn--ghost', href: row[1], text: 'Download', download: '' }) + : el('span', { class: 'status-pill status-pill--muted', text: 'Not produced' })) + ]); + })) + ]); + detail.appendChild(outTable); + + main.appendChild(detail); + } + + /* ------------------------------------------------------------ edit screen */ + + function renderEdit() { + var main = q('#main'); + api.clear(main); + + var program = state.programId ? (state.programById[String(state.programId)] || null) : null; + var langLabel = state.languageCode ? languageTagLabel(state.languageCode) : ''; + var isSource = state.languageCode === 'en'; + + main.appendChild(el('div', { class: 'umbra-heading umbra-heading--row' }, [ + el('div', {}, [ + el('p', { class: 'eyebrow', text: 'Edit' }), + el('h1', { text: program ? program.name : 'Edit' }), + el('p', { class: 'lede', text: langLabel ? ('Front matter and text for ' + langLabel + '.') : 'Front matter and manuscript structure.' }) + ]), + el('div', { class: 'versions' }, [ + el('span', { class: 'versions__label', text: 'Versions' }), + el('button', { class: 'version-pill', type: 'button', text: 'current' }), + el('button', { class: 'btn', type: 'button', text: 'Save version', disabled: true, title: 'Revision storage is not implemented yet.' }) + ]) + ])); + + // Load project status to determine what this language project shows. + if (!state.languageCode) { + main.appendChild(el('div', { class: 'empty-state' }, [ + el('span', { class: 'empty-state__icon', text: '◇' }), + el('p', { class: 'empty-state__title', text: 'No language selected' }), + el('p', { class: 'empty-state__body', text: 'Use the program selector above to choose a language project.' }) + ])); + return; + } + + if (typeof api.projectStatus === 'function') { + api.projectStatus(state.programId).then(function (data) { + var langs = (data && data.languages) || []; + var match = null; + langs.forEach(function (l) { if (l.language_tag === state.languageCode) { match = l; } }); + renderEditContent(main, program, state.languageCode, match); + }, function (err) { + renderEditContent(main, program, state.languageCode, null); + }); + } else { + renderEditContent(main, program, state.languageCode, null); + } + } + + function renderEditContent(main, program, code, status) { + var isSource = code === 'en'; + var tr = status && status.translation || {}; + var trState = tr.state || 'not_started'; + + // ---------- Translation not started: only queue translation ---------- + if (trState !== 'complete') { + main.appendChild(el('div', { class: 'empty-state' }, [ + el('span', { class: 'empty-state__icon', text: '◇' }), + el('h2', { text: languageTagLabel(code) + ' — not yet translated' }), + el('p', { class: 'lede', text: 'No translation exists for this language yet. Queue a translation job to begin.' }) + ])); + main.appendChild(el('div', { class: 'umbra-inline-actions' }, [ + el('button', { class: 'btn', type: 'button', text: 'Queue translation', + on: { click: function () { + api.queueTranslation(program.id, code).then( + function () { api.showToast('Translation queued for ' + languageTagLabel(code), 'success'); applyRoute(); }, + function (err) { api.showToast(api.describeError(err), 'error'); } + ); + } } + }), + el('button', { class: 'btn btn--ghost', type: 'button', text: '← Back', on: { click: function () { writeHash('program', program.id || ''); } } }) + ])); + return; + } + + // ---------- Translation complete: fetch chapters, build sidebar tree, render editor ---------- + var sidebar = q('#sidebar'); + var chapterTree = el('div', { class: 'binder__tree', 'aria-label': 'Binder' }); + chapterTree.appendChild(el('div', { class: 'tree__group', text: isSource ? 'Source text' : 'Translation' })); + + var editorSurface = el('div', { + class: 'binder__surface', contenteditable: 'true', spellcheck: 'true', + 'aria-label': 'Editable text' + }, [el('p', { text: 'Select a chapter from the left menu to edit its text.' })]); + editorSurface.style.display = 'none'; + + var uttPanel = el('div', { class: 'utterance-review-panel' }); + uttPanel.style.display = 'none'; + uttPanel.appendChild(el('h3', { text: isSource ? 'English source — review before validation' : 'Translated text — review before audio' })); + var uttList = el('div', { class: 'utterance-list' }); + uttPanel.appendChild(uttList); + + // Front matter items panel (metadata like ISBN, cover, etc.). + var fmPanel = el('div', { class: 'front-matter-panel' }); + fmPanel.style.display = 'none'; + fmPanel.appendChild(el('h3', { text: 'Front matter metadata' })); + + var currentChapterId = null; + var currentParagraphs = []; // {id, speaker, type, text} + + function buildChapterTree(chapters) { + api.clear(chapterTree); + // Audio format label at the top + chapterTree.appendChild(el('div', { class: 'tree__group', text: 'Audio Format' })); + chapterTree.appendChild(el('div', { class: 'tree__group', text: isSource ? 'Source text' : 'Translation' })); + + // Front matter section (matter starts with 'front') + var fmChapters = chapters.filter(function (c) { return c.matter && c.matter.startsWith('front'); }); + if (fmChapters.length) { + var fmGroup = el('div', { class: 'tree__group', text: 'Front matter' }); + fmChapters.forEach(function (ch) { + var item = el('div', { class: 'tree__item tree__item--child tree__item--chapter', 'data-chapter': ch.id, text: ch.title }); + fmGroup.appendChild(item); + }); + chapterTree.appendChild(fmGroup); + } + + // Back matter section (matter starts with 'back') + var bmChapters = chapters.filter(function (c) { return c.matter && c.matter.startsWith('back'); }); + if (bmChapters.length) { + var bmGroup = el('div', { class: 'tree__group', text: 'Back matter' }); + bmChapters.forEach(function (ch) { + var item = el('div', { class: 'tree__item tree__item--child tree__item--chapter', 'data-chapter': ch.id, text: ch.title }); + bmGroup.appendChild(item); + }); + chapterTree.appendChild(bmGroup); + } + + // Body section + var bodyChapters = chapters.filter(function (c) { return !c.matter || c.matter === 'body'; }); + if (bodyChapters.length) { + var bodyGroup = el('div', { class: 'tree__group', text: 'Body' }); + bodyChapters.forEach(function (ch) { + var item = el('div', { class: 'tree__item tree__item--child tree__item--chapter', 'data-chapter': ch.id, text: ch.title }); + bodyGroup.appendChild(item); + }); + chapterTree.appendChild(bodyGroup); + } + + // Click handlers for chapter items + chapterTree.querySelectorAll('.tree__item--chapter').forEach(function (item) { + item.addEventListener('click', function () { + chapterTree.querySelectorAll('.tree__item--chapter').forEach(function (i) { + i.classList.remove('tree__item--active'); + }); + item.classList.add('tree__item--active'); + currentChapterId = item.getAttribute('data-chapter'); + loadChapter(currentChapterId); + }); + }); + + // Place chapter tree in the LEFT SIDEBAR + api.clear(sidebar); + sidebar.appendChild(chapterTree); + } + + function loadChapter(chapterId) { + editorSurface.style.display = 'none'; + uttPanel.style.display = 'none'; + fmPanel.style.display = 'none'; + + // Find chapter metadata to determine matter + var chapterInfo = allChapters.find(function (c) { return c.id === chapterId; }); + var isFrontMatter = chapterInfo && chapterInfo.matter && chapterInfo.matter !== 'body'; + + if (isFrontMatter && typeof api.getChapterContent === 'function') { + // Front matter: show editable text with pandoc formatting + editorSurface.style.display = 'block'; + api.clear(editorSurface); + editorSurface.appendChild(el('p', { class: 'lede', text: 'Loading chapter content…' })); + api.getChapterContent(program.id, code, chapterId).then(function (data) { + api.clear(editorSurface); + var paragraphs = (data && data.paragraphs) || []; + if (!paragraphs.length) { + editorSurface.appendChild(el('p', { class: 'lede', text: 'No content for ' + chapterId + '.' })); + return; + } + currentParagraphs = paragraphs.map(function (p) { + return { chapterId: chapterId, id: p.id, speaker: p.speaker || 'narrator', type: p.type || 'narration', text: p.text || '' }; + }); + currentParagraphs.forEach(function (p) { + var paraEl = el('p', { 'data-para': p.id }); + paraEl.textContent = p.text; + paraEl.addEventListener('input', function () { + p.text = paraEl.textContent; + }); + editorSurface.appendChild(paraEl); + }); + editorSurface.style.display = 'block'; + toolbar.style.display = 'block'; + }, function (err) { + api.clear(editorSurface); + editorSurface.appendChild(el('p', { class: 'list-summary list-summary--error', text: 'Chapter content could not be loaded: ' + api.describeError(err) })); + editorSurface.style.display = 'block'; + toolbar.style.display = 'block'; + }); + return; + } + + // Body chapter: utterance-by-chapter review + var paraData = currentParagraphs.filter(function (p) { return p.chapterId === chapterId; }); + + if (!paraData.length && typeof api.getUtteranceReview === 'function') { + editorSurface.style.display = 'none'; + api.clear(editorSurface); + uttPanel.style.display = 'block'; + api.clear(uttList); + uttList.appendChild(el('p', { class: 'lede', text: 'Loading utterances…' })); + api.getUtteranceReview(program.id, code, chapterId).then(function (data) { + api.clear(uttList); + var utterances = (data && data.utterances) || []; + var speakers = (data && data.speakers) || []; + if (!utterances.length) { + uttList.appendChild(el('p', { class: 'lede', text: 'No utterances found for ' + chapterId + '.' })); + return; + } + uttList.appendChild(el('div', { class: 'utterance-header' }, [ + el('div', { class: 'utterance__speaker', text: 'Speaker' }), + el('div', { class: 'utterance__variation', text: 'Variation' }), + el('div', { class: 'utterance__text', text: 'Text' }), + el('div', { class: 'utterance__actions', text: 'Actions' }) + ])); + utterances.forEach(function (utt) { + uttList.appendChild(el('div', { class: 'utterance-row' }, [ + el('div', { class: 'utterance__speaker' }, [ + el('select', { class: 'input', 'data-utt': utt.id }, speakers.map(function (spk) { + var sel = spk.id === (utt.speaker || 'narrator'); + return el('option', { value: spk.id, text: spk.name, selected: sel }); + })) + ]), + el('div', { class: 'utterance__variation' }, [ + el('select', { class: 'input', 'data-utt': utt.id }, [ + el('option', { value: 'neutral', text: 'Neutral', selected: true }), + el('option', { value: 'angry', text: 'Angry' }), + el('option', { value: 'joyful', text: 'Joyful' }), + el('option', { value: 'terrified', text: 'Terrified' }), + el('option', { value: 'griefstricken', text: 'Griefstricken' }), + el('option', { value: 'shocked', text: 'Shocked' }), + el('option', { value: 'furious', text: 'Furious' }), + el('option', { value: 'tense', text: 'Tense' }), + el('option', { value: 'urgent', text: 'Urgent' }), + el('option', { value: 'horror', text: 'Horror' }), + el('option', { value: 'slow', text: 'Slow' }), + el('option', { value: 'moderate', text: 'Moderate' }), + el('option', { value: 'fast', text: 'Fast' }) + ]) + ]), + el('div', { class: 'utterance__text', text: utt.text }), + el('div', { class: 'utterance__actions' }, [ + el('button', { class: 'btn btn--ghost', type: 'button', text: '▶', title: 'Play audio', disabled: true, 'data-utt': utt.id }), + el('button', { class: 'btn btn--ghost', type: 'button', text: '⟳', title: 'Regenerate audio', disabled: true, 'data-utt': utt.id }), + el('button', { class: 'btn btn--ghost', type: 'button', text: '✕', title: 'Delete', disabled: true, 'data-utt': utt.id }) + ]) + ])); + }); + currentParagraphs = utterances.map(function (utt) { + return { chapterId: chapterId, id: utt.id, speaker: utt.speaker || 'narrator', type: 'dialogue', text: utt.text }; + }); + }, function (err) { + api.clear(uttList); + uttList.appendChild(el('p', { class: 'list-summary list-summary--error', text: 'Utterances could not be loaded: ' + api.describeError(err) })); + }); + return; + } + + if (paraData.length && paraData.some(function (p) { return p.text; })) { + editorSurface.style.display = 'block'; + api.clear(editorSurface); + paraData.forEach(function (p) { + var paraEl = el('p', { 'data-para': p.id }); + paraEl.textContent = p.text; + paraEl.addEventListener('input', function () { + p.text = paraEl.textContent; + }); + editorSurface.appendChild(paraEl); + }); + editorSurface.style.display = 'block'; + toolbar.style.display = 'block'; + } else { + uttPanel.style.display = 'block'; + api.clear(uttList); + uttList.appendChild(el('p', { class: 'lede', text: 'No editable text for ' + chapterId + '.' })); + } + } + + function applyFormat(openTag, closeTag) { + var sel = window.getSelection(); + if (!sel.rangeCount) return; + var range = sel.getRangeAt(0); + var selectedText = range.toString(); + if (!selectedText) return; + var newText = openTag + selectedText + closeTag; + range.deleteContents(); + range.insertNode(document.createTextNode(newText)); + var selEl = sel.anchorNode.parentElement; + var paraEl = selEl.closest('p[data-para]'); + if (paraEl) { + var paraId = paraEl.getAttribute('data-para'); + var p = currentParagraphs.find(function (x) { return x.id === paraId; }); + if (p) p.text = paraEl.textContent; + } + } + + // Fetch chapters and build sidebar tree + var allChapters = []; + if (typeof api.getChapters === 'function') { + api.getChapters(program.id, code).then(function (chapters) { + allChapters = chapters; + buildChapterTree(chapters); + if (chapters.length) { + var firstChapter = chapters[0]; + currentChapterId = firstChapter.id; + var firstItem = chapterTree.querySelector('.tree__item--chapter[data-chapter="' + firstChapter.id + '"]'); + if (firstItem) { + firstItem.classList.add('tree__item--active'); + } + loadChapter(firstChapter.id); + } + }, function (err) { + api.clear(main); + main.appendChild(el('p', { class: 'list-summary list-summary--error', text: 'Chapters could not be loaded: ' + api.describeError(err) })); + }); + } + + main.appendChild(el('div', { class: 'edit-main-content' }, [editorSurface, uttPanel])); + + // Format toolbar (shown when front matter text is editable). + var toolbar = el('div', { class: 'format-toolbar' }); + toolbar.style.display = 'none'; + toolbar.appendChild(el('button', { class: 'btn btn--ghost', type: 'button', text: 'Bold', title: 'Bold (**text**)', + on: { click: function () { applyFormat('**', '**'); } } })); + toolbar.appendChild(el('button', { class: 'btn btn--ghost', type: 'button', text: 'Italic', title: 'Italic (*text*)', + on: { click: function () { applyFormat('*', '*'); } } })); + toolbar.appendChild(el('button', { class: 'btn btn--ghost', type: 'button', text: 'Underline', title: 'Underline (text)', + on: { click: function () { applyFormat('', ''); } } })); + toolbar.appendChild(el('button', { class: 'btn btn--ghost', type: 'button', text: 'Heading', title: 'Heading (# text)', + on: { click: function () { applyFormat('# ', '\n'); } } })); + main.appendChild(toolbar); + + // Show toolbar when editor is visible + var _origLoad = loadChapter; + loadChapter = function (chapterId) { + _origLoad(chapterId); + toolbar.style.display = (editorSurface.style.display === 'block') ? 'block' : 'none'; + }; + + // Front matter items per language (metadata). + if (typeof api.getFrontMatter === 'function') { + api.getFrontMatter(program.id, code).then(function (data) { + var items = (data && data.items) || []; + fmPanel.style.display = 'block'; + items.forEach(function (item) { + fmPanel.appendChild(el('div', { class: 'fm-item' }, [ + el('label', { text: item.label || item.key }), + el('input', { + class: 'input', type: 'text', value: item.value || '', + 'data-key': item.key, + 'data-output': item.output_page || '' + }), + el('select', { class: 'input', 'data-key': item.key }, [ + el('option', { value: '', text: 'None' }), + el('option', { value: 'epub', text: 'EPUB' }), + el('option', { value: 'pdf', text: 'PDF' }), + el('option', { value: 'audio', text: 'Audio' }) + ]) + ])); + }); + fmPanel.appendChild(el('button', { + class: 'btn', type: 'button', text: 'Save front matter metadata', + on: { click: function () { + var out = []; + fmPanel.querySelectorAll('.fm-item input').forEach(function (inp) { + var key = inp.getAttribute('data-key'); + var sel = fmPanel.querySelector('.fm-item select[data-key="' + key + '"]'); + out.push({ + key: key, + label: inp.previousElementSibling ? inp.previousElementSibling.textContent : key, + value: inp.value, + output_page: sel ? sel.value : '' + }); + }); + api.saveFrontMatter(program.id, code, out).then( + function () { api.showToast('Front matter metadata saved', 'success'); }, + function (err) { api.showToast(api.describeError(err), 'error'); } + ); + } } + })); + main.appendChild(fmPanel); + }, function () { + // Non-fatal: front matter may not exist yet. + }); + } + + main.appendChild(el('div', { class: 'umbra-inline-actions' }, [ + el('button', { class: 'btn btn--ghost', type: 'button', text: '← Back', on: { click: function () { writeHash('program', program.id || ''); } } }) + ])); + } + + /* ----------------------------------------------------------- audio screen */ + + function renderAudio() { + var main = q('#main'); + api.clear(main); + + var program = state.programId ? (state.programById[String(state.programId)] || null) : null; + var langLabel = state.languageCode ? languageTagLabel(state.languageCode) : ''; + + main.appendChild(el('div', { class: 'umbra-heading' }, [ + el('div', {}, [ + el('p', { class: 'eyebrow', text: 'Audio' }), + el('h1', { text: langLabel ? (langLabel + ' audio') : 'Audio production' }), + el('p', { class: 'lede', text: program ? (program.name + ' · ' + langLabel) : 'Select a language project.' }) + ]) + ])); + + main.appendChild(el('div', { class: 'audio-layout' }, [ + el('div', { class: 'audio-sidebar' }, [ + el('p', { class: 'eyebrow', text: 'Audio sections' }), + el('ul', { class: 'audio-section-list' }, [ + el('li', { text: 'Overview' }), + el('li', { text: 'Speakers (voices)' }), + el('li', { text: 'Proper names' }), + el('li', { text: 'Chapters' }), + el('li', { text: 'Assignments' }), + el('li', { text: 'Runs' }), + el('li', { text: 'Queue' }) + ]) + ]), + el('div', { class: 'audio-main' }, [ + el('div', { class: 'empty-state' }, [ + el('span', { class: 'empty-state__icon', text: '▶' }), + el('p', { class: 'empty-state__title', text: 'Audio workspace not yet wired' }), + el('p', { class: 'empty-state__body', text: 'Audio sections follow the UmbraAudio build system: speakers/voices, proper names, chapters, assignments, runs, queue. Real run data needs the audio backend endpoints.' }) + ]) + ]) + ])); + } + + /* ---------------------------------------------------------- format screen */ + + function renderFormat() { + var main = q('#main'); + api.clear(main); + + var program = state.programId ? (state.programById[String(state.programId)] || null) : null; + var langLabel = state.languageCode ? languageTagLabel(state.languageCode) : ''; + + main.appendChild(el('div', { class: 'umbra-heading umbra-heading--row' }, [ + el('div', {}, [ + el('p', { class: 'eyebrow', text: 'Format' }), + el('h1', { text: langLabel ? (langLabel + ' metadata') : 'Format' }), + el('p', { class: 'lede', text: program ? (program.name + ' · ' + langLabel) : 'Select a language project.' }) + ]), + el('div', { class: 'format-tabs' }, [ + el('button', { class: 'format-tab format-tab--active', type: 'button', text: 'EPUB' }), + el('button', { class: 'format-tab', type: 'button', text: 'PDF' }) + ]) + ])); + + main.appendChild(el('div', { class: 'form-grid' }, [ + el('div', { class: 'form-grid__full' }, [ + el('label', { text: 'Cover' }), + el('input', { class: 'input', type: 'text', placeholder: 'Cover image' }) + ]), + el('div', { class: 'form-grid__full' }, [ + el('label', { text: 'ISBN' }), + el('input', { class: 'input', type: 'text', placeholder: 'ISBN-13' }) + ]), + el('div', { class: 'form-grid__full' }, [ + el('label', { text: 'Translated blurb' }), + el('textarea', { class: 'input', placeholder: 'Translated blurb' }) + ]), + el('div', { class: 'form-grid__full' }, [ + el('label', { text: 'Translated Amazon search terms' }), + el('input', { class: 'input', type: 'text', placeholder: 'Comma-separated keywords' }) + ]), + el('div', { class: 'form-grid__full' }, [ + el('label', { text: 'Print profile' }), + el('input', { class: 'input', type: 'text', placeholder: 'Paperback / Hardback' }) + ]), + el('div', { class: 'form-grid__full' }, [ + el('label', { text: 'Page count / spine' }), + el('input', { class: 'input', type: 'text', placeholder: 'Measured on render' }) + ]) + ])); + + // Load existing format metadata per language. + if (state.languageCode && typeof api.getFormatMetadata === 'function') { + api.getFormatMetadata(state.programId, state.languageCode).then(function (data) { + if (data && data.saved) { + var cover = main.querySelector('input[placeholder="Cover image"]'); + var isbn = main.querySelector('input[placeholder="ISBN-13"]'); + var blurb = main.querySelector('textarea[placeholder="Translated blurb"]'); + var terms = main.querySelector('input[placeholder="Comma-separated keywords"]'); + var profile = main.querySelector('input[placeholder="Paperback / Hardback"]'); + var pages = main.querySelector('input[placeholder="Measured on render"]'); + if (cover) cover.value = data.cover || ''; + if (isbn) isbn.value = data.isbn || ''; + if (blurb) blurb.value = data.blurb || ''; + if (terms) terms.value = data.amazon_search_terms || ''; + if (profile) profile.value = data.print_profile || ''; + if (pages) pages.value = data.page_count || ''; + } + }, function () { + // Non-fatal: format metadata may not exist yet. + }); + } + + main.appendChild(el('div', { class: 'umbra-inline-actions' }, [ + el('button', { class: 'btn', type: 'button', text: 'Save', on: { click: function () { + var cover = main.querySelector('input[placeholder="Cover image"]').value; + var isbn = main.querySelector('input[placeholder="ISBN-13"]').value; + var blurb = main.querySelector('textarea[placeholder="Translated blurb"]').value; + var terms = main.querySelector('input[placeholder="Comma-separated keywords"]').value; + var profile = main.querySelector('input[placeholder="Paperback / Hardback"]').value; + var pages = main.querySelector('input[placeholder="Measured on render"]').value; + api.setFormatMetadata(state.programId, state.languageCode, { + cover: cover || null, + isbn: isbn || null, + blurb: blurb || null, + amazon_search_terms: terms || null, + print_profile: profile || null, + page_count: pages || null + }).then( + function () { api.showToast('Format metadata saved', 'success'); }, + function (err) { api.showToast(api.describeError(err), 'error'); } + ); + } } }), + el('button', { class: 'btn btn--ghost', type: 'button', text: 'Cancel', on: { click: function () { writeHash('program', state.programId || '', state.languageCode || ''); } } }) + ])); + } + + /* ----------------------------------------------------------- review screen */ + + function renderReview() { + var main = q('#main'); + api.clear(main); + + var program = state.programId ? (state.programById[String(state.programId)] || null) : null; + var langLabel = state.languageCode ? languageTagLabel(state.languageCode) : ''; + var chapterId = state.chapterId || 'ch-6'; + var isSource = state.languageCode === 'en'; + + main.appendChild(el('div', { class: 'umbra-heading' }, [ + el('div', {}, [ + el('p', { class: 'eyebrow', text: 'Review utterances by chapter' }), + el('h1', { text: langLabel ? (langLabel + ' · ' + chapterId) : 'Utterance review' }), + el('p', { class: 'lede', text: program ? (program.name + ' · ' + langLabel + ' · ' + chapterId) : 'Select a language project.' }) + ]) + ])); + + main.appendChild(el('div', { class: 'umbra-inline-actions' }, [ + el('button', { class: 'btn btn--ghost', type: 'button', text: '← Back', on: { click: function () { writeHash('program', state.programId || '', state.languageCode || ''); } } }) + ])); + + // Check translation state before showing utterances. + // Only languages with complete translation can be reviewed. + if (typeof api.projectStatus === 'function') { + api.projectStatus(program.id).then(function (data) { + var langs = (data && data.languages) || []; + var match = null; + langs.forEach(function (l) { if (l.language_tag === state.languageCode) { match = l; } }); + var trState = match ? (match.translation && match.translation.state) : 'not_started'; + if (trState !== 'complete') { + renderReviewNotTranslated(main, program, state.languageCode, trState); + return; + } + renderReviewUtterances(main, program, state.languageCode, chapterId, isSource); + }, function (err) { + // Fall through to showing utterances if status can't be loaded. + renderReviewUtterances(main, program, state.languageCode, chapterId, isSource); + }); + } else { + renderReviewUtterances(main, program, state.languageCode, chapterId, isSource); + } + } + + function renderReviewNotTranslated(main, program, code, trState) { + var isSource = code === 'en'; + var label = languageTagLabel(code); + var stateText = trState === 'in_progress' ? 'in progress' : 'not yet started'; + + main.appendChild(el('div', { class: 'empty-state' }, [ + el('span', { class: 'empty-state__icon', text: '◇' }), + el('h2', { text: label + ' — translation ' + stateText }), + el('p', { class: 'lede', text: 'No translation exists for this language yet. Queue a translation job to begin.' }) + ])); + main.appendChild(el('div', { class: 'umbra-inline-actions' }, [ + el('button', { class: 'btn', type: 'button', text: 'Queue translation', + on: { click: function () { + api.queueTranslation(program.id, code).then( + function () { api.showToast('Translation queued for ' + label, 'success'); applyRoute(); }, + function (err) { api.showToast(api.describeError(err), 'error'); } + ); + } } + }), + el('button', { class: 'btn btn--ghost', type: 'button', text: '← Back', on: { click: function () { writeHash('program', program.id || ''); } } }) + ])); + } + + function renderReviewUtterances(main, program, code, chapterId, isSource) { + var uttPanel = el('div', { class: 'utterance-review-panel' }); + uttPanel.appendChild(el('h3', { text: isSource ? 'English source — review before validation' : 'Translated text — review before audio' })); + var uttList = el('div', { class: 'utterance-list' }); + uttPanel.appendChild(uttList); + main.appendChild(uttPanel); + + var allChapters = []; + var currentChapterId = chapterId; + + function buildChapterTree(chapters) { + allChapters = chapters; + var sidebar = q('#sidebar'); + api.clear(sidebar); + + var chapterTree = el('div', { class: 'binder__tree', 'aria-label': 'Binder' }); + chapterTree.appendChild(el('div', { class: 'tree__group', text: 'Audio Format' })); + chapterTree.appendChild(el('div', { class: 'tree__group', text: isSource ? 'Source text' : 'Translation' })); + + var fmChapters = chapters.filter(function (c) { return c.matter && c.matter.startsWith('front'); }); + if (fmChapters.length) { + var fmGroup = el('div', { class: 'tree__group', text: 'Front matter' }); + fmChapters.forEach(function (ch) { + var active = ch.id === currentChapterId ? ' tree__item--active' : ''; + var item = el('div', { class: 'tree__item tree__item--child tree__item--chapter' + active, 'data-chapter': ch.id, text: ch.title }); + fmGroup.appendChild(item); + }); + chapterTree.appendChild(fmGroup); + } + + var bmChapters = chapters.filter(function (c) { return c.matter && c.matter.startsWith('back'); }); + if (bmChapters.length) { + var bmGroup = el('div', { class: 'tree__group', text: 'Back matter' }); + bmChapters.forEach(function (ch) { + var active = ch.id === currentChapterId ? ' tree__item--active' : ''; + var item = el('div', { class: 'tree__item tree__item--child tree__item--chapter' + active, 'data-chapter': ch.id, text: ch.title }); + bmGroup.appendChild(item); + }); + chapterTree.appendChild(bmGroup); + } + + var bodyChapters = chapters.filter(function (c) { return !c.matter || c.matter === 'body'; }); + if (bodyChapters.length) { + var bodyGroup = el('div', { class: 'tree__group', text: 'Body' }); + bodyChapters.forEach(function (ch) { + var active = ch.id === currentChapterId ? ' tree__item--active' : ''; + var item = el('div', { class: 'tree__item tree__item--child tree__item--chapter' + active, 'data-chapter': ch.id, text: ch.title }); + bodyGroup.appendChild(item); + }); + chapterTree.appendChild(bodyGroup); + } + + chapterTree.querySelectorAll('.tree__item--chapter').forEach(function (item) { + item.addEventListener('click', function () { + chapterTree.querySelectorAll('.tree__item--chapter').forEach(function (i) { + i.classList.remove('tree__item--active'); + }); + item.classList.add('tree__item--active'); + currentChapterId = item.getAttribute('data-chapter'); + writeHash('review', program.id || '', state.languageCode || '', currentChapterId); + loadChapter(currentChapterId); + }); + }); + + sidebar.appendChild(chapterTree); + } + + function loadChapter(chapId) { + api.clear(uttList); + uttList.appendChild(el('p', { class: 'lede', text: 'Loading utterances…' })); + if (typeof api.getUtteranceReview === 'function') { + api.getUtteranceReview(program.id, code, chapId).then(function (data) { + api.clear(uttList); + var utterances = (data && data.utterances) || []; + var speakers = (data && data.speakers) || []; + if (!utterances.length) { + uttList.appendChild(el('p', { class: 'lede', text: 'No utterances found for ' + chapId + '.' })); + return; + } + // UmbraAudio column layout: speaker, variation, text, play, regenerate, delete + uttList.appendChild(el('div', { class: 'utterance-header' }, [ + el('div', { class: 'utterance__speaker', text: 'Speaker' }), + el('div', { class: 'utterance__variation', text: 'Variation' }), + el('div', { class: 'utterance__text', text: 'Text' }), + el('div', { class: 'utterance__actions', text: 'Actions' }) + ])); + utterances.forEach(function (utt) { + uttList.appendChild(el('div', { class: 'utterance-row' }, [ + el('div', { class: 'utterance__speaker' }, [ + el('select', { class: 'input', 'data-utt': utt.id }, speakers.map(function (spk) { + var sel = spk.id === (utt.speaker || 'narrator'); + return el('option', { value: spk.id, text: spk.name, selected: sel }); + })) + ]), + el('div', { class: 'utterance__variation' }, [ + el('select', { class: 'input', 'data-utt': utt.id }, [ + el('option', { value: 'neutral', text: 'Neutral', selected: true }), + el('option', { value: 'angry', text: 'Angry' }), + el('option', { value: 'joyful', text: 'Joyful' }), + el('option', { value: 'terrified', text: 'Terrified' }), + el('option', { value: 'griefstricken', text: 'Griefstricken' }), + el('option', { value: 'shocked', text: 'Shocked' }), + el('option', { value: 'furious', text: 'Furious' }), + el('option', { value: 'tense', text: 'Tense' }), + el('option', { value: 'urgent', text: 'Urgent' }), + el('option', { value: 'horror', text: 'Horror' }), + el('option', { value: 'slow', text: 'Slow' }), + el('option', { value: 'moderate', text: 'Moderate' }), + el('option', { value: 'fast', text: 'Fast' }) + ]) + ]), + el('div', { class: 'utterance__text', text: utt.text }), + el('div', { class: 'utterance__actions' }, [ + el('button', { class: 'btn btn--ghost', type: 'button', text: '▶', title: 'Play audio', disabled: true, 'data-utt': utt.id }), + el('button', { class: 'btn btn--ghost', type: 'button', text: '⟳', title: 'Regenerate audio', disabled: true, 'data-utt': utt.id }), + el('button', { class: 'btn btn--ghost', type: 'button', text: '✕', title: 'Delete', disabled: true, 'data-utt': utt.id }) + ]) + ])); + }); + }, function (err) { + api.clear(uttList); + uttList.appendChild(el('p', { class: 'list-summary list-summary--error', text: 'Utterances could not be loaded: ' + api.describeError(err) })); + }); + } + } + + if (typeof api.getChapters === 'function') { + api.getChapters(program.id, code).then(function (chapters) { + buildChapterTree(chapters); + loadChapter(currentChapterId); + }, function (err) { + api.clear(main); + main.appendChild(el('p', { class: 'list-summary list-summary--error', text: 'Chapters could not be loaded: ' + api.describeError(err) })); + }); + } else { + loadChapter(currentChapterId); + } + + main.appendChild(el('div', { class: 'umbra-inline-actions' }, [ + el('button', { class: 'btn', type: 'button', text: 'Save speaker assignments', disabled: true, title: 'Speaker assignment persistence needs the audio backend.' }) + ])); + } + + /* ------------------------------------------------------------- rendering */ + + var RENDERERS = { + control: renderControlPanel, + program: function () { + if (state.programId) { + renderProgramDetail(); + } else { + renderProgramList(); + } + }, + edit: renderEdit, + audio: renderAudio, + format: renderFormat, + review: renderReview + }; + + function applyRoute() { + var route = readHash(); + + var activity = route.activity; + if (!activity) { + activity = isAdmin(state.session && state.session.user) ? 'control' : 'program'; + } + if (activity === 'control' && !isAdmin(state.session && state.session.user)) { + activity = 'program'; + } + + state.activity = activity; + state.section = route.section; + state.programId = route.programId; + state.languageCode = route.languageCode; + + // Resolve program from workspace snapshot. + var programs = (state.workspace && state.workspace.projects) || []; + state.programs = programs; + state.programById = {}; + programs.forEach(function (p) { state.programById[String(p.id)] = p; }); + + var select = q('#activity-select'); + if (select && select.value !== activity) { + select.value = activity; + } + + // Program selector visibility: only meaningful for program/edit/audio/format/review. + var programSelect = q('#program-select'); + if (programSelect) { + if (activity === 'program' || activity === 'edit' || activity === 'audio' || activity === 'format' || activity === 'review') { + programSelect.classList.remove('umbra-topbar__mode--hidden'); + // Populate options when a program is selected, regardless of activity. + var prog = state.programById[String(state.programId)] || null; + if (prog) { + api.clear(programSelect); + programSelect.appendChild(el('option', { value: '', text: prog.name + ' — select a language' })); + languageDropdownOptions(prog).forEach(function (opt) { programSelect.appendChild(opt); }); + programSelect.value = state.languageCode || ''; + } + } else { + programSelect.classList.add('umbra-topbar__mode--hidden'); + } + } + + // Topbar: show language tag + program name when a language project is selected. + var nameEl = q('#topbar-language-name'); + if (state.languageCode && state.programId) { + var program = state.programById[String(state.programId)] || null; + api.setText(nameEl, state.languageCode.toUpperCase() + ' — ' + (program ? program.name : '')); + } else { + api.setText(nameEl, ''); + } + + state.chapterId = route.chapterId || null; + + renderSidebar(); + + if (activity === 'program' && !route.programId) { + renderProgramList(); + } else { + (RENDERERS[activity] || renderProgramList)(); + } + } + + /* -------------------------------------------------------------------- init */ + + function bindBanner() { + var select = q('#activity-select'); + if (select) { + select.addEventListener('change', function () { + var value = this.value; + if (value === 'control') { + writeHash('control', state.section || 'dashboard'); + } else if (value === 'program') { + writeHash('program', state.programId || ''); + } else if (value === 'review') { + writeHash('review', state.programId || '', state.languageCode || '', state.chapterId || 'ch-6'); + } else { + writeHash(value, state.programId || '', state.languageCode || ''); + } + }); + } + + var programSelect = q('#program-select'); + if (programSelect) { + programSelect.addEventListener('change', function () { + var code = this.value; + if (!code) { + writeHash('program', state.programId || ''); + return; + } + if (state.activity === 'program') { + writeHash('program', state.programId || '', code); + } else if (state.activity === 'review') { + writeHash('review', state.programId || '', code, state.chapterId || 'ch-6'); + } else { + writeHash(state.activity, state.programId || '', code); + } + }); + } + + var menu = q('#account-menu'); + if (menu) { + menu.addEventListener('change', function () { + var value = this.value; + this.value = ''; + if (value === 'settings') { + global.location.href = 'settings.html'; + } else if (value === 'signout') { + api.logout().then( + function () { global.location.href = 'index.html'; }, + function () { global.location.href = 'index.html'; } + ); + } + }); + } + + global.addEventListener('hashchange', applyRoute); + } + + function init() { + bindBanner(); + + api.me().then(function (session) { + state.session = session; + paintIdentity(session); + return api.workspace().then( + function (data) { state.workspace = data; }, + function () { state.workspace = null; } + ); + }).then(function () { + applyRoute(); + }, function () { + // Session could not be loaded; api.js navigates away. + }); + } + + if (document.readyState === 'loading') { + document.addEventListener('DOMContentLoaded', init); + } else { + init(); + } +})(window); diff --git a/web/assets/workspace-project.js b/web/assets/workspace-project.js new file mode 100644 index 0000000..88dd99d --- /dev/null +++ b/web/assets/workspace-project.js @@ -0,0 +1,166 @@ +(function () { + 'use strict'; + + var api = window.UmbraAPI; + var nav = window.UmbraNav; + + function query(sel) { + return document.querySelector(sel); + } + + function el(tag, attrs, children) { + return api.el(tag, attrs || {}, children || []); + } + + function getParam(name) { + try { + var url = new URL(window.location.href); + return url.searchParams.get(name); + } catch (e) { + return null; + } + } + + function formatDate(s) { + if (!s) return '—'; + try { + var d = new Date(s); + if (isNaN(d.getTime())) return s; + return d.toLocaleString(); + } catch (e) { + return s; + } + } + + function statusPill(status) { + var s = (status || '').toString(); + var tone = s === 'active' ? 'success' : s === 'on_hold' ? 'warning' : s === 'released' ? 'info' : s === 'queued' || s === 'running' ? 'warning' : s === 'done' || s === 'complete' || s === 'available' ? 'success' : 'muted'; + return api.el('span', { class: 'status-pill status-pill--' + tone, text: api.label(s) }); + } + + function load() { + if (!api) return; + api.getSession({ cache: 'no-store' }) + .then(function (session) { + var user = session && session.user ? session.user : null; + var role = user && user.role ? user.role : ''; + if (query('#rail-display-name')) query('#rail-display-name').textContent = user && user.display_name ? user.display_name : (user && user.username ? user.username : 'Account'); + if (query('#rail-role')) query('#rail-role').textContent = api.label(role) || '—'; + var avatar = query('#rail-avatar'); + if (avatar) { + var initials = (user && user.display_name ? user.display_name : (user && user.username ? user.username : 'UT')).trim().split(/\s+/).slice(0, 2).map(function (p) { return p[0] ? p[0].toUpperCase() : ''; }).join(''); + avatar.textContent = initials || 'UT'; + } + + var projectId = getParam('id'); + if (!projectId) { + if (query('#project-title')) query('#project-title').textContent = 'No project selected'; + if (query('#project-subtitle')) query('#project-subtitle').textContent = 'Return to workspace and select a project.'; + return; + } + + // Try to load project details from admin API if available, else from workspace snapshot + Promise.all([ + api.fetch('/api/uadmin/projects/' + encodeURIComponent(projectId), { method: 'GET' }).catch(function(){ return null; }), + api.fetch('/api/workspace', { method: 'GET', cache: 'no-store' }).catch(function(){ return null; }) + ]).then(function (results) { + var projRes = results[0]; + var wsRes = results[1]; + var project = null; + if (projRes && projRes.ok) { + project = projRes.project || projRes.data || projRes; + } + if (!project && wsRes && wsRes.ok) { + var data = wsRes.data || wsRes; + var list = (data.projects || []).concat(data.assigned_to_me || []); + for (var i = 0; i < list.length; i++) { + if (String(list[i].id) === String(projectId)) { project = list[i]; break; } + } + } + if (!project) { + if (query('#project-title')) query('#project-title').textContent = 'Project'; + if (query('#project-subtitle')) query('#project-subtitle').textContent = 'Project details unavailable from API.'; + return; + } + if (query('#breadcrumb-identity')) query('#breadcrumb-identity').textContent = project.name; + if (query('#project-title')) query('#project-title').textContent = project.name; + if (query('#project-subtitle')) query('#project-subtitle').textContent = (project.slug ? '/' + project.slug : '') + (project.organization_id ? '' : ''); + var pill = query('#project-status-pill'); + if (pill) { pill.className = 'status-pill status-pill--' + (project.status === 'active' ? 'success' : project.status === 'on_hold' ? 'warning' : project.status === 'released' ? 'info' : 'muted'); pill.textContent = api.label(project.status); } + + // English source table (placeholder based on known structure) + var tbody = query('#english-tbody'); + if (tbody) { + api.clear(tbody); + var rows = [ + {ch: 'Front', title: 'Front matter', words: '—', status: 'available'}, + {ch: 'Body', title: 'Chapters (Deal Hunter)', words: '~66,884', status: 'available'}, + {ch: 'Back', title: 'Back matter', words: '—', status: 'available'} + ]; + rows.forEach(function(r) { + var tr = document.createElement('tr'); + tr.appendChild(el('td', {}, [document.createTextNode(r.ch)])); + tr.appendChild(el('td', {}, [document.createTextNode(r.title)])); + tr.appendChild(el('td', {}, [document.createTextNode(r.words)])); + tr.appendChild(el('td', {}, [statusPill(r.status)])); + tr.appendChild(el('td', {}, [api.el('button', { class: 'text-button', type: 'button', text: 'Edit/View' })])); + tbody.appendChild(tr); + }); + } + + // Translations + var tlist = query('#translations-list'); + var tempty = query('#translations-empty'); + if (tlist) api.clear(tlist); + var trans = [ + {lang: 'English', code: 'en', status: 'source'}, + {lang: 'Simplified Chinese (zh-CN)', code: 'zh-Hans', status: 'in_progress'} + ]; + if (tlist && trans.length) { + trans.forEach(function(t) { + var card = api.el('article', { class: 'project-card project-card--' + (t.status === 'in_progress' ? 'on_hold' : 'active') }, [ + api.el('div', { class: 'project-card__main' }, [ + api.el('span', { class: 'project-card__name', text: t.lang + ' (' + t.code + ')' }), + api.el('div', { class: 'project-card__meta' }, [api.el('span', { text: 'Status: ' + api.label(t.status) })]) + ]), + api.el('div', { class: 'project-card__side' }, [statusPill(t.status), api.el('button', { class: 'text-button', type: 'button', text: 'View / Tag for review' })]) + ]); + tlist.appendChild(card); + }); + if (tempty) tempty.hidden = true; + } else if (tempty) { + tempty.hidden = false; + } + + // Outputs + var oepub = query('#out-epub'); + var opdf = query('#out-pdf'); + var oaudio = query('#out-audio'); + if (oepub) oepub.textContent = 'Not generated'; + if (opdf) opdf.textContent = 'Not generated'; + if (oaudio) oaudio.textContent = 'Not generated'; + + // Buttons + var bq = query('#btn-queue-translation'); + if (bq) bq.addEventListener('click', function(){ api.showToast && api.showToast('Translation queued (placeholder)'); }); + var bep = query('#btn-generate-epub'); + if (bep) bep.addEventListener('click', function(){ api.showToast && api.showToast('EPUB generation queued (placeholder)'); }); + var bpdf = query('#btn-generate-pdf'); + if (bpdf) bpdf.addEventListener('click', function(){ api.showToast && api.showToast('PDF generation queued (placeholder)'); }); + var ba = query('#btn-generate-audio'); + if (ba) ba.addEventListener('click', function(){ api.showToast && api.showToast('Audio generation queued (placeholder)'); }); + var bal = query('#btn-add-lang'); + if (bal) bal.addEventListener('click', function(){ api.showToast && api.showToast('Add language (placeholder)'); }); + }); + }) + .catch(function () { + // ignore + }); + } + + if (document.readyState === 'loading') { + document.addEventListener('DOMContentLoaded', load); + } else { + load(); + } +})(); diff --git a/web/index.html b/web/index.html index 7872555..c102c07 100644 --- a/web/index.html +++ b/web/index.html @@ -67,7 +67,7 @@
Administrator?
- Open control panel + Open control panel

An account issued a temporary password must replace it at first sign-in before any workspace access is granted.

diff --git a/web/umbra-app.html b/web/umbra-app.html new file mode 100644 index 0000000..abc0bea --- /dev/null +++ b/web/umbra-app.html @@ -0,0 +1,80 @@ + + + + + + + Umbra Tools + + + + + +
+
+
+ + + Umbra Tools + + +
+
+ +
+
+
+
+ + + Loading… + Checking session + +
+ +
+
+ + + +
+ +
+
+ +
+ + + + + diff --git a/web/workspace-project.html b/web/workspace-project.html new file mode 100644 index 0000000..c277cec --- /dev/null +++ b/web/workspace-project.html @@ -0,0 +1,143 @@ + + + + + + Project · UmbraTrans + + + + +
+ + +
+
+ +
WorkspaceLoading…
+
+ — + Workspace + +
+
+ +
+
+

Project workspace

+

Loading…

+

—

+
+
+ + + + +
+
+ +
+
+
+

Source

+

English Source

+
+
+

Original input book text (editable in line with Umbra Audio UI).

+
+ + + + + +
ChapterTitleWordsStatus
+
+
+ +
+
+
+

Translations

+

Translated Versions

+
+
+ +
+
+

View translations, tag for review/updates. English and Chinese shown when available.

+
+
+ +

No translations yet

+

Queue a translation to start. Chinese translation may be in progress.

+
+
+ +
+
+
+

Outputs

+

Output Formats

+
+
+
+
+ EPUB + — + Status / availability +
+
+ PDF + — + Status / availability +
+
+ Audio + — + Status / availability +
+
+
+ +
+
+
+

Audio

+

Audio Production

+
+
+

Audio generation queued separately. Uses translated text when available.

+
+
+
+ +
+ + + + +