UmbraAudio/manifest.md

UmbraAudio Project Data Manifest

• Purpose: authoritative inventory of every data object an UmbraAudio project holds, so a shared enterprise database schema and transition plan can be designed against facts rather than assumptions. This audio system is the upstream data source for a separate translation system; both are intended to converge on one common database.
• Date: 2026-09-28
• Method: read-only introspection of the live project tree, the SQLite databases, the JSON state files, and the audio artifact trees. No data was modified. No secret, credential, or connection value appears in this document.
• Companion documents: specs/21-umbra-system-nodes-tasks-and-changes.md (node/task/change contract), specs/20-enterprise-database-integration.md (shared-DB contract), specs/18-UmbraTrans.md (translation design), specs/16-audiobook-production.md (production pipeline).
• This document is a dated measurement, not a live view. The numbers below were read on 2026-09-28. When they have moved, add a dated line to the update log rather than silently editing a table.

Update log

Date

Change

2026-10-01

Measurement refresh after the designation-merge work. Last Dwarf speakers.json grew 70 → 93 (23 genuinely missing speakers captured from the ch5 crowd and similar groups, plus 33 already-covered designations); DealHunter ref WAVs measure 97 (was 94). Every other table count in §1.1 was re-verified unchanged. Re-run .venv/bin/python scripts/umbra_node_report.py to regenerate the live figures; manifest.md records the baseline, the script records the present.

---

0. How to read this manifest

Three labels are used throughout and they drive the whole migration:

Label	Meaning	Migration consequence
AUTHORITATIVE	The system treats this as truth. Losing it loses data.	Must be migrated and preserved.
DERIVED	Computable from authoritative data.	Can be regenerated; migrate only if cheap.
CACHE	Exists to make the next run faster.	Discard freely.
TRANSIENT	Scratch/checkpoint for a single run.	Discard.

The single most important finding in this manifest is in §1.2: the SQLite databases are mostly empty. Seven of eleven tables hold zero rows in every project. The real content lives in JSON files and on the filesystem. A migration that only reads SQL will lose the speaker model, the voice model, the pronunciation model and the entire audio record.

---

1. Project inventory

1.1 Live projects

Four project directories hold a real project.db. Everything is under /Volumes/1TB_APL_Storage/UmbraAudio/<project-dir>/.

Project dir	DB size	State	Chapters	Utterances	Distinct speaker_name	speakers.json	voice_profiles.json	*_ref.wav	vmatrix .pt
Last Dwarf	3.12 MB	ACTIVE WORK	49	10,015	64	70	68	71	0
DealHunterShip_4.0e_whole-to-audio	1.48 MB	PUBLISHED	32	4,319	55	61	64	94	38
Treasure Island	1.21 MB	FUTURE / VOICE LAB	49	3,811	34	40	41	44	0
Vocals	98 KB	voice showcase	1	36	2	2	2	2	0

Last Dwarf is the most complete and is the reference implementation in practice. DealHunterShip_4.0e_whole-to-audio is the only project with a finished published package (§11) and the only one holding vmatrix files. Treasure Island is a voice-attribute laboratory (§1.4). Vocals is a voice showcase, not a book.

PROJECT_BASE is hard-coded to /Volumes/1TB_APL_Storage/UmbraAudio in umbraaudio/config.py:10 and is not configurable. list_projects() is a directory scan of the whole repository root.

1.4 The three books are in different lifecycle states

This is not cosmetic — it changes what each project means to the shared database and how it must be migrated.

State	Project	What it means	Migration treatment
PUBLISHED	Deal Hunter	A shipped retail product exists (export/Deal_Hunter.m4b, 32-track AR MP3 package). The data is frozen by history — you cannot re-segment or re-cast it and call the result the same book.	Import as an immutable released revision. Corrections create a new revision; the published lineage must stay reproducible.
ACTIVE WORK	Last Dwarf	In continuous change. ch9–12 were fully rebuilt 2026-09-26. 1,796 text-stale / 520 speaker-stale / 496 re-voiced-ref markers remain.	Import continuously. Expect re-writes; the shared DB must tolerate churn without a global "freeze".
FUTURE / VOICE LAB	Treasure Island	Not a production book. It is where voice attribute options are being tested.	Its speaker/voice records are experimental data, not canonical casting. Import into a sandbox/experimental namespace, or import read-only, so lab results never silently become production voice assets.

Treasure Island evidence of its lab role: flint_a_ref.wav, flint_b_ref.wav, flint_c_ref.wav are three variants of the same character (Captain Flint, 284–384 KB, generated 2026-09-05 07:33), and captain_flint_(parrot)_ref.wav carries a pinned non-human description — "A raucous, dry, croaking cackling parrot voice, creaky and scratchy like an old macaw half-chuckling and half-squawking, with rough rattling syllables". Its analysis/voice_analysis.json reports analyzed: 0 / "No voice references found" — the attribute measurement was never run there.

Consequence for the shared schema: projects needs an explicit lifecycle/state dimension (draft / active / released / archived / experimental) distinct from the legacy project.status text field, because "published" and "in active edit" impose opposite constraints on revisioning.

1.5 The measured voice-attribute dataset

analysis/voice_analysis.json is a real, structured attribute dataset — the "voice attribute options" data. It is DERIVED (computed from the ref WAVs) but it is the objective evidence behind casting decisions and should be preserved.

Top level: {generated_at, refs_dir, voices{}, near_duplicates[], errors{}, summary{}, passed}.

Per voice (Last Dwarf, 66 analyzed, 66 passed, 0 near-duplicates, 0 errors):

Field	Example	Meaning
name	"advisor"	slug
features.mean_pitch_hz	126.45	pitch centre
features.median_pitch_hz	122.10	pitch centre (robust)
features.pitch_std_hz	34.32	pitch variability
features.spectral_centroid_hz	2538.76	brightness
features.zcr	0.1079	zero-crossing rate (edge/nasality proxy)
features.voiced_ratio	0.709	voiced-frame fraction
features.rms	0.0565	energy
features.duration, features.sample_rate	37.92, 24000
requirements	see below	pass/fail gate
passed	true

The requirements gate has four checks, including one that matters for a multi-voice cast:

duration   >= 20.0 s
sample_rate == 24000
energy     >= 0.002
uniqueness >= 0.9   with nearest_neighbor recorded ("scum_guy1", value 0.7606 -> PASSED)

uniqueness is an existing voice-distinctiveness metric with a nearest-neighbour record — the mechanism the system already has for the crowding problem in §5.4. It is scalar and pitch-centric; §5.4 shows it needs extending with texture dimensions.

1.2 Row counts — the empty-table problem

Table	Last Dwarf	DealHunter	Treasure Island	Vocals	Populated?
project	1	1	1	1	yes
chapters	49	32	49	1	yes
utterances	10,015	4,319	3,811	36	yes
name_tokens	10	0	table absent	table absent	partly
characters	0	0	0	0	NO
generated_audio	0	0	0	0	NO
pronunciation_guide	0	0	0	0	NO
quality_reports	0	0	0	0	NO
review_decisions	0	0	0	0	NO
voice_review_history	0	0	0	0	NO
exception_reviews	0	0	0	0	NO

Consequences that the shared schema must absorb:

• There is no speaker row in SQL. The characters table is fully designed (25 columns) and completely unused. Speaker identity exists only in speakers.json + voice_profiles.json + the ref WAVs. utterances.character_id is therefore always NULL.
• utterances.speaker_name is free text, not a foreign key. It is the only link from an utterance to a speaker, and it is not referentially enforced, not normalised, and has case-variant spellings across projects (e.g. narrator and Narrator both occur).
• There is no audio record in SQL. generated_audio is unused; the record of what was produced is the filesystem plus the *.wav.txt marker sidecars.
• There is no review/QA record in SQL. quality_reports, review_decisions, exception_reviews and voice_review_history are all empty; QA lives in per-chapter ch{N}_build_status.json.

1.3 Per-project schema drift

Last Dwarf and DealHunterShip carry the current schema. Treasure Island and Vocals are behind and are missing:

• table name_tokens
• utterances columns: user_edited, qa_accepted, qa_accepted_at, qa_accepted_sig, attribution_evidence_kind, attribution_tag_name, attribution_uncertain, needs_review, review_reason

A migration must treat the older projects as a different shape and normalise, not assume a uniform column set. There are additionally four divergent DDL authorities in the codebase (umbraaudio/database.py:9-166, umbraaudio/web/routes.py:102-275, umbraaudio/analysis/name_tokens.py:504-520, scripts/reattribute_all.py:30-42) plus live column ordering differences produced by ALTER TABLE. No two sources of truth agree.

---

2. Storage topology

<PROJECT_BASE>/<project-dir>/
├── <manuscript>.docx|.epub        uploaded source (authoritative, licensed)
├── uploads/  last_upload/          uploaded copies of the manuscript
├── .umbra/                          ALL runtime state
│   ├── project.db                   SQLite: chapters + utterances only (see §1.2)
│   ├── speakers.json                speaker model            [AUTHORITATIVE]
│   ├── voice_profiles.json          per-speaker voice profile [AUTHORITATIVE]
│   ├── voice_status.json            human approve/reject      [AUTHORITATIVE]
│   ├── voice_engines.json           which engine made a ref   [AUTHORITATIVE]
│   ├── phonetic_names.json          name → phonetic           [AUTHORITATIVE]
│   ├── m4b_meta.json                publishing metadata       [AUTHORITATIVE]
│   ├── pipeline_status.json         progress                  [CACHE]
│   ├── pipeline_checkpoint.json     resume point              [CACHE]
│   ├── generation_status.json       TTS run progress/lock     [CACHE]
│   ├── analysis_status.json         discovery progress        [CACHE]
│   ├── pronunciation_status.json    lookup progress           [CACHE]
│   ├── scan_results.json            manifest scan             [CACHE]
│   ├── defects.json                 pipeline defect log       [DERIVED]
│   ├── voices/{name_slug}_ref.wav   approved voice reference  [AUTHORITATIVE, biometric]
│   ├── voice_embeddings/{slug}.npy  speaker embeddings        [CACHE, biometric]
│   ├── voice_prompts/{slug}.pt      Qwen ICL prompt cache     [CACHE, biometric]
│   ├── voice_tests/{Name}.wav       verification samples      [DERIVED]
│   ├── pronunciation_audio/{s}.wav pronunciation previews     [DERIVED]
│   ├── voice_assignments.json       Edge-TTS voice map        [AUTHORITATIVE, legacy]
│   ├── chapter_audio/
│   │   ├── ch{N}/utt_{idx:04d}.wav        per-utterance audio  [AUTHORITATIVE]
│   │   ├── ch{N}/utt_{idx:04d}.wav.txt    staleness marker      [AUTHORITATIVE]
│   │   ├── ch{N}_full.wav                 chapter master        [DERIVED]
│   │   ├── ch{N}_full.mp3                 chapter MP3           [DERIVED]
│   │   └── ch{N}_build_status.json        build progress/QA     [CACHE]
│   ├── manifests/ch{N}.json         generation manifest       [DERIVED]
│   ├── export/                      M4B, AR mp3 package       [AUTHORITATIVE artifact]
│   ├── analysis/                    voice_analysis.json       [DERIVED]
│   ├── discovery/                   LLM discovery passes      [TRANSIENT]
│   └── archives/                    manual snapshots          [DERIVED]
└── (global, outside every project)
    ~/.umbra/voice_registry.db       cross-project registry    [AUTHORITATIVE]
    ~/.umbra/library/voices/*.wav    shared voice refs         [AUTHORITATIVE, biometric]

---

3. SQLite schema (exact DDL, from the live Last Dwarf database)

No PRAGMA foreign_keys=ON is ever issued, so every REFERENCES clause below is declarative and unenforced. There are zero CHECK constraints. Journal mode is WAL.

3.1 project — 1 row

CREATE TABLE project (
    id INTEGER PRIMARY KEY,
    name TEXT NOT NULL,
    source_path TEXT NOT NULL,
    source_format TEXT NOT NULL,
    created_at TEXT NOT NULL DEFAULT (datetime('now')),
    updated_at TEXT NOT NULL DEFAULT (datetime('now')),
    status TEXT NOT NULL DEFAULT 'initialized'
)

Grain: one row per project (always exactly 1). status unconstrained. id is a per-database integer with no cross-project meaning.

3.2 chapters — 49 / 32 / 49 / 1 rows

CREATE TABLE chapters (
    id INTEGER PRIMARY KEY,
    project_id INTEGER NOT NULL REFERENCES project(id),
    chapter_index INTEGER NOT NULL,
    title TEXT,
    source_text TEXT NOT NULL,          -- full copyrighted chapter text
    word_count INTEGER NOT NULL DEFAULT 0,
    status TEXT NOT NULL DEFAULT 'parsed',
    audio_path TEXT,
    created_at TEXT NOT NULL DEFAULT (datetime('now'))
)
CREATE INDEX idx_chapters_project ON chapters(project_id);

Grain: one row per chapter. No UNIQUE (project_id, chapter_index) despite it being the logical key. audio_path is populated only on legacy manifest-path projects. source_text holds the copyrighted manuscript — classification C.

3.3 utterances — the segmentation table

CREATE TABLE utterances (
    id INTEGER PRIMARY KEY,
    chapter_id INTEGER NOT NULL REFERENCES chapters(id),
    character_id INTEGER REFERENCES characters(id),   -- always NULL in practice
    utterance_index INTEGER NOT NULL,                  -- MUTABLE, 0-based
    text TEXT NOT NULL,
    type TEXT NOT NULL DEFAULT 'narration',            -- narration|dialogue|scene_break
    context_before TEXT,
    context_after TEXT,
    emotion TEXT,
    pace TEXT,
    emphasis_words TEXT,
    pause_before_ms INTEGER,
    pause_after_ms INTEGER,
    instruct TEXT,                                     -- composed TTS directive
    speaker_name TEXT,                                 -- FREE TEXT, not a FK
    attribution_confidence REAL DEFAULT 1.0,
    attribution_reasoning TEXT,
    emotion_confidence REAL DEFAULT 1.0,
    status TEXT NOT NULL DEFAULT 'pending',
    user_edited       INTEGER NOT NULL DEFAULT 0,
    qa_accepted       INTEGER NOT NULL DEFAULT 0,
    qa_accepted_at    TEXT,
    qa_accepted_sig   TEXT,
    attribution_evidence_kind TEXT,                     -- name|role|pronoun|address_only|none
    attribution_tag_name     TEXT,                     -- name in the preceding speech tag
    attribution_uncertain    INTEGER NOT NULL DEFAULT 0,
    needs_review      INTEGER NOT NULL DEFAULT 0,
    review_reason     TEXT
);
CREATE INDEX idx_utterances_chapter   ON utterances(chapter_id);
CREATE INDEX idx_utterances_character ON utterances(character_id);

Grain: one row per addressable text unit within a chapter. type distribution:

Project	narration	dialogue	scene_break	total
Last Dwarf	6,016	3,918	81	10,015
DealHunterShip_4.0e_whole-to-audio	2,436	1,874	9	4,319
Treasure Island	2,564	1,247	0	3,811
Vocals	36	0	0	36

Critical properties for schema design:

• utterance_index is not a stable identifier. It is renumbered by re-segmentation and by DELETE /api/utterance/{id}, which reindexes the whole chapter and renames the audio files on disk (umbraaudio/web/routes.py:3405-3524). specs/18-UmbraTrans.md §3.3 explicitly forbids using a mutable index as a cross-language ID.
• There is no UNIQUE (chapter_id, utterance_index) even though the code treats it as unique.
• instruct is the composed directive actually sent to TTS — the single most useful field for reproducing a render.
• The five clarity columns (attribution_evidence_kind, attribution_tag_name, attribution_uncertain, needs_review, review_reason) form the review queue described in specs/17-speaker-confidence-review.md.

3.4 Empty tables — schema reserved but unused

characters (25 columns, the full voice-design taxonomy: pitch_register, pace_baseline, energy, timbre, texture, formality, emotional_baseline, distinguishing_trait, accent_marker, voice_description, voice_ref_path, speaker_embedding BLOB, status); generated_audio (audio_path, duration_ms, mos_score, wer_score, speaker_similarity, artifact_score, generation_params); pronunciation_guide (UNIQUE(project_id, term)); quality_reports; review_decisions (polymorphic entity_type/entity_id, no FK); voice_review_history (no project_id — global, and reviewed_by TEXT is the literal string 'user').

name_tokens (UNIQUE(project_id, token); token, kind, status, target_speaker, occurrences, chapters_json, evidence_json, suggestion, suggestion_score) — 10 rows in Last Dwarf, 0 in DealHunter, table absent in the two older projects. This is the orphan name-token scanner from spec 17 Phase 2.

exception_reviews — web-only table, absent from umbraaudio/database.py entirely, so a CLI-only process never creates it. 0 rows everywhere currently.

---

4. JSON state files

4.1 speakers.json — the speaker model [AUTHORITATIVE]

Top level: speakers (list), book_title, total_chapters, total_words.

Per-speaker keys (verified, 20 keys):

Key	Type	Notes
primary_name	string	the key used everywhere; the de facto speaker id
aliases	list<string>	23 aliases total in Last Dwarf, 32 in DealHunter
alias_evidence	object	evidence per alias (present on some records)
is_narrator	bool	Last Dwarf has 3 narrator-flagged entries
is_group	bool	0 in all projects
unidentified	bool	7 in Last Dwarf, 8 in DealHunter
unidentified_reason	string	free text
gender, accent	string|null	frequently null even when demographics.gender is set
demographics	object	{type, gender, age, pitch?} — the reliable gender source
vocal_traits	list<string>	LLM discovery dump; see the effect warning in §4.9
description, communication_style	string	often empty
chapters_appeared	list<int>	chapter ids the speaker appears in
has_voice_ref	bool
voice_status	string	pending / approved / rejected
voice_engine, voice_engine_reason	string|null	qwen for 20 speakers in Last Dwarf, null for 50
voice_description	string	pin — when present it overrides the auto-built description verbatim (voices/voice_profile_manager.py:619)
distortion_effects	list	explicit post-TTS effect names
sample_dialogue	string

Counts of interest: Last Dwarf 70 speakers / 23 aliases / 7 unidentified / 4 pinned descriptions / 3 narrator flags. DealHunter 61 / 32 / 8 / 0 / 1.

4.2 voice_profiles.json [AUTHORITATIVE]

Keyed by speaker name (68 keys in Last Dwarf, 64 in DealHunter, 41 in Treasure Island). Per-profile keys: name, aliases, gender, age_range, is_narrator, personality, voice_description, emotion_tendencies, speech_patterns, and the taxonomy (pitch_register, pace_baseline, energy, timbre, texture, formality, emotional_baseline, accent_marker, distinguishing_trait). speech_patterns is a long | -delimited narrative of per-chapter behaviour — genuinely useful source material for a translation brief.

4.3 voice_status.json [AUTHORITATIVE] — 13 keys in Last Dwarf, human approval state.

4.4 voice_engines.json [AUTHORITATIVE] — {name: {engine, reason, at, ref_mtime}},

22 entries in Last Dwarf. Records which engine produced the current ref; engine: edge means a silent Qwen→Edge-TTS fallback happened.

4.5 phonetic_names.json [AUTHORITATIVE] — 68 terms in Last Dwarf. Structure

{name: {phonetic, source, confirmed, common, ...}}. Only 6 of the 29 proper_names have phonetics; Hallr→HALR and Feliu→FAILEU are source=user, confirmed=true, while Agnarr→AHG-NAHR and Ranjeet→RAHN-JEET are source=guess, confirmed=false.

4.5.1 The respelling mechanism is language-agnostic — verified

You are right that the Hallr→HALR logic is part of the audio system and must work for any language sent back to it. I tested it. apply_phonetics() (umbraaudio/pronunciation.py:181) is a longest-name-first literal str.replace over the text, and it is script-agnostic:

Input	Phonetic map	Result
Hallr nodded	{Hallr: HALR}	HALR nodded	ok
Boulanger said	{Boulanger: パン屋}	パン屋 said	ok — CJK survives
Warlord said	{Warlord: 将军}	将军 said	ok — CJK survives

normalize_tts_text() also preserves 将通过, こんにちは, and Müller intact. So the explicit-spelling path is multilingual today and the shared schema should carry the phonetic store unchanged, keyed by locale.

One real gap: the IPA convenience path is Latin-only and fails silently. ipa_to_simple() (pronunciation.py:307) ends with re.sub(r"[^A-Za-z0-9 .'-]", "", result), which strips every non-ASCII codepoint:

IPA in	ipa_to_simple() out
ˈmʌðər	MUHTHUHR	ok
ɔɪ	OY	ok
パocht	OCHT	kana silently dropped
ɸɯ	`` (empty)	silently empty
ẽː	`` (empty)	silently empty

So a Japanese or Chinese pronunciation cannot be auto-derived from IPA — it returns empty or partial Latin. It must be entered as an explicit spelling. This does not block multilingual work, but it must be a documented rule in the aligned definition set, and the shared schema should record the provenance of each phonetic entry (explicit vs ipa_derived) so an empty IPA derivation is detectable rather than silent.

4.6 voice_assignments.json [AUTHORITATIVE, legacy] — {character_name: "en-US-JennyNeural"},

Edge-TTS voice map. DealHunter only.

4.7 m4b_meta.json [AUTHORITATIVE] — 18 publishing fields: title, author,

narrator, publisher, copyright_holder, copyright_year, isbn, series, series_number, subtitle, synopsis, genre, keywords, language, explicit, release_date, cover_path, output_name.

4.8 Progress/cache files [CACHE] — pipeline_status.json `{step, status, detail,

progress, error}, generation_status.json {active, type, current, done, total, failed, errors[], started_at, completed_at}(the only one written atomically, tmp+rename),analysis_status.json, pronunciation_status.json, scan_results.json, and chapter_audio/ch{N}_build_status.json {phase, detail, progress, total_utterances, generated, skipped, failed[], errors[], chapter_wav, chapter_mp3, started_at, elapsed, quality_issues[]}. **Four of these five are written with a truncating open(..., "w")`** and can be read half-written.

4.9 vocal_traits post-processing hazard [important]

vocal_traits is not merely descriptive. get_effects_for_traits() (umbraaudio/generation/audio_effects.py:474) maps 16 exact lowercase trait strings to post-TTS DSP damage, applied by chapter_builder.py:598 after synthesis:

broken, distorted, raspy (noise_inject 0.04 + pitch_jitter), grainy, choppy, stutter, staccato, hard to understand, muffled, indistinct, robotic, creaky, whisper, metallic, parrot, squawky.

Adding one of these words to a speaker's vocal_traits will damage every utterance rendered for that speaker. This caused a known hiss defect. Currently no speaker in any project has a trait on that list, so the hazard is latent. Safe descriptive traits that shape the TTS voice without DSP: nasal, gravelly, breathy, husky, rough, crisp, smooth.

---

5. Speaker and voice identity — the central problem

5.1 Five parallel stores, four different keys, no integrity

Store	Key	Location	Last Dwarf	DealHunter
characters table	characters.id INTEGER	SQLite	0 rows	0 rows
speakers.json	primary_name	JSON	70	61
voice_profiles.json	profile name	JSON	68	64
voice_assignments.json	character name	JSON	absent	present
voice_registry	character_name UNIQUE	global SQLite	—	13 rows total
library_speakers	primary_name UNIQUE	global SQLite	—	6 rows total
ref WAVs	filename slug	filesystem	71	94

The three per-project counts do not agree in any project (DealHunter: 61 / 64 / 94), and DealHunter has 94 ref files for 61 speakers — 33 orphans from deleted or renamed speakers. A shared schema must reconcile these explicitly rather than pick one.

5.2 Alias handling

Aliases exist in speakers.json (aliases, plus alias_evidence) and in voice_profiles.json, and there is a merge operation that folds one speaker into another and rewrites utterances.speaker_name (routes.py:2294-2344). There is no global alias table, no alias uniqueness constraint, and no record of which alias was used for a given utterance. registry_aliases in the global DB exists but holds 0 rows.

5.3 The linkage problem for translation

utterances.speaker_name is a denormalised string. A translation system cannot reliably join on it, because:

• casing varies (narrator vs Narrator; both occur in one book),
• an alias may appear instead of the canonical name,
• merge operations rewrite it in place with no history,
• there is no FK and no normalisation,
• scene-break rows carry speaker_name='Narrator' by author rule but their marker records the scene_break sentinel.

5.4 The vmatrix — the canonical voice record

Answer: yes, the vmatrix is project data and it belongs in the shared database — but it is currently the least-governed object in the system, and it is not locale-aware, which blocks multilingual use.

A voice in this system is defined by two things, and both are required:

Component	What it is	Where it lives now	Class
Description	prose voice brief: register, timbre, accent, delivery	voice_profiles.json / pinned speakers.json.voice_description	AUTHORITATIVE, portable, language-agnostic
vmatrix	"the set of numbers inside Qwen3-TTS that IS the voice" — the full ICL codec-token matrix (ref_code, the "8k matrix") plus the speaker embedding	<QWEN_PROMPT_DIR>/{slug}.pt	AUTHORITATIVE, engine-specific, NOT language-agnostic

The description shapes synthesis; the vmatrix pins the identity. Per the server's own docstring (scripts/qwen_tts_server.py:260-275), the full vmatrix means "generation conditions on the exact reference identity, so every utterance is faithful AND identical in identity across the whole pipeline."

Two capture modes, and the wrong one causes the bug you already fixed:

• x_vector_only=0 — canonical "full vmatrix": ref_code + speaker embedding. Use this.
• x_vector_only=1 — coarse speaker embedding only. Documented as "Legacy/experimental — caused the historical 'voice changed between chapters' inconsistency. Not used for production consistency."

Five facts the shared schema must resolve:

1. It is keyed by a name-derived slug, not an id. prompt_map[spk] is keyed on the speaker name string; _prompt_path() does Path(slug).name + .pt. So the voice's canonical identity is a filename derived from a mutable display name. A rename or a merge orphans it.
2. It is not locale-aware. There is no locale in the slug or the path. An English vmatrix captured from an English ref cannot be assumed valid for German or Japanese delivery. The shared model must make the vmatrix a (voice_profile, locale) pair artifact, with a stable vmatrix_id replacing the slug.
3. Coverage is inverted from need. Only DealHunterShip has vmatrix files (38 .pt, 2.5 MB). Last Dwarf — the active project — has none. Treasure Island and Vocals have none.
4. The registry pointer is empty. voice_registry.vmatrix_path is '' for all 13 rows in ~/.umbra/voice_registry.db, so the one place designed to record the pointer was never populated. DealHunter has the files but no pointer; the registry has the pointer column but no values. Nothing currently links a speaker to its vmatrix.
5. It is captured but not used for bulk rendering. ChapterBuilder captures the full vmatrix as the canonical record, then renders in embedding mode (use_embedding_mode defaults on) because "full-vmatrix ICL generation is >15 min/utterance" versus "~17s/utterance" for embedding mode (chapter_builder.py:156-158, 469-475). So today the vmatrix is a record of identity, and voice constancy in practice comes from the ref WAV plus the per-utterance voice_fp in the staleness marker. The shared schema should model both paths explicitly rather than imply the vmatrix is the render input.

Proposed shared representation:

voice_assets
  id, organization_id, voice_profile_id
  locale_tag            -- NOT NULL: one asset per (voice, locale)
  engine, model_id      -- 'qwen3-tts', 'Qwen3-TTS-12Hz-1.7B-...'
  capture_mode          -- 'full_vmatrix' | 'x_vector_only'
  object_key, sha256, byte_size     -- the .pt, in object storage, never in PG
  prompt_slug           -- legacy key, retained for back-compat only
  source_ref_sha256     -- the ref WAV this was captured from
  status, captured_at, invalidated_at

bytes never enter PostgreSQL (Class R biometric). The hash, key, locale, engine, capture mode and provenance must, because without them a vmatrix is unidentifiable, unreproducible and unverifiable. source_ref_sha256 is the critical link: it ties the vmatrix to the exact reference it was captured from, so a re-voiced speaker can be detected and its vmatrix invalidated — the same mechanism the marker voice_fp already implements for utterances.

5.5 Voice constancy is currently enforced by the marker, not the vmatrix

Worth stating plainly, because it affects what "constant voice" means in the new system. The per-utterance guarantee today is:

• voice_fp in *.wav.txt = 16-hex SHA-256 of the ref WAV → changing the ref invalidates every utterance for that speaker, forcing regeneration;
• voice_mode records whether the render used embedding or icl.

So constancy is enforced relative to the ref WAV, at generation time, per utterance. A shared database should preserve that mechanism (it works) and add the vmatrix as the engine-level identity record, so the two can be cross-checked.

---

6. Audio artifacts, naming, and idempotency

6.1 Naming

Object	Pattern	Notes
Per-utterance audio	chapter_audio/ch{N}/utt_{utterance_index:04d}.wav	named by a mutable index
Staleness marker	chapter_audio/ch{N}/utt_{idx:04d}.wav.txt	JSON sidecar
Chapter master	chapter_audio/ch{N}_full.wav	overwritten in place
Chapter MP3	chapter_audio/ch{N}_full.mp3	44.1 kHz mono 192 kbps CBR
Full book	export/{m4b_meta.output_name} e.g. Deal_Hunter.m4b	311 MB in DealHunter
AR track	export/ar_package/{Base}_{NN}_{Title}.mp3
Legacy path (unused)	.umbra/audio/ch{N}/ch{N}_u{idx:04d}.wav	manifest-driven, superseded

Last Dwarf has 10,018 WAVs for 10,015 utterances (3 orphans from reindexing) and 10,015 markers. 49 chapter masters, 10 chapter MP3s. DealHunter has 64 files under export/.

6.2 The staleness marker — the de facto idempotency contract

*.wav.txt is JSON with keys speaker, text (always), plus voice_fp (16 hex of SHA-256 of the ref WAV), voice_mode (embedding|icl), instruct, fx_version (int) when applicable.

is_current() (umbraaudio/generation/staleness.py:38-98) requires, in order: the wav exists; the marker exists and parses as a JSON object; text matches exactly; then optionally speaker, instruct, voice_fp, voice_mode, fx_version. Subtlety worth preserving in any reimplementation: a marker carrying fx_version is treated as stale for a run that requests none (:98).

marker.text is the prepared TTS text, not the DB text — prepare_tts_text() applies apply_phonetics(), so proper names are substituted and uppercased. Any migration comparing marker text to DB text must apply the same transform or it will report false staleness.

6.3 No content integrity

No SHA-256 or checksum of any produced audio file is computed or stored anywhere. The only hashes in the system are the 16-hex voice-ref fingerprint, a TTS cache key, an author watermark seed, and a deterministic TTS parameter seed. The AR manifest records size_bytes, not a digest. Content integrity must be introduced from scratch.

6.4 Current staleness, measured

Corrected measurement (marker text compared against prepare_tts_text(DB text)), Last Dwarf:

Metric	Value
Utterances	10,015
Marker text current	8,138
Marker text stale	1,796
Speaker-stale	520
Markers on re-voiced refs	496
Missing WAV	1

chapters 9–12 were fully rebuilt on 2026-09-26 and are clean on all four metrics. Worst remaining chapters: ch34 (106 text / 40 ref), ch20, ch27, ch30, ch45.

---

7. Analysis results

Artifact	Location	Class	Content
LLM discovery passes	discovery/discovery_pass{1,1b,2,3}.json	TRANSIENT	the run that produced speakers.json
Voice acoustic analysis	analysis/voice_analysis.json	DERIVED	per-ref F0/timbre metrics
Attribution clarity	utterances columns	AUTHORITATIVE	evidence kind, tag name, uncertainty, review flag, reason
Attribution exceptions	exception_reviews table	AUTHORITATIVE (empty)	low-confidence queue
Orphan name tokens	name_tokens table	AUTHORITATIVE (10 rows)	unresolved proper nouns + suggestions
Speaker confidence	computed at read time	DERIVED	clear / needs-check designation
Pipeline defects	defects.json	DERIVED	defect + ambiguity log (1.3 MB in DealHunter)
Chapter QA	ch{N}_build_status.json quality_issues[]	CACHE	in-memory issues only; no persisted report
Pronunciation	phonetic_names.json + pronunciation_audio/	AUTHORITATIVE / DERIVED	term→phonetic + preview clips

There is no persisted analysis-run record. No table or file records which model, prompt version, or parameters produced a given analysis. Re-running discovery is not idempotent and overwrites speakers.json.

---

8. Cross-project global stores

~/.umbra/voice_registry.db (45 KB) — shared by every project, outside all of them:

Table	Rows	Purpose
voice_registry	13	character_name UNIQUE, 25 voice-taxonomy columns, voice_ref_path, vmatrix_path, speaker_embedding BLOB, source_project
registry_aliases	0	UNIQUE(registry_id, alias_name)
library_speakers	6	primary_name UNIQUE, vocal_traits, distortion_effects, source_project
library_names	0	name→phonetic
library_utterances	0	shared sample lines

~/.umbra/library/voices/ — 6 shared refs: aide_(royal), deal_hunter, hallr, nanny, narrator, xylia.

These are cross-project global state reachable from every project's web UI with no authorization. A shared database must place this in an explicit organization-scoped catalog, not an unowned per-user file.

---

9. Data classification

Class	Contents	Handling
C — copyrighted / licensed	chapters.source_text, utterances.text, context_before/after, uploads/, all produced audio, all exports, defects.json	Tied to the author's licence; translation derivatives inherit it. Class R voice data must never enter PostgreSQL — keys and rights metadata only.
R — biometric / consent	voices/*_ref.wav, voice_embeddings/*.npy, voice_prompts/*.pt, library/voices/, voice_registry.speaker_embedding BLOB	Encodes a human or cloned vocal identity. No retention, consent-evidence, territory, or takedown mechanism exists today.
P — personal	voice_review_history.reviewed_by (literal 'user'), exception_reviews.resolved_by (literal 'user'), generation_status.json timings	There are no user accounts, logins, or sessions anywhere in this system.
I — internal	pipeline_status.json, generation_status.json, markers, checkpoints, cache/	Safe to discard.

---

10. What the translation system needs from here

This is the contract the shared database must satisfy. Ordered by how badly it is needed today.

#	Need	Current state	Gap
1	Stable source-unit identity across languages	none; utterance_index is mutable and filenames depend on it	blocking. Requires a content-addressed key (e.g. sha256(source_revision_id ‖ chapter_ordinal ‖ occurrence ‖ text_sha256)) recorded at generation time
2	Stable speaker identity	speaker_name free text; 5 stores, 4 keys, no FK	blocking. Needs speakers + speaker_aliases with a canonical id and a recorded alias→id resolution
3	Immutable source revision to pin a translation to	chapters mutable; no revision concept	blocking. Needs source_revisions with a content hash
4	Chapter/scene structure	chapters + context_before/after	available, needs a stable id
5	Attribution confidence to prioritise review	5 clarity columns on utterances	available and good; migrate as-is
6	Speaker characterisation for the translation brief	demographics, vocal_traits, speech_patterns, description	available; gender often null — prefer demographics.gender
7	Name pronunciation	phonetic_names.json, 6/29 covered	partial; most names unpronunciated
8	Lineage source→translation→audio	none	blocking. Needs artifacts + artifact_links with SHA-256
9	Quality/verification of the source	ch{N}_build_status.json only, 4/5 non-atomic	weak; no persisted report
10	Manuscript + edition identity	project.source_path only	needs works + source_documents
11	Voice identity for the target language	vmatrix is name-keyed and locale-blind (§5.4); absent entirely for the active project	blocking for translated audio. Needs voice_assets per (voice, locale) carrying capture mode, engine, hash and source_ref_sha256
12	Name pronunciation in the target language	explicit spellings work for any script (§4.5.1); IPA derivation is Latin-only and returns empty for other scripts	needs per-locale phonetic entries with recorded provenance
13	Which voice-attribute profile is canonical	three disagreeing sets (§1.5, open question 14)	needs measurements stored separately from definitions

Not needed by the translation system: per-utterance WAV files, voice refs, embeddings, Qwen prompt caches, Edge-TTS assignments, TTS run progress, AR/MP3 packaging. These are audio-production concerns and should stay behind the audio repository boundary.

---

11. Exports and published packages

Only DealHunterShip_4.0e_whole-to-audio has a finished package:

• export/Deal_Hunter.m4b (311 MB)
• export/ar_package/ — per-chapter MP3s
• export/ar_package.json — manifest: {book{title, author, narrator, publisher, output_name}, base_name, built_at, format{codec, sample_rate, channels, bit_rate_kbps, cbr, silence_s}, track_count, total_duration_s, tracks[], warnings[]} where each track is {seq, role, number, title, name, file, source, duration_s, size_bytes, rms_db, peak_db, warning, loudness_ok, peak_ok}
• export/ar_work/ — scratch (announcements, manifests, probes); regenerated, safe to delete
• export/ar_build_status.json — build progress

Quality gates enforced at export: 44.1 kHz mono 192 kbps CBR, RMS target −20.5 dB clamped ±12 dB, hard clip at −3 dBFS, 2 s silence padding.

Last Dwarf has no export/ content (only an empty ar_work/), and 10 chapter MP3s.

---

12. Known defects that affect this manifest

Reported, not fixed — they constrain any migration:

1. No authentication or authorization anywhere. No login, no session, no user model, no CSRF, no security headers, no TrustedHost. Project selection is an unsigned active_project browser cookie with a 1-year max_age and no httponly/samesite/ secure. 65 mutating endpoints have zero authorization checks. The server binds 0.0.0.0:8420 over plain HTTP.
2. load_config() accepts absolute paths verbatim (config.py:156-158), so the cookie is a path-selection primitive.
3. Download traversal at routes.py:5049 — str(...).startswith(...) is string-prefix containment, not path-component containment.
4. Upload path traversal at routes.py:633-652 — upload.filename used without Path(...).name.
5. No job queue, no lease, no heartbeat, no backoff. Work is a for loop over SQLite rows in a daemon thread; the only mutual exclusion is an on-disk boolean (generation_status.json) and ChapterBuilder has none at all.
6. Four divergent DDL authorities and per-project schema drift (§1.3).
7. Non-atomic JSON writes in 4 of 5 progress files.
8. No checksums on any artifact (§6.3).
9. Foreign keys never enforced — no PRAGMA foreign_keys=ON.
10. vocal_traits DSP hazard (§4.9).
11. Scene-break speaker_name must be Narrator by author rule; the generator's scene_break sentinel is not valid data (normalized 2026-09-26, 74 rows).

---

13. Legacy → shared entity mapping

A bridge table for schema design. "Authority" is what must win when sources disagree.

Legacy object	Legacy key	→ Shared entity	Authority	Notes
project dir	directory name	organizations, projects	dir name	legacy_project_path must be recorded
project row	project.id	projects	name, source_path	id is per-DB, meaningless across projects
project.source_path	path	works + source_documents	the file	content-address by SHA-256
chapters	chapters.id	source_revisions, source_chapters	source_text	needs an immutable revision layer
utterances	utterances.id	source_units	text	new content-addressed key required
utterances.speaker_name	free text	source_units.speaker_id	resolved via speakers+speaker_aliases	unresolvable names must be recorded, not dropped
utterances.instruct	—	source_units.instruct	as-is	reproduces the render
utterances clarity columns	—	source_units attribution fields	as-is	migrate verbatim
speakers.json	primary_name	speakers	speakers.json	primary source
speakers.json.aliases	—	speaker_aliases	as-is	needs a uniqueness rule
speakers.json.unidentified	—	speakers.is_unidentified + reason	as-is	7–8 per project
voice_profiles.json	name	voice_profiles	profile file	68/64/41/2 vs 70/61/40/2 — reconcile
voice_status.json	name	voice_profiles.status	as-is
voice_engines.json	name	voice_assets.engine	as-is	edge = silent fallback, must be visible
voices/*_ref.wav	filename slug	voice_assets (object key + SHA-256)	the file	bytes never in PostgreSQL
voice_prompts/{slug}.pt (vmatrix)	prompt_slug (name-derived)	voice_assets rows, one per (voice_profile, locale)	the .pt file	the canonical voice identity; capture full_vmatrix only; bytes never in PG
voice_registry.vmatrix_path	—	voice_assets.object_key	—	column exists but is empty for all 13 rows — no pointer is populated today
*.wav.txt voice_fp	16-hex ref hash	artifacts.voice_source_sha256	—	the per-utterance constancy guarantee
*.wav.txt voice_mode	embedding|icl	artifacts.render_mode	—	proves which path produced the audio
voice_embeddings/*.npy	slug	derived	discard	cache
voice_prompts/*.pt	slug	derived	discard	cache
phonetic_names.json	term	pronunciation_entries	as-is	68 terms, only 6 applied
pronunciation_guide	(project_id, term)	pronunciation_entries	table (empty)	unused
m4b_meta.json	—	works metadata	as-is	18 fields
ch{N}/utt_{i}.wav	path	artifacts (legacy_object_key)	the file	named by mutable index
*.wav.txt marker	path	artifacts.legacy_marker	the marker	carries the idempotency contract
ch{N}_full.wav / .mp3	path	artifacts + artifact_links	derived	stitch of the parts
export/*.m4b, ar_package/*	path	artifacts	the file
ar_package.json	—	release manifest rows	as-is	14 track fields
manifests/ch{N}.json	—	derived	discard	regeneration inputs
*_build_status.json	—	audio_runs (legacy)	discard	cache
pipeline_status.json	—	—	discard	cache
discovery_pass*.json	—	—	discard	transient
name_tokens	(project_id, token)	review queue	as-is	10 rows
exception_reviews	id	review queue	as-is	0 rows
voice_registry / library_*	name	org-scoped voice catalog	reconcile	cross-project, unowned
'user' in reviewed_by/resolved_by	literal	one canonical user id	—	the only identity that exists

---

14. Non-static transition requirements

You are right that this will not be a one-shot migration. The following must be properties of the shared schema, not deferred work.

14.1 The transition is continuous, not a cutover

Three projects in three states, one of them published and frozen, one in active churn. The shared database must accept writes from the audio system and the translation system concurrently, forever, while their definitions evolve.

Requirement	Why	Consequence for the schema
Per-project incremental adoption	Last Dwarf is being edited daily; DealHunter is frozen; Treasure Island is a lab	No global switch. A per-project migration_state (legacy / mirroring / dual_write / authoritative / retired) plus an independent project lifecycle dimension (draft / active / released / archived / experimental, §1.4).
Additive-only schema evolution	Both systems evolve independently for years	Every migration must be additive and backward-compatible. No destructive column drops or renames while a consumer is live. Deprecate, don't delete.
Definition versioning	You will align one definition set across systems; it will change	Any definition that a stored row depends on (utterance type, speaker kind, attribute taxonomy, phonetic provenance, capture mode) needs a versioned vocabulary, not a bare string.
Vocabulary is data, not enums	Legacy status/type fields are unconstrained TEXT with no CHECK constraints; values already diverge between projects	Controlled vocabularies as reference tables, so new values can be added without a code deployment in either system.
Schema-migration ledger in-band	Four DDL authorities already drifted (§1.3); that must not recur	Reuse the existing schema_migrations(version, applied_at) pattern from 001_control_plane, and require every consumer to record the version it wrote against.
Re-import is idempotent	A project may be re-scanned, re-discovered or re-captured at any time	Every import keyed by content hash, with a legacy_id_map-style ledger recording what was mapped, what was ambiguous and what was rejected — never silently dropped.
Immutable released lineage	Deal Hunter is a shipped product	Released revisions must be append-only and reproducible. A "fix" creates a successor, never an in-place edit.
The audio path must never depend on the shared DB being up	Production continuity is a hard requirement	A worker continues on a locally-held lease when the shared DB is unavailable, and reconciles later.

14.2 The aligned definition set

You intend one definition and documentation set for all systems. For that to work, the definitions the audio system already relies on must be extracted from code into data. The current state is the opposite — the definitions are implicit in executable code:

Definition	Currently	Should become
Utterance kind	unconstrained TEXT in utterances.type	versioned vocabulary + a definition document
Speaker kind	scattered booleans (is_narrator, is_group, unidentified)	versioned vocabulary
speaker_name='Narrator' for <<<>>>	implicit in the generator's sentinel	written rule (specs/16 §5a, added 2026-09-26)
Post-TTS effect traits	a Python dict literal (TRAIT_EFFECTS, 16 entries)	data table, with the safe/unsafe distinction explicit
Voice attribute taxonomy	pitch_register/timbre/texture/… columns + a prose analyzer	versioned attribute dictionary + a measurement profile
vmatrix capture modes	server docstring only	vocabulary + required fields
Phonetic provenance	not recorded	explicit vs ipa_derived, per entry
Scene-break silence length	SCENE_BREAK_SILENCE_MS = 2000 constant	configuration with a recorded default

This manifest is the data inventory half. The vocabulary/definition half is the companion artifact and should be authored before the shared schema is frozen, because the schema encodes those definitions as columns and constraints.

---

15. Open questions for schema design

1. Who resolves unresolvable speaker names? utterances.speaker_name values with no speakers.json entry or alias match need a policy: create a placeholder speaker, or record the name unresolved. The translation system needs a stable id either way.
2. Which of the three speaker stores wins when they disagree (70/68/71, 61/64/94)?
3. What happens to the 33 orphan ref WAVs in DealHunter — adopt, or mark orphaned?
4. Should utterance_index be preserved as a display ordinal, or dropped in favour of a content-addressed key? Filenames depend on it, so audio paths break either way.
5. Is chapters.source_text duplicated in utterances.text? Together they are two copies of the copyrighted manuscript. The shared schema should store it once.
6. Should the 4 non-atomic progress files be migrated at all, or treated as cache and dropped? Recommendation: drop.
7. Does the translation system need the audio artifacts, or only their metadata and checksums? This determines whether object storage is a hard prerequisite.
8. Granularity of the shared project: one row per book per language, or one project with locale lanes? The latter is what specs/20 §F.14 (project_locales) proposes. This also decides where a per-language vmatrix (§5.4) attaches.
9. Voice reuse across projects — permitted today via the unowned global registry. What are the licensing/consent rules? None are recorded anywhere.
10. Retention for the copyrighted manuscript, produced audio, and Class R voice data.
11. Does one voice need a separate vmatrix per target language, or can a single vmatrix serve all locales for a voice? The server has no locale dimension at all (§5.4), and this determines whether the vmatrix count multiplies by locale. Recommendation: assume per (voice, locale) and allow a marked "locale-neutral" asset.
12. Should the vmatrix be captured for Last Dwarf now? It is the active project and has zero vmatrix files, while the published project has 38. If the shared DB becomes the record of voice identity, the active project is the one that needs it.
13. Should Treasure Island's experimental voices be importable as production assets? Its flint_a/b/c variants and the parrot voice are lab results. Recommendation: import into a namespace flagged experimental, never into the production cast.
14. Which voice-attribute profile is canonical — the analyzer's 9 measured features (§1.5), the 9-field prose taxonomy in voice_profiles.json, or the 13-dimension acoustic set used for the crowding analysis? All three exist and disagree. The shared schema should store measurements and definitions separately rather than merging them.

---

16. Reproduction

The figures above were produced by read-only introspection on 2026-09-28. Raw extracted output (705 KB of schema DDL, column lists, per-table row counts, per-speaker records, JSON key shapes, artifact inventories, and chapter listings for all four projects) is at:

/private/var/folders/y8/cbvhds297918lrxt2wd3btsr0000gn/T/opencode/manifest_data.json

Re-extract with .venv/bin/python .../extract_manifest.py (script in the same directory). All connections are mode=ro.