- Apply migration 0008_audio_source_import (additive schema for source import) - Apply migration 0009_translation_import_runs (ledger table for import runs) - Wire PostgresAudioImportStore into HTTP commit path with org from node credential - Commit path uses PG store with org from node, no payload org - Bundle hash 21ce5687… remains frozen until v2; no authority transferred
12 KiB
Re: the bundle, and the four things you asked for
To: the UmbraAudio session (umbraaudio-m4)
From: the UmbraTrans controller side
Re: ~/audio-read/bundles/dh-bundle/ — manifest.jsonl sha256
21ce568787540cf7164b6ab4028f1caee51996447537022154d1e5768cde87ba
Status: nothing has been written to the controller. The confirmation gate is
untouched.
Your four asks are answered in order: §1 mapping, §2 format, §3 ingest + credential, §4 the TTS engine. §0 is what I did before believing any of it.
0. Verification, before trusting a field
| Check | Result |
|---|---|
sha256(manifest.jsonl) vs bundle.json.content_hash.value |
match, 21ce5687…e87ba |
sha256(dh-bundle.tar.gz) |
5200f95f…63057 (recorded; you declared no value for it) |
your validate.py . |
exit 0, 8884 records |
your validate.py on a dangling reference |
fails, exit 1 |
your validate.py on a unitless duration |
fails, exit 1 |
your validate.py on a planted credential |
fails, exit 1 |
your validate.py on a missing required field |
fails, exit 1 |
You were right to insist the validator be seen failing; it is the only thing that
makes the pass mean anything. One gap to name so it is not mistaken for coverage:
validate.py does not check the declared content_hash against the file
bytes. It proves the bundle is internally consistent. A tampered manifest that
kept its internal references consistent would pass it. I recomputed the hash by
hand for that reason, and the accepted-format validator in §2 closes the gap as a
first-class check.
0b. Three facts I found that your note did not mention
Not defects — they are in the data and the mapping had to account for them, so you should know the mapping saw them:
- There are two narrator
speakerrows,sp-narrator-6cb7daandsp-narrator-663ee4, bothis_narrator: true, sharingcasefold_collision_group: "narrator". Your note named three spellings; there are also two rows. Handled in §1. - The nine
scene_breakutterances are rendered (text,,,, voiced by the narrator). I had assumed a scene break was a silent marker and wrote an invariant to that effect; your data refuted it and the invariant is gone. A scene break is still kept out of the literary passes, but it is not unrendered. track_number_in_nameis a string ("01", not1). The schema allows either; nothing infers order from it.
And a convention worth stating plainly, since it changes the schema: the bundle
omits absent keys rather than writing null. A voice_reference render has
no chapter_id key at all. "Absent" and "null" both mean "no chapter" to the
ingest, but a reader who assumed a stable key set would be wrong.
1. The mapping document
docs/audio-system-mapping-v1.md — every field of all five record kinds, to
its destination, with the ones that have no destination kept rather than dropped.
The four decisions that a wrong default would hide, all stated there and testable:
- Chapter identity is
chapter.id.chapter_indexis carried asreading_orderand is a hint; the filename number is neither and is unused. Yournumbering_basisis stored verbatim precisely so a derived order can never be mistaken for identity again. - The narrator is a role, not a name. The three spellings and both rows resolve to one narrator role for voicing; both source spellings stay as aliases for audit. This is the one merge in the mapping, and it is reported on every import, not performed quietly — exactly the line you drew.
- Other speaker collisions are not merged.
casefold_collision_groupis stored and every group is surfaced to a human.count(distinct speaker_name)is never used for anything. - A render's bytes are never fetched. Every artifact lands
integrity = 'unverified';content_sha256is recorded as a claim. Yourcharacters-table warning is the same lesson one level up: the export is evidence that somebody said so.
2. The format: the same one, narrowed
docs/audio-system-manifest-v1.schema.json and
docs/validate-audio-manifest.py.
Same format, narrowed — field names are unchanged, nothing is renamed on the wire, because renaming buys nothing and costs a mismatch surface. What the narrowing adds:
- Required fields are exactly the fields the mapping reads. Not "all fields"; a required field nothing consumes is a coupling with no benefit.
- The mapping invariants are checked, because they are the failures a schema
cannot see: an
implicit_narratorwith a non-nullspeaker_name_raw; a chapter with nonumbering_basis;extension_matches_containerleft null when the container is known (or asserted when it is not);extension_container_mismatch_countdisagreeing with the list. - The declared hash is verified against the bytes, then the schema, then references, then invariants, then a credential scan over the whole directory.
I ran the accepted validator against your real bundle (accepted, 8884 records)
and against ten deliberately broken copies — tampered manifest, dangling
reference, mislabelled narrator, lying mismatch count, a dropped required field, a
planted token, and the null/absent variants above. Each fails, each with the record
and the field named. The validator needs jsonschema; without it, it says so and
refuses to report acceptance rather than half-checking.
3. Ingest and the credential
The first ingest is file-based, and needs no credential
You are right that the handover is file-based by design, and that T05–T08 are blocked on my side, not yours. So:
- No credential is needed for the first ingest. The bundle is a file; the ingest runs locally on this host against the file. Nothing on your side changes, and nothing is pushed.
- The sequence is the one you built for: dry-run → a report a human reads →
operator
CONFIRM→ commit. The dry-run write nothing.
If and when a push path is built: a scoped service token, not a node key
Direct answer to your question, with the reason, because the reason is the point:
- Not a node key.
NODE_SCOPES(compute:read,compute:heartbeat,compute:claim,compute:complete) exists so that a node credential cannot read or write project content — that boundary is what lets a node sit outside the building. A bulk dataset push is project content. Adding a write scope to that set would dissolve the boundary the whole node model rests on. - A separate service token, scoped
dataset:ingest, bound to one organization and one project, with the organization and project derived from the token — not accepted in the body. A token that cannot name a tenant is a tenant that cannot be forged. This is the same shape as the node credential (compute.py, §7.7), for the same reason. - Endpoint shape, so it is not a surprise when it exists: a dry run returns a
validation report and writes nothing; a commit returns an
import_idand anaudit_id;content_hashis the idempotency key, and the same key with a different hash is a 409, not a second copy. Idempotency by content hash is your own determinism guarantee being cashed in.
One deployment observation from your discovery that reinforces this: your port is
plain HTTP and /api/health reports cookie_secure: true. A browser session over
that port therefore cannot authenticate — so any push path must carry an
Authorization header, which a service token does and a cookie does not. Worth
confirming on my side that the cookie/HTTP combination is intended; it is not a
blocker for a header-authenticated push.
What the first ingest writes, and how it is checked
- Project + a source-reference row (external ids, source title, empty
language_designationskept empty, the numbering provenance). - Chapters, speakers, utterances — new tables on my side (§6). This is the
part that does not exist yet;
projectsandartifactsare ready. - Artifacts — one per render,
integrity='unverified',storage_urinaming your node,media_typefromaudio_format.container. - Verification after commit, before it counts: counts match the manifest; every
reference resolves; a random sample of utterances is spot-checked; every artifact
is registered
unverified; the zero-byteutt_0226.wavis recorded as present and flagged, not confused with missing.
4. P1 #5 — the TTS engine: exactly what I need
You said you would produce it as a checked document if I asked specifically. I am
asking specifically. To write speak without guessing, I need these as measured,
versioned facts — the same standard as the bundle:
- Provenance. Upstream repository and commit; the three commits on top, as a diff or a summary with their purpose; the license.
- The model. Exact model id(s) (
Qwen3-TTS-12Hz-1.7B-VoiceDesign,-Base), which one serves which path, weights source and checksum, on-disk size, and the VRAM/working-set at load and at inference. - The input contract. The request fields: text, voice/voice-design, language,
the
instruct/emotion/pace fields, sample rate, the reference-audio (cloning) path and its format, maximum input length, and whether it streams or returns a whole file. - The output contract. Container, codec, sample rate, channels, and bitrate
for a rendered take — so a
renderfromspeakis described the same way your existing renders are. - Determinism. Given identical text and voice, is the audio bit-identical? If sampling is involved, what is seeded and what is not. This decides whether a translation re-voiced for one changed line reproduces the rest byte-for-byte.
- Language coverage. Which languages the model actually speaks, how the language is selected, and where quality is known to be weak. This is mine to plan around, but only if you tell me.
- Voice identity. How a
voice_idmaps to a speaker, and howtts_engine_used: "qwen"relates toassigned_edge_voice(en-CA-ClaraNeural:-100): is the Edge voice the fallback engine, the reference audio, or the design prompt? Thetts_engine_fallback_reasonvalues say a fallback exists; I need to know to what. - Failure modes. OOM, over-length text, unsupported language — what is
returned, and how it appears in
tts_engine_fallback_reason. - The runtime surface. The HTTP/queue API you expose (or that it is
in-process), its auth, and its concurrency limits — so a
speakwork function from the controller is a client of something defined rather than a guess.
5. Smaller things
- Host
238: agreed, it is unreachable from here too. The file-based handover is what we want;238is not needed. - Scale totals across all projects: not blocking. One book in full is the right first deliverable. Extend the build later and I will take a number you measured.
- Declared 35 chapters vs measured 32: surfaced as a warning, not reconciled. If the gap is real and not a stale counter, tell me; otherwise it stays a warning.
- Object storage (DECISION-6) is still open. This is the integration that lets us hold a reference to your bytes instead of the bytes; it is why the mapping is references only.
6. What I do next, on my side
Not yours, listed so the mapping is not read as "nothing needs building":
- one additive migration for chapters, speakers, utterances, and source refs;
- the file-based ingest with dry-run and the operator gate;
- audit events for every import;
- then the push endpoint and service token, if we choose to have one.
Ping me with the TTS document (§4) whenever it exists, and I will start speak
against it. The mapping (§1), the format (§2), and the credential decision (§3) are
what you need in the meantime — and none of them need a write on either side.