umbragit/UmbraAudio
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Action the controller alignment proposal: source_unit_key + phase reports

Browse Source 
The controller (oldbox) sent a data-model alignment proposal and asked for six
additive changes on this node, the validation protocol, and an honest report.
Six of the seven changes are done; the seventh is declared missing rather than faked.

Stage 0 - reachability verified exactly as documented: health 200, heartbeat 401
(the correct answer - auth is refusing), missing Host 400. Nothing else was sent.

source_unit_key (controller 5.1, the blocking item)
  umbraaudio/source_units.py. sha256("uk1" | chapter_source_text_sha256 |
  occurrence | text_sha256). Nothing positional in the key: "occurrence"
  disambiguates repeated text in document order.
  - 18,181/18,181 units keyed across all four projects, 0 collisions
  - chapter 1 renumbered +100 -> 0 keys changed (the load-bearing proof)
  - backfill 10,015 rows in 0.11s, second run stamps 0 (idempotent)
  - alignment phase 3: 200/200 probe units resolved with used_index: false
  - tolerates the older projects that have no chapters.chapter_index
  Recorded at generation time in chapter_builder, BEFORE the rows are read and
  BEFORE the marker is written, so every path that renders audio also produces
  identity and the clip can name its source unit.

5.7 recorded: utterance_index is KEPT as a display ordinal and made
  non-load-bearing. spec 21 §10.

Blob hashes and effect chain (5.2)
  Markers now carry the full sha256 of the delivered bytes and the effects that
  produced them. Verified the existing order is already correct - write_marker
  runs AFTER apply_effects - and closed the remaining hole: a changed trait list
  changed the hash with no recorded cause. sha256 verification is opt-in, since
  hashing every file on every validation pass would re-read gigabytes.

Vocabularies published (5.4) - umbraaudio/definitions.py, 19 sets, 126 values,
  each value carrying whether it is 'declared' (spec/controller) or 'extracted'
  (read out of code), so a diff shows a difference of meaning, not of wording.

Alignment report tool (6.6) - scripts/alignment_report.py --phase all. Emits the
  controller's envelope with the per-phase evidence shapes, and obeys the two
  rules they set: no evidence means blocked, never pass; a field we cannot produce
  is null PLUS a finding, never omitted. Phase 2 is deliberately 'fail' - the
  three-way store disagreement is real, and a pass there would be a check that
  passed by doing nothing.

Fingerprint discipline (5.5) - the drift detector reported its OWN fingerprint
  change when source_unit_key was added rather than hiding it. Node profile
  updated with the new fingerprints and an honest note that they moved.

Node report extended - per-project source-unit-key coverage, per-voice-asset
  sha256 + object_key, definition_set_version, controller stage reached.

Six gaps found in the proposal (spec 21 §11), including the one they invited plus
five more. The sharpest: "hash at generation" ignores that audio is
post-processed afterwards, and our key is anchored to chapters.source_text - a
second copy of the manuscript - so any change to chapter storage silently
invalidates every key in that chapter.

5.6 deliberately NOT faked: personality has no versioned artifact here, only
  unversioned LLM prose in speakers.json. Reported as phase 6b finding B-03
  instead, because wrapping unversioned inputs in a version field would produce
  artifacts claiming a lineage they do not have.

Answered the controller's two open questions (infra/controller-handoff-2026-10-03.md):
forward-only link for a revealed recurring character, and yes - a new edition may
contradict a frozen state, since this node's correction path IS a regeneration.

Full suite 484 passed / 8 failed - the same 8 pre-existing and unrelated failures.
ruff clean on every file touched. The parallel session's chapter_builder hunk
(helper_url) was filtered out of the commit and left intact in the working tree.
This commit is contained in:
B.D. Murphy
2026-10-02 20:16:06 -07:00
parent 893dc57574
commit 829a03b89f
15 changed files with 2083 additions and 16 deletions
Download Patch File Download Diff File Expand all files Collapse all files

69
specs/21-umbra-system-nodes-tasks-and-changes.md
View File

@@ -568,11 +568,14 @@ criterion so it can be *confirmed* rather than reported done.
| `M4-01` | Node profile + read-only node report + `--check` | Make this machine describable to the system | `infra/umbra-node.json`; `--check` exits 0 | **done 2026-10-01** |
| `M4-02` | Commit this session's artifacts and document them | AGENTS.md cadence rule; nothing left uncommitted | Commit exists; session recorded in `AGENTS.md` | 15 min |
| `M4-03` | Reconcile the dirty working tree | 20 files uncommitted from parallel sessions; a lost day of work is a real loss here | Working tree clean, or every uncommitted hunk accounted for | 1 h |
| `M4-04` | Add `source_unit_key` (content-addressed, additive) | The one blocking identity concept that needs **no** controller. Without it, the mirror imports unstable IDs | Column exists, populated for new generations, filenames unchanged, `is_current` unaffected, tests pass | 3-4 h |
| `M4-05` | Extract the aligned definition set into data | `manifest.md` §14.2 — vocabularies currently live in code, so every product reimplements them | A versioned `definitions/` file with utterance kinds, speaker kinds, capture modes, phonetic provenance; code reads it | 3-5 h |
| `M4-04` | Add `source_unit_key` (content-addressed, additive) | The one blocking identity concept that needs **no** controller. Without it, the mirror imports unstable IDs | **done 2026-10-03** — `umbraaudio/source_units.py`; 18,181/18,181 units keyed across 4 projects, 0 collisions; renumbering changes 0 keys; 200/200 probe units resolve without the index; marker carries the key | 3-4 h |
| `M4-05` | Extract the aligned definition set into data | `manifest.md` §14.2 — vocabularies currently live in code, so every product reimplements them | **half done 2026-10-03** — `umbraaudio/definitions.py` + published `definitions/umbraaudio-defs.json` (19 sets, 126 values, per-value provenance). **Remaining: make the code read the sets instead of its own literals** | 3-5 h |
| `M4-06` | Fix the cheap security defects | Precondition for any exposure change, and for an honest node posture | Traversal + filename fixes with tests | 2 h |
| `M4-07` | Resolve the speaker-count disagreement | 70/93/71 — the single most confusing fact about this machine's data | One authoritative count per project, with the others derived or explicitly deprecated | 2-3 h |
| `M4-08` | Add a read-only `/api/node` health endpoint | Lets the controller scrape node health without an agent; LAN-only | Endpoint returns the node report; no secrets; LAN-only documented | 1-2 h |
| `M4-09` | Controller Stage 0 — reachability | Prove the node is reachable and that auth actually refuses | **done 2026-10-03** — 200 / 401 / 400, exactly as the controller documented | 15 min |
| `M4-10` | Controller Stage 1 — send the node report as a heartbeat | Prove the node runs the code it claims and reports the schema it has | `scripts/alignment_report.py` + `umbra_node_report.py` produce the payloads; **blocked on the controller issuing a `compute:read` + `compute:heartbeat` credential** | 30 min once credentialed |
| `M4-11` | Alignment report tool (controller §6.6) | Agreement must be mechanical, not prose | **done 2026-10-03** — `scripts/alignment_report.py --phase all`, per-phase evidence shapes, `blocked` never `pass` without evidence | 1 h |
### 9.2 NEXT — needs a decision or another machine
@@ -604,8 +607,66 @@ nothing else, do those two.
---
## 10. Change log
## 10. Decision record — `utterance_index` (controller §5.7)
The controller asked this node to record the decision rather than make it for us.
> **`utterance_index` is KEPT as a display ordinal, and is made non-load-bearing.**
- **Kept** because a reviewer needs "utt 231" to find a line, and because deleting or
renumbering it now breaks nothing that matters: `source_unit_key` is the identity, the
column is a label.
- **Non-load-bearing** means: no artifact name, no lineage record, no cross-reference and
no cross-language link may use it. Audio filenames still do (`utt_{index:04d}.wav`) as a
*local filesystem convenience*; the marker now also carries `source_unit_key`, so the
audio record can name its source unit without the filename being load-bearing.
- Recorded by: the UmbraAudio node, 2026-10-03. Status: node decision, open to reversal.
Proof that the decision was actually implemented, not just written: renumbering a chapter by
+100 changed **0** of its keys, and 200/200 probe units resolved with `used_index: false`.
---
## 11. Gaps found in the controller's alignment proposal
The controller asked to be told about a fifth gap rather than have it worked around. Six,
in the order they matter.
| ID | Severity | Gap | What this node did |
|---|---|---|---|
| **G-A** | major | *"Every blob gets its hash at generation"* — but audio is **post-processed after generation** (`apply_effects_to_file`, distortion effects, scene-break silence). A hash taken at generation would identify bytes that never shipped, and a changed trait list would silently change the artifact with no recorded cause. | **Verified the existing order is correct** (`write_marker` runs *after* effects), and closed the remaining hole: the marker now records `effects: [...]` alongside the hash, so the chain from raw TTS output to delivered bytes is explicit. |
| **G-B** | major | `source_unit_key` is anchored to **`chapters.source_text`** — a *second stored copy* of the copyrighted manuscript. Any change to how chapters are stored (unicode normalisation, trimming, re-extraction) silently invalidates **every** key in that chapter. | Documented as a deliberate, load-bearing coupling: content identity needs a content anchor. Mitigation is the anchor's own hash, now reported per project. Recommend the controller treat the chapter-text hash as the **source revision identity** rather than assuming a separate revision id. |
| **G-C** | major | The controller's key spec includes `source_revision_id`. This node has **no revision concept**, so it substitutes a chapter-text hash. Two sides will therefore compute **different keys for the same unit**. | Answered rather than worked around: `source_unit_key` is the audio-side key and the controller must store it verbatim as `source_units.source_unit_key` rather than recomputing it. Recomputation would be impossible anyway — the controller cannot know our occurrence rule. |
| **G-D** | minor | The marker did not record which source unit produced a clip, so an audio artifact could not be traced back to a unit revision. | Marker now records `source_unit_key`. |
| **G-E** | major | Voice assets are keyed by **name-derived slugs** (`officer_ref.wav`). Two projects can hold the same slug for different voices, so a slug is not a portable identity. | The node report now emits each ref WAV's full `sha256` + `object_key`. `voice_assets` must key on the bytes and carry the project scope; the slug stays a local label. |
| **G-F** | minor | `source_unit_key` is **project-agnostic by content**: identical chapter text in two projects yields identical keys. That answers the controller's open §8.3 question — and it is a tenancy consideration, not only a design one. | Deliberately left unsalted and documented. If per-tenant isolation is required, the controller salts at ingest (`sha256(tenant || key)`); doing it here would have made keys non-portable for no security benefit. |
---
## 12. Alignment status as of 2026-10-03
| Phase | Verdict | Note |
|---|---|---|
| 0 (reachability) | **pass** | `health` 200, `heartbeat` 401, missing-`Host` 400 — all three exactly as documented |
| 1 (vocabulary) | **pass** | 19 sets, 126 values, published to `definitions/umbraaudio-defs.json` with per-value provenance |
| 2 (cardinality) | **fail** | Evidence present; the three-way store disagreement is real and unresolved — a pass here would be a check that passed by doing nothing |
| 3 (identity) | **pass** | 200/200 units resolved by content key, **0** used `utterance_index` |
| 4 (lineage) | blocked | needs the controller's chosen artifact |
| 5 (timeline) | blocked | no time-scoped attribute model exists here yet |
| 6 (voice matrix) | blocked | voices are not captured as structured matrix values |
| 6b (persistence/partial capture) | blocked | **persistence is not captured at all**; absence is NULL, not typed |
| 6c (editions) | blocked | no edition concept |
| 7 (round trip) | blocked | needs the ingest API, which does not exist |
Regenerate any row with `scripts/alignment_report.py --phase all`.
---
## 13. Change log
| Date | Change |
|---|---|
| 2026-10-01 | Initial draft. Created by the UmbraAudio node. Added `infra/umbra-node.json`, `scripts/umbra_node_report.py`. Status: awaiting ratification. |
| 2026-10-01 | Initial draft. Created by the UmbraAudio node. Added `infra/umbra-node.json`, `scripts/umbra_node_report.py`. Status: awaiting ratification. |
| 2026-10-02 | Controller (oldbox) alignment proposal received; D1-D16 recorded, 2 questions open. This document is the node's response. |
| 2026-10-03 | Controller alignment proposal actioned. Implemented `source_unit_key` (§10 decision, §5.1), blob hashes + effect chain (§5.2), published vocabularies (§5.4), fingerprint discipline (§5.5), and the alignment report tool (§6.6). Recorded six gaps (§11) and phase-by-phase status (§12). Stage 0 verified. |
| 2026-10-03 | Controller alignment proposal received and actioned. Implemented `source_unit_key` (§10 decision, §5.1), blob hashes + effect chain (§5.2), published vocabularies (§5.4), fingerprint discipline (§5.5), and the alignment report tool (§6.6). Recorded six gaps (§11) and the phase-by-phase status (§12). |