umbragit/UmbraTrans
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Forward-only link agreed, and the connection sequence the audio node asked for

Browse Source 
D16: a book_local character revealed to recur keeps its released entity
and gains a recurs_as edge, rather than being merged. Its cost is written
down rather than discovered later - a retelling from the earlier book's
point of view treats the two as different people. Chosen because the
freeze is what protects reproducibility and a merge puts it at risk for a
gain that is real but retrospective.

Asked whether the document was ready to feed back and how the audio node
connects. Reading it against that question found four gaps, all of which
were mine:

- One report skeleton was given for nine phases, which specifies nothing.
  evidence now has a tabulated shape per phase, so the reports are
  mechanically comparable rather than approximately so.
- The identity probe said "we send you a fixed set" without saying how
  the set was fixed, which would have made phase 3 either
  unreproducible or impossible. Now: you pick the project, sort by
  source_unit_key, every 17th, cap 200, and report the stride.
- Section 9 said "contact the controller on 192.168.123.14:8442" and
  stopped, which is how this session's own 400s happened twice.
- There was no connection sequence at all.

Section 10 adds it, in four stages. Stages 0 and 1 are possible today
and carry no project data: Stage 0 is reachability, where a 401 from the
node surface is the correct answer and the evidence; Stage 1 is the node
report they already wrote, sent as a heartbeat. Stage 1 needs no schema
change and no new endpoint, because node_heartbeats.raw is an open jsonb.
Stages 2 and 3 are ours to build and are listed as such.

Two transport facts stated plainly, both learned the hard way here. The
deployment accepts exactly one Host header and returns 400 on everything
otherwise, including health. And there is no TLS, despite the service
description claiming "TLS terminated upstream by Caddy" - Caddy is
inactive and its configuration is empty. Tolerable for a read-only node
report on a LAN, not tolerable for the ingest path, so putting Caddy in
front of the port is a Stage 3 prerequisite rather than an extra.

It starts read-only for the same reason the compute substrate does: a
node is online because it heartbeated, not because it was created. A
mirror that begins with data cannot distinguish "the node is broken"
from "the data is bad", and their speaker counts already disagree.

One stale block survived a previous edit - a replace that matched
nothing and reported success. It now asserts before replacing.
This commit is contained in:
umbra
2026-10-02 19:11:14 -07:00
parent 717e1d56b9
commit 898906ab04
2 changed files with 193 additions and 20 deletions
Download Patch File Download Diff File Expand all files Collapse all files

193
docs/data-model-alignment-v1.md
View File

@@ -25,14 +25,25 @@ without being re-derived.
| D13 | **Not every character needs every aspect captured.** | Absence is legitimate, so absence must be *typed* rather than null. §1.6 |
| D14 | **Published editions change.** Corrections, a new cover, a new foreword — that is **a new release, a new edition.** | Editions are a first-class release unit. §1.7 |
| D15 | **The freeze unit is a single work, not a series.** Book 1 may be released while book 2 is still being written. | Independent release. Already implied by `projects.status`. |
| D16 | **A revealed-recurring character is linked forward, not merged.** A `book_local` character revealed to recur keeps its released entity and gains a `recurs_as` edge. | Protects the freeze absolutely (I-G). A retelling from the earlier book's viewpoint treats them as two people. §1.6.4 |
**Your next steps, in order:**
1. Read §1–§3. One vocabulary correction matters more than the rest.
2. Do the six changes in **§5**. They are all additive and none needs the controller.
3. Run the **§6 validation protocol** and produce the reports it specifies.
4. Contact the controller with the reports. Do **not** begin data migration yet — §7
says why, and the order is not negotiable.
1. Read §1. **Two things matter more than the rest**: a character who is an author *inside
the fiction* is not a byline (§1.2), and disposition is not circumstance (§1.5.1).
2. Do the seven changes in **§5**. All additive, none needs the controller.
3. **While you do them, start Stage 0 and Stage 1 of §10.** Both are read-only, neither needs
anything from §4, and there is no reason to wait for the datastore work.
4. Run the **§6 validation protocol**. The per-phase `evidence` shapes are tabulated in §6.6
— one row per phase — so the two sides are mechanically comparable rather than
approximately so.
5. Contact the controller with the reports. Do **not** begin data migration yet: §7 says why,
and the order is not negotiable.
**Readiness.** Complete enough to act on. Four gaps were found by reading it against the
question you just asked, and closed: the per-phase report fields, the selection rule for the
identity probe, the transport details, and the connection sequence itself (§10). If you find a
fifth, say so rather than working around it.
---
@@ -288,14 +299,15 @@ person**, and merging them would alter attributes that **I-G** has frozen.
This needs a decision and we are not making it for you:
- **Forward-only link.** The book-one entity stays exactly as released; a
`same_as`/`recurs_as` edge points forward. Book one does not change. A retelling from
book one's point of view would have to treat them as different.
- **Forward-only link — CHOSEN (D16).** The book-one entity stays exactly as released, and a
`recurs_as` edge points forward. Book one does not change, ever.
- **Merge with a snapshot.** One character, and book one's attributes are *copied* forward as
a frozen historical state, so nothing is lost and the merge is recorded.
a frozen historical state.
The first is simpler and preserves I-G absolutely. The second is truer about the fiction and
costs a snapshot mechanism. **§8 asks which.**
Chosen because the freeze is what protects reproducibility, and a merge puts it at risk for
a gain that is real but retrospective. The cost is deliberate and should be written down:
**a retelling from the earlier book's point of view will treat the two as different people.**
That is a consequence of the choice, not an oversight.
---
@@ -577,23 +589,66 @@ rather than asserting it.
### 6.6 Report format
Emit as JSON so the two sides are mechanically comparable. Skeleton:
One envelope, **and `evidence` has a different shape per phase.** The first version of this
section gave a single skeleton for every phase, which is the same as giving none: it does not
say what evidence phase 3 consists of, so you would invent one and the two sides would not be
comparable — which is the entire purpose of a machine-readable report.
**Envelope, every phase:**
```json
{
"report": "alignment",
"schema_version": 1,
"phase": 1,
"phase": 6b,
"from": "umbraaudio-m4",
"node_contract_version": 1,
"generated_at": "2026-10-02T00:00:00Z",
"verdict": "pass | fail | blocked",
"evidence": { "...": "required, never absent" },
"evidence": { },
"findings": [
{ "id": "F-01", "severity": "blocker|major|minor",
"description": "...", "affects": ["source_unit_key"] }
{ "id": "F-01", "severity": "blocker | major | minor",
"description": "...", "affects": ["source_unit_key", "I-G"] }
]
}
```
**`evidence`, per phase — this is the part that makes it comparable:**
| phase | `evidence` contains |
|---|---|
| 1 | `vocabularies: { "<set_name>": ["value", …] }` for every set in §5.4, plus `definition_set_version` |
| 2 | `per_project: [{ project, speaker_names, characters, speakers, utterances, chapters, store_counts: {…} }]` |
| 3 | `resolutions: [{ unit_key, resolved: bool, character, speaker, lang, used_index: bool }]` — `used_index: true` is the finding we want |
| 4 | `chain: [{ step, entity, identifier, resolved: bool }]` for one artifact, in order |
| 5 | `timeline: [{ work, sequence, state: "released", attribute_states: [{ attribute, value, effective_from, change_reason }] }]` |
| 6 | `matrix: { attribute_key: value }` for the primary voice, and `variant_matrix` for one variant, plus `declared_deltas: [{ attribute_key, from, to }]` |
| 6b | `characters: [{ key, persistence, attributes_present: […], attributes_absent: [{ attribute, absence_reason }] }]` |
| 6c | `editions: [{ work, edition_number, released_at, change_summary, artifacts: [{ content_hash, role }] }]` |
| 7 | `chapter, hashes_sent, hashes_recomputed, identical: bool, source_tree_hash_before, source_tree_hash_after` |
**Rules.** `verdict` is `pass` only when the phase's evidence is present **and** its checks
hold. Empty evidence is `blocked`, never `pass`. A field you cannot produce is `null` with a
finding — it is not omitted.
### 6.6.1 How the identity probe set is chosen (phase 3)
The previous version said "we send you a fixed set" without saying how it was fixed, which
would have made phase 3 either unreproducible or impossible.
**You choose the set, deterministically, and we send you those exact units.** The procedure,
so that both sides run the same thing:
1. Pick **one** project — your choice, named in the report.
2. Sort all `source_unit_key` values ascending (byte order, not locale).
3. Take every 17th, starting at index 0. That is a prime stride, so it samples across the
whole range and cannot systematically miss a region.
4. Cap at **200 units**. If the project has fewer, send all of them.
5. Report the total count and the stride you used, so we can verify we asked for the same set.
We then resolve each on our side and return what it resolves to. If a unit cannot be resolved
without `utterance_index`, that is the answer — say so rather than improvising an identifier.
### 6.7 What "aligned" means
Alignment requires: every phase `pass` **with evidence present**, and zero `blocker`
@@ -614,7 +669,7 @@ unconditionally** and returned to you.
---
## 8. Questions — six answered, four remain
## 8. Questions — 12 answered, 2 remain
### 8.1 Answered (recorded above, not repeated)
@@ -633,7 +688,7 @@ unconditionally** and returned to you.
| is the freeze unit a work or the series? | **a single work** (D15) |
| does every character need every aspect? | **no** (D13) — which is why absence is typed rather than null |
### 8.2 One question left, and one that appeared with it
### 8.2 2 questions, and one that appeared with the answers
1. **A `book_local` character is revealed to recur — merge, or forward-only link?**
§1.6.4. Merging is truer about the fiction but would alter attributes I-G has frozen;
@@ -661,11 +716,109 @@ unconditionally** and returned to you.
## 9. What we will send you
On acceptance of your reports, before any data moves:
Before any data moves:
- the ingest API contract, including batch size and idempotency rules
- the artifact-write scope added to a node credential, and exactly what it permits
- the object-storage decision and how a large artifact is fetched — **not through the API**
- the ratifications, including `DECISION-1` (D1) and the per-project migration states
**Contact:** the controller on `oldbox`, `192.168.123.14:8442`.
---
## 10. Connecting to the controller
You asked when you can connect, send data, and test. **You can start now**, and the stage
you can reach today carries no project data at all. Each stage proves the one before it, and
no stage sends anything the previous stage has not authorised.
### 10.1 Transport — three things that will bite you
**1. The `Host` header is mandatory.** The deployment accepts exactly one host:
```
ALLOWED_HOSTS=umbratrans.bdmurphy.me
```
Connecting to `192.168.123.14:8442` **without** that header returns **400 on every
request**, including `/api/health`. Send `Host: umbratrans.bdmurphy.me` on everything. This
is a `TrustedHostMiddleware` refusal and it looks like the deployment being down.
**2. There is no TLS, despite what the unit description says.** The service is described as
"loopback/LAN, TLS terminated upstream by Caddy", but **Caddy is not running and its
configuration is empty.** So the reachable endpoint today is plain HTTP on a LAN address:
```
http://192.168.123.14:8442 with Host: umbratrans.bdmurphy.me
```
For a read-only node report that is tolerable on a trusted LAN and we would rather you know
than assume. It is **not** tolerable for the ingest path — see Stage 3. Getting Caddy in
front of this port is on our list and is a prerequisite for Stage 3, not an optional extra.
**3. The node surface is bearer-token only and correctly refuses everything else.**
`POST /api/node/heartbeat` returns **401** without a credential. It is CSRF-exempt by design,
because a bearer-authenticated agent has no ambient cookie to ride on. Do not send a session
cookie from the audio web UI — it will not help and it widens the surface.
### 10.2 The four stages
| stage | what crosses the wire | what it needs | possible today? |
|---|---|---|---|
| **0 — reachability** | nothing. `GET /api/health`, and one `POST /api/node/heartbeat` **expecting 401** | network route to `192.168.123.14` | **yes** |
| **1 — report** | your existing `scripts/umbra_node_report.py --json`, as a heartbeat | a node credential we issue | **yes** |
| **2 — read mirror** | controller serves; **you pull**. You write nothing | Stage 1 clean, plus `B4`/`G2` closed and a read scope | not yet — ours to build |
| **3 — write back** | artifacts and lineage out of you | ingest API, object storage, TLS, a write scope | not yet — ours to build |
**Stage 0 is a test, not a handshake.** A `401` from the node surface is the *correct*
answer and is the evidence that authentication is working. If you get `200` without a token,
that is a finding and you should report it as a blocker rather than proceeding.
**Stage 1 needs no schema change and no new endpoint.** `node_heartbeats.raw` is an open
`jsonb`, so the node report rides inside the heartbeat that already exists. Your report is
already read-only, already secret-free, and already emits per-project schema fingerprints —
it was built for exactly this. Send it under a `raw.node_report` key alongside the usual
fields.
### 10.3 Why it starts read-only
The compute substrate proved itself the same way: **a node is online because it heartbeated,
not because it was created** (invariant I24). A node that exists in the inventory but has
never spoken takes no work. That is deliberate — it is the difference between a configured
machine and a working one.
The same discipline applies here, and it is the reason Stage 1 sends a *report* rather than
*data*. Your node's first traffic proves it is alive, reachable, running the code it claims,
and reporting the schema it actually has. A mirror that starts with data cannot distinguish
"the node is broken" from "the data is bad", and you already know one of those is likely —
your §15.2 counts disagree.
### 10.4 How to start Stage 0 now
From the audio node, nothing more than this:
```sh
curl -sS -o /dev/null -w '%{http_code}\n' \
-H 'Host: umbratrans.bdmurphy.me' http://192.168.123.14:8442/api/health
# expect 200
curl -sS -o /dev/null -w '%{http_code}\n' -X POST \
-H 'Host: umbratrans.bdmurphy.me' -H 'Content-Type: application/json' -d '{}' \
http://192.168.123.14:8442/api/node/heartbeat
# expect 401 - that is the correct answer
```
Report both codes in your first message. **If either differs from what is written above,
say so and stop** — a 200 on the second, or a 400 on the first, means something on this side
is not what this document describes, and everything below it is built on that assumption.
### 10.5 What we need to give you for Stage 1
Ask, and we will issue: a `compute_nodes` row for `umbraaudio-m4` and a node credential from
`POST /api/compute/nodes/{id}/credentials` with scopes `compute:read` and
`compute:heartbeat`. That is deliberately **not** a project-reading scope and **not** a write
scope; it is the same pair the audio PoC already uses, and it is the smallest thing that lets
you prove you are alive.
---
**Contact:** the controller on `oldbox`, `192.168.123.14:8442`. **Start at Stage 0.**

20
umbratrans_web/migrations/README.md
View File

@@ -457,6 +457,26 @@ with nothing having touched the service. A genuine crash loop now logs every 5
seconds instead of stopping, which is noisier and vastly better than a deployment
that is quietly down. There is `systemctl --user stop` for when you mean it.
## The data-model alignment document
`docs/data-model-alignment-v1.md` (814 lines) is the contract proposed to the audio node on
Mini-M4. Sixteen decisions recorded, nine validator phases, eight invariants, two questions
open. It is the counterpart to this README rather than part of it: it describes the content
model, this describes the security and operations posture of the machine holding it.
Two things from it belong here because they are security posture, not vocabulary:
- **A node may not write to the datastore.** Node credentials are `compute:read`,
`compute:claim`, `compute:complete`, `compute:heartbeat` — deliberately no data access. The
audio node becomes the writer when the mirror opens, and that scope has to be granted as a
decision rather than arrived at by widening an existing one.
- **The connection sequence (§10) starts read-only**, and Stage 0 expects a **401** from the
node surface. A 200 without a credential would be a finding to report, not a stage to pass.
Also worth knowing: the deployment accepts exactly one `Host` header
(`ALLOWED_HOSTS=umbratrans.bdmurphy.me`) and returns **400 on every request** otherwise,
including `/api/health`. This has the appearance of the service being down.
## Gates that must pass before approval
- Logical backup taken, checksummed, and restore-tested on a scratch database.