Files
UmbraAudio/specs/21-umbra-system-nodes-tasks-and-changes.md
B.D. Murphy 507ed95424 Declare this machine an Umbra node + cross-machine system contract (spec 21)
A parallel effort is moving UmbraTrans into the main Umbra system on oldbox
(192.168.123.14). This commit makes this machine describable to that system and
records what the system still needs to decide. Everything is a proposal for the
controller to ratify; no architecture is decided here.

specs/21-umbra-system-nodes-tasks-and-changes.md
  Four workstreams, each with what exists, what is missing, and a proposal:
  - WS1 data model alignment / copy / verify / transition. Design is far along
    already (manifest.md + spec 20); the missing piece is ratification.
  - WS2 every machine is a node that gets assigned any task. Job types, leases
    and pools exist (spec 19 §12); the node half does not. Proposes node
    identity, capability advertisement, task/result envelopes, and refusal as a
    valid audited outcome.
  - WS3 cross-product change management. Nothing exists today. Proposes a
    requirement catalog owned by the controller, four change classes, and a
    distribute-then-confirm-with-evidence loop. Includes the worked example of a
    controller-owned entity (designation, pronunciation, voice attribute) that
    must apply across every product.
  - WS4 UI topology. Recommends hybrid: controller gets the system portal,
    the audio review UI stays on the worker because it is artifact-native and
    has no authentication at all. Adds a git-tag deploy + verify + confirm flow.

  Also: 25 questions grouped by who can answer, each naming what it blocks, and
  a task list for this machine (NOW / NEXT / LATER) where every task has a
  machine-checkable acceptance criterion.

infra/umbra-node.json
  This machine's declared profile: node_id umbraaudio-m4, role worker, services,
  capabilities, resource pools, constraints, project list with lifecycle, schema
  fingerprints, and an honest facts_for_the_controller block (empty tables, no
  stable utterance id, no auth, no shared-postgres client). No secrets - ids,
  names, labels and counts only.

scripts/umbra_node_report.py
  Read-only node report. Git state, listening ports, .env KEY NAMES only, per
  project row counts, and a schema fingerprint hashed over sorted
  table.column:type so ALTER TABLE reordering does not read as drift. --json for
  machine consumption; --check compares declared profile to measured reality and
  exits 2 on drift. Drift is deliberately limited to stable facts (contract
  version, project set, schema fingerprints) - git HEAD and live ports are
  reported but never drift.

manifest.md
  Now explicitly a dated measurement with an update log, because other machines
  read it as current. 2026-10-01 refresh: Last Dwarf speakers.json 70 -> 93
  after the designation merge, DealHunter ref WAVs 94 -> 97. Every other §1.1
  count re-verified unchanged.

Measured this machine: 4 projects, 2 distinct schema fingerprints (real
per-project drift), only 4 of 11 tables populated, 461 tests passing / 8 known
pre-existing failures. No UmbraAudio behaviour changed.
2026-10-01 07:14:54 -07:00

33 KiB
Raw Blame History

Spec 21 — The Umbra System: Worker Nodes, Task Assignment, Cross-Product Change Management, and UI Topology

  • Status: PROPOSAL. Nothing here is ratified. Every table marked proposed is a draft contract for the controller to accept, amend, or reject.
  • Date: 2026-10-01
  • Audience: every machine running an Umbra product, plus the control plane.
  • Owner of ratification: the main Umbra system (controller, currently oldbox, 192.168.123.14). This document was written by a worker and proposes; it does not decide.
  • Companion documents (do not duplicate them, link to them):
    • manifest.md — authoritative inventory of what UmbraAudio actually holds (898 lines).
    • specs/20-enterprise-database-integration.md — shared-database contract, blockers, decisions.
    • specs/19-enterprise-control-plane.md — tenancy, roles, jobs, lineage.
    • specs/18-UmbraTrans.md — translation/multilingual production.
    • infra/README-enterprise-bootstrap.md — the control-plane database bootstrap that already exists on oldbox.
    • AGENTS.md — this machine's operational rules and hazards.

0. How to use this document

This machine (a worker) has written it so that the other machines and the controller do not have to re-derive the same facts. Three concrete things every other machine should do:

  1. Read §5 and §6 of manifest.md — that is where the real data inventory is. Do not re-invent it.
  2. Produce a node report with scripts/umbra_node_report.py (this machine's implementation, §8.2). It is read-only, prints no secrets, and emits a schema fingerprint per project. Copy the script; it has no product-specific imports beyond umbraaudio.config.
  3. Declare your node in your own infra/umbra-node.json in the shape of §4.4, then run scripts/umbra_node_report.py --check and get exit 0 before claiming you are aligned.

1. Executive summary

# Workstream Where it is What is missing
WS1 Align the data model, copy UmbraAudio data in, verify, later transition Design is unusually far along: manifest.md + spec 20 Ratification. Nine open decisions, five hard blockers, zero implementation.
WS2 Every machine is a node that gets assigned tasks (not only audio) Job types, leases, heartbeats and pools are specified (spec 19 §12) Node contract: no node identity, no capability advertisement, no task envelope, no reporting rule. Proposed in §4.
WS3 How changes are managed across products Nothing exists. Specs 19/20 cover data changes, not code/requirement changes Requirement catalog + distribution + confirmation loop. Proposed in §5.
WS4 Does a central server take over the UI? How do updates get deployed and tested? DECISION-8 is open and unanswered A decision and a deployment/update flow. Recommendation + options in §6.

The single most important cross-cutting fact: this machine already contains a complete, measured data inventory and a verified shared-database contract. The system does not need another discovery phase. It needs someone to ratify decisions, then implement. Every week spent re-discovering is a week not spent building.


2. System shape (proposed, matches spec 19 §4)

                        ┌──────────────────────────────────────┐
                        │  CONTROLLER  (oldbox 192.168.123.14) │
                        │  • authoritative data (PostgreSQL)   │
                        │  • identity, tenancy, authorization   │
                        │  • requirements catalog  (§5)         │
                        │  • definition sets        (§5.7)      │
                        │  • task queue + leases     (§4)       │
                        │  • lineage, artifacts, releases       │
                        │  • portal UI (status only,  §6)      │
                        └───────┬──────────────┬───────────────┘
                                │ pull tasks  │ report results
              ┌─────────────────┴───┐      ┌───┴──────────────┐
              ▼                     ▼      ▼                  ▼
      ┌───────────────┐     ┌──────────────┐  ┌────────────────────┐
      │ node: audio   │     │ node: trans  │  │ node: <future>     │
      │ mini.bdmurphy │     │ <other host> │  │                    │
      │               │     │              │  │                    │
      │ UmbraAudio    │     │ UmbraTrans   │  │ other products     │
      │ web UI :8420  │     │              │  │                    │
      │ TTS  :8881    │     │              │  │                    │
      └───────────────┘     └──────────────┘  └────────────────────┘

Authority rule (already proposed in spec 20 §A.3–A.4, needs ratification):

  • The controller is authoritative for identity, tenancy, requirements, definitions, task state, lineage and release decisions.
  • A node is authoritative for its own working area (SQLite + filesystem) while it is in legacy or mirroring state, and it must keep working when the controller is unreachable.
  • Nothing is ever a cutover for the whole system at once. Migration state is per project (legacy → mirroring → dual_write → authoritative → retired; manifest §14.1).

3. WS1 — Align the data model, copy the data, verify, then transition

3.1 What already exists (do not redo)

  • manifest.md — every data object, exact DDL, JSON shapes, artifact naming, the legacy→shared entity mapping (§13), 14 open questions (§15), and the aligned-definition-set list (§14.2).
  • specs/20 — table-by-table contract, cutover phases P1–P4, RLS review, acceptance tests.
  • infra/postgres/001_control_plane.sql — 13 real tables, already bootstrapped on oldbox.
  • Measured state of this machine, reproducible today: scripts/umbra_node_report.py.

3.2 The five headline facts any aligning machine must know

  1. The SQLite is mostly empty. Only project, chapters, utterances, name_tokens hold data. characters, generated_audio, pronunciation_guide, quality_reports, review_decisions, exception_reviews, voice_review_history are empty in every project. A SQL-only migration loses the speaker model, the voice model, the pronunciation model and the entire audio record.
  2. There is no durable utterance identity. Files are utt_{utterance_index:04d}.wav and utterance_index is renumbered by re-segmentation and by utterance deletion. Spec 18 §3.3 forbids mutable indices as cross-language IDs. Spec 20 §E.1 proposes a content-addressed source_unit_key recorded at generation time with filenames unchanged — this machine can build that today with no dependency on the controller.
  3. Speaker identity is free text. utterances.speaker_name is the only link to a speaker; it is not a foreign key, not normalised, and has case variants. Five parallel stores, four key schemes, three-way count disagreements (Last Dwarf 93/68/71 and DealHunter 61/64/97 as of 2026-10-01).
  4. The control-plane worker role is a blocker. umbracontrol_worker matches zero RLS rows and cannot attribute an audit event (spec 20 B1). Needs service_identities + job_leases, not a GUC workaround. B4 (functions EXECUTE-able by PUBLIC with no ALTER DEFAULT PRIVILEGES) must be fixed before any further migration.
  5. Per-project schema drift is real and measurable. This machine currently has two schema fingerprints: 0078569a0fa6bc70 (Last Dwarf, DealHunter) and 367d5df9e546dbba (Treasure Island, Vocals — missing name_tokens and nine utterance columns). A migration must introspect per database, never assume a uniform shape.

3.3 Sequence (proposed)

Phase Name Gate to pass Who
P0 Ratify the nine decisions in §6.1 Decisions recorded in the controller Controller
P1 Adapter/repository boundary inside UmbraAudio; no behaviour change Existing tests unchanged; "where state lives" becomes configuration Worker
P2 Read-only mirror of the audio data into the controller §3.4 verification passes on all 4 projects Worker + controller
P3 Dual-write; controller authoritative, worker still functional offline No divergence beyond an agreed window; leases heartbeat Both
P4 Per-project authoritative; local SQLite becomes read-only cache Lineage reproducible; released lineage immutable Controller

3.4 Verification gates for "it all works" (proposed)

  1. Every project imports with zero dropped rows; ambiguous/unresolved values are recorded, not discarded (legacy_id_map rows with a reason).
  2. Row counts match the node report for every tracked table.
  3. Schema fingerprints are recorded per source database, and drift is reported, not normalised silently.
  4. Every utterance resolves to a speaker_id or an explicit needs_source_review row.
  5. Every voice has a SHA-256 and an object_key; no binary in PostgreSQL.
  6. A sample chapter can be re-imported and produces an identical hash set (idempotency).
  7. Proven: importing does not create or modify a single file in the source project tree (before/after tree hash).
  8. The audio worker completes a real chapter with the controller deliberately stopped.

4. WS2 — Every machine is a node that receives tasks

4.1 Why this needs its own contract

Spec 19 §12 already defines job types, leases, heartbeats, pools and idempotency keys. What is missing is the node half: how a machine announces what it is, what it can do, what it refuses, and how it reports a finished task. Without that, "assign a task to a machine" cannot be expressed, let alone executed.

4.2 Node identity (proposed)

  • node_id: stable, human-readable, lowercase-hyphenated, never reused. This machine: umbraaudio-m4.
  • role: controller | worker.
  • products: which products the node runs (["umbraaudio"]).
  • node_contract_version: the version of this contract the node implements. Bump on breaking change. A node that reports an unknown version is not eligible for tasks.
  • Identity is declared in the repository, not configured by hand in a UI (§4.4).

4.3 Capability advertisement (proposed)

A node advertises capabilities (what it can do), resource_pools (what it has, with concurrency), and constraints (what it must not do). The scheduler matches a task's required capability list against this advertisement — it does not hard-code hostnames.

Constraint declaration matters as much as capability declaration. This machine, for example, must refuse any task requiring an LLM and local TTS simultaneously, because 16 GB of unified memory cannot hold both and doing so silently degrades output to Edge-TTS.

4.4 Node profile file (proposed, implemented here)

Each node keeps infra/umbra-node.json — identity, services, capabilities, pools, constraints, project list, schema fingerprints, and the honest gaps. This machine's file is infra/umbra-node.json; it contains no secrets (ids, names, labels, counts only).

4.5 Task envelope (proposed)

task:
  task_id: tsk_01J...            # opaque, controller-issued, globally unique
  requirement_id: REQ-...        # §5 — every task traces to a requirement
  product: umbraaudio
  capability: analysis.speaker_attribution
  target_node: umbraaudio-m4     # null = scheduler chooses by capability
  scope:                          # what the task operates on
    project: "Last Dwarf"
    locale: null
    revision: <source_revision_id or null>
  instruction: "Re-run attribution for ch11-49 after the designation merge."
  inputs:                          # immutable references, never inline blobs
    definition_set_version: defs-2026-10-01
    source_snapshot_id: snap_...
  acceptance_criteria:            # machine-checkable where possible
    - "pytest tests/ -q exits 0"
    - "node report --check exits 0"
    - "utterance.chapter_id in (11..49) all have attribution_confidence NOT NULL"
  idempotency_key: sha256(...)    # re-delivery must not duplicate work
  lease_ttl_s: 900
  deadline: 2026-10-08T00:00:00Z
  report_back:                    # required in the result, not optional
    - changed_row_count
    - before_after_fingerprint
    - test_evidence
    - definition_set_version_used
    - notes

4.6 Result envelope (proposed)

result:
  task_id: tsk_01J...
  status: succeeded | failed | refused | partial
  refusal_reason: "..."           # required when refused
  evidence: { ... }               # acceptance_criteria satisfied, with proof
  node: { node_id, node_contract_version, git_head, dirty }
  facts: { ... }                  # the read-only node report, or the relevant subset

A node may refuse a task and must say why; refusing is a valid, audited outcome. Silently producing wrong output is not.

4.7 What this machine can accept today

audio.parse_manuscript, audio.segment_utterances, analysis.{character_discovery,speaker_attribution,speaker_clarity,name_token_scan}, voice.{reference_generation,attribute_measurement}, audio.{render_utterances,stitch_chapters,normalize_loudness}, export.{m4b,mp3_package}, review.speaker_QA.

It cannot accept: translation, TTS on CUDA, anything needing an authenticated UI, anything requiring concurrent LLM + local TTS.


5. WS3 — How changes are managed across products

This is the least-specified part of the system today and the one the operator cares most about. Proposal below.

5.1 The model

The controller owns requirements and definitions. Products own implementation. A change is never pushed into a product. It is distributed as a requirement, implemented locally, and then confirmed with evidence. The requirement cannot close until every affected product confirms or explicitly waives.

This preserves two real facts: (a) some products make changes locally and commit in git; (b) some changes must apply everywhere. Neither needs the other to be overruled.

5.2 Requirement record (proposed)

requirement:
  requirement_id: REQ-0042
  title: "Unresolved speaker designations must exist as speakers"
  rationale: "A designation with no profile resolves to a random named character."
  change_class: cross_product      # local_product | cross_product | shared_definition | operational
  owned_by: controller
  definition_set_version: defs-2026-10-01   # null for local_product
  scope:
    products: [umbraaudio, umbratrans]
    nodes: [umbraaudio-m4]                    # empty = all nodes running those products
  acceptance_criteria: ["each product reports its own conformance measurement"]
  state: proposed | triaged | distributed | in_progress | confirmed | waived | released
  confirmations:                             # one per product/node
    - product: umbraaudio
      node: umbraaudio-m4
      state: confirmed
      evidence: {name_tokens_resolved: 10, unresolved_designations: 0, report_at: "..."}
      commit: "4775e3b"

5.3 The four change classes (proposed)

Class What changes Distribution Confirmation evidence
local_product One product's internals (UI, performance, refactor) Requirement records intent and outcome; no code is sent Product reports resulting version/commit
cross_product Same behaviour needed in N products Requirement per product, each with its own criteria Each product's own measurement
shared_definition Vocabulary/schema/attribute definitions Versioned data, not code. Products pull the set and record which version they used definition_set_version_used + applied row counts
operational Env, config, deploy, credentials-ish Per-node task; the node reports state Node report facts (e.g. ports, fingerprints)

5.4 The worked example the operator asked about

"A user added or changed something that must apply across the system, owned and managed by the main controller."

The real cases in this system are a speaker designation, a name pronunciation, a voice attribute, or a role kind. Proposed handling:

  1. The controller holds the entity as master data with an identity, evidence, and a revision.
  2. It emits a shared_definition requirement: "add term X with spelling Y, provenance explicit, effective for products P1 and P2, definition set defs-2026-10-02".
  3. Each product applies it in its own representation — UmbraAudio writes phonetic_names.json + speakers.json; UmbraTrans writes its own pronunciation table. The controller never writes another product's files.
  4. Each product confirms with a measurement: "term present, spelling matches, N utterances affected" — not "done".
  5. Until every product confirms, the requirement stays in_progress and is visible as such.
  6. A local product-side discovery of the same fact (e.g. the designation scanner found 23 missing speakers here) is reported upward as evidence, and the controller decides whether it becomes master data. This is how the worker's discovery becomes system knowledge.

5.5 Code updates: distribution and testing (proposed)

Code ships through git, per product repository. The controller's job is not to carry code; it is to carry the compatibility matrix and the requirement.

controller: REQ "move umbraaudio to tag v0.9 + definition set defs-2026-10-02"
   ↓ distributed (one task per node)
node: git fetch && git checkout v0.9
      run the product's own verification:
        - pytest tests/ -q                      (this machine: 461 passing)
        - scripts/umbra_node_report.py --check  (profile/contract still valid)
        - a product smoke test (start web, hit one endpoint)
      record: commit SHA, test counts, report fingerprint, definition_set_version_used
   ↓ confirm
controller: closes REQ only when every target node confirms or explicitly waives

Rules:

  • A node that cannot reach the controller keeps working and queues its confirmation.
  • A node never auto-updates itself without a task. Audio production must not stop because someone merged.
  • Definition sets are pulled by version and recorded; a node using an old set is visible, not silently wrong.
  • Every product repo keeps its own AGENTS.md hazards. The system-level hazard list (§8.6) is additive, never a replacement.

5.6 What must NOT be centrally managed

Product-internal implementation, local caches, generated audio, model weights, TTS prompt caches, per-node environment tuning, and anything machine-specific (this machine's external volume, TCC rules, its own launchd layout).


6. WS4 — Where does the UI live?

6.1 The decision (proposed, answers the operator's question)

Neither extreme. Hybrid, with the controller authoritative for data and the worker keeping the program UI.

The controller does take on the UI function, but only the system UI: a portal showing node health, task state, requirement conformance, definition-set versions, project/work registry and release status. It does not host the audio review UI.

Why:

  • The audio review UI is bound to local artifacts — WAV players, per-utterance regeneration, staleness markers, local voice references. A central UI would have to stream every artifact across the network to reproduce it.
  • That UI has no authentication at all (spec 20 §J.6). Centralising it would publish an unauthenticated mutation surface to the internet. It must stay on the LAN regardless.
  • Its real risk is staleness and fragility, not location. A node can be perfectly robust in place and useless if it cannot report what version it is and whether its data still agrees with the controller.

6.2 Options considered

Option Pros Cons
Central UI only One place, one auth, one version Streams all artifacts; couples UI downtime to production; biggest rewrite; publishes attack surface today
Worker UI only (today) Fast, local, artifact-native No cross-project view; a dead node is invisible; updates drift silently
Hybrid (recommended) Portal on the controller; program UI stays on the worker; signed deep links; version + conformance shown on both Two UIs to keep coherent

6.3 How the two UIs stay coherent (proposed)

  • The worker UI displays, in its footer, node_id, definition_set_version, and the last successful confirmation timestamp. A reviewer can always see which system's view they are in.
  • The portal deep-links into the worker UI with a short-lived signed link. No credentials travel in a URL.
  • Any authoritative decision (approve, merge speaker, release, publish) is written in the controller and read by the worker UI; the worker UI never becomes the source of truth for one of those.

6.4 How worker UI robustness is assured (proposed)

  1. Node report as the health probe. scripts/umbra_node_report.py --check is the definition of "this node is healthy and matches its declared profile".
  2. Existing process discipline. This machine already runs the UI under a launchd KeepAlive agent with a health-gated startup. That pattern is the baseline for every node.
  3. Read-only by default. Sync and pull operations never mutate project data without an explicit task.
  4. Degradation is normal. If the controller is down, the worker keeps producing audio from its local lease and reconciles later. Production availability outranks synchronization.
  5. No silent exposure. While the UI is unauthenticated, it must remain LAN-only. The reverse proxy must not publish mutation endpoints. This is a precondition, not a nice-to-have.

6.5 Decisions this workstream needs (proposed answer)

  • Ratify the hybrid (portal on controller, program UI on worker).
  • Decide whether the audio UI is retired later or kept permanently. Recommendation: keep. It is the program UI; it is what a director uses while a book is in production.
  • Make the open security defects (§8.5) a precondition for any change in exposure.

7. Questions that need answers

Grouped by who can answer. Each says what it blocks.

7.1 System topology and authority

  1. Is the controller's database authoritative for the audio system? (spec 20 DECISION-1; this document §2 assumes yes). Blocks: everything in WS1.
  2. Does UmbraTrans read the authoritative database instead of snapshotting? (DECISION-2). If yes, spec 18 §4.2.1 is superseded and a hard class of bugs disappears. Blocks: the adapter.
  3. One node per product, or one node running several products? This machine currently runs only UmbraAudio but already hosts the GPU box's relay. Affects leases and blast radius.
  4. What is the official home of the requirement catalog? The control-plane PostgreSQL is the proposal; it must be decided before any requirement tooling is built.

7.2 Data model

  1. Who resolves an unresolvable speaker name — create a placeholder, or record it unresolved? (manifest §15.1). Blocks: the translation identity gate, which is fail-closed on exactly this.
  2. Which of the three speaker stores wins when they disagree (Last Dwarf 93/68/71, DealHunter 61/64/97 — manifest §15.2)?
  3. Is utterance_index preserved as a display ordinal or retired in favour of a content-addressed key (manifest §15.4)? Recommendation: keep it as a display ordinal, add source_unit_key.
  4. Is the manuscript stored once or duplicated in chapters.source_text and utterances.text (manifest §15.5)? Affects copyright handling.
  5. Which voice-attribute profile is canonical — the 9 measured features, the prose taxonomy, or the 13-dimension acoustic set (manifest §15.14)? Recommendation: store measurements and definitions separately.
  6. Does one voice need a separate vmatrix per target language? (manifest §15.11). Recommendation: per (voice, locale), with a marked locale-neutral option.

7.3 Tasks and nodes

  1. What is the node identity scheme, and who issues it? (§4.2).
  2. Does the scheduler match on declared capability only, or may a task name a node? Recommendation: capability-first, node as a hint.
  3. What happens when a node refuses a task? Recommendation: valid, audited, visible, and the requirement stays open.
  4. Are tasks ever allowed to require LLM + TTS co-residency on a 16 GB machine? Recommendation: forbid it as a capability constraint, not a per-node convention.

7.4 Change management

  1. Is the requirement catalog part of the control-plane database, or a separate system?
  2. What is the confirmation standard of proof — a test result, a measurement, a human approval, or all three? (§5.2).
  3. Who arbitrates when two products need incompatible versions of a shared definition?
  4. How long may a requirement stay in_progress before it is escalated? There is no clock in this proposal.

7.5 UI and deployment

  1. Ratify the hybrid UI topology (§6.1).
  2. Does every product deploy by git tag pull, or is there a package registry?
  3. Who may push a definition-set version, and does it require review?
  4. Which host runs object storage, and is it the same host as PostgreSQL? (spec 20 DECISION-6). Until it exists, artifacts stay on local filesystems and the database holds only resolvable keys.

7.6 Security preconditions

  1. Raterify the B1 remedy (service_identities, not a GUC workaround).
  2. Raterify B4 — revoke EXECUTE on app functions and set default privileges before any further migration.
  3. Who signs off on exposing any worker UI beyond the LAN, given zero authentication today?

8. What UmbraAudio already provides for the whole system

8.1 The high-value assets (not just data)

Asset Where Reuse value across machines
Complete measured data inventory manifest.md Removes the discovery phase from every other machine
Verified shared-DB contract + blockers specs/20 Saves an entire architecture exercise
Control-plane schema, live infra/postgres/001_control_plane.sql, bootstrapped on oldbox The system already has a database; it lacks domain tables
Data-manifest generator manifest.md + scripts/umbra_node_report.py Any machine can produce the same fingerprint report
Node profile format infra/umbra-node.json + scripts/umbra_node_report.py Copyable node declaration
Legacy→shared entity mapping manifest.md §13 The actual migration mapping table, already written
Aligned-definition set list manifest.md §14.2 The exact list of definitions to extract from code into data
Verification culture 447→461 passing tests, ruff, mypy, per-session acceptance criteria A node can confirm with evidence rather than assertion
Operational hazard knowledge AGENTS.md Saves every machine from rediscovering TCC/PATH/co-residency traps

8.2 Reproducible facts about this machine (today)

.venv/bin/python scripts/umbra_node_report.py           # human-readable
.venv/bin/python scripts/umbra_node_report.py --json    # machine-readable
.venv/bin/python scripts/umbra_node_report.py --check   # confirm profile; exit 2 on drift
  • 4 projects, 2 distinct schema fingerprints (real drift).
  • Only 4 of 11 tables carry data.
  • Speaker identity lives in JSON + WAVs; audio record lives on the filesystem + markers.
  • No authentication, no shared-Postgres client, no lease/heartbeat.
  • 461 tests passing; 8 known pre-existing/parallel failures.

8.3 Requirements of this data that the shared model must satisfy

From manifest.md §10, the blocking ones are: stable source-unit identity, stable speaker identity, an immutable source revision, and artifact lineage with SHA-256. The rest are available today and can migrate as-is.

8.4 What the worker must never do

Write into another product's data area; renumber or rewrite utterances; reuse English audio markers as target-language markers; call another product's private functions; require a migration before the controller is ready.

8.5 Defects that constrain any exposure or migration (from manifest.md §12)

No authentication anywhere (65 mutating endpoints, plain HTTP, 0.0.0.0 bind); project selection is an unsigned cookie; download traversal via string-prefix containment; upload filename not sanitised; no job queue/lease/heartbeat; four divergent DDL authorities; non-atomic JSON writes; no artifact checksums; foreign keys never enabled.

8.6 Hazards on this machine specifically

External-volume TCC (only the venv python may touch the volume under launchd); launchd's bare PATH; macOS Local Network TCC blocking launchd processes from LAN IPs; Qwen and LM Studio never co-resident; PROJECT_BASE hard-coded; concurrent editors in this repository.


9. This machine's plan — task list

Legend: NOW = no dependency on the controller, safe today. NEXT = needs a decision or another machine. LATER = needs the shared database to exist. Each task has an acceptance criterion so it can be confirmed rather than reported done.

9.1 NOW

ID Task Why Acceptance Est
M4-01 Node profile + read-only node report + --check Make this machine describable to the system infra/umbra-node.json; --check exits 0 done 2026-10-01
M4-02 Commit this session's artifacts and document them AGENTS.md cadence rule; nothing left uncommitted Commit exists; session recorded in AGENTS.md 15 min
M4-03 Reconcile the dirty working tree 20 files uncommitted from parallel sessions; a lost day of work is a real loss here Working tree clean, or every uncommitted hunk accounted for 1 h
M4-04 Add source_unit_key (content-addressed, additive) The one blocking identity concept that needs no controller. Without it, the mirror imports unstable IDs Column exists, populated for new generations, filenames unchanged, is_current unaffected, tests pass 3-4 h
M4-05 Extract the aligned definition set into data manifest.md §14.2 — vocabularies currently live in code, so every product reimplements them A versioned definitions/ file with utterance kinds, speaker kinds, capture modes, phonetic provenance; code reads it 3-5 h
M4-06 Fix the cheap security defects Precondition for any exposure change, and for an honest node posture Traversal + filename fixes with tests 2 h
M4-07 Resolve the speaker-count disagreement 70/93/71 — the single most confusing fact about this machine's data One authoritative count per project, with the others derived or explicitly deprecated 2-3 h
M4-08 Add a read-only /api/node health endpoint Lets the controller scrape node health without an agent; LAN-only Endpoint returns the node report; no secrets; LAN-only documented 1-2 h

9.2 NEXT — needs a decision or another machine

ID Task Blocked by
M4-09 Resolve remaining orphan name tokens (10 in Last Dwarf) manifest.md §15.1 policy; §7.2 Q5
M4-10 Merge designations that are the same person ("The Big One" / "The Biggest One") Human review
M4-11 Re-run attribution after the designation merge M4-10
M4-12 Make PROJECT_BASE configurable Touches config.py, which a parallel session is editing — coordinate first
M4-13 Decide the object-storage host DECISION-6 (§7.5 Q22)
M4-14 Backfill source_unit_key for existing utterances M4-04 + a definition of the key inputs

9.3 LATER — needs the shared database

ID Task Blocked by
M4-15 Implement the read-only mirror (P2) DECISION-1 ratified; B1/B4 fixed on the controller
M4-16 Prove the import creates no file in the source tree M4-15
M4-17 Lease + heartbeat client so a task can be claimed service_identities + job_leases exist
M4-18 Definition-set pull and version reporting Controller serves the set
M4-19 Per-project authoritative cutover P2/P3 verification green for that project
M4-20 Per-locale voice assets (vmatrix per (voice, locale)) manifest.md §15.11 decision

9.4 Sequencing note

M4-04 and M4-05 are the highest-leverage items on this list. Both are additive, both are independent of the controller, and both make every later phase cheaper. If the system does nothing else, do those two.


10. Change log

Date Change
2026-10-01 Initial draft. Created by the UmbraAudio node. Added infra/umbra-node.json, scripts/umbra_node_report.py. Status: awaiting ratification.