umbragit/UmbraAudio
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Declare this machine an Umbra node + cross-machine system contract (spec 21)

Browse Source 
A parallel effort is moving UmbraTrans into the main Umbra system on oldbox
(192.168.123.14). This commit makes this machine describable to that system and
records what the system still needs to decide. Everything is a proposal for the
controller to ratify; no architecture is decided here.

specs/21-umbra-system-nodes-tasks-and-changes.md
  Four workstreams, each with what exists, what is missing, and a proposal:
  - WS1 data model alignment / copy / verify / transition. Design is far along
    already (manifest.md + spec 20); the missing piece is ratification.
  - WS2 every machine is a node that gets assigned any task. Job types, leases
    and pools exist (spec 19 §12); the node half does not. Proposes node
    identity, capability advertisement, task/result envelopes, and refusal as a
    valid audited outcome.
  - WS3 cross-product change management. Nothing exists today. Proposes a
    requirement catalog owned by the controller, four change classes, and a
    distribute-then-confirm-with-evidence loop. Includes the worked example of a
    controller-owned entity (designation, pronunciation, voice attribute) that
    must apply across every product.
  - WS4 UI topology. Recommends hybrid: controller gets the system portal,
    the audio review UI stays on the worker because it is artifact-native and
    has no authentication at all. Adds a git-tag deploy + verify + confirm flow.

  Also: 25 questions grouped by who can answer, each naming what it blocks, and
  a task list for this machine (NOW / NEXT / LATER) where every task has a
  machine-checkable acceptance criterion.

infra/umbra-node.json
  This machine's declared profile: node_id umbraaudio-m4, role worker, services,
  capabilities, resource pools, constraints, project list with lifecycle, schema
  fingerprints, and an honest facts_for_the_controller block (empty tables, no
  stable utterance id, no auth, no shared-postgres client). No secrets - ids,
  names, labels and counts only.

scripts/umbra_node_report.py
  Read-only node report. Git state, listening ports, .env KEY NAMES only, per
  project row counts, and a schema fingerprint hashed over sorted
  table.column:type so ALTER TABLE reordering does not read as drift. --json for
  machine consumption; --check compares declared profile to measured reality and
  exits 2 on drift. Drift is deliberately limited to stable facts (contract
  version, project set, schema fingerprints) - git HEAD and live ports are
  reported but never drift.

manifest.md
  Now explicitly a dated measurement with an update log, because other machines
  read it as current. 2026-10-01 refresh: Last Dwarf speakers.json 70 -> 93
  after the designation merge, DealHunter ref WAVs 94 -> 97. Every other §1.1
  count re-verified unchanged.

Measured this machine: 4 projects, 2 distinct schema fingerprints (real
per-project drift), only 4 of 11 tables populated, 461 tests passing / 8 known
pre-existing failures. No UmbraAudio behaviour changed.
This commit is contained in:
B.D. Murphy
2026-10-01 07:14:54 -07:00
parent 34c221ea85
commit 507ed95424
4 changed files with 1170 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

164
infra/umbra-node.json Normal file
View File

@@ -0,0 +1,164 @@
{
"_comment": [
"Umbra node profile - WHAT THIS MACHINE IS, declared by this machine.",
"Produced under spec 21 (specs/21-umbra-system-nodes-tasks-and-changes.md).",
"Read-only fact check: .venv/bin/python scripts/umbra_node_report.py --check",
"This file contains NO secrets. Only ids, names, labels and counts.",
"'status: draft' means the controller has not yet ratified this node."
],
"node_contract_version": 1,
"status": "draft",
"identity": {
"node_id": "umbraaudio-m4",
"display_name": "UmbraTools - Mac mini M4 (UmbraAudio)",
"organization": "umbra",
"role": "worker",
"products": ["umbraaudio"],
"primary_product": "umbraaudio",
"timezone": "America/Los_Angeles",
"operator": "single human operator (B.D. Murphy)"
},
"host": {
"hostname": "mini.bdmurphy.me",
"os": "macOS 26.6.2 (arm64)",
"lan_ipv4": ["192.168.123.12", "192.168.123.154"],
"storage_root": "/Volumes/1TB_APL_Storage",
"home_relocated": "/Volumes/Dock/worstauthor",
"python": "3.12.13"
},
"services": {
"web_ui": {
"bind": "0.0.0.0:8420",
"stack": "FastAPI + HTMX",
"public_name": "umbraaudio.bdmurphy.me",
"exposure": "Caddy reverse proxy on ANOTHER LAN machine -> this port",
"auth": "NONE (unsigned active_project cookie). Do not expose beyond the LAN.",
"autostart": "launchd LaunchAgent com.umbraaudio.web (login-scanned, KeepAlive)"
},
"tts_local": {
"url": "http://localhost:8881",
"engine": "Qwen3-TTS (transformers, MPS)",
"autostart": "NONE - manual nohup .venv/bin/python scripts/qwen_tts_server.py"
},
"tts_relay_listener": {
"url": "http://127.0.0.1:18881",
"note": "root LaunchDaemon on this host forwards to a LAN GPU box; has no upstream fallback"
},
"llm_local": {
"url": "http://localhost:1234/v1",
"engine": "LM Studio, google/gemma-4-e4b",
"note": "MUST NOT be co-resident with local Qwen3-TTS (16 GB unified memory)"
}
},
"capabilities": [
"audio.parse_manuscript",
"audio.segment_utterances",
"analysis.character_discovery",
"analysis.speaker_attribution",
"analysis.speaker_clarity",
"analysis.name_token_scan",
"voice.reference_generation",
"voice.attribute_measurement",
"audio.render_utterances",
"audio.stitch_chapters",
"audio.normalize_loudness",
"export.m4b",
"export.mp3_package",
"review.speaker_QA"
],
"resource_pools": {
"cpu": {"capacity": "bounded; shares the same 16 GB as TTS", "concurrency": 1},
"llm": {"capacity": "1 x gemma-4-e4b (MPS, ~21 tok/s)", "concurrency": 1},
"tts_mps": {"capacity": "1 x Qwen3-TTS-12Hz-1.7B", "concurrency": 1, "note": "one orphan generation can stall the model lock for ~20 min"},
"gpu_cuda": {"capacity": 0, "note": "no local NVIDIA GPU; GPU work is delegated to 192.168.123.14"},
"ffmpeg": {"capacity": 1, "note": "launchd PATH must be extended or export fails"}
},
"constraints": [
"macOS Local Network TCC blocks launchd-spawned processes from all LAN IPs (EHOSTUNREACH); loopback is exempt",
"external-volume TCC: only .venv/bin/python may read/write /Volumes/1TB_APL_Storage under launchd; a shell cannot even open a script there",
"Qwen3-TTS and LM Studio must never be co-resident",
"PROJECT_BASE is hard-coded in umbraaudio/config.py and is not configurable",
"four divergent DDL authorities in this repo (see manifest.md 1.3)"
],
"projects": [
"DealHunterShip_4.0e_whole-to-audio",
"Last Dwarf",
"Treasure Island",
"Vocals"
],
"project_lifecycle": {
"DealHunterShip_4.0e_whole-to-audio": "released",
"Last Dwarf": "active",
"Treasure Island": "experimental",
"Vocals": "showcase"
},
"schema_fingerprints": ["0078569a0fa6bc70", "367d5df9e546dbba"],
"_schema_fingerprints_note": [
"Two distinct fingerprints = real per-project schema drift.",
"0078569a0fa6bc70: current shape (has name_tokens + the 5 clarity columns) - Last Dwarf, DealHunter.",
"367d5df9e546dbba: older shape (no name_tokens, missing 9 utterance columns) - Treasure Island, Vocals.",
"Produced by scripts/umbra_node_report.py; re-run --check after any migration."
],
"authoritative_data_summary": {
"in_sql": ["project", "chapters", "utterances", "name_tokens"],
"in_json": [
"speakers.json",
"voice_profiles.json",
"voice_status.json",
"voice_engines.json",
"phonetic_names.json",
"m4b_meta.json"
],
"on_filesystem_only": ["voices/*_ref.wav", "voice_prompts/*.pt (vmatrix)", "chapter_audio/**/*.wav", "*.wav.txt markers"],
"never_import_as_authoritative": ["pipeline_status.json", "*_build_status.json", "manifests/*.json", "discovery_pass*.json"]
},
"facts_for_the_controller": {
"empty_sql_tables": [
"characters",
"generated_audio",
"pronunciation_guide",
"quality_reports",
"review_decisions",
"exception_reviews",
"voice_review_history"
],
"has_stable_utterance_id": false,
"has_stable_speaker_fk": false,
"has_user_accounts": false,
"has_shared_postgres_client": false,
"has_heartbeat_or_lease": false,
"test_suite": "pytest, 461 passing at 2026-10-01 (8 pre-existing/parallel failures)",
"open_defects_blocking_exposure": [
"no authentication on any endpoint (spec 20 J.6)",
"download path traversal via str.startswith (routes.py ~5049)",
"upload filename not sanitised with Path(...).name",
"no CSRF, no security headers, no TrustedHost"
]
},
"contracts": {
"informational_only": {
"min_node_contract_commit": "4775e3b",
"note": "git HEAD moves constantly; this field is documentation, never drift-checked"
},
"specs": [
"specs/21-umbra-system-nodes-tasks-and-changes.md",
"specs/20-enterprise-database-integration.md",
"specs/19-enterprise-control-plane.md",
"specs/18-UmbraTrans.md",
"manifest.md"
]
}
}

12
manifest.md
View File

@@ -8,9 +8,19 @@
- **Method:** read-only introspection of the live project tree, the SQLite databases, the JSON - **Method:** read-only introspection of the live project tree, the SQLite databases, the JSON
state files, and the audio artifact trees. No data was modified. No secret, credential, or state files, and the audio artifact trees. No data was modified. No secret, credential, or
connection value appears in this document. connection value appears in this document.
- **Companion documents:** `specs/20-enterprise-database-integration.md` (shared-DB contract), - **Companion documents:** `specs/21-umbra-system-nodes-tasks-and-changes.md` (node/task/change
contract), `specs/20-enterprise-database-integration.md` (shared-DB contract),
`specs/18-UmbraTrans.md` (translation design), `specs/16-audiobook-production.md` `specs/18-UmbraTrans.md` (translation design), `specs/16-audiobook-production.md`
(production pipeline). (production pipeline).
- **This document is a dated measurement, not a live view.** The numbers below were read on
2026-09-28. When they have moved, add a dated line to the update log rather than silently
editing a table.
## Update log
| Date | Change |
|---|---|
| 2026-10-01 | Measurement refresh after the designation-merge work. `Last Dwarf` `speakers.json` grew **70 → 93** (23 genuinely missing speakers captured from the ch5 crowd and similar groups, plus 33 already-covered designations); `DealHunter` ref WAVs measure **97** (was 94). Every other table count in §1.1 was re-verified unchanged. Re-run `.venv/bin/python scripts/umbra_node_report.py` to regenerate the live figures; `manifest.md` records the baseline, the script records the present. |
--- ---

405
scripts/umbra_node_report.py Normal file
View File

@@ -0,0 +1,405 @@
#!/usr/bin/env python3
"""Read-only Umbra node report: the facts another machine needs to align with.
Every Umbra machine is a *node* in the Umbra system (spec 21). Before any data
model can be aligned, or any task assigned, the controller needs a factual
description of what a node actually holds: its git state, its projects, the
per-project SQLite shape (row counts plus a schema fingerprint), the JSON state
files, and the audio artifact counts.
This script produces exactly that, and nothing else. It is strictly read-only:
it never opens a database for writing, never mutates a project, and never reads
secret values (only the *names* of ``.env`` keys).
Usage:
python scripts/umbra_node_report.py # human-readable
python scripts/umbra_node_report.py --json # machine-readable
python scripts/umbra_node_report.py --check # profile vs reality
python scripts/umbra_node_report.py --project-dir "Last Dwarf"
``--check`` compares ``infra/umbra-node.json`` (the node's declared profile)
against measured reality and exits non-zero on drift. That is the "confirmation"
half of spec 21 §5: a node confirms the profile it advertises. The check is
deliberately limited to *stable* facts -- contract version, project set, and the
per-project schema fingerprints. Volatile facts (git HEAD, which ports happen to
be listening right now) are reported but never treated as drift.
"""
from __future__ import annotations
import argparse
import hashlib
import json
import platform
import socket
import sqlite3
import subprocess
import sys
from datetime import UTC, datetime
from pathlib import Path
from typing import Any
sys.path.insert(0, str(Path(__file__).parent.parent))
from umbraaudio.config import list_projects, slugify_project_name
REPO_ROOT = Path(__file__).resolve().parent.parent
NODE_PROFILE = REPO_ROOT / "infra" / "umbra-node.json"
# Version of the node-report contract itself. Bump when the shape below changes;
# `infra/umbra-node.json` must declare the same value for `--check` to confirm.
NODE_CONTRACT_VERSION = 1
# Ports that carry meaning for this system. Reported only; never contacted.
MEANINGFUL_PORTS = {
8420: "umbraaudio web UI (FastAPI/HTMX)",
8881: "Qwen3-TTS local API",
18881: "qwen-relay listener (loopback -> LAN GPU box)",
1234: "LM Studio OpenAI-compatible server",
5432: "PostgreSQL (control plane, not hosted here)",
}
# Tables whose row counts matter for alignment. Seven of eleven tables are empty
# in every project today (manifest.md §1.2) - that emptiness is itself a fact
# the shared schema must absorb.
TRACKED_TABLES = (
"project",
"chapters",
"utterances",
"name_tokens",
"characters",
"generated_audio",
"pronunciation_guide",
"quality_reports",
"review_decisions",
"exception_reviews",
"voice_review_history",
)
# JSON state files, classified per manifest.md §0.
STATE_FILES = {
"speakers.json": "AUTHORITATIVE",
"voice_profiles.json": "AUTHORITATIVE",
"voice_status.json": "AUTHORITATIVE",
"voice_engines.json": "AUTHORITATIVE",
"voice_assignments.json": "AUTHORITATIVE (legacy)",
"phonetic_names.json": "AUTHORITATIVE",
"m4b_meta.json": "AUTHORITATIVE",
"analysis_status.json": "CACHE",
"pipeline_status.json": "CACHE",
"generation_status.json": "CACHE",
}
def _run(*cmd: str) -> str | None:
"""Return stdout of a command, or None if it is unavailable/fails."""
try:
out = subprocess.run(
cmd, cwd=REPO_ROOT, capture_output=True, text=True, timeout=20, check=False
)
except (OSError, subprocess.SubprocessError):
return None
return out.stdout.strip() or None
def _git_state() -> dict[str, Any]:
head = _run("git", "rev-parse", "--short", "HEAD")
branch = _run("git", "rev-parse", "--abbrev-ref", "HEAD")
dirty = _run("git", "status", "--porcelain")
changed = [line[3:] for line in (dirty or "").splitlines() if len(line) > 3]
return {
"head": head,
"branch": branch,
"clean": dirty == "",
"uncommitted_files": changed,
}
def _listening_ports() -> dict[str, Any]:
"""Report which meaningful ports this host listens on (best effort)."""
listening: list[int] = []
raw = _run("lsof", "-nP", "-iTCP", "-sTCP:LISTEN")
if raw:
for line in raw.splitlines()[1:]:
for token in line.split():
if token.count(":") == 1:
port_text = token.rsplit(":", 1)[1]
if port_text.isdigit():
listening.append(int(port_text))
return {
"listening": sorted(set(listening)),
"meaningful": {
str(port): role
for port, role in MEANINGFUL_PORTS.items()
if port in listening
},
}
def _env_key_names() -> list[str]:
"""Return ``.env`` KEY NAMES only. Values are never read or reported."""
env_path = REPO_ROOT / ".env"
if not env_path.exists():
return []
names = set()
for line in env_path.read_text(encoding="utf-8", errors="replace").splitlines():
line = line.strip()
if not line or line.startswith("#") or "=" not in line:
continue
names.add(line.split("=", 1)[0].strip())
return sorted(names)
def _schema_fingerprint(db: sqlite3.Connection) -> tuple[str, int]:
"""Hash the logical schema (tables + columns + declared types).
``ALTER TABLE`` reorders columns, so the hash is computed over *sorted*
tuples: it detects added/dropped/renamed/retyped columns without reporting
spurious drift from ordering alone.
"""
rows = db.execute(
"SELECT name FROM sqlite_master WHERE type='table' AND name NOT LIKE 'sqlite_%'"
).fetchall()
parts: list[str] = []
for (table,) in rows:
for col in db.execute(f'PRAGMA table_info("{table}")'):
parts.append(f"{table}.{col[1]}:{col[2]}")
digest = hashlib.sha256("\n".join(sorted(parts)).encode()).hexdigest()
return digest[:16], len(parts)
def _project_fingerprint(project_dir: Path) -> dict[str, Any]:
data_dir = project_dir / ".umbra"
db_path = data_dir / "project.db"
report: dict[str, Any] = {
"project_dir": str(project_dir.relative_to(REPO_ROOT)),
"db_exists": db_path.exists(),
"db_size_bytes": db_path.stat().st_size if db_path.exists() else 0,
"state_files": {},
"voice_refs": 0,
"vmatrix_pt": 0,
"voice_embeddings": 0,
"utterance_wavs": 0,
"markers": 0,
"full_stitches": 0,
"exports": [],
}
for name, classification in STATE_FILES.items():
path = data_dir / name
if path.exists():
report["state_files"][name] = {
"classification": classification,
"size_bytes": path.stat().st_size,
}
voices = data_dir / "voices"
if voices.is_dir():
report["voice_refs"] = sum(1 for _ in voices.rglob("*_ref.wav"))
prompts = data_dir / "voice_prompts"
if prompts.is_dir():
report["vmatrix_pt"] = sum(1 for _ in prompts.rglob("*.pt"))
embeddings = data_dir / "voice_embeddings"
if embeddings.is_dir():
report["voice_embeddings"] = sum(1 for _ in embeddings.rglob("*.npy"))
audio = data_dir / "chapter_audio"
if audio.is_dir():
report["utterance_wavs"] = sum(1 for _ in audio.rglob("utt_*.wav"))
report["markers"] = sum(1 for _ in audio.rglob("utt_*.wav.txt"))
report["full_stitches"] = sum(1 for _ in audio.rglob("ch*_full.*"))
export_dir = data_dir / "export"
if export_dir.is_dir():
report["exports"] = sorted(
p.name for p in export_dir.iterdir() if p.is_file() and p.suffix in {".m4b", ".mp3"}
)
if not db_path.exists():
report["schema_fingerprint"] = None
report["tables"] = {}
return report
# Read-only URI: never creates, migrates, or writes. The UmbraTrans spec 18
# §4.2.1 forbids helpers whose side effects touch the source project.
con = sqlite3.connect(f"file:{db_path.resolve()}?mode=ro", uri=True)
try:
fingerprint, column_count = _schema_fingerprint(con)
report["schema_fingerprint"] = fingerprint
report["schema_columns"] = column_count
present = {
row[0]
for row in con.execute(
"SELECT name FROM sqlite_master WHERE type='table'"
)
}
counts: dict[str, int] = {}
for table in TRACKED_TABLES:
if table not in present:
counts[table] = -1 # table absent = a different schema shape
continue
try:
counts[table] = con.execute(f'SELECT COUNT(*) FROM "{table}"').fetchone()[0]
except sqlite3.Error:
counts[table] = -2
report["tables"] = counts
finally:
con.close()
return report
def build_report(project_dir: str | None = None) -> dict[str, Any]:
"""Collect the full node report. Read-only."""
projects = list_projects()
if project_dir:
projects = [p for p in projects if p["slug"] == project_dir] or [
{"slug": slugify_project_name(project_dir), "name": project_dir}
]
fingerprints = [
_project_fingerprint(REPO_ROOT / p["slug"])
for p in projects
if (REPO_ROOT / p["slug"]).is_dir()
]
report: dict[str, Any] = {
"report_version": NODE_CONTRACT_VERSION,
"node_contract_version": NODE_CONTRACT_VERSION,
"generated_at": datetime.now(UTC).isoformat(timespec="seconds"),
"node_profile_path": str(NODE_PROFILE.relative_to(REPO_ROOT)),
"host": {
"hostname": socket.gethostname(),
"platform": platform.platform(),
"python": platform.python_version(),
},
"git": _git_state(),
"listening": _listening_ports(),
"env_key_names": _env_key_names(),
"project_count": len(projects),
"projects": fingerprints,
"schema_fingerprint_set": sorted(
{p["schema_fingerprint"] for p in fingerprints if p["schema_fingerprint"]}
),
}
# Facts the controller must know that are *not* derivable from the repo.
report["facts"] = {
"project_base_hard_coded": True,
"project_base_value": "/Volumes/1TB_APL_Storage/UmbraAudio",
"authoritative_sql_tables": ["project", "chapters", "utterances", "name_tokens"],
"empty_sql_tables": [
"characters",
"generated_audio",
"pronunciation_guide",
"quality_reports",
"review_decisions",
"voice_review_history",
"exception_reviews",
],
"speaker_identity_lives_in": "speakers.json + voice_profiles.json + ref WAVs",
"audio_record_lives_in": "filesystem + *.wav.txt markers",
"authentication": "none; active_project is an unsigned browser cookie",
"shared_database_present_here": False,
"secret_values_reported": False,
}
return report
def check_profile(report: dict[str, Any]) -> list[str]:
"""Compare the declared node profile with measured reality.
Only stable facts are checked (contract version, project set, schema
fingerprints). Returns a list of drift messages; empty means confirmed.
"""
if not NODE_PROFILE.exists():
return [f"missing node profile: {NODE_PROFILE.relative_to(REPO_ROOT)}"]
declared = json.loads(NODE_PROFILE.read_text(encoding="utf-8"))
drift: list[str] = []
declared_version = declared.get("node_contract_version")
if declared_version != NODE_CONTRACT_VERSION:
drift.append(
f"node_contract_version declared {declared_version} != supported "
f"{NODE_CONTRACT_VERSION}"
)
declared_projects = set(declared.get("projects", []))
measured_projects = {p["project_dir"] for p in report["projects"]}
if declared_projects != measured_projects:
drift.append(
f"projects declared {sorted(declared_projects)} != measured {sorted(measured_projects)}"
)
if declared.get("schema_fingerprints"):
declared_set = sorted(declared["schema_fingerprints"])
if declared_set != report["schema_fingerprint_set"]:
drift.append(
f"schema fingerprints declared {declared_set} != measured "
f"{report['schema_fingerprint_set']}"
)
return drift
def _print_human(report: dict[str, Any], drift: list[str], checked: bool) -> None:
host = report["host"]
git = report["git"]
print("UMBRA NODE REPORT (read-only)")
print(f" generated_at {report['generated_at']}")
print(f" host {host['hostname']} {host['platform']}")
dirty_note = "clean" if git["clean"] else f"{len(git['uncommitted_files'])} uncommitted"
print(f" python {host['python']}")
print(f" git {git['branch']} @ {git['head']} ({dirty_note})")
print(f" listening {report['listening']['meaningful'] or 'no known ports'}")
print(f" env keys {len(report['env_key_names'])} (names only, no values)")
print(f" schema fingerprints {report['schema_fingerprint_set']}")
print("\n projects:")
for proj in report["projects"]:
populated = {t: n for t, n in proj["tables"].items() if n > 0}
empty = [t for t, n in proj["tables"].items() if n == 0]
absent = [t for t, n in proj["tables"].items() if n == -1]
print(f" - {proj['project_dir']} db={proj['db_size_bytes']}B "
f"schema={proj['schema_fingerprint']}")
print(f" populated: {populated}")
if empty:
print(f" empty: {empty}")
if absent:
print(f" absent: {absent}")
print(f" audio: refs={proj['voice_refs']} vmatrix={proj['vmatrix_pt']} "
f"wavs={proj['utterance_wavs']} markers={proj['markers']} "
f"stitches={proj['full_stitches']} exports={proj['exports']}")
print("\n system facts:")
for key, value in report["facts"].items():
print(f" {key}: {value}")
if drift:
print("\n PROFILE DRIFT (declared profile does not match measured reality):")
for d in drift:
print(f" ! {d}")
elif checked:
print("\n profile check: CONFIRMED (declared profile matches measured reality)")
def main() -> int:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--json", action="store_true", help="machine-readable output")
parser.add_argument("--check", action="store_true",
help="compare infra/umbra-node.json to reality; exit 2 on drift")
parser.add_argument("--project-dir", default=None, help="limit to one project directory")
args = parser.parse_args()
report = build_report(project_dir=args.project_dir)
drift = check_profile(report) if args.check else []
if args.json:
report["profile_drift"] = drift
report["profile_checked"] = args.check
print(json.dumps(report, indent=2, sort_keys=True))
else:
_print_human(report, drift, args.check)
return 2 if drift else 0
if __name__ == "__main__":
raise SystemExit(main())

590
specs/21-umbra-system-nodes-tasks-and-changes.md Normal file
View File

@@ -0,0 +1,590 @@
# Spec 21 — The Umbra System: Worker Nodes, Task Assignment, Cross-Product Change Management, and UI Topology
- **Status:** PROPOSAL. Nothing here is ratified. Every table marked *proposed* is a draft
contract for the controller to accept, amend, or reject.
- **Date:** 2026-10-01
- **Audience:** every machine running an Umbra product, plus the control plane.
- **Owner of ratification:** the main Umbra system (controller, currently `oldbox`,
`192.168.123.14`). This document was written by a worker and proposes; it does not decide.
- **Companion documents** (do not duplicate them, link to them):
- `manifest.md` — authoritative inventory of what UmbraAudio actually holds (898 lines).
- `specs/20-enterprise-database-integration.md` — shared-database contract, blockers,
decisions.
- `specs/19-enterprise-control-plane.md` — tenancy, roles, jobs, lineage.
- `specs/18-UmbraTrans.md` — translation/multilingual production.
- `infra/README-enterprise-bootstrap.md` — the control-plane database bootstrap that
already exists on `oldbox`.
- `AGENTS.md` — this machine's operational rules and hazards.
---
## 0. How to use this document
This machine (a worker) has written it so that the other machines and the controller do not
have to re-derive the same facts. Three concrete things every other machine should do:
1. **Read §5 and §6 of `manifest.md`** — that is where the real data inventory is. Do not
re-invent it.
2. **Produce a node report** with `scripts/umbra_node_report.py` (this machine's
implementation, §8.2). It is read-only, prints no secrets, and emits a schema fingerprint
per project. Copy the script; it has no product-specific imports beyond `umbraaudio.config`.
3. **Declare your node** in your own `infra/umbra-node.json` in the shape of §4.4, then run
`scripts/umbra_node_report.py --check` and get exit 0 before claiming you are aligned.
---
## 1. Executive summary
| # | Workstream | Where it is | What is missing |
|---|---|---|---|
| WS1 | Align the data model, copy UmbraAudio data in, verify, later transition | Design is unusually far along: `manifest.md` + spec 20 | **Ratification.** Nine open decisions, five hard blockers, zero implementation. |
| WS2 | Every machine is a node that gets assigned tasks (not only audio) | Job types, leases, heartbeats and pools are specified (spec 19 §12) | **Node contract**: no node identity, no capability advertisement, no task envelope, no reporting rule. Proposed in §4. |
| WS3 | How changes are managed across products | Nothing exists. Specs 19/20 cover *data* changes, not *code/requirement* changes | **Requirement catalog + distribution + confirmation loop.** Proposed in §5. |
| WS4 | Does a central server take over the UI? How do updates get deployed and tested? | `DECISION-8` is open and unanswered | **A decision and a deployment/update flow.** Recommendation + options in §6. |
**The single most important cross-cutting fact:** this machine already contains a complete,
measured data inventory and a verified shared-database contract. The system does not need
another discovery phase. It needs someone to **ratify decisions**, then implement. Every week
spent re-discovering is a week not spent building.
---
## 2. System shape (proposed, matches spec 19 §4)
```text
┌──────────────────────────────────────┐
│ CONTROLLER (oldbox 192.168.123.14) │
│ • authoritative data (PostgreSQL) │
│ • identity, tenancy, authorization │
│ • requirements catalog (§5) │
│ • definition sets (§5.7) │
│ • task queue + leases (§4) │
│ • lineage, artifacts, releases │
│ • portal UI (status only, §6) │
└───────┬──────────────┬───────────────┘
│ pull tasks │ report results
┌─────────────────┴───┐ ┌───┴──────────────┐
▼ ▼ ▼ ▼
┌───────────────┐ ┌──────────────┐ ┌────────────────────┐
│ node: audio │ │ node: trans │ │ node: <future> │
│ mini.bdmurphy │ │ <other host> │ │ │
│ │ │ │ │ │
│ UmbraAudio │ │ UmbraTrans │ │ other products │
│ web UI :8420 │ │ │ │ │
│ TTS :8881 │ │ │ │ │
└───────────────┘ └──────────────┘ └────────────────────┘
```
Authority rule (already proposed in spec 20 §A.3–A.4, **needs ratification**):
- The controller is authoritative for **identity, tenancy, requirements, definitions, task
state, lineage and release decisions**.
- A node is authoritative for **its own working area** (SQLite + filesystem) while it is in
`legacy` or `mirroring` state, and it must keep working when the controller is unreachable.
- Nothing is ever a cutover for the whole system at once. Migration state is **per project**
(`legacy → mirroring → dual_write → authoritative → retired`; manifest §14.1).
---
## 3. WS1 — Align the data model, copy the data, verify, then transition
### 3.1 What already exists (do not redo)
- `manifest.md` — every data object, exact DDL, JSON shapes, artifact naming, the legacy→shared
entity mapping (§13), 14 open questions (§15), and the aligned-definition-set list (§14.2).
- `specs/20` — table-by-table contract, cutover phases P1–P4, RLS review, acceptance tests.
- `infra/postgres/001_control_plane.sql` — 13 real tables, already bootstrapped on `oldbox`.
- Measured state of this machine, reproducible today: `scripts/umbra_node_report.py`.
### 3.2 The five headline facts any aligning machine must know
1. **The SQLite is mostly empty.** Only `project`, `chapters`, `utterances`, `name_tokens`
hold data. `characters`, `generated_audio`, `pronunciation_guide`, `quality_reports`,
`review_decisions`, `exception_reviews`, `voice_review_history` are **empty in every
project**. A SQL-only migration loses the speaker model, the voice model, the pronunciation
model and the entire audio record.
2. **There is no durable utterance identity.** Files are `utt_{utterance_index:04d}.wav` and
`utterance_index` is renumbered by re-segmentation and by utterance deletion. Spec 18 §3.3
forbids mutable indices as cross-language IDs. Spec 20 §E.1 proposes a content-addressed
`source_unit_key` recorded at generation time with filenames unchanged — **this machine can
build that today with no dependency on the controller.**
3. **Speaker identity is free text.** `utterances.speaker_name` is the only link to a speaker;
it is not a foreign key, not normalised, and has case variants. Five parallel stores, four
key schemes, three-way count disagreements (`Last Dwarf` 93/68/71 and `DealHunter` 61/64/97
as of 2026-10-01).
4. **The control-plane worker role is a blocker.** `umbracontrol_worker` matches zero RLS rows
and cannot attribute an audit event (spec 20 `B1`). Needs `service_identities` +
`job_leases`, not a GUC workaround. `B4` (functions `EXECUTE`-able by `PUBLIC` with no
`ALTER DEFAULT PRIVILEGES`) must be fixed **before any further migration**.
5. **Per-project schema drift is real and measurable.** This machine currently has two schema
fingerprints: `0078569a0fa6bc70` (Last Dwarf, DealHunter) and `367d5df9e546dbba` (Treasure
Island, Vocals — missing `name_tokens` and nine utterance columns). A migration must
introspect per database, never assume a uniform shape.
### 3.3 Sequence (proposed)
| Phase | Name | Gate to pass | Who |
|---|---|---|---|
| P0 | Ratify the nine decisions in §6.1 | Decisions recorded in the controller | Controller |
| P1 | Adapter/repository boundary inside UmbraAudio; no behaviour change | Existing tests unchanged; "where state lives" becomes configuration | Worker |
| P2 | Read-only mirror of the audio data into the controller | §3.4 verification passes on all 4 projects | Worker + controller |
| P3 | Dual-write; controller authoritative, worker still functional offline | No divergence beyond an agreed window; leases heartbeat | Both |
| P4 | Per-project `authoritative`; local SQLite becomes read-only cache | Lineage reproducible; released lineage immutable | Controller |
### 3.4 Verification gates for "it all works" (proposed)
1. Every project imports with zero dropped rows; ambiguous/unresolved values are recorded, not
discarded (`legacy_id_map` rows with a reason).
2. Row counts match the node report for every tracked table.
3. Schema fingerprints are recorded per source database, and drift is reported, not normalised
silently.
4. Every utterance resolves to a `speaker_id` **or** an explicit `needs_source_review` row.
5. Every voice has a SHA-256 and an `object_key`; no binary in PostgreSQL.
6. A sample chapter can be re-imported and produces an identical hash set (idempotency).
7. Proven: importing does not create or modify a single file in the source project tree
(before/after tree hash).
8. The audio worker completes a real chapter with the controller deliberately stopped.
---
## 4. WS2 — Every machine is a node that receives tasks
### 4.1 Why this needs its own contract
Spec 19 §12 already defines job types, leases, heartbeats, pools and idempotency keys. What is
missing is the *node* half: how a machine announces what it is, what it can do, what it refuses,
and how it reports a finished task. Without that, "assign a task to a machine" cannot be
expressed, let alone executed.
### 4.2 Node identity (proposed)
- `node_id`: stable, human-readable, lowercase-hyphenated, never reused. This machine:
`umbraaudio-m4`.
- `role`: `controller | worker`.
- `products`: which products the node runs (`["umbraaudio"]`).
- `node_contract_version`: the version of this contract the node implements. Bump on breaking
change. A node that reports an unknown version is not eligible for tasks.
- Identity is **declared in the repository, not configured by hand in a UI** (§4.4).
### 4.3 Capability advertisement (proposed)
A node advertises `capabilities` (what it can do), `resource_pools` (what it has, with
concurrency), and `constraints` (what it must not do). The scheduler matches a task's required
capability list against this advertisement — it does not hard-code hostnames.
Constraint declaration matters as much as capability declaration. This machine, for example,
must refuse any task requiring an LLM and local TTS simultaneously, because 16 GB of unified
memory cannot hold both and doing so silently degrades output to Edge-TTS.
### 4.4 Node profile file (proposed, implemented here)
Each node keeps `infra/umbra-node.json` — identity, services, capabilities, pools, constraints,
project list, schema fingerprints, and the honest gaps. This machine's file is
`infra/umbra-node.json`; it contains **no secrets** (ids, names, labels, counts only).
### 4.5 Task envelope (proposed)
```yaml
task:
task_id: tsk_01J... # opaque, controller-issued, globally unique
requirement_id: REQ-... # §5 — every task traces to a requirement
product: umbraaudio
capability: analysis.speaker_attribution
target_node: umbraaudio-m4 # null = scheduler chooses by capability
scope: # what the task operates on
project: "Last Dwarf"
locale: null
revision: <source_revision_id or null>
instruction: "Re-run attribution for ch11-49 after the designation merge."
inputs: # immutable references, never inline blobs
definition_set_version: defs-2026-10-01
source_snapshot_id: snap_...
acceptance_criteria: # machine-checkable where possible
- "pytest tests/ -q exits 0"
- "node report --check exits 0"
- "utterance.chapter_id in (11..49) all have attribution_confidence NOT NULL"
idempotency_key: sha256(...) # re-delivery must not duplicate work
lease_ttl_s: 900
deadline: 2026-10-08T00:00:00Z
report_back: # required in the result, not optional
- changed_row_count
- before_after_fingerprint
- test_evidence
- definition_set_version_used
- notes
```
### 4.6 Result envelope (proposed)
```yaml
result:
task_id: tsk_01J...
status: succeeded | failed | refused | partial
refusal_reason: "..." # required when refused
evidence: { ... } # acceptance_criteria satisfied, with proof
node: { node_id, node_contract_version, git_head, dirty }
facts: { ... } # the read-only node report, or the relevant subset
```
A node **may refuse** a task and must say why; refusing is a valid, audited outcome. Silently
producing wrong output is not.
### 4.7 What this machine can accept today
`audio.parse_manuscript`, `audio.segment_utterances`,
`analysis.{character_discovery,speaker_attribution,speaker_clarity,name_token_scan}`,
`voice.{reference_generation,attribute_measurement}`,
`audio.{render_utterances,stitch_chapters,normalize_loudness}`, `export.{m4b,mp3_package}`,
`review.speaker_QA`.
It cannot accept: translation, TTS on CUDA, anything needing an authenticated UI, anything
requiring concurrent LLM + local TTS.
---
## 5. WS3 — How changes are managed across products
This is the least-specified part of the system today and the one the operator cares most about.
Proposal below.
### 5.1 The model
> **The controller owns requirements and definitions. Products own implementation.**
> A change is never *pushed* into a product. It is *distributed* as a requirement, implemented
> locally, and then **confirmed** with evidence. The requirement cannot close until every
> affected product confirms or explicitly waives.
This preserves two real facts: (a) some products make changes locally and commit in git; (b) some
changes must apply everywhere. Neither needs the other to be overruled.
### 5.2 Requirement record (proposed)
```yaml
requirement:
requirement_id: REQ-0042
title: "Unresolved speaker designations must exist as speakers"
rationale: "A designation with no profile resolves to a random named character."
change_class: cross_product # local_product | cross_product | shared_definition | operational
owned_by: controller
definition_set_version: defs-2026-10-01 # null for local_product
scope:
products: [umbraaudio, umbratrans]
nodes: [umbraaudio-m4] # empty = all nodes running those products
acceptance_criteria: ["each product reports its own conformance measurement"]
state: proposed | triaged | distributed | in_progress | confirmed | waived | released
confirmations: # one per product/node
- product: umbraaudio
node: umbraaudio-m4
state: confirmed
evidence: {name_tokens_resolved: 10, unresolved_designations: 0, report_at: "..."}
commit: "4775e3b"
```
### 5.3 The four change classes (proposed)
| Class | What changes | Distribution | Confirmation evidence |
|---|---|---|---|
| `local_product` | One product's internals (UI, performance, refactor) | Requirement records *intent and outcome*; no code is sent | Product reports resulting version/commit |
| `cross_product` | Same behaviour needed in N products | Requirement per product, each with its own criteria | Each product's own measurement |
| `shared_definition` | Vocabulary/schema/attribute definitions | **Versioned data**, not code. Products pull the set and record which version they used | `definition_set_version_used` + applied row counts |
| `operational` | Env, config, deploy, credentials-ish | Per-node task; the node reports state | Node report facts (e.g. ports, fingerprints) |
### 5.4 The worked example the operator asked about
*"A user added or changed something that must apply across the system, owned and managed by the
main controller."*
The real cases in this system are a **speaker designation**, a **name pronunciation**, a **voice
attribute**, or a **role kind**. Proposed handling:
1. The controller holds the entity as **master data** with an identity, evidence, and a revision.
2. It emits a `shared_definition` requirement: "add term `X` with spelling `Y`, provenance
`explicit`, effective for products P1 and P2, definition set `defs-2026-10-02`".
3. Each product applies it **in its own representation** — UmbraAudio writes
`phonetic_names.json` + `speakers.json`; UmbraTrans writes its own pronunciation table. The
controller never writes another product's files.
4. Each product confirms with a measurement: "term present, spelling matches, N utterances
affected" — not "done".
5. Until every product confirms, the requirement stays `in_progress` and is visible as such.
6. A local product-side discovery of the same fact (e.g. the designation scanner found 23
missing speakers here) is *reported upward* as evidence, and the controller decides whether it
becomes master data. This is how the worker's discovery becomes system knowledge.
### 5.5 Code updates: distribution and testing (proposed)
Code ships through **git**, per product repository. The controller's job is not to carry code;
it is to carry the **compatibility matrix** and the requirement.
```text
controller: REQ "move umbraaudio to tag v0.9 + definition set defs-2026-10-02"
↓ distributed (one task per node)
node: git fetch && git checkout v0.9
run the product's own verification:
- pytest tests/ -q (this machine: 461 passing)
- scripts/umbra_node_report.py --check (profile/contract still valid)
- a product smoke test (start web, hit one endpoint)
record: commit SHA, test counts, report fingerprint, definition_set_version_used
↓ confirm
controller: closes REQ only when every target node confirms or explicitly waives
```
Rules:
- A node that cannot reach the controller keeps working and **queues** its confirmation.
- A node never auto-updates itself without a task. Audio production must not stop because
someone merged.
- Definition sets are pulled by version and recorded; a node using an old set is visible, not
silently wrong.
- Every product repo keeps its own `AGENTS.md` hazards. The system-level hazard list
(§8.6) is additive, never a replacement.
### 5.6 What must NOT be centrally managed
Product-internal implementation, local caches, generated audio, model weights, TTS prompt
caches, per-node environment tuning, and anything machine-specific (this machine's external
volume, TCC rules, its own launchd layout).
---
## 6. WS4 — Where does the UI live?
### 6.1 The decision (proposed, answers the operator's question)
**Neither extreme. Hybrid, with the controller authoritative for data and the worker keeping
the program UI.**
The controller **does** take on the UI function, but only the *system* UI: a portal showing node
health, task state, requirement conformance, definition-set versions, project/work registry and
release status. It does **not** host the audio review UI.
Why:
- The audio review UI is bound to local artifacts — WAV players, per-utterance regeneration,
staleness markers, local voice references. A central UI would have to stream every artifact
across the network to reproduce it.
- That UI has **no authentication at all** (spec 20 §J.6). Centralising it would publish an
unauthenticated mutation surface to the internet. It must stay on the LAN regardless.
- Its real risk is *staleness and fragility*, not location. A node can be perfectly robust in
place and useless if it cannot report what version it is and whether its data still agrees
with the controller.
### 6.2 Options considered
| Option | Pros | Cons |
|---|---|---|
| Central UI only | One place, one auth, one version | Streams all artifacts; couples UI downtime to production; biggest rewrite; publishes attack surface today |
| Worker UI only (today) | Fast, local, artifact-native | No cross-project view; a dead node is invisible; updates drift silently |
| **Hybrid (recommended)** | Portal on the controller; program UI stays on the worker; signed deep links; version + conformance shown on both | Two UIs to keep coherent |
### 6.3 How the two UIs stay coherent (proposed)
- The worker UI displays, in its footer, `node_id`, `definition_set_version`, and the last
successful confirmation timestamp. A reviewer can always see which system's view they are in.
- The portal deep-links into the worker UI with a short-lived signed link. No credentials travel
in a URL.
- Any *authoritative* decision (approve, merge speaker, release, publish) is written in the
controller and read by the worker UI; the worker UI never becomes the source of truth for one
of those.
### 6.4 How worker UI robustness is assured (proposed)
1. **Node report as the health probe.** `scripts/umbra_node_report.py --check` is the definition
of "this node is healthy and matches its declared profile".
2. **Existing process discipline.** This machine already runs the UI under a launchd
`KeepAlive` agent with a health-gated startup. That pattern is the baseline for every node.
3. **Read-only by default.** Sync and pull operations never mutate project data without an
explicit task.
4. **Degradation is normal.** If the controller is down, the worker keeps producing audio from
its local lease and reconciles later. Production availability outranks synchronization.
5. **No silent exposure.** While the UI is unauthenticated, it must remain LAN-only. The reverse
proxy must not publish mutation endpoints. This is a precondition, not a nice-to-have.
### 6.5 Decisions this workstream needs (proposed answer)
- Ratify the hybrid (portal on controller, program UI on worker).
- Decide whether the audio UI is retired later or kept permanently. **Recommendation: keep.**
It is the program UI; it is what a director uses while a book is in production.
- Make the open security defects (§8.5) a precondition for any change in exposure.
---
## 7. Questions that need answers
Grouped by who can answer. Each says what it blocks.
### 7.1 System topology and authority
1. **Is the controller's database authoritative for the audio system?** (spec 20 `DECISION-1`;
this document §2 assumes yes). Blocks: everything in WS1.
2. **Does UmbraTrans read the authoritative database instead of snapshotting?** (`DECISION-2`).
If yes, spec 18 §4.2.1 is superseded and a hard class of bugs disappears. Blocks: the adapter.
3. **One node per product, or one node running several products?** This machine currently runs
only UmbraAudio but already hosts the GPU box's relay. Affects leases and blast radius.
4. **What is the official home of the requirement catalog?** The control-plane PostgreSQL is the
proposal; it must be decided before any requirement tooling is built.
### 7.2 Data model
5. **Who resolves an unresolvable speaker name** — create a placeholder, or record it unresolved?
(manifest §15.1). Blocks: the translation identity gate, which is fail-closed on exactly this.
6. **Which of the three speaker stores wins** when they disagree (`Last Dwarf` 93/68/71,
`DealHunter` 61/64/97 — manifest §15.2)?
7. **Is `utterance_index` preserved as a display ordinal** or retired in favour of a
content-addressed key (manifest §15.4)? Recommendation: keep it as a display ordinal, add
`source_unit_key`.
8. **Is the manuscript stored once** or duplicated in `chapters.source_text` and
`utterances.text` (manifest §15.5)? Affects copyright handling.
9. **Which voice-attribute profile is canonical** — the 9 measured features, the prose taxonomy,
or the 13-dimension acoustic set (manifest §15.14)? Recommendation: store measurements and
definitions separately.
10. **Does one voice need a separate vmatrix per target language?** (manifest §15.11).
Recommendation: per `(voice, locale)`, with a marked locale-neutral option.
### 7.3 Tasks and nodes
11. **What is the node identity scheme, and who issues it?** (§4.2).
12. **Does the scheduler match on declared capability only**, or may a task name a node?
Recommendation: capability-first, node as a hint.
13. **What happens when a node refuses a task?** Recommendation: valid, audited, visible, and
the requirement stays open.
14. **Are tasks ever allowed to require LLM + TTS co-residency** on a 16 GB machine? Recommendation:
forbid it as a capability constraint, not a per-node convention.
### 7.4 Change management
15. **Is the requirement catalog part of the control-plane database**, or a separate system?
16. **What is the confirmation standard of proof** — a test result, a measurement, a human
approval, or all three? (§5.2).
17. **Who arbitrates when two products need incompatible versions of a shared definition?**
18. **How long may a requirement stay `in_progress` before it is escalated?** There is no clock
in this proposal.
### 7.5 UI and deployment
19. **Ratify the hybrid UI topology** (§6.1).
20. **Does every product deploy by git tag pull**, or is there a package registry?
21. **Who may push a definition-set version**, and does it require review?
22. **Which host runs object storage**, and is it the same host as PostgreSQL? (spec 20
`DECISION-6`). Until it exists, artifacts stay on local filesystems and the database holds
only resolvable keys.
### 7.6 Security preconditions
23. **Raterify the `B1` remedy** (`service_identities`, not a GUC workaround).
24. **Raterify `B4`** — revoke `EXECUTE` on `app` functions and set default privileges *before*
any further migration.
25. **Who signs off on exposing any worker UI beyond the LAN**, given zero authentication today?
---
## 8. What UmbraAudio already provides for the whole system
### 8.1 The high-value assets (not just data)
| Asset | Where | Reuse value across machines |
|---|---|---|
| Complete measured data inventory | `manifest.md` | Removes the discovery phase from every other machine |
| Verified shared-DB contract + blockers | `specs/20` | Saves an entire architecture exercise |
| Control-plane schema, live | `infra/postgres/001_control_plane.sql`, bootstrapped on `oldbox` | The system already has a database; it lacks domain tables |
| Data-manifest generator | `manifest.md` + `scripts/umbra_node_report.py` | Any machine can produce the same fingerprint report |
| Node profile format | `infra/umbra-node.json` + `scripts/umbra_node_report.py` | Copyable node declaration |
| Legacy→shared entity mapping | `manifest.md` §13 | The actual migration mapping table, already written |
| Aligned-definition set list | `manifest.md` §14.2 | The exact list of definitions to extract from code into data |
| Verification culture | 447→461 passing tests, ruff, mypy, per-session acceptance criteria | A node can *confirm* with evidence rather than assertion |
| Operational hazard knowledge | `AGENTS.md` | Saves every machine from rediscovering TCC/PATH/co-residency traps |
### 8.2 Reproducible facts about this machine (today)
```bash
.venv/bin/python scripts/umbra_node_report.py # human-readable
.venv/bin/python scripts/umbra_node_report.py --json # machine-readable
.venv/bin/python scripts/umbra_node_report.py --check # confirm profile; exit 2 on drift
```
- 4 projects, 2 distinct schema fingerprints (real drift).
- Only 4 of 11 tables carry data.
- Speaker identity lives in JSON + WAVs; audio record lives on the filesystem + markers.
- No authentication, no shared-Postgres client, no lease/heartbeat.
- 461 tests passing; 8 known pre-existing/parallel failures.
### 8.3 Requirements of this data that the shared model must satisfy
From `manifest.md` §10, the blocking ones are: stable source-unit identity, stable speaker
identity, an immutable source revision, and artifact lineage with SHA-256. The rest are
available today and can migrate as-is.
### 8.4 What the worker must never do
Write into another product's data area; renumber or rewrite utterances; reuse English audio
markers as target-language markers; call another product's private functions; require a
migration before the controller is ready.
### 8.5 Defects that constrain any exposure or migration (from `manifest.md` §12)
No authentication anywhere (65 mutating endpoints, plain HTTP, `0.0.0.0` bind); project
selection is an unsigned cookie; download traversal via string-prefix containment; upload
filename not sanitised; no job queue/lease/heartbeat; four divergent DDL authorities; non-atomic
JSON writes; no artifact checksums; foreign keys never enabled.
### 8.6 Hazards on this machine specifically
External-volume TCC (only the venv python may touch the volume under launchd); launchd's bare
`PATH`; macOS Local Network TCC blocking launchd processes from LAN IPs; Qwen and LM Studio
never co-resident; `PROJECT_BASE` hard-coded; concurrent editors in this repository.
---
## 9. This machine's plan — task list
Legend: **NOW** = no dependency on the controller, safe today. **NEXT** = needs a decision or
another machine. **LATER** = needs the shared database to exist. Each task has an acceptance
criterion so it can be *confirmed* rather than reported done.
### 9.1 NOW
| ID | Task | Why | Acceptance | Est |
|---|---|---|---|---|
| `M4-01` | Node profile + read-only node report + `--check` | Make this machine describable to the system | `infra/umbra-node.json`; `--check` exits 0 | **done 2026-10-01** |
| `M4-02` | Commit this session's artifacts and document them | AGENTS.md cadence rule; nothing left uncommitted | Commit exists; session recorded in `AGENTS.md` | 15 min |
| `M4-03` | Reconcile the dirty working tree | 20 files uncommitted from parallel sessions; a lost day of work is a real loss here | Working tree clean, or every uncommitted hunk accounted for | 1 h |
| `M4-04` | Add `source_unit_key` (content-addressed, additive) | The one blocking identity concept that needs **no** controller. Without it, the mirror imports unstable IDs | Column exists, populated for new generations, filenames unchanged, `is_current` unaffected, tests pass | 3-4 h |
| `M4-05` | Extract the aligned definition set into data | `manifest.md` §14.2 — vocabularies currently live in code, so every product reimplements them | A versioned `definitions/` file with utterance kinds, speaker kinds, capture modes, phonetic provenance; code reads it | 3-5 h |
| `M4-06` | Fix the cheap security defects | Precondition for any exposure change, and for an honest node posture | Traversal + filename fixes with tests | 2 h |
| `M4-07` | Resolve the speaker-count disagreement | 70/93/71 — the single most confusing fact about this machine's data | One authoritative count per project, with the others derived or explicitly deprecated | 2-3 h |
| `M4-08` | Add a read-only `/api/node` health endpoint | Lets the controller scrape node health without an agent; LAN-only | Endpoint returns the node report; no secrets; LAN-only documented | 1-2 h |
### 9.2 NEXT — needs a decision or another machine
| ID | Task | Blocked by |
|---|---|---|
| `M4-09` | Resolve remaining orphan name tokens (10 in Last Dwarf) | `manifest.md` §15.1 policy; §7.2 Q5 |
| `M4-10` | Merge designations that are the same person ("The Big One" / "The Biggest One") | Human review |
| `M4-11` | Re-run attribution after the designation merge | `M4-10` |
| `M4-12` | Make `PROJECT_BASE` configurable | Touches `config.py`, which a parallel session is editing — coordinate first |
| `M4-13` | Decide the object-storage host | `DECISION-6` (§7.5 Q22) |
| `M4-14` | Backfill `source_unit_key` for existing utterances | `M4-04` + a definition of the key inputs |
### 9.3 LATER — needs the shared database
| ID | Task | Blocked by |
|---|---|---|
| `M4-15` | Implement the read-only mirror (P2) | `DECISION-1` ratified; `B1`/`B4` fixed on the controller |
| `M4-16` | Prove the import creates no file in the source tree | `M4-15` |
| `M4-17` | Lease + heartbeat client so a task can be claimed | `service_identities` + `job_leases` exist |
| `M4-18` | Definition-set pull and version reporting | Controller serves the set |
| `M4-19` | Per-project `authoritative` cutover | P2/P3 verification green for that project |
| `M4-20` | Per-locale voice assets (vmatrix per `(voice, locale)`) | `manifest.md` §15.11 decision |
### 9.4 Sequencing note
`M4-04` and `M4-05` are the highest-leverage items on this list. Both are additive, both are
independent of the controller, and both make every later phase cheaper. If the system does
nothing else, do those two.
---
## 10. Change log
| Date | Change |
|---|---|
| 2026-10-01 | Initial draft. Created by the UmbraAudio node. Added `infra/umbra-node.json`, `scripts/umbra_node_report.py`. Status: awaiting ratification. |