d6cebe4d55
Replaces claiming with assignment. The controller hands a job to one or more nodes and a node asks what it has been given; jobs.lease_owner could name only one holder, so a job fanned out to several machines was inexpressible, and 'what was assigned and what is not complete' had no answer. job_assignments is one row per (job, node, attempt), so both are now readable. Nodes poll rather than being pushed, which is not a simplification: nodes sit behind NAT and the controller has no route to them. The only path that exists is a node calling out. The rule with teeth: a job is complete when the controller has the artifacts to show it. Reporting is not believing. A node that finishes and hands over nothing gets its assignment failed, with the reason naming what was missing, and the work stays available for another machine. The tempting implementation - mark complete, then verify afterwards - cannot fail, because the only thing it could compare against is the report it just trusted. artifacts_verified_at is written by verification alone, so 'the node says it is done' and 'the controller confirmed it' are different facts that cannot look alike. fan_out defaults to 1 and is never inferred. More than one is redundant execution: several machines do the same work and the first verified result wins, the rest are cancelled. Worth having for cheap idempotent work, wasteful for anything costing GPU-seconds, so a caller has to say so. The losers are cancelled rather than left running - otherwise every machine on the list burns its time on a result nobody will read. Four defects found while building it, all by tests written first: - Naming a node explicitly bypassed the drain check. An administrator could drain a machine, see it read as drained, and still hand it work: the control doing nothing while appearing to work. Status is now checked on both routes. - attempts were counted from jobs.attempt, which only claim_job incremented, so under assignments max_attempts was never reached and a failing job was re-offered forever. Counted from failed assignments now, so a fan-out to three costs one attempt, not three. - start_assignment reported 'not startable' when the real problem was that the assignment belonged to another node. Ownership is checked first, because an operator handed the first will look in the wrong place. - The artifact API accepted values the database rejects: role 'output' where job_artifacts_role_check allows only 'consumed' and 'produced', an unbounded status, a content_hash that was not 64 hex characters, and kind at 80 where the limit is 64. Every completion would have been a 500. The schema had fixed this vocabulary all along and the API had to learn it rather than invent it. Artifacts are recorded even when verification fails: a short run still produced evidence, and discarding it would destroy the record of why. storage_key is never treated as proof the bytes exist, because there is no object storage to ask yet (DECISION-6). Claim still exists. It is listed in the pinned node surface as pending removal and the next change takes it out - two paths handing out work is the state this project keeps finding and criticising. Mutation-verified: trusting the node fails 3 tests, dropping the drain check fails 1, leaving fan-out losers running fails 1. 674 passed, 4 skipped.
UmbraTrans local bootstrap workspace
This workspace contains the unprivileged agent-comm server requested by the Oldbox handoff. It does not contain the UmbraTrans application source: no application bundle was present locally. A staged archive, if present, is left untouched.
Layout
agent_comm/— Python 3.11+ standard-library, stdio-only MCP server.agent_comm/tests/— dependency-free unit tests.umbratrans_web/— phase-1 account/user-management backend slice (FastAPI, Argon2id, sessions, admin routes, in-memory and local JSON development repositories). It does not connect to PostgreSQL; seeumbratrans_web/README.md.umbratrans_web/migrations/— proposed, unapplied phase-1 identity SQL and its rollback.opencode.json— non-secret OpenCode registration for the confirmed NAS2 mailbox.agent-comm.local.example.json— non-secret local fallback registration..env.example— non-secret environment template.DATA-SEPARATION.md— development/production boundary.
Confirmed NAS2 mailbox
The oldbox desktop account mounted NAS2 netbackup through its user GVfs session and confirmed the UmbraTrans directory readable, searchable, and writable. The server requires the explicit NAS deployment mode, the enclosing GVfs mount, and an affirmative mount confirmation:
/run/user/1002/gvfs/smb-share:server=nas2.local,share=netbackup/UmbraTrans/agent-mailbox
Run read-only health checks:
cd /home/umbra/src/UmbraTrans
AGENT_COMM_MODE=nas \
AGENT_COMM_NAS_MOUNT=/run/user/1002/gvfs \
AGENT_COMM_NAS_CONFIRMED=1 \
AGENT_COMM_ROOT='/run/user/1002/gvfs/smb-share:server=nas2.local,share=netbackup/UmbraTrans/agent-mailbox' \
/home/umbra/.local/bin/uv run python -m agent_comm --check
If the user GVfs session is absent, do not guess another path or broaden share permissions. Use the private local fallback documented in agent-comm.local.example.json, or have a privileged operator restore the confirmed user mount.
The server speaks newline-delimited MCP JSON-RPC over stdin/stdout. It opens no network socket and never executes queued instructions.
OpenCode registration
opencode.json contains the exact NAS configuration and paths only. It must never contain passwords, tokens, private keys, or other secrets. A Mac-side reader still needs separate configuration and verification before an outbox message can be considered received.
Development workflow
The project uses uv and a workspace-local virtual environment:
cd /home/umbra/src/UmbraTrans
/home/umbra/.local/bin/uv sync --python /usr/bin/python3
/home/umbra/.local/bin/uv run python -m unittest discover -s agent_comm/tests -v
/home/umbra/.local/bin/uv run ruff check .
/home/umbra/.local/bin/uv run ruff format --check .
/home/umbra/.local/bin/uv run python -m compileall -q agent_comm
make check
The tests cover schema and size validation, atomic no-overwrite writes, restrictive modes, path traversal, symlink refusal, secret-pattern redaction, inbox processing, and read-only initialization.
The account-management slice has its own targets:
cd /home/umbra/src/UmbraTrans
/home/umbra/.local/bin/uv run pytest umbratrans_web/tests
make web-check
Its tests use the in-memory repository and generated per-run passwords; they never read a production configuration, never connect to a database, and contain no credential material.
Backups
scripts/umbratrans_backup.sh is installed root-owned as /usr/local/sbin/umbratrans-backup and scheduled by the enabled umbratrans-backup.timer. It writes only to the confirmed NAS2 netbackup/UmbraTrans/backups tree:
- PostgreSQL:
pg_dumpcustom-format logical dump, never a live data-directory copy. - Configuration: PostgreSQL configuration files only.
- Workspace: source/configuration archive excluding
.venv, Git internals, runtime data, local mailboxes, and secret-bearing files.
Each file is staged outside the share, published with an atomic rename as umbra, checksummed, and verified after publication. Retention is 30 days for PostgreSQL/configuration/manifests and 14 days for workspace archives; stale temporary files older than one hour are removed. The timer fails closed if the user GVfs mount is absent and does not mount or broaden the NAS share.
The current first-run backup set is present under:
/run/user/1002/gvfs/smb-share:server=nas2.local,share=netbackup/UmbraTrans/backups
Check timer status with systemctl status umbratrans-backup.timer and the last run with journalctl -u umbratrans-backup.service.