master
99 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
umbra audio
|
11f27f5ef9 |
Capability profile for node 238, measured over SSH
Reached 238 via public-key enrolment - no password anywhere. The first attempt failed because the key had never landed in authorized_keys; the second worked once the fingerprint matched. Findings worth keeping beyond this node: The VM reports 'AMD Ryzen 9 3900X' and then lists 65 CPUID flags that stop at SSE4.2. No avx, no avx2, no fma. That silicon has AVX2 and FMA, so VirtualBox is masking them and /proc/cpuinfo is a declaration, not a measurement. Recorded as *unverified* rather than absent, because confirming it needs an actual AVX instruction executed and there is no gcc on the guest - a flag list cannot settle it. This is the defect the compute design already has, one layer down. The agent sets throughput_factor by string-matching a GPU model (40.0 if '4070' in name else 20.0), which is a declared factor wearing a measured one's name. Here the hypervisor makes the same kind of lie. Capability discovery therefore has to probe by executing instructions and treat an unverifiable feature as unavailable. Also: cpu.max is 'max 100000' so the guest is not CPU-capped, memory is 47.0 GiB unballooned, no GPU and no nested virt flags, and umbra has no sudo. Recorded in docs/ node-238-capability-profile.md with how each value was obtained. |
||
|
umbra audio
|
0c35eb2d03 |
Allow the address the service binds to as a Host value
The deployment listens on 192.168.123.14 and TrustedHostMiddleware rejected that same address, so every request by IP returned a bare 400 Invalid host header - including /api/health, which is indistinguishable from the service being down. It has cost this project twice. Found when the first pairing attempt from 192.168.123.238 came back 400 with no body and no logged detail. The pairing code was fine and went unspent; the request never reached the route. Both umbratrans.bdmurphy.me and 192.168.123.14 are now allowed. The 0600 on the env file is unchanged, and the correction is recorded in the alignment doc's transport note so the audio node is not sent looking for a service outage. |
||
|
umbra audio
|
32dc580c74 |
Pair nodes with a short code instead of a copied credential
Issuing a node credential by hand required writing the key into a 0600 file on the controller and having a person copy it onto another machine. That key landed in shell history and a terminal buffer, and it made the controller a courier: nothing could be enrolled without me in the middle, which is the opposite of what a fleet of heterogeneous nodes needs. `POST /api/compute/pairing-codes` (admin session) mints a six-digit code for a *named* node. `POST /api/node/pair` exchanges it, once, for a credential returned in the response. The node never sees a secret typed by a person, and the controller is no longer in the path. ## Why a six-digit code is defensible 1,000,000 possibilities is nothing on its own. What makes it work is everything around it, and any one of these alone would be too thin: - **Single use.** Redemption is `SELECT ... FOR UPDATE` then `UPDATE ... WHERE used_at IS NULL`, in one transaction. - **Expiring.** Default ten minutes, hard-bounded to 60-1800s. Long enough to walk to a machine; short enough that a code left in a terminal buffer is not a credential for the day. - **Bound to a node name.** A code read out of a terminal buffer cannot pair a machine called something else. Without this, "confirm it is the machine" is decoration. - **Argon2id at rest.** A database read or a support transcript yields nothing replayable. - **Rate limited per source address**, charged *before* any code lookup, so guessing across many codes costs the same as guessing at one. Twenty per quarter-hour against a million-code space is about 21 years. This is the control that makes the length defensible; the others are cheap additions to it. Every rejection is worded identically - "no such code", "wrong code", "expired", "already used" all read the same - so the response is not an oracle for the rate limiter to work against. The one exception is a name mismatch, which says so plainly: it is not secret, and an operator staring at a failure needs to be told the code is for a different machine rather than left guessing. A name mismatch does *not* spend the code, because the obvious implementation spends it and then the retry with the right name cannot work. ## Why not a SECURITY DEFINER function The obvious shape is a plpgsql function that verifies and redeems, so the node surface never touches the tables. It cannot be done in SQL here: the codebase hashes with Argon2id in Python, and the only hashing PostgreSQL offers in this database is `crypt()` (bcrypt and MD5). Writing pairing against bcrypt would claim an algorithm choice the rest of the system does not make. So verification is in the repository, which means the pairing tables need grants for the application role when G2 lands, and the path needs re-testing as a non-owner role. Recorded in the migration header. ## Three defects found by writing the tests - **Redemption order.** Claiming the code before inserting the node it references violates the foreign key, so *every* redemption failed. Found by nine failing tests with an FK error that had nothing to do with pairing logic. - **A race that produced a 500.** Two nodes verifying one code simultaneously both created a node, and the loser hit a unique-constraint violation on `(organization_id, name)` - an unhandled error for an entirely ordinary condition. Fixed with `SELECT ... FOR UPDATE` on the code row before the insert, so the second caller blocks, re-evaluates `used_at IS NULL`, and gets the ordinary refusal. The concurrency test fails when the lock is removed and passes when it is present. - **A test that could not fail.** Removing the `used_at IS NULL` from the *claim* left all tests passing, because the candidate query's `used_at IS NULL` filter masks it. Sequential redemption cannot distinguish the two guards; only a concurrent one can. That is now `test_a_concurrent_redemption_produces_exactly_one_node`, and `test_a_spent_code_is_not_offered_as_a_candidate` pins the first guard directly. The reuse trigger needed a correction too. It originally refused any change to `expires_at`, which would have made it impossible to revoke an unused code - the normal way to withdraw a code issued to the wrong person. It now refuses identity changes and expiry *extension*, and permits expiry to move earlier only. Two existing tests flagged the new unauthenticated route, correctly: `test_the_node_surface_is_exactly_what_a_node_may_do` and `test_every_node_route_uses_node_credential_authentication` both assumed every `/api/node` route needs a credential. Rather than weaken either, `/pair` is recorded as a named exception in both, with a check that the named route still exists - so a second unauthenticated node route has to be added deliberately with a reason. 509 passed, 27 skipped. Deployment: 0004 applied after a backup, health 200, and the live endpoint verified to refuse a bogus code with 403 and a wrong node name with a `pairing_name_mismatch` that does not spend the code. G2 is still open, and the pairing tables will need grants when it closes. |
||
|
umbra audio
|
382ca8fc78 |
Derive node status on read; a heartbeat loop for the audio node
Two problems, both found by the audio node running Stage 1.
## A node that went quiet still looked available
`compute_nodes.status` is promoted to `online` by a heartbeat and demoted to
`offline` by an administrator. Nothing demoted it on the passage of time: grep found
`offline` appearing in exactly one place, the heartbeat's promotion CASE. So an agent
that was killed left the row reading `online` indefinitely - measured on the
deployment, where `umbraaudio-m4` showed `online` against a last heartbeat nine hours
old.
That is the failure invariant I24 exists to prevent. `create_node` refuses to mark a
node ready before its agent has ever spoken, and then nothing noticed when it stopped
speaking.
Derived on read instead, in `effective_status`, called from `_node_from_row` so every
caller - console, heartbeat response, placement - gets the same answer:
offline: no recent heartbeat (derived; the stored status is still 'online')
Three decisions inside it:
- Only `online` is derived away. `draining` and `retired` are administrator
decisions about intent, and a machine going quiet does not retract an order to stop
taking work. Demoting a draining node would make the console report it as gone
rather than winding down, and blame the wrong thing.
- Nothing is written back. The stored value stays as-is and is carried on the model as
`stored_status`, so a GET has no side effect and two readers cannot disagree about
what the column says. `NodeResponse` reports it only when it differs from `status`.
- The stale-window check that was hand-rolled in the placement loop is gone. Two copies
of one rule is how they drift; the loop now reads the same function as everything
else, which is what makes the console and the scheduler agree.
A sweeper was the alternative and is how expired leases are handled, so it was not a
strange idea. It loses because it is a periodic task that must actually run on a
deployment that has already been taken down three boots by a stray process - and while
it is not running, the stored value is wrong again. Computing on read has no window and
no background dependency.
## The document was wrong about what happens when a node stops
It said "miss that window and the node goes offline". That was false, which is how this
was found. The audio node followed the instruction, sent one heartbeat, and nothing
told it or anybody that a second one was expected.
`poc/umbra_stage1.py` is a loop rather than a description of one, so the audio node can
drop it in unchanged. One `curl` is one request: it returns to the prompt when done,
which is correct for curl and indistinguishable from a healthy node that never started.
The script runs in the foreground so closing the terminal stops it, which makes the
first few minutes obvious - press Ctrl-C and the node goes quiet, and about 45 seconds
later the controller says so. It exits on 401 and 403 rather than retrying once every
ten seconds for an hour, because a credential problem does not resolve itself, and it
sends an empty report rather than nothing when the report script is missing, since a
node that stops heartbeating because its report broke is indistinguishable from a dead
machine.
Verified against the live controller: three consecutive 200s at a 10s interval, and a
wrong key producing 401 and a clean exit.
Verified on the deployment: a node quiet for 50s reads `offline` while the column still
says `online`, and reads `online` again one second after a heartbeat.
488 passed, 27 skipped. The three staleness tests were confirmed to fail with the
derivation neutered, and the export test caught `effective_status` missing from
`__all__`.
|
||
|
umbra audio
|
2ff322dcd2 |
Close all thirteen app functions in the migration, not by hand
B4 was recorded as open against the deployment. It was not open there - the ten original functions had been closed by hand - but it was open everywhere else. A database restored from a backup of the deployment comes back with 10 of 13 PUBLIC-executable, because a hand-applied GRANT is a property of one database and does not survive a restore. Observed on scratch, built from the pre-0003 dump. So 0003 now revokes from PUBLIC on all thirteen, by full signature taken from pg_proc.oid::regprocedure: a bare () does not resolve for a function taking one argument, and REVOKE on a nonexistent name aborts the whole migration. Three findings while doing it, none of which would have shown up without running the tests: - My own three guard functions went in PUBLIC-executable, reproducing the exact defect B4 is about. - Revoking PUBLIC without regranting broke five tenancy tests with a permission denied error on the platform-admin helper. Every RLS policy calls it, so a missing grant here looks like a broken policy rather than a missing grant. Regranted to the roles that actually read and write, guarded by EXISTS so a role added later is covered by the next migration rather than by luck. - The revoke block originally sat before the functions were created, so it failed on a fresh restore and passed on the already-migrated database - passing by accident, the failure mode this project keeps hitting. Both revoke blocks now sit after the triggers, and the fresh-restore path is the one verified. Verified by restoring the pre-0003 backup into scratch, applying 0003, and running the suite: 0 of 13 PUBLIC-executable, 4 guards, 474 passed, 27 skipped. Re-applying is a no-op. Deployment data unchanged (users=2 orgs=3 audit=43 perms=20); health 200, heartbeat 401 without a credential and 200 with one, so the revoke broke nothing. G2 is still open: 26 tables have RLS enabled but not forced, and the application connects as the owner. |
||
|
umbra audio
|
c0a1971a52 |
Stage 1 credential issued; record what it was verified not to do
The doc previously said 'ask and we will issue'. It now carries the node id, the key prefix, the four observed checks, and the heartbeat payload shape. The key itself is not in the document. It is in a 0600 file on oldbox, because a credential in a git-tracked design document is a credential in every clone and in every paste of it. The two rows worth reading are the third and fourth: a heartbeat-only credential claiming a job is refused 403, and the job is still queued with no lease owner. The refusal happens at the scope check, before the job is touched - a refusal that had already half-claimed would leave the queue believing a node is working when it is not. |
||
|
umbra audio
|
0501739a88 |
append-only guards, and a fixture that could not set itself up
Three defects, found by applying 0003 to the deployment after a restore-tested backup. Each was caught by a check failing for a reason that had nothing to do with what it claimed to check. artifacts had no immutability guard, so content addressing was a convention. Its identity is the bytes: an UPDATE that changes content_hash or storage_key rewrites history for every job that consumed the row, and the lineage in job_artifacts would point at content that never existed. job_artifacts and job_events had the same gap - provenance and the event trail behind I25/I26, both editable in place. The guard on artifacts is UPDATE only, and that is a decision rather than an oversight. Guarding DELETE as well would make storage reclamation impossible: no artifact row could ever be removed, and the table would grow for the life of the deployment. The provenance guarantee does not rest on that guard. job_artifacts references artifacts ON DELETE CASCADE, so deleting an artifact any job consumed fires the guard on job_artifacts and is refused. An artifact nothing consumed is by definition nothing anyone's record depends on. Four cases are tested in both directions: each test was confirmed to fail when its trigger was disabled or widened, so none of them passes because its mechanism failed. test_rls_tenancy's fixture inserted a role with a fixed id and ON CONFLICT DO NOTHING. The unique constraint is on (scope, name), so the insert silently did nothing - organization_admin already existed under a different id - and the fixture then assigned user_roles to a role that was not there. validate_user_role_scope raised inside the fixture and all eight tests errored rather than failed, which reads as "the thing under test is broken". It now looks the role up and says so if the role is absent. test_append_only_guards.py needs no teardown: job_artifacts guards DELETE, so its rows cannot be removed, and fixed ids would collide on the next run. Ids are generated per run instead. The database is disposable by construction, which is what UMBRATRANS_TEST_DATABASE_IS_DISPOSABLE=1 asserts. 474 passed, 27 skipped, 0 errors. |
||
|
umbra
|
898906ab04 |
Forward-only link agreed, and the connection sequence the audio node asked for
D16: a book_local character revealed to recur keeps its released entity and gains a recurs_as edge, rather than being merged. Its cost is written down rather than discovered later - a retelling from the earlier book's point of view treats the two as different people. Chosen because the freeze is what protects reproducibility and a merge puts it at risk for a gain that is real but retrospective. Asked whether the document was ready to feed back and how the audio node connects. Reading it against that question found four gaps, all of which were mine: - One report skeleton was given for nine phases, which specifies nothing. evidence now has a tabulated shape per phase, so the reports are mechanically comparable rather than approximately so. - The identity probe said "we send you a fixed set" without saying how the set was fixed, which would have made phase 3 either unreproducible or impossible. Now: you pick the project, sort by source_unit_key, every 17th, cap 200, and report the stride. - Section 9 said "contact the controller on 192.168.123.14:8442" and stopped, which is how this session's own 400s happened twice. - There was no connection sequence at all. Section 10 adds it, in four stages. Stages 0 and 1 are possible today and carry no project data: Stage 0 is reachability, where a 401 from the node surface is the correct answer and the evidence; Stage 1 is the node report they already wrote, sent as a heartbeat. Stage 1 needs no schema change and no new endpoint, because node_heartbeats.raw is an open jsonb. Stages 2 and 3 are ours to build and are listed as such. Two transport facts stated plainly, both learned the hard way here. The deployment accepts exactly one Host header and returns 400 on everything otherwise, including health. And there is no TLS, despite the service description claiming "TLS terminated upstream by Caddy" - Caddy is inactive and its configuration is empty. Tolerable for a read-only node report on a LAN, not tolerable for the ingest path, so putting Caddy in front of the port is a Stage 3 prerequisite rather than an extra. It starts read-only for the same reason the compute substrate does: a node is online because it heartbeated, not because it was created. A mirror that begins with data cannot distinguish "the node is broken" from "the data is bad", and their speaker counts already disagree. One stale block survived a previous edit - a replace that matched nothing and reported success. It now asserts before replacing. |
||
|
umbra
|
717e1d56b9 |
Editions, character persistence, and a freeze anchored per character
Five more decisions, and the second one closes the last three questions plus reveals two things the model did not have. Not every character recurs. A subset persist across books and the rest are local to one, and persistence is *declared* rather than inferred - which turns out to bear directly on a data quality problem we already knew about. The three-way speaker-count disagreements may be partly different characters who share a name: "The Big One" in book one is not necessarily "The Big One" in book two. Declaring persistence gives a way to tell those cases apart instead of reconciling them into one row. I-A holds for every character; I-B is only meaningful for a persistent one, and vacuously true otherwise, which is correct rather than unmodelled. The freeze is anchored once, per character, at the release of the first book they appear in - not per book and not per series. New invariant I-G, which is stricter than I-F for a recurring character and identical for a book-local one. Capture is selective, which has one sharp edge. If not every character needs every aspect, then a missing personality artifact means one of three quite different things: not attempted, attempted and not applicable, or attempted and failed. Only the third is a defect, and a nullable model cannot tell them apart - so it reports a healthy system as incomplete or an incomplete one as healthy. Absence is therefore typed, not null, and the validator treats in_progress as never satisfied. Published editions change, and a correction, a new cover or a new foreword is a new release rather than an edit. So editions sit between the work and the artifacts, and projects.status = released moves onto the edition: the work is not released, it has releases. Content- addressing makes this nearly free - an edition that fixes a paragraph mostly reuses the same artifacts by hash, and only genuinely changed content becomes new. That resolves the obvious tension with the freeze: a correction in edition 2 that contradicts edition 1 is handled by a new attribute state only later editions reference, which is I-H. The freeze unit is a single work, so book 1 can be released while book 2 is still being written. That leaves one question the answers created rather than closed: a book-local character revealed to recur is two entities for one person, and merging them would alter attributes I-G has frozen. Forward-only link, or merge with a snapshot? Recommended the forward-only link, because the freeze is what protects reproducibility and the gain from a merge is retrospective. It is a question about the fiction. Two new validator phases: persistence and partial-capture, where a mismatch in declared persistence is a blocker and disagreement in counts is the expected finding; and an edition probe, where unchanged content must appear with the same hash across editions. Two lines in 1.5.2 still described the older per-work freeze and were corrected rather than left to contradict the new anchoring. 670 lines, 15 decisions, 9 validator phases, 8 invariants each checked to have a home, 2 questions open. |
||
|
umbra
|
21d0949b84 |
Character attributes change over the story: the largest model change yet
Four more decisions, and the last one reshapes the model. Personality is one thing per character: global across books and language-neutral, because the disposition the reader meets should not change with the language or the situation. Scope is the character alone, so `lang` is not part of a character_profile's identity. A translation that cannot carry the disposition is a finding about the performance, not a second personality - and modelling it as a second personality would make exactly that inconsistency undetectable. A voice is a matrix of parameter values, and a variant is a small documented change to that matrix. That answers the scope question better than anything the document proposed: same identity, different values, difference recorded as data. New invariant I-E, because a variant that changes an undeclared value is the failure mode. The large one: a work is mutable until publication and frozen after it, like software before deployment. Character attributes are defined in the first work and change only explicitly - John goes through trauma in book one, and in book two the situation costs him a leg. So attributes are time-scoped and changes are events, never silent edits. The trap here is one `characters.attributes` column, and the operator's own two answers already separate what it would have merged. Disposition is global and stable; circumstance is scoped to the timeline. The amputation is circumstance, and whether it alters disposition is an authorial judgement recorded as an explicit change, never a consequence derived automatically. If the two classes shared a table, those rules would collide in the schema and someone would resolve it by re-deriving personality from circumstance - destroying precisely the consistency that was designed. Three classes: identity, disposition, circumstance. The freeze needed no migration: `projects.status` already carries `released`, which is exactly the release boundary described. Added `series_works` for the ordering, `character_attribute_defs` for the classes, `character_attribute_states` append-only with an effective position, and `voice_matrix_values` plus `voice_variant_deltas`. Six invariants now, each checked to have a home rather than being an aspiration with no column behind it. Two new validator phases: a timeline probe where a circumstance change altering an accepted personality is a blocker, and a voice matrix probe for undeclared variant changes. One question left, because D9 says "a set of items must remain the same" without naming the set. We assumed the minimum that makes the rest work and flagged the assumption as load-bearing: do released character attributes freeze, or only the released rendering? Does a published edition ever change - is there an edition concept? And is the freeze unit a single work or the whole series? Corrected two stale rows now that D1 is answered: one still said DECISION-1 was unratified, and one still offered it to them. |
||
|
umbra
|
838733d51d |
Data model alignment: six decisions recorded, character model rewritten
The alignment document for the audio system now carries what the operator decided, and the character model is materially different from the one I first wrote. The correction that mattered: I had framed speaker and author as two names for one thing that might get conflated. They are not adjacent, they are a chain, and the expensive trap is a third thing. A character may be an author *inside the fiction* - a character who writes a book within the story. That is the character's occupation and it does not reclassify them; the book author can make a character an author and the character remains a character. So "author" appears in two unrelated places - the byline on a work, and a job inside a story - and merging them puts every in-story author on the credit pages. Decisions recorded: the controller is authoritative for data but does not specify algorithms; compute and capability design is deferred to its own phase and does not block this one; a character may belong to several series; a character has an LLM-derived personality; a character's voice is consistent across every utterance and every book; and a character may have additional voices, each designed, captured and stored with a reason. Three things fell out of that which the model did not have: - Personality is LLM output, so it is an artifact with lineage, not a field. Re-analysis creates a version; the accepted one is a pointer. - Speaker is derived - a character with lines in a work - so it is a flag on an appearance, not an entity. Modelling it as an entity is what forces every language to have its own cast. - Voice variants are passage-shaped, not character-shaped. All three of the reasons given are scoped to a run of utterances, so a variant needs a range or it silently becomes the character's voice everywhere. Four invariants now have a home, checked rather than asserted: one primary voice per (character, language); that primary voice keyed so no book can change it; no variant without a reason; personality never edited in place. Corrected a stale row in section 4 that still named the old table scheme, and re-checked that no invariant is an aspiration without a column behind it. Nine new tables are required and none is claimed as existing. Six of the open questions are answered and four remain, three of them sharper now: whether personality is per character or per language, whether it is global or genuinely differs per book, what a variant's scope is expressed in, and whether a source revision belongs to a project or to a series. |
||
|
umbra
|
a032e4f542 |
Record the GPU boundary as measured, on all three accounts
Fourth consecutive boot with all five nodes 0660 root:video, the timer firing, and the unit's journal showing the helper working - so the mechanism holds. What was still inference is now measurement. ollama and umbra-poc both report the RTX 3060, and umbra - which runs the controller - is refused. The device bits and the video memberships were identical to earlier boots where nvidia-smi had been run under those accounts, but identical configuration is not an observed result. Worth noting how this was reached: nvidia-device-access.sh show had been reporting 'could not check' for the two non-root accounts since the reboot, because there was no cached sudo credential. That is the honest answer, and it is why the direct run mattered - the tool had been accurately describing its own blindness while the boundary was being claimed from configuration alone. |
||
|
umbra
|
de08cfe38e |
Stop rate-limiting the deployment: any finite budget is exhaustible
The GPU permissions survived a third consecutive boot - all five nodes 660 root:video - so that is now established rather than hopeful. The deployment was down again for the same reason, and this time the widened start limit I committed two boots ago turned out not to have fixed it. It exhausted twenty restarts and returned to the same permanent failed, with the port free and nobody touching the service. That fix's verification missed it. The port was freed after three restarts, inside the window, so the test exercised the easy case and the commit said "fixed". The real case is a fault that outlasts the budget, which is the only case that matters. The rate limit is the wrong mechanism, not the wrong value. Any finite budget can be exhausted by a fault that lasts longer than it, and the failure is silent and permanent - `failed` looks like a decision rather than a give-up. StartLimitIntervalSec=0 disables it: the unit retries for as long as the fault lasts and binds the moment it clears. Verified against the case that broke it twice. Held the port for 200 seconds, which drove 31 restarts - past the old limit of 20 - with the unit never reaching failed. Freeing the port gave 200 from /api/health within eight seconds, with nothing having started it. A genuine crash loop now logs every 5s instead of stopping: noisier, and much better than a deployment that is quietly down. 469 pass with a database, 428 without. Deployment healthy, GPU boundary intact, database unchanged at users=2 orgs=3 projects=0 audit=43 with no compute or datastore tables, ollama serving, persistenced up, perms timer firing, zero stray agents. |
||
|
umbra
|
1dbf5fb384 |
The GPU permissions survive a reboot, and the deployment stops giving up
Second boot since the permission fix, and it held: all five nodes 0660 root:video, the timer firing on schedule, and umbra - which runs the controller - still refused the card. That claim is now an observation rather than an inference. The same reboot surfaced an unrelated fault worth fixing. The deployment did not come up: a `python3 -m http.server` from a development session held port 8442, serving web/ - which the backend mounts itself and does not need - and the service failed five times in ten seconds. With StartLimitIntervalSec=120 and StartLimitBurst=5 that exhausted the budget, and systemd then never tried again. The deployment stayed down after the port was free and needed someone to notice, which is the second reboot in a row to end this way. A transient condition had become a permanent outage. Widened to 500s and a burst of 20, which keeps retrying across a slow boot or a settling reverse proxy while still stopping a genuine crash loop. Verified by holding the port with a squatter and watching: the unit cycles active/activating without giving up, and the instant the port is freed the next retry binds and /api/health returns 200. My first attempt at this test was void - the squatter could not bind because the service still held the port, so it proved nothing. Note that two consecutive reboots have been taken down by a background shell in this session holding the deployment's port. It returns after each boot. Stopping that shell, or moving it to another port, is the durable fix; killing it again only defers it. 469 pass with a database, 428 without. Deployment healthy, database unchanged at users=2 orgs=3 projects=0 audit=43 with no compute or datastore tables, ollama serving, agent account intact, zero stray agents. |
||
|
umbra
|
4937b9beed |
Record that the GPU permissions now hold, and how that was verified
Cleared the unit's stale active(exited) state so the corrected version could run, then broke the permission deliberately the way nvidia-persistenced does and watched the timer repair it: 666 -> 660 root:video, with the helper reporting 'restricted 5 node(s)' in the journal. ollama and umbra-poc can measure the GPU; umbra, which runs the controller, cannot. Deployment healthy, ollama serving on 11434, database unchanged. Records what is still unverified: no boot has been observed end to end since the change. The unit, timer and udev hooks are installed and exercised individually, and the persistenced interaction is handled by ordering rather than luck, but 'works after a reboot' is a claim until a reboot confirms it. show answers it in one command. |
||
|
umbra
|
0ab1f0f1de |
The GPU boundary did not survive a reboot, and neither did the fix for it
This is the verification I said could not be done until the machine rebooted. The reboot proved the boot path wrong, and uncovering why took three rounds. **The boundary was gone.** /dev/nvidiactl came back 0666 root:root, so the controller could open the card again - the exact thing the udev rule was applied to prevent. **Why: the unit ran before anything existed.** umbra-nvidia-devices ran at 05:38:24, ordered After=systemd-udev-settle, globbed /dev, found no nodes, and exited 0. udev-settle means "udev has finished its queue", not "the proprietary driver has finished initialising". The nodes appeared moments later at 0666 and nothing revisited them. The helper ended in an unconditional `exit 0` - I had written that deliberately, so an early boot would not fail a unit - and it is precisely what made "I fixed the permissions" indistinguishable from "there were nothing to fix". **Why the first repair also failed: nvidia-persistenced resets the mode.** It chmods /dev/nvidiactl back to 0666 on every initialisation, via nvidia-modprobe. That is out of udev's reach entirely, because udev does not manage these nodes. So the fix is no longer an ordering problem but an eventual one: the unit now runs After=nvidia-persistenced.service, and a two-minute timer re-asserts the permissions because the trigger that breaks them can fire at any time. **Why the timer did nothing either.** RemainAfterExit=yes on the oneshot keeps it "active (exited)", so systemd treated every activation as already satisfied: the timer fired on schedule, the service reported Result=success and ExecMainStatus=0, and the permission was still wrong two minutes later. That is the original bug repeated one level out, inside the mechanism built to repair it. Removed, and a test now pins it - by directive name, with comments stripped, since the unit documents the omission by naming it and a substring search failed on the explanation. **I also broke nvidia-persistenced.** It drops to uid 122 with setuid and without initgroups(), so it never receives the video supplementary group and failed with "Insufficient Permissions" once the nodes were 0660 root:video. A systemd drop-in runs it as root - upstream's own default - which works without widening the device permissions. It was running fine before. The helper now waits for the nodes, verifies the modes afterwards, and exits non-zero under --strict rather than reporting a no-op as success. It takes UMBRA_NVIDIA_DEVDIR so its three paths can be tested without root and without a GPU; the first attempt at that test pointed CANARY at a non-existent path and still exited 0, because the fix loop globbed /dev directly and the override only reached the wait. Also: nvidia-device-access.sh `show` reported "cannot measure the GPU" when a cached sudo credential had simply expired - a finding about the machine when the truth was that the tool could not ask. It now distinguishes no from could-not-check, because I was misled by it during this check. **Still outstanding, and I could not finish it.** The repository is correct; the installed state is stale, because clearing the unit's active (exited) state needs root and that authorization was declined. As of now /dev/nvidiactl is 0666 and the controller can open it. To finish: systemctl stop umbra-nvidia-devices.service systemctl start umbra-nvidia-devices.service stat -c '%a %U:%G %n' /dev/nvidiactl and expect 660 root:video. Otherwise the system came through the reboot intact: the deployment was down only because a leftover `python3 -m http.server` from an earlier session was holding port 8442, and the backend mounts the same files itself. Deployment healthy, database unchanged at users=2 orgs=3 projects=0 audit=43 with no compute or datastore tables, ollama serving, agent account intact, zero stray agents. 471 pass with a database, 430 without. |
||
|
umbra
|
1ab9c2595d |
Verify the datastore policies as a role that does not own the tables
They had never been observed to refuse anything, and until now every
statement this project made about tenant isolation was a statement about
the adapter's own WHERE clauses.
The reason was structural. RLS does not apply to a table's owner until
FORCE ROW LEVEL SECURITY, and every connection here is the owner. A probe
run as the owner showed organization B's administrator successfully
updating organization A's imprint - exactly what WITH CHECK was written
to prevent. A policy that cannot refuse is indistinguishable from a policy
that is absent, which is why the previous commit said so plainly rather
than claiming the migration was verified.
test_rls_tenancy.py runs as rls_probe, a role with plain table privileges
and no ownership. Eight tests, each checked against a mutation: each
organization sees only its own artifacts, an unattached principal sees
nothing on any datastore table, a publisher is readable across tenants but
writable only by its owner, a cross-tenant UPDATE changes zero rows, and a
cross-tenant INSERT is refused. Two of the eight also assert the positive
cases, because a policy that refuses everything would satisfy every
refusal above.
Two behaviours asserted so a change is noticed: a filtered UPDATE or
DELETE raises nothing and simply affects zero rows, while an INSERT
violating WITH CHECK raises InsufficientPrivilege. The first is a trap
for application code that checks "did it raise?" instead of "did it
change a row?", and it is why the earlier manual probe reported a
cross-tenant update as "allowed" when the statement had matched nothing.
Two mistakes in the tests, both worth recording. A policy violation aborts
the transaction, so the second refusal in a test could not even be
attempted and reported as "DID NOT RAISE" when nothing had run; each
assertion now gets a savepoint. And a module-level pytest.skip() is a
collection error rather than a skip, which broke the whole suite instead
of quietly omitting the file - the guard now evaluates to a boolean.
This does not close gate G2. The application still connects as the owner,
so in the running system these policies still enforce nothing. What is
established is that they are correct, which makes FORCE ROW LEVEL SECURITY
plus a non-owner application role a configuration change with a
known-good target rather than an untested hope. README.md says exactly
that, and the gate stays open.
Creating the role needs superuser and was authorized this time:
sudo -u postgres psql -d umbratrans_scratch -c \
"CREATE ROLE rls_probe NOLOGIN; GRANT rls_probe TO umbra;"
465 pass with a database, 424 without. Deployment untouched: users=2
orgs=3 projects=0 audit=43, no compute or datastore tables.
|
||
|
umbra
|
5fc38d4708 |
The datastore, and two parties that never sign in
Compute capability is not LLM work. It is mechanical split, analysis,
text to speech, PDF formatting and audio analysis, assigned as needed,
with the code on every node. That needs a capability vocabulary rather
than VRAM - which is noted and not yet built - and, more urgently, a
place to put the output. jobs.result is a jsonb column: right for
{"echo": "hello from the queue"}, wrong for "the English audiobook, four
hours twelve minutes, derived from these thirty-seven chapters and this
cast".
So 0003 gains section 5: artifacts, job_artifacts, publishers,
project_publishers, authors, project_authors, and eight permission
keys. Folded in here rather than deferred to 0004 because 0003 has still
never been applied to the deployment, and an unapplied migration can
still be edited.
The word "account" is now absent from every new column, table and
comment. It has meant two incompatible things in this system - a person
who signs in, and a house a work is published under - and a schema
using one word for both will use it for both in one constraint. An
organization owns the work. A user is a worker and owns nothing. A
publisher is the party a work is published under and has no password, no
session and no row in user_roles, because a name that appears on a work
is not a party that can be asked to prove who it is.
A byline is a third party and is neither. In Umbra -> WorstAuthor ->
B.D. Murphy, WorstAuthor is the publisher and B.D. Murphy is the
byline; different lifetimes, and the spec had been calling both "brand".
project_authors carries a credit_role rather than a single author_id,
because the author of the English edition is the translator of the
French one and the narrator is frequently neither.
owner_organization_id is the one deliberately cross-tenant column, and
the policies are asymmetric about it: reading a publisher is open to any
authenticated principal because publishers are shared reference data,
while setting that column is refused to anyone but a platform
administrator. There is no second policy carving that out, because
PostgreSQL combines permissive policies with OR and an "admins may also"
policy widens access for everyone. The WITH CHECK on the single FOR ALL
policy is doing that job alone.
Found while testing rather than while writing: RLS was not enabled on
four of the six new tables. PostgreSQL creates a policy on a table where
it cannot apply and reports success, so pg_policies listed them and
nothing was enforced - and a check for "does the policy exist" would
have passed throughout. test_rls_coverage.py asserts relrowsecurity
instead, and each of its six assertions has been checked against a
mutation that breaks it. One needed strengthening: dropping the
dedicated SELECT policy left the table looking fully policed, because
FOR ALL includes SELECT, so the surviving policy quietly gated reading on
organization ownership and hid shared reference data.
Also: the nvidia tests failed for want of a sudo credential rather than
because anything was wrong. They skip now, with the reason stated. A
test that fails for environmental reasons teaches the reader to ignore
red.
The policies have never been observed to refuse anything, and README.md
says so plainly. A probe as the owner showed organization B's
administrator editing organization A's imprint - what WITH CHECK was
written to prevent - because RLS does not apply to a table's owner until
FORCE ROW LEVEL SECURITY. That is gate G2, not a new problem. Testing it
for real needs a non-owner role and superuser; the request was not
granted, so it is not done.
455 pass with a database, 422 without.
|
||
|
umbra
|
c1741456e2 |
Apply the GPU boundary, and fix three checks that verified themselves
The udev rule I wrote last commit could not have worked. The nvidia driver creates /dev/nvidia* in the kernel without registering them in sysfs, so `udevadm info --name=/dev/nvidiactl` answers "Unknown device" and udev never evaluates a rule for them - which is also why they are 0666, since nothing sets a MODE. The rule hooks the driver bind event instead and runs a script, by which point the nodes exist. The first version of that hook triggered 0000:01:00.0, which on this machine is an AMD root complex, vendor 0x1022 rather than 0x10de. The trigger fired on the wrong device, the RUN key never ran, and the permissions looked right because the apply script had just fixed them itself. The address is discovered now. Verified by resetting to 0666, firing only the bind event, and watching it come back 0660 - the script never invoked. Applied and verified: ollama can open the card, umbra-poc can, umbra cannot. Traced, nvidia-smi opens /dev/nvidia0, /dev/nvidiactl and /dev/nvidia-uvm, so umbra-poc is in video or the agent starts, heartbeats, and reports a machine with no GPU - a failure that presents as a scheduling problem and is a permissions one. It is the agent's only supplementary group and grants a device, not a filesystem, so setup's "no supplementary groups" check now allows exactly video and rejects everything else, verified by adding users and watching it fail. Three tests, each verified against a mutation. The rule test resets the permissions and fires only the event, so it fails if the rule is inert. The end-state test fails rather than skips when the rule is missing - skipping made deleting the rule quieter, three passed becoming one passed and two skips, which reads as an improvement in the wrong direction. Also: the check "the controller cannot open the card" was testing root, which bypasses permissions entirely, and reported a working rule as a failure. The account is named now. Then four runs in a row leaked agents, and the cause was upstream of the leak. The victim reused the first agent's credential, and a node is identified by its credential, not by its config's node_name - so it heartbeated as poc-agent-node, was never offered the long job, and sat there for the whole step while a correct liveness check reported that the worker it had killed had survived. It has a third node and its own credential now, so the requeued job is not immediately reclaimed by an agent already stopped for the step. The last leak was a worker asleep in a 600-second job, unable to see a stop file because the check is at the top of its loop rather than inside the work. Four runs left one each. The job is 10 seconds now: the victim exits the instant it claims it and never runs it, so the length was never the point. PoC node names and idempotency keys carry a per-run suffix, so a second run against the same database does not die on a UniqueViolation - and there is no route to delete a node. Three consecutive runs, zero failures, zero leaks. 451 pass with a database, 424 without. Deployment untouched: users=2 orgs=3 projects=0 audit=43, no compute tables. |
||
|
umbra
|
220bc00823 |
An upgrade path for the agent, and the GPU device boundary
Two questions worth answering properly rather than by assertion. How an agent is updated. `setup` has always doubled as an upgrade - it installs the file unconditionally, which is most of what an upgrade needs and the one thing it should not do: a broken file replaces a working agent on every node with no way back. `update` names the version being replaced, keeps the previous file, and verifies that the new agent *starts as umbra-poc* rather than that it parses. A file that imports cleanly under umbra's virtualenv may not be readable by the account that runs it. Verified against a deliberately broken agent: installed, detected, previous restored. The agent does not update itself. An agent that can fetch and execute code has a remote-code-execution channel, and the point of the account is that it holds no credentials and reaches nothing. Updates are an out-of-band deploy per node. Nothing enforces a minimum agent version either - the console can report "behind", but refusing a heartbeat would strand a node mid-job for tidiness. The GPU. /dev/nvidia* is 0666 root:root because the driver creates it that way and its own 71-nvidia.rules sets no MODE or GROUP. Measured, not assumed: nvidia-smi -L succeeds for umbra, ollama and umbra-poc alike - so the controller can open the card today, and the only thing stopping it from loading a model on top of the node's work is the scheduler choosing not to. 72-umbra-nvidia.rules sets 0660 root:video, sorting after the driver's rule so it wins. video already contains ollama, which is right: ollama is the account that opens the GPU. umbra-poc must be added to it, and traced nvidia-smi opens /dev/nvidia0, /dev/nvidiactl and /dev/nvidia-uvm - so without the membership the agent starts, heartbeats, and reports a machine with no GPU. That fails silently, as a scheduling problem rather than a permissions one. It is the agent's only supplementary group and grants a device, not a filesystem, so it is added by the apply script rather than by setup. What it does not buy: separation between two agents. Both are umbra-poc, both in video, both could open the card. On one box with one GPU, placement is the control and the kernel is not. What it buys is that every other account stops being able to touch the card, the controller above all. apply verifies the rule took, that ollama and the agent can still measure the GPU, and that ollama still serves models - and reverts itself if any of those fail. Also fixes a check this session broke: a sed meant for new code rewrote verify's agent-start line to run a command called `x`, so it had been failing on every run and I read past it. `bash -n` cannot see that class of damage; verify did, which is the argument for it. |
||
|
umbra
|
0f5c1e3408 |
Run the agent as the service account it claims to be
The compute proof of concept had never been run with the agent as umbra-poc. Running it that way found three defects, all the same shape: a check that passed because its mechanism did nothing. The demonstration never reached the agent at all. It held the pid it spawned, which under `sudo -n -u` is the sudo wrapper - sudo forks the agent before exec'ing. Signalling that killed the wrapper and left the worker running, renewing its lease and holding the job indefinitely, under a printed note saying "exactly like a power cut". An unprivileged process cannot signal another account's process at all, and kill(2) on a process group succeeds as soon as one member is signalled, so the wrapper's death read as the agent's. Every run leaked an agent. Lease recovery was a diagnosis rather than a mechanism. sweep_expired_leases documented itself as "called by the controller's own timer" and there was no timer; a killed node's job came back only when somebody noticed a stalled job and posted to an administrative route by hand. The proof of concept swept by hand and recorded the result as automatic. app.py now runs the loop for the life of the application, building the compute repository at startup so a restart mid-job still recovers it. The power cut is now the agent's own: os._exit(137) the instant it claims a job, before running it, because a process that does not own the agent cannot signal it. A clean stop goes through a stop_file the agent polls, for the same reason. Liveness is measured by finding the worker's own pid in /proc matched on its config path, sampled repeatedly and accumulated - the wrapper and the worker are not simultaneous, so a single sample finds only the wrapper. The node key travels by `sudo --preserve-env` rather than as VAR=value in argv, because argv is world-readable through /proc/<pid>/cmdline and the environment is not. The demonstration checks whether the key will survive before starting and falls back to a pipe, which node_agent.py --node-key-stdin accepts. Neither a file nor argv ever holds it. Eight tests, each verified to fail against a mutation of the code it covers: three for the sweeper (it must return an expired lease, must survive a failing round, must not invent a repository it has no database for) and five driving a real agent process against a stub controller (stops on the stop file, exits 137 holding an uncompleted job, takes the key from both routes and neither, reports an empty pipe as an empty pipe). 448 pass with a database, 421 without. The deployment database is untouched: users=2 orgs=3 projects=0 audit=43, no compute tables. |
||
|
umbra audio
|
1ab0fa2498 |
A service-account setup and boundary check for the node agent
The agent is stdlib-only - no package imports, no third-party dependencies - so it needs no virtualenv and no access to the controller's home directory. That is the whole reason the setup is short: an account with no group memberships at all, running one file from /opt on the system python. poc/poc_node_user.sh has four subcommands. setup needs root and creates the account, installs the agent, and writes a config that deliberately contains no key. verify asserts the boundary. run does both. status reports what exists and what is reachable, needing no privilege at all. The check that matters: the agent must not be able to read ~/.config/umbratrans/umbratrans.env. It cannot, and there is a second layer behind it - peer authentication maps a unix user to a database role, and umbra-poc has none, so even a leaked DSN would not connect. The two are independent and neither alone is the boundary. Writing verify produced the third instance of a bug class that has come up repeatedly in this work: a check that passes because its mechanism failed. When sudo -u could not run, every positive check correctly failed and every negative check passed - because `test -r` on an unreadable file returns false, which is exactly what "correctly denied" looks like. The script would have reported a boundary as holding while testing nothing. It now confirms the privilege drop once, up front, and abandons the run rather than reporting; status prints "unknown (no privilege drop)" rather than "no". Found by running the checks against an account it could not drop to, which is the only way to see it. Also: the agent's config now travels in a 0750 directory, because mkdtemp gives 0700 and a different user could not have read its own config out of it. The script is dry-run against an absent account, an existing unprivileged account, a missing controller, and a rootless setup invocation. It has not been run end to end, because umbra-poc does not exist yet. |
||
|
umbra audio
|
d13664a607 |
A compute substrate: node inventory, a leased queue, and an agent
Implements §6.16-6.20 and §7 of the specification, and proves it end to end on this box: a controller and an agent, the agent with no database credential at all, ten checks against the properties whose failure modes are silent. Migration 0003 adds compute_nodes, compute_devices, node_credentials, model_registry, node_models, node_heartbeats and job_events, and creates jobs - which §6.8 had specified and 0001 had never migrated. Every column added to jobs is nullable or defaulted, every policy is written against the same app.* helpers the existing eleven tables use, and re-applying is safe: CREATE POLICY has no IF NOT EXISTS, which is how 0001 learned the same lesson for ADD CONSTRAINT. Four properties, and each exists because the alternative fails silently: - **I25** claiming is FOR UPDATE SKIP LOCKED. Two agents claiming at once get different jobs. The proof of purpose counts the 'claimed' events rather than counting successes, because two agents both succeeding is not evidence. - **I26** a lease that expires returns the job to the queue, and the proof of purpose SIGKILLs the holder rather than stopping it, because a clean stop would release the lease and a clean stop is not the failure being tested. - **I24** a node is online because it heartbeated. Creation is not readiness. - **I28** a node credential reaches queue operations and nothing else. The organization comes from the credential, so a forged tenant identifier has nothing to attach to. And the one that is hardest to get right in the cheap direction: **a device that is slow is eligible, and one that is too small is not.** A model that does not fit in VRAM runs on system RAM, slowly, and a job nothing can run is 'blocked' with the reason rather than left looking like it is progressing. The proof of purpose found six defects, listed in §7.8. The two worth naming here: - Mounting one router at two prefixes published the administrator's routes behind node-credential authentication, so a node could create nodes and issue itself more authority. Two routers now, with a test that pins the surfaces disjoint and the node surface exactly. - **The correction I was given about unified memory was in the specification and not in the code.** Eligibility was VRAM-only, so every CPU and every shared-memory machine was ineligible, which would have reduced the fleet to one machine. Writing the implementation is what kept the error alive. Also: the CSRF middleware blocked the whole node API, because a bearer-authenticated agent has no ambient cookie for a cross-site request to ride on. Exempted by prefix, with a test asserting that no cookie-authenticated route joined it. 419 tests pass with the integration suite against a real database. The deployment database is untouched and has no compute tables. |
||
|
umbra audio
|
bfde763f00 |
Fix a 500 on /api/health from an uninterpolated SQL placeholder
The first deploy of the previous commit took the health endpoint down. The
tenant-row predicate had been added to `count_users_with_role` as a literal
`{TENANT_ROW_PREDICATE}` inside a plain triple-quoted string rather than an
f-string, so the braces were sent to PostgreSQL as text and the statement failed
to parse. That query backs `setup_is_required`, which every unauthenticated
health probe calls, so `/api/health` returned 500 and the deployment reported
itself broken.
Nothing caught it before production, and the reasons are worth keeping:
- the unit tests use the in-memory repository, which has no SQL;
- the integration tests did not call `count_users_with_role`;
- and the source-level test I wrote to pin the predicate's presence was
satisfied just as well by the broken version, because the broken version does
contain the text. A test that checks a predicate was *written down* cannot tell
a working predicate from an inert one.
Two tests now, each catching the regression on its own:
- `test_no_sql_string_carries_an_uninterpolated_placeholder` walks the parse tree
rather than the text, so a `{...}` in a non-f-string fails whether or not the
surrounding code mentions the constant. It reports the offending query, not a
line number: `inspect.getsource` returns a class body lifted out of the file, so
its line numbers do not point anywhere a reader could open.
- `test_the_health_probe_queries_run_against_the_real_schema` runs the probe's
own queries against the real schema, and
`test_setup_is_required_ignores_project_assignments` holds the predicate's
purpose: a project row is never mistaken for a platform administrator.
413 tests pass with the integration suite; 388 without a database.
|
||
|
umbra audio
|
ed72f61975 |
Separate organizations, and give members their organization's projects
An organization's work is separate from every other organization's, and the separation is now one module rather than a property each route remembers. `tenancy.py` resolves the caller's scope in a single place. A member is pinned to the organization on their own assignment, expressed as a concrete identifier rather than a flag, so "no filter" is unreachable except for a platform administrator. Every read and write resolves through it, so there is no per-route variation to drift. Within an organization, every account sees every project. The brief is explicit that two people working on different books both see both, and assignment says who is *working* on a project rather than who may *know* it exists. This diverges from `app.can_access_project`, which restricts visibility to assigned members; the divergence, and what it means if row-level security is ever switched on, are recorded in migrations/README.md because that would otherwise be a silent tenant-internal regression no test here would catch. Two rules, both learned by getting them wrong first: - A read naming another organization is ignored, because refusing it would confirm the identifier exists. A write is refused rather than retargeted, because a project landing in an organization the caller did not ask for is somewhere they will not look. - A missing project and a foreign one get the same error, so an identifier cannot be probed across a boundary. Projects: the `projects` table, its statuses, and the project-scoped arm of `user_roles` were already in the schema and unused. No migration. The dashboard is one server-scoped call, `/api/workspace`, rather than several assembled in the browser — pieces from different tenants that nobody notices are not a failure mode worth shipping. Filters narrow the list already in the page, so switching cannot mix two fetches, and an empty list is distinguishable from a failed one. **A sign-in lockout, found by the integration suite.** An assignment is a second active `user_roles` row, and `_resolve_assignment` refused an account with more than one. So assigning anybody to anything made them unable to authenticate — locked out of the page the assignment was meant to show them. `_sync_assignment` had the same defect. Six queries were scope-blind; three were harmless only because the assignment role is named `viewer` and never matched a tenant role name, which is luck rather than a property, so a structural test now pins the predicate on every one. The guard against two *tenant* rows survives, narrowed to the ambiguity that actually matters. Also fixed: `get_project_by_slug` cast `slug::text`, making citext's case-insensitivity a case-*sensitive* comparison that appeared to work everywhere else. An assignment grants the `viewer` project role, since no `member` role exists at project scope; it is a named constant, not a query literal, because the project workflow is not defined yet. 410 tests pass with the integration suite against a real database; 386 without. |
||
|
umbra audio
|
b12c702c66 |
Show whether an account's address is confirmed, and let an admin re-send
The accounts page now has an Email column carrying the address and its verification state. A confirmed address shows a plain marker; an unconfirmed one shows a button that sends another confirmation link. The field was already on the response, and it was lying. AccountResponse.is_email_verified defaults to True and from_view never set it, so every account in the list reported itself as confirmed, invited ones included. A badge that always reads "Confirmed" is worse than no badge, and about to hang a re-send control off it made that worth fixing first. It is passed explicitly now, with a test covering an invited account. The address moves out of the Account cell into its own column, because the verification state is a fact about the address and reads better next to it than stacked under a name. The badge is a button only when unverified. Offering to re-send a confirmation for an address that is already confirmed would imply a pending one might exist, and would put a live control on every row for no reason. The re-send posts to the endpoint the sign-in page already uses, which answers identically whether or not an account is awaiting confirmation. That is deliberate: the console holds a list of accounts, so "is this address verified?" is exactly the question that should not be askable of an arbitrary address, and an administrator-specific endpoint would answer it precisely. A test asserts the two responses are byte-identical. Verified end to end against a scratch database: an invited account renders as "Not confirmed", the control sends a verification link rather than a reset link, and an address with no account produces the same message. |
||
|
umbra audio
|
fa781cf708 |
Stop the reset page calling a dead link when it cannot reach the service
The reset link was reported as "no longer valid" while it was still live.
The token in the store was unconsumed and had not expired, so the link was
fine and the page was wrong.
Two mistakes, and the second is the one that mattered.
The first was a double question mark. passwordResetAvailable did
'/api/auth/reset-password?' + buildQuery(...), and buildQuery already
returns a leading '?', so the request went out as:
GET /api/auth/reset-password??token=... -> 422
The server read an empty parameter name and rejected the request, which is
correct. Now the call site appends buildQuery's output instead.
The second is why it took this long to find. The page had one failure
path, so every outcome became "this link is no longer valid": a malformed
request, a 500, a dropped connection, and a genuinely expired link were
all reported the same way. That is a confident false statement to make to
someone holding a good link, and it is exactly what happened.
There is now a third state, shown when the service could not be asked. It
says the check could not be completed and that this is not the same as the
link having expired, because it is not. Only the server's own "not
available" answer claims the link is dead, which is what keeps the
non-enumeration property intact: the page still cannot be used to learn
whether a link ever existed.
Two tests. One asserts no request site adds a '?' to buildQuery's output,
and that the named method uses the right shape; reverting the fix fails it.
The other asserts the unreachable state exists and is distinct.
The token from the earlier attempt is still valid until 06:57, so the same
link can be used once this is deployed.
|
||
|
umbra audio
|
e034061ec1 |
Add password reset and self-service account settings
An administrator could not reset a password for bdmurph73@gmail.com, and there was no way for a signed-in person to change their own. Both are now in place. Migration 0002 gives single-use tokens a purpose. 0001 reused the control plane's unused bootstrap_tokens for email verification, and the table has no notion of what a token is for, which was invisible while there was only one kind. With a second kind it stops being invisible: a reset link clicked by anyone holding it would confirm an address, and a verification link would set a password. The column defaults to the existing purpose, so no row is rewritten and verification is unchanged. Every token lookup now filters on purpose, and the adapter reads the purpose from the row rather than hardcoding it. An administrator may start a reset but never finish one. The link goes to the address on the account, only its holder can complete it, and the endpoint returns a bool rather than a password. An administrator who can set someone's password can also sign in as them indefinitely. A reset revokes every session, because a reset is what someone reaches for when they think their password is compromised, and revoking only the session that completed it would leave a stolen cookie working. Reset links live one hour against the two days a verification link gets. A verification link confirms an address the person already controls; a reset link is a live credential for as long as it is valid. Three bugs found while building it, all mine: - request_password_reset did not call _authorize_target, so an organization administrator could start a reset on an account in another tenant. Caught by a test written for the scoping rule that revoke_sessions already followed. - The in-memory repository never checked token expiry, only PostgreSQL did. So an expired link was honoured in every test and refused in production - the wrong direction for a divergence between two stores of the same rule. - The PostgreSQL adapter's get_live_verification_token took a purpose but not the now its in-memory counterpart needed, so the first call against a real database raised TypeError and returned a 500. The conformance test only checked that a method existed. It now compares signatures, which immediately caught that the protocol itself was missing the parameter. The reported failure was a deadlock rather than a missing feature. The account was invited: it could not sign in because the address was unverified, and I refused to reset anything not yet active. It is now allowed, and completing the reset also confirms the address, since holding the link proves control of it. Without that the flow would not converge - reset the password, sign in, refused again, using a verification link that was never sent. Suspended and deactivated accounts are still refused, because a reset restores sign-in and would otherwise undo a suspension without going through the status controls. Verified live: the account that could not be reset is now active, verified and able to sign in, with password.reset_requested, password.reset_completed and email.verified all in the audit trail. A reset token is refused by the verification endpoint, an invented token gives the same answer as a used one, and a token in a reset URL is redacted from the access log. 327 tests pass, 337 with the integration suite. |
||
|
umbra audio
|
7696c9d2e0 |
Open the organization editor when the edit button is used
The "Edit name / slug" button on the organizations page did nothing. It calls openOrganizationEditor, which filled the dialog in and returned without displaying it. Every sibling helper in the file - the account status dialog, the revoke dialog, the organization status dialog - opens its modal, so the name was carrying a promise the body did not keep. It went unnoticed because the "New organization" button works, and that one is a static [data-open] control whose handler calls openOrganizationEditor and then opens the modal itself. The row button is built in JavaScript, so the handler bound to [data-open] at startup never saw it, and nothing else displayed the dialog. Both paths share the helper, so one worked and one did not, which is what made it look like a dead button rather than a missing feature. The editor now opens the modal as well as filling it. The generic handler still calls open() afterwards, which is a no-op because the modal guards on being open already. A new test asserts that every helper named open* in the file opens a modal, so the next one cannot break the same way. Verified it fails with the fix reverted and passes with it. 302 tests pass, 311 with the integration suite. |
||
|
umbra audio
|
60aa2c9ba8 |
Keep verification tokens out of the access log
Checking the deployment after first-run setup turned up a leak I had not looked for. Uvicorn's access log records the full request line, so a request to /verify-email.html?token=<secret> wrote the verification token to the journal verbatim. Five of them were sitting there. Anyone able to read the journal could confirm an address they do not control, which is the one thing email verification exists to prevent. The tokens are single-use and expire, so the window is short, but it is a window and it did not need to exist. The redaction filter was installed only on the application's own loggers; uvicorn's is a separate tree. The filter now covers the server's loggers as well, and two things were needed rather than one. Existing loggers are patched directly, and a `_RedactingLogger` is installed as the logger class so loggers created later are covered too, because Python does not apply a parent's filters to a child's records. Without the second, a module imported lazily after startup logs in the clear, silently. The first attempt at the filter broke the log format: rendering the record, redacting it and clearing the arguments scrubbed the token but cost uvicorn's level prefix and trailing status text, so access lines became `Message: '...' 200`. Arguments are now redacted in place and the format string is left alone. One case still collapses the record, when the format string itself names a secret as in "created password=%s", because redacting it would consume the %s and break the substitution; that is confined to application loggers, since the server's own format strings carry nothing secret-shaped. Verified live: a request with a distinctive marker in the query string logs as `?token=[redacted]`, the marker appears nowhere in the journal, and the line keeps its prefix, request and status. Ordinary query strings are untouched, since a log nobody can read is not a log. |
||
|
umbra audio
|
b27915bc68 |
Fix the confirmation link, and stop the test suite eating a database
Two separate problems, one of them mine and serious. The confirmation link did nothing. `request` in api.js refuses a state-changing call with no CSRF token *in the browser*, before fetch is reached, so no request was sent and nothing appeared in the service log. The confirmation page is the only page that arrives without having made a safe request first, because it is opened straight from a link in an email, so it was the only one with no token. Added `ensureCsrfToken`, which fetches one if there is none, and called it before the POST. The alternative was exempting /api/auth/verify-email from CSRF the way login and first-run setup are. Those are exempt because nothing about them relies on a session; this one is state-changing and a session can exist, so the exemption list stays short. Verified against a live token: page loads, no cookie, priming GET, then POST returns 200 and the account becomes active. Replaying the same link then returns 400, so single-use still holds. The second problem is worse. This commit's earlier state truncated users, organizations and audit_events after every test, and I ran it against the deployment database. It destroyed the two accounts and the organization that were in it. My own "leave the database clean" change from two commits ago caused it: TRUNCATE removes the suite's rows and every other row equally, and the check that had been passing only asserted the suite's rows were gone, which is also true of a full database. It passed because I had just emptied that database by hand, so it never tested the case that mattered. Tried fixing it by deleting selectively instead. That cannot work either: audit_events has no ON DELETE CASCADE, and app.prevent_audit_mutation raises on delete, so the rows cannot be removed one at a time at all. So the safety is not in cleaning up carefully, it is in refusing to start. Two guards: the DSN must be accompanied by UMBRATRANS_TEST_DATABASE_IS_DISPOSABLE=1, because a database name is not evidence and an empty database is indistinguishable from an unconfigured deployment; and the suite then refuses to run if the target holds any rows, naming them. Verified both ways, with a planted row that survives. The deployment database is cleared and first-run setup is open again. |
||
|
umbra audio
|
2b3a093e96 |
Configure the domain mail transport and run in production
bdmurphy.me's MX records point at privateemail.com. Its submission host is smtp.privateemail.com:587 with STARTTLS, offering PLAIN and LOGIN over TLS 1.3. Not the MX host: an MX serves inbound mail and its submission ports are closed, so the obvious place to look has nothing listening. A test message to umbra@bdmurphy.me was delivered, and so was a real confirmation message through the provisioning path, leaving the account in `invited` awaiting confirmation. Flipping to production surfaced two guards that had become limitations rather than protections, and both are fixed rather than routed around: - `serve` refused any production configuration. It is now the deployment command as well as the development one, so it takes --allow-production by name. That flag is an acknowledgement, not the control: the configuration gate independently refuses a deployment without a PostgreSQL DSN, Secure cookies, CSRF, an absolute verification base URL, and a mail transport. - `create_app` refused to build a production app unless a repository was injected, with a message saying the PostgreSQL adapter "is not part of this slice". It is. The check is now about which adapter is in use rather than where it came from, stated twice: once against the configured kind before anything is opened, and once against the built object, because an injected repository bypasses the first. Two tests asserted the old behaviour and were rewritten rather than deleted. One claimed a production app must be refused for want of an injected repository, which would have made a production deployment impossible to start; it now covers both routes to a development store. The other built a production app on an in-memory repository to assert a Set-Cookie header, which the corrected rule refuses; it now asserts the cookie flags on settings that ask for them, with the production half covered by the configuration gate that refuses insecure cookies. /api/health now reports environment=production, repository=postgres, mail_deliverable=true, setup_required=false, and the service survives a restart with its data intact. |
||
|
umbra audio
|
17f8e81fb7 |
Verify the mail transport instead of trusting its configuration
Gmail rejects an account password for SMTP and wants an app password instead, which it says in a reply nobody reads. Configured with smtp.gmail.com:587, a port, and a password, the deployment looked healthy: `_mail_is_deliverable` asked whether the transport was smtp and a host was set, both true, so /api/health reported mail_deliverable=true and the control panel allowed account creation. Every account created that way would have been stranded in `invited` with a confirmation link that was never delivered and cannot be re-sent, and the administrator would have seen a success response each time. That is a data problem created by a configuration problem, invisible until somebody tried to sign in. So the transports report whether they can actually deliver: - `Mailer.verify()` on all three, sharing the connection path with `send` so a probe cannot succeed by a route real delivery would not take. - SmtpMailer authenticates and disconnects, and logs the server's own rejection code, because "wrong password" and "server unreachable" are different problems and a bare False throws that away. The reply contains no credential. - health reports the verified state, not the configured one. And enrollment refuses when mail cannot be delivered, in the service rather than the browser. The control panel already disabled its own buttons, but a client-side check is a courtesy, not a control: anything reaching the process would still have created the account. Exempt is the operator path, which attests the address instead of mailing it, because guarding that would deadlock first-run setup behind SMTP configuration, and development, where the logging transport is the point and the flow would otherwise be untestable. Verified against smtp.gmail.com on both 587 (STARTTLS) and 465 (implicit TLS): reachable, TLS established, AUTH offered, and 535 5.7.8 "Username and Password not accepted" on both. The network and the server are right; the credential is not. 286 tests pass without a database, 295 with one. |
||
|
umbra audio
|
c8b8cdb118 |
Deploy the service: dedicated database, unit, and the public hostname
Brings up a working deployment and, in the process, fixes a unit that could not start and a test suite that had been writing into it. - Created the dedicated `umbratrans` database owned by the non-superuser role `umbra`, applied the control-plane schema and the phase-1 migration. The migration is idempotent, verified by applying it twice. The existing `umbracontrol` control plane is untouched, which is the point of a separate database. - The unit could not start: `ProtectHome=read-only` forbids the write that `uv run` makes to ~/.cache, so it exited 2 and restart-looped until systemd's start limit stopped it. Switched to the venv interpreter directly, which is the right shape for a service anyway: a fixed interpreter, nothing to resolve at boot, and no developer tool's cache able to change what runs. - The integration suite leaves the database as it found it. It creates real accounts, sessions, credentials and audit rows and used to leave every one behind, which was only visible because the scratch database was disposable. Running it against the deployment database filled it with ten test accounts, and first-run setup then reported that an administrator already existed — a confusing failure with no visible cause. Teardown truncates the rows the suite's code creates and leaves the schema's reference data alone. - Bound to 192.168.123.14:8442, the private LAN address the reverse proxy reaches, replacing a static file server that had no API behind it. The public hostname now serves the real application. First-run setup is open, and the deployment runs staging with the mail requirement explicitly waived: /api/health reports mail_deliverable=false, the control panel shows a standing banner, and account creation is refused there. Sign-in and setup work; nothing needing a delivered email does, until SMTP credentials arrive. 288 tests pass against this database, and leave it clean. |
||
|
umbra audio
|
ef6bf1e6a3 |
Add first-run setup so a deployment can be finished in the browser
The gap this closes: the operator command that creates the first administrator refuses to run in production, and production refuses to start without SMTP credentials that only the operator has. So a real deployment could not be brought up at all. Everything else already worked. Setup is offered only when both of these hold, checked against the store on every request rather than cached at boot: * no platform administrator exists, and * the token generated at that boot is presented. The token is what keeps this from being first-requester-wins: anyone who reached the URL before the operator would otherwise become the platform administrator. It is written to the journal, so control of the service is still what authorises the first administrator, and it is regenerated on restart. It is spent when it authorises rather than when the account is created, so a password the policy rejects cannot leave a shown token usable. Five failed attempts burn it. Comparison is constant-time. The status endpoint never returns it, and there is no endpoint that would. One deliberate exception to the log filter: the announcement uses no key=value shape, so redact_text leaves it alone, and a test pins both halves — the setup code still gets through, and `setup_token=` still does not. The filter is not weakened, disabled, or special-cased. Also: - a `staging` environment, so a deployment can be brought up before SMTP exists. It holds every production rule except that one, and the waiver that permits it is refused in production outright, reported by /api/health as mail_deliverable, and shown as a standing banner in the control panel, which then refuses to create accounts. A staging deployment cannot become a convenient way to run something production would not allow. - `serve` accepts `--allow-lan-bind` for a named private address, because the reverse proxy runs on another host. It refuses a wildcard and a public address, so the flag cannot open the port to the internet. AF_UNIX was added to the unit's address-family set, without which the PostgreSQL socket connection cannot be made. - `count_users_with_role` on the repository, counting active assignments rather than profile rows, because a suspended administrator is not one. - setup.html and the sign-in link that appears only while setup is outstanding, plus the deployment runbook and a rootless unit. A test of mine was asserting over every credential row in the shared scratch database rather than the one it created, so it passed only while no account had replaced its one-time password. Scoped it, and added the other half of the property. 279 pass without a database, 288 against one. |
||
|
umbra audio
|
ef9785a5da |
Give the confirmation email a page that resolves
The link the service builds pointed at /verify-email?token=..., but the frontend is mounted as static files and that mount resolves verify-email.html and not a bare /verify-email, so the link 404'd. Following the confirmation email did nothing at all, which is the one step of the flow that cannot be worked around. The existing test followed /api/auth/verify-email, which is the API route and not the link that goes in the message. Two different URLs, so the page a recipient clicks was never exercised. The new test builds the app with a static root the way a deployment does and requests the path taken out of the delivered body, so the two can no longer drift apart. web/verify-email.html and web/assets/verify.js are the landing page. It consumes the token from the query or the fragment on arrival, accepts a pasted link or a bare token, and strips the token from the address bar once used so it is not left in history. It reuses the sign-in card's styling and the shared API client, and carries no inline script or style, to stay inside the content security policy. The duplicate verification view that had been added to the sign-in page is removed in favour of this one, and its dead bindings with it. The sign-in page keeps the panel that appears when the API answers email_not_verified, since that is only ever reached once the password has already been proven correct. |
||
|
umbra audio
|
d1b19fbdb3 |
Update the site icon and wire the UI to email login
Icon: the supplied artwork is now UmbraTrans_Icon2_899x920.jpg, gold and
cyan on white, with the wordmark corrected to "UmbraTrans". It is copied
verbatim to web/assets/umbratrans-icon.jpg as the source, and
web/assets/umbratrans-emblem.png is derived from it and is what the pages
load: the book cropped to its ink bounds, the white page made transparent
by flood-filling inward from the corners so the white lettering inside
the book survives, letterboxed square rather than side-trimmed so no part
of the emblem is cut, and emitted at 256px.
A square crop of a 827x695 emblem has to choose between trimming the
sides and letterboxing, and trimming was cutting the book's frame. The
glow also moved from box-shadow to filter, because a box-shadow follows
the border box rather than the artwork's alpha and would have drawn a
glowing rectangle behind a book-shaped emblem.
The UI was still written against username login, so the sign-in form
could not have worked against the API: it posted {username, password} and
LoginRequest requires {email, password}. Fixed, along with the rest of
the email-verification flow that had no UI at all:
- the sign-in form collects an email address, and its copy says so;
- a 403 email_not_verified, which is only ever reached once the password
is already proven correct, reveals a panel offering a new link. The
resend wording is the server's and is identical for real and unknown
addresses;
- a confirmation view accepts a pasted link or token, and one is opened
automatically when the URL carries a token, which is then stripped from
the address bar so it is not left in history;
- the password-policy hint read the sign-in field to tell the user their
password must avoid their "account name", but the rule checks the
account's username. It now uses the signed-in account's handle, and
says nothing rather than the wrong thing when there is no handle;
- the provision form called email optional while the API requires it,
and let a member be created with no organization, which the store
rejects. Both are caught client-side now with wording that names the
field to fix;
- `serve` accepts --repository postgres, so the app can be exercised
against the authoritative store.
Verified against a live server on PostgreSQL: sign-in with the email
address, forced first-password change, provisioning an invited member,
401 for a wrong password and 403 email_not_verified for the right one,
and byte-identical resend answers for a real and an unknown address.
|
||
|
umbra audio
|
64af39eb68 |
Run the PostgreSQL integration suite against the real schema
The integration suite had never been executed. Running it against a PostgreSQL 16 instance carrying an exact copy of the control-plane schema found seven real defects, all of which were invisible without a database. Migration: - `ADD CONSTRAINT ... UNIQUE USING INDEX` failed when the index was already attached, for both `organizations_slug_key` and `users_username_key`. Because the file runs in one transaction, that error rolled back every other change, leaving a half-applied schema rather than an obviously untouched one. Both are now guarded, so the file applies cleanly to the real schema and is idempotent on re-apply. The adapter: - `get_user_by_email` was never implemented. The protocol stub returned `None`, so email login found no account against a real database while working perfectly against the in-memory one. - The four verification-token methods were never implemented either, so email verification links never worked against PostgreSQL. - The RLS identity GUC was wrong. `app.current_user_id()` reads `app.user_id`, not `app.current_user_id`; setting the similarly named GUC left every policy helper resolving to NULL, so every policy denied and every query returned zero rows. Added `verify_rls_contract()`, called at startup, which proves the two agree. It has teeth: pointed at a wrong GUC it refuses. - `SET` takes no bind parameters, so the statement timeout was a syntax error on every transaction. The identity GUC now uses `set_config`, which is parameterisable and keeps the value bound. - `append_audit_event` returned only `id` and then mapped the whole row, and passed a text `target_id` where a `UUID` was required. The domain: - A member with no organization is storable in the development JSON store and rejected by `user_roles_check`, so the same call succeeded in development and failed in production. Enforced in the service now, and 43 tests were relying on the JSON store's leniency. - `create_account` accepted `require_email_verification` and silently dropped it on the operator path, returning an unverified account to a caller that had asked for a verified one. Now refused, with the reason. - All 26 protocol stubs returned `None` instead of raising, so a missing method was indistinguishable from an empty database. They now raise, and a conformance test checks each repository implements every declared method rather than merely inheriting it. The old conformance test used `hasattr`, which inheritance always satisfies; it passed while five methods were missing. - The bootstrap CLI refused an unsatisfiable role only after the operator had typed a one-time password. The migration README now records which review questions are answered and which remain, including that RLS is enabled but not forced, so the adapter's own scoping is currently the only tenant control and the isolation tests below it are still outstanding. 255 pass without a database, 263 against one. |
||
|
umbra audio
|
0461dff9d2 |
Use the email address as the login identifier and require confirmation
Sign-in now takes the email address rather than a username. This needed no migration: users.email is already citext NOT NULL UNIQUE in the control plane, so it is a case-insensitively unique login key on its own. username survives as an optional display handle for audit lines and is never a login path, which removes the redundancy of asking an administrator for two identifiers. An account created by an administrator now starts in the control plane's invited state and cannot sign in until a single-use link is consumed. The token maps onto bootstrap_tokens, which was already shaped for exactly this and had been unused, so that needed no migration either. Modelling invited as a real state also retires the adapter's earlier fail-closed refusal of that value, which existed because invited was previously unrepresentable. Three decisions worth stating plainly. The "email not verified" message is sent only after the password has been verified. Reporting it earlier would turn the login form into an oracle for which addresses have accounts, which is the one thing the uniform authentication error exists to prevent. A test asserts that a wrong password and an unknown account are byte-identical, and it caught a real bug: the check was first written before the password comparison, and the generic "not active" branch swallowed invited accounts. Both are now ordered deliberately. The operator bootstrap path does not require verification. The operator is attesting to the address themselves, which is the standing an administrator already has, and requiring a delivered email there would deadlock first-time setup on SMTP configuration that does not exist yet. The logging mail transport is refused in production. A deployment that logged links instead of sending them would appear to work while every new account sat permanently unverified. The development transport writes the link with the token redacted, because a log is where live credentials end up. That means a human cannot complete the flow by reading the log; the test harness captures the message in memory instead. This is a deliberate limitation rather than an oversight, and it is documented. Also: production now requires a mail transport and an absolute verification base URL alongside the existing DSN gate, because an account that cannot be verified cannot be used at all. 269 tests pass, 3 skipped. Verified end to end against a real uvicorn server: bootstrap login, organization creation, an invited account with its one-time password, and a handle rejected as a login path. |
||
|
umbra audio
|
31fd8720ec |
Add the PostgreSQL adapter for the authoritative store
Implements AccountRepository against the control-plane schema plus the phase-1 identity migration, and wires it in so production can select it. The adapter is not a thin SQL wrapper, because the control plane does not store the two things the application needs on the users row. Role and organization come from user_roles. There is no role column and no organization column on users; a role is assigned through user_roles with a scope, and app.validate_user_role_scope() enforces that the role's scope matches the assignment. Resolving one role per user is therefore a join, and a user holding more than one active assignment is refused rather than resolved to an arbitrary row. The organization on that assignment becomes the scope of every administrative decision, so guessing would hand out the wrong tenant's authority. Reconciling this with project-scoped membership stays an open review question, recorded as such. Email became a required field on account creation. users.email is NOT NULL UNIQUE, so accepting a missing address would let account creation succeed against the development repositories and then fail on the authoritative store. It is now required in the service, the API schema, and the bootstrap command, so it fails identically everywhere. Audit identifiers now belong to the store. audit_events.id is a bigint identity, so the adapter reports that value; the application correlation UUID is preserved in the detail and never displaces the row key. AuditEvent.id is a string for that reason, which removes a type the two stores disagreed about. Three further fail-closed behaviours: a users.status of invited or locked is refused rather than coerced onto active; a row with mfa_required is refused, because this phase cannot satisfy a second factor and authenticating it anyway would grant access to someone who was supposed to present one; and a missing organization blocks access rather than granting it. save_user deliberately does not write username or email, since both are identity keys with no reviewed change path and no audit event behind them. Every operation runs in one transaction on one pooled connection and publishes the caller with SET LOCAL app.current_user_id, so a returned connection cannot carry one request's identity into the next. That is inert until the reviewed RLS policies exist; the explicit WHERE clauses enforce scope today. A statement timeout bounds how long a pooled connection can be held. Production fails closed at two independent gates: the settings layer refuses without a DSN, and the repository boundary refuses to build a PostgreSQL store without one. Verified by running both, including that an unreachable host refuses to start rather than degrading. The migration gains an index on audit_events (target_type, target_id), which the administrator audit filter needs and nothing covered, and the review questions now include the three things that must be checked before provisioning: the roles rows and their scopes, any row with mfa_required set, and any user holding more than one active assignment. Verification, and its limit. 236 tests pass with 3 skipped. The adapter's row mapping, role resolution, conflict mapping, and configuration are unit tested and run everywhere. Its SQL, transactions, and pooled reuse are NOT executed: this environment has no database credentials, because sudo to the postgres user needs a password, there is no umbra database role, and there is no .pgpass. The integration suite is written and selected by UMBRATRANS_TEST_DSN, and its fixture refuses any DSN whose database name contains umbracontrol. A green run here says nothing about whether the SQL is correct. |
||
|
umbra audio
|
08a44dbacd |
Wire the browser client to the real account and organization API
The pages under web/ were a static mockup with hardcoded names and counts. They now call the real backend, and the areas with no backend are explicit empty states rather than invented numbers. Real: sign in, the mandatory first-login password change, sign out, the account table with filters and lifecycle actions, organization create/rename/re-slug/ status, the audit log, and dashboard counts scoped to the caller. The project workspace keeps its Studio identity and its real session wiring, but its project, maintenance, and statistics areas now say which backend work is pending instead of showing sample figures. Removed the mock release-readiness pipeline and the fake production metrics rather than restyling them. The client depends on properties the server already enforces, and states them where a future editor will look: - No secret is sent by a client. Account provisioning has no password field; the service generates the one-time password and returns it once. - That one-time password lives in exactly one variable, is cleared when the reveal panel closes and on pagehide, and is never written to storage, a cookie, the URL, the page title, or the console. - Every unsafe request carries X-CSRF-Token read fresh from the readable cookie, and GET /api/health on page load bootstraps it because the middleware issues none on an unsafe method and login is exempt. - Nothing is trusted from the browser. Controls a role cannot use are hidden, and the service re-checks each one. Rendered markup is built by an el() helper that routes all server-supplied text through textContent, with no innerHTML anywhere. The response CSP is script-src 'self' and style-src 'self', so an inline script or style attribute would be refused by the browser; there is none. Verification, and its limits. The banned-pattern grep for inline script, inline handlers, style attributes, innerHTML, storage, and console output comes back empty. Replaying the exact request sequence the client performs against a real uvicorn server confirmed all 86 contract assertions: every field the JavaScript reads is present, the CSRF cookie is issued by the health probe, the one-time password is absent from every later response, and the error codes the UI branches on are the ones the API returns. The client was NOT executed in a browser. No desktop browser is attached to this session and no JavaScript runtime is installed on this host, so the one-time-password reveal, the modal focus trap, and the role gating are verified by reading and by the contract replay, not by driving a real browser. That gap needs closing before this is trusted. web/README.md replaces the mock description, including the run command and the fact that this is not a production deployment: create_app still refuses to start in production because the PostgreSQL adapter does not exist. |
||
|
umbra audio
|
0b9c4d4220 |
Add organization tenancy and same-origin frontend mounting
The administrator interface needs organizations to manage, and the slice had none: users.organization_id was a bare UUID pointing at nothing, so there was no tenant to create, list, or validate an account against. Tenancy: - Organization record with name, slug, and status active|suspended|archived, matching the control-plane vocabulary rather than reusing the account one. - Reads are open to any administrator; every write is platform-admin only, so the tenant boundary stays under a single authority. An organization admin is pinned to the organization on their own principal. - A derived slug is normalized from the display name, but a slug the operator typed must already be canonical, so nothing is silently rewritten behind their back. archived is terminal. - GET /api/uadmin/summary for the dashboard, recomputed per request from the same scoped queries the list endpoints use, so a summary cannot disclose a number the caller could not reach by listing. Two integrity gaps closed: 1. Account creation accepted any UUID as an organization. It now requires the organization to exist and to be active. 2. Suspending an organization revoked member sessions but did not stop new logins, because each member's own account status stays active. Both authenticate and resolve_session now refuse a member of a non-active organization, and a missing organization record blocks access rather than granting it. A test asserts the member is still active yet locked out. Migration, still unapplied: section 4 creates organizations with CREATE TABLE IF NOT EXISTS so a control-plane table is skipped, never redefined, and the rollback drops it only when this migration actually created it. The membership divergence is recorded as the top open review question rather than decided: user_roles is the control plane's membership source, this phase uses a column, and the migration adds no membership column so the choice stays open. Serving: UMBRATRANS_STATIC_ROOT mounts the browser client at / after the API routes, so the session cookie and CSRF header work without CORS. HTML responses get their own CSP allowing only same-origin script and style, which is why the frontend carries no inline script or style. Verification: 204 tests pass (21 new organization tests plus the existing suite), ruff check and format clean. Also exercised against a real uvicorn server rather than only the test client: bootstrap, forced password change, organization creation, one-time password issue, member lockout on organization suspension, and rejection of an account assigned to a suspended tenant. |
||
|
umbra audio
|
6482921d0b |
Run the static prototype under systemd
The prototype was served by a bare `python3 -m http.server` in a foreground shell. It has no restart policy, so it died silently and the public URL returned a 502 from Caddy with valid DNS and a valid certificate already in place. This unit makes the failure mode visible and recoverable. umbratrans-web-prototype.service: - Restart=on-failure with a 5s delay and a 120s/5-attempt start limit, so a crash loop stops retrying but a clean stop always starts again. - Binds 0.0.0.0:8442 serving web/; Caddy keeps terminating TLS upstream. - Sandboxed: runs unprivileged with an empty capability bounding set, no new privileges, read-only home and system, private tmp and devices, system call filtering, and no access to kernel or control-group interfaces. - Journal output to the user journal for inspection. PrivateUsers=yes is documented as deliberately omitted. It hides /etc/group from the manager, so systemd cannot resolve supplementary groups and the unit fails with status=216/GROUP. A probe unit confirmed the remaining sandbox directives are all valid in user scope. Verified: all three routes and the icon return 200 locally and the public HTTPS URL returns 200. SIGKILL of the main PID recovered automatically within the restart delay. `systemd-analyze verify` is clean. Two follow-ups requiring root are recorded in web/README.md: the rootless install must drop User= because a --user manager cannot setuid, and `loginctl enable-linger umbra` is needed for the service to survive logout and start at boot. |
||
|
umbra audio
|
b474995414 |
Add phase-1 account management backend slice
Implements the account and user-management system behind the new web UI against development repositories only. Nothing here connects to, reads, or modifies the production umbracontrol database, and no account is created in any approved environment. Account service and API: - Local temporary-password and OIDC enrollment, separated by type. - Argon2id credentials, must-change enforcement, server-side sessions with hashed tokens, idle and absolute deadlines, rotation and revocation. - Administrator account management: list, get, create, suspend, reactivate, deactivate, revoke sessions, org-scoped audit view. - Hidden-prompt operator command for the first administrator, refusing production mode and refusing to run without a TTY. Temporary passwords are never accepted from a client. The request schema has no password field and refuses unknown fields; the administrator path generates the one-time password in process, hashes it immediately, and returns it once in the creation response. The account can reach no route, including /api/auth/me, until it replaces that password, after which the one-time value no longer authenticates. Security posture: fail-closed configuration, Secure cookies in production, double-submit CSRF, constant-time token comparison, generic authentication errors, audit detail allow-listing, and logger-tree secret redaction. Reviewed but unapplied: migrations/0001_phase1_identity.sql adds users.username, a separate user_credentials table, and additive session columns, with a rollback and open review questions. The PostgreSQL adapter is deliberately absent, so the application refuses to start in production rather than run against an unreviewed store. Verification: 169 tests pass (148 new, 21 pre-existing); ruff check, ruff format, compileall, and make web-check are clean. |
||
|
umbra audio
|
37b2056808 | Add UmbraTrans UI prototype and integration design | ||
|
umbra audio
|
6a3baacb34 | Harden agent communication validation | ||
|
umbra audio
|
127a048cfe | Restrict package discovery to agent_comm | ||
|
umbra audio
|
8a61d9298b | Add verified NAS backup automation | ||
|
umbra audio
|
fc6c36331b | Bootstrap UmbraTrans agent communication workspace |