de08cfe38e92c4c4381911ec5f94c1714dbc238b
37 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
umbra
|
de08cfe38e |
Stop rate-limiting the deployment: any finite budget is exhaustible
The GPU permissions survived a third consecutive boot - all five nodes 660 root:video - so that is now established rather than hopeful. The deployment was down again for the same reason, and this time the widened start limit I committed two boots ago turned out not to have fixed it. It exhausted twenty restarts and returned to the same permanent failed, with the port free and nobody touching the service. That fix's verification missed it. The port was freed after three restarts, inside the window, so the test exercised the easy case and the commit said "fixed". The real case is a fault that outlasts the budget, which is the only case that matters. The rate limit is the wrong mechanism, not the wrong value. Any finite budget can be exhausted by a fault that lasts longer than it, and the failure is silent and permanent - `failed` looks like a decision rather than a give-up. StartLimitIntervalSec=0 disables it: the unit retries for as long as the fault lasts and binds the moment it clears. Verified against the case that broke it twice. Held the port for 200 seconds, which drove 31 restarts - past the old limit of 20 - with the unit never reaching failed. Freeing the port gave 200 from /api/health within eight seconds, with nothing having started it. A genuine crash loop now logs every 5s instead of stopping: noisier, and much better than a deployment that is quietly down. 469 pass with a database, 428 without. Deployment healthy, GPU boundary intact, database unchanged at users=2 orgs=3 projects=0 audit=43 with no compute or datastore tables, ollama serving, persistenced up, perms timer firing, zero stray agents. |
||
|
umbra
|
1dbf5fb384 |
The GPU permissions survive a reboot, and the deployment stops giving up
Second boot since the permission fix, and it held: all five nodes 0660 root:video, the timer firing on schedule, and umbra - which runs the controller - still refused the card. That claim is now an observation rather than an inference. The same reboot surfaced an unrelated fault worth fixing. The deployment did not come up: a `python3 -m http.server` from a development session held port 8442, serving web/ - which the backend mounts itself and does not need - and the service failed five times in ten seconds. With StartLimitIntervalSec=120 and StartLimitBurst=5 that exhausted the budget, and systemd then never tried again. The deployment stayed down after the port was free and needed someone to notice, which is the second reboot in a row to end this way. A transient condition had become a permanent outage. Widened to 500s and a burst of 20, which keeps retrying across a slow boot or a settling reverse proxy while still stopping a genuine crash loop. Verified by holding the port with a squatter and watching: the unit cycles active/activating without giving up, and the instant the port is freed the next retry binds and /api/health returns 200. My first attempt at this test was void - the squatter could not bind because the service still held the port, so it proved nothing. Note that two consecutive reboots have been taken down by a background shell in this session holding the deployment's port. It returns after each boot. Stopping that shell, or moving it to another port, is the durable fix; killing it again only defers it. 469 pass with a database, 428 without. Deployment healthy, database unchanged at users=2 orgs=3 projects=0 audit=43 with no compute or datastore tables, ollama serving, agent account intact, zero stray agents. |
||
|
umbra
|
4937b9beed |
Record that the GPU permissions now hold, and how that was verified
Cleared the unit's stale active(exited) state so the corrected version could run, then broke the permission deliberately the way nvidia-persistenced does and watched the timer repair it: 666 -> 660 root:video, with the helper reporting 'restricted 5 node(s)' in the journal. ollama and umbra-poc can measure the GPU; umbra, which runs the controller, cannot. Deployment healthy, ollama serving on 11434, database unchanged. Records what is still unverified: no boot has been observed end to end since the change. The unit, timer and udev hooks are installed and exercised individually, and the persistenced interaction is handled by ordering rather than luck, but 'works after a reboot' is a claim until a reboot confirms it. show answers it in one command. |
||
|
umbra
|
0ab1f0f1de |
The GPU boundary did not survive a reboot, and neither did the fix for it
This is the verification I said could not be done until the machine rebooted. The reboot proved the boot path wrong, and uncovering why took three rounds. **The boundary was gone.** /dev/nvidiactl came back 0666 root:root, so the controller could open the card again - the exact thing the udev rule was applied to prevent. **Why: the unit ran before anything existed.** umbra-nvidia-devices ran at 05:38:24, ordered After=systemd-udev-settle, globbed /dev, found no nodes, and exited 0. udev-settle means "udev has finished its queue", not "the proprietary driver has finished initialising". The nodes appeared moments later at 0666 and nothing revisited them. The helper ended in an unconditional `exit 0` - I had written that deliberately, so an early boot would not fail a unit - and it is precisely what made "I fixed the permissions" indistinguishable from "there were nothing to fix". **Why the first repair also failed: nvidia-persistenced resets the mode.** It chmods /dev/nvidiactl back to 0666 on every initialisation, via nvidia-modprobe. That is out of udev's reach entirely, because udev does not manage these nodes. So the fix is no longer an ordering problem but an eventual one: the unit now runs After=nvidia-persistenced.service, and a two-minute timer re-asserts the permissions because the trigger that breaks them can fire at any time. **Why the timer did nothing either.** RemainAfterExit=yes on the oneshot keeps it "active (exited)", so systemd treated every activation as already satisfied: the timer fired on schedule, the service reported Result=success and ExecMainStatus=0, and the permission was still wrong two minutes later. That is the original bug repeated one level out, inside the mechanism built to repair it. Removed, and a test now pins it - by directive name, with comments stripped, since the unit documents the omission by naming it and a substring search failed on the explanation. **I also broke nvidia-persistenced.** It drops to uid 122 with setuid and without initgroups(), so it never receives the video supplementary group and failed with "Insufficient Permissions" once the nodes were 0660 root:video. A systemd drop-in runs it as root - upstream's own default - which works without widening the device permissions. It was running fine before. The helper now waits for the nodes, verifies the modes afterwards, and exits non-zero under --strict rather than reporting a no-op as success. It takes UMBRA_NVIDIA_DEVDIR so its three paths can be tested without root and without a GPU; the first attempt at that test pointed CANARY at a non-existent path and still exited 0, because the fix loop globbed /dev directly and the override only reached the wait. Also: nvidia-device-access.sh `show` reported "cannot measure the GPU" when a cached sudo credential had simply expired - a finding about the machine when the truth was that the tool could not ask. It now distinguishes no from could-not-check, because I was misled by it during this check. **Still outstanding, and I could not finish it.** The repository is correct; the installed state is stale, because clearing the unit's active (exited) state needs root and that authorization was declined. As of now /dev/nvidiactl is 0666 and the controller can open it. To finish: systemctl stop umbra-nvidia-devices.service systemctl start umbra-nvidia-devices.service stat -c '%a %U:%G %n' /dev/nvidiactl and expect 660 root:video. Otherwise the system came through the reboot intact: the deployment was down only because a leftover `python3 -m http.server` from an earlier session was holding port 8442, and the backend mounts the same files itself. Deployment healthy, database unchanged at users=2 orgs=3 projects=0 audit=43 with no compute or datastore tables, ollama serving, agent account intact, zero stray agents. 471 pass with a database, 430 without. |
||
|
umbra
|
1ab9c2595d |
Verify the datastore policies as a role that does not own the tables
They had never been observed to refuse anything, and until now every
statement this project made about tenant isolation was a statement about
the adapter's own WHERE clauses.
The reason was structural. RLS does not apply to a table's owner until
FORCE ROW LEVEL SECURITY, and every connection here is the owner. A probe
run as the owner showed organization B's administrator successfully
updating organization A's imprint - exactly what WITH CHECK was written
to prevent. A policy that cannot refuse is indistinguishable from a policy
that is absent, which is why the previous commit said so plainly rather
than claiming the migration was verified.
test_rls_tenancy.py runs as rls_probe, a role with plain table privileges
and no ownership. Eight tests, each checked against a mutation: each
organization sees only its own artifacts, an unattached principal sees
nothing on any datastore table, a publisher is readable across tenants but
writable only by its owner, a cross-tenant UPDATE changes zero rows, and a
cross-tenant INSERT is refused. Two of the eight also assert the positive
cases, because a policy that refuses everything would satisfy every
refusal above.
Two behaviours asserted so a change is noticed: a filtered UPDATE or
DELETE raises nothing and simply affects zero rows, while an INSERT
violating WITH CHECK raises InsufficientPrivilege. The first is a trap
for application code that checks "did it raise?" instead of "did it
change a row?", and it is why the earlier manual probe reported a
cross-tenant update as "allowed" when the statement had matched nothing.
Two mistakes in the tests, both worth recording. A policy violation aborts
the transaction, so the second refusal in a test could not even be
attempted and reported as "DID NOT RAISE" when nothing had run; each
assertion now gets a savepoint. And a module-level pytest.skip() is a
collection error rather than a skip, which broke the whole suite instead
of quietly omitting the file - the guard now evaluates to a boolean.
This does not close gate G2. The application still connects as the owner,
so in the running system these policies still enforce nothing. What is
established is that they are correct, which makes FORCE ROW LEVEL SECURITY
plus a non-owner application role a configuration change with a
known-good target rather than an untested hope. README.md says exactly
that, and the gate stays open.
Creating the role needs superuser and was authorized this time:
sudo -u postgres psql -d umbratrans_scratch -c \
"CREATE ROLE rls_probe NOLOGIN; GRANT rls_probe TO umbra;"
465 pass with a database, 424 without. Deployment untouched: users=2
orgs=3 projects=0 audit=43, no compute or datastore tables.
|
||
|
umbra
|
5fc38d4708 |
The datastore, and two parties that never sign in
Compute capability is not LLM work. It is mechanical split, analysis,
text to speech, PDF formatting and audio analysis, assigned as needed,
with the code on every node. That needs a capability vocabulary rather
than VRAM - which is noted and not yet built - and, more urgently, a
place to put the output. jobs.result is a jsonb column: right for
{"echo": "hello from the queue"}, wrong for "the English audiobook, four
hours twelve minutes, derived from these thirty-seven chapters and this
cast".
So 0003 gains section 5: artifacts, job_artifacts, publishers,
project_publishers, authors, project_authors, and eight permission
keys. Folded in here rather than deferred to 0004 because 0003 has still
never been applied to the deployment, and an unapplied migration can
still be edited.
The word "account" is now absent from every new column, table and
comment. It has meant two incompatible things in this system - a person
who signs in, and a house a work is published under - and a schema
using one word for both will use it for both in one constraint. An
organization owns the work. A user is a worker and owns nothing. A
publisher is the party a work is published under and has no password, no
session and no row in user_roles, because a name that appears on a work
is not a party that can be asked to prove who it is.
A byline is a third party and is neither. In Umbra -> WorstAuthor ->
B.D. Murphy, WorstAuthor is the publisher and B.D. Murphy is the
byline; different lifetimes, and the spec had been calling both "brand".
project_authors carries a credit_role rather than a single author_id,
because the author of the English edition is the translator of the
French one and the narrator is frequently neither.
owner_organization_id is the one deliberately cross-tenant column, and
the policies are asymmetric about it: reading a publisher is open to any
authenticated principal because publishers are shared reference data,
while setting that column is refused to anyone but a platform
administrator. There is no second policy carving that out, because
PostgreSQL combines permissive policies with OR and an "admins may also"
policy widens access for everyone. The WITH CHECK on the single FOR ALL
policy is doing that job alone.
Found while testing rather than while writing: RLS was not enabled on
four of the six new tables. PostgreSQL creates a policy on a table where
it cannot apply and reports success, so pg_policies listed them and
nothing was enforced - and a check for "does the policy exist" would
have passed throughout. test_rls_coverage.py asserts relrowsecurity
instead, and each of its six assertions has been checked against a
mutation that breaks it. One needed strengthening: dropping the
dedicated SELECT policy left the table looking fully policed, because
FOR ALL includes SELECT, so the surviving policy quietly gated reading on
organization ownership and hid shared reference data.
Also: the nvidia tests failed for want of a sudo credential rather than
because anything was wrong. They skip now, with the reason stated. A
test that fails for environmental reasons teaches the reader to ignore
red.
The policies have never been observed to refuse anything, and README.md
says so plainly. A probe as the owner showed organization B's
administrator editing organization A's imprint - what WITH CHECK was
written to prevent - because RLS does not apply to a table's owner until
FORCE ROW LEVEL SECURITY. That is gate G2, not a new problem. Testing it
for real needs a non-owner role and superuser; the request was not
granted, so it is not done.
455 pass with a database, 422 without.
|
||
|
umbra
|
c1741456e2 |
Apply the GPU boundary, and fix three checks that verified themselves
The udev rule I wrote last commit could not have worked. The nvidia driver creates /dev/nvidia* in the kernel without registering them in sysfs, so `udevadm info --name=/dev/nvidiactl` answers "Unknown device" and udev never evaluates a rule for them - which is also why they are 0666, since nothing sets a MODE. The rule hooks the driver bind event instead and runs a script, by which point the nodes exist. The first version of that hook triggered 0000:01:00.0, which on this machine is an AMD root complex, vendor 0x1022 rather than 0x10de. The trigger fired on the wrong device, the RUN key never ran, and the permissions looked right because the apply script had just fixed them itself. The address is discovered now. Verified by resetting to 0666, firing only the bind event, and watching it come back 0660 - the script never invoked. Applied and verified: ollama can open the card, umbra-poc can, umbra cannot. Traced, nvidia-smi opens /dev/nvidia0, /dev/nvidiactl and /dev/nvidia-uvm, so umbra-poc is in video or the agent starts, heartbeats, and reports a machine with no GPU - a failure that presents as a scheduling problem and is a permissions one. It is the agent's only supplementary group and grants a device, not a filesystem, so setup's "no supplementary groups" check now allows exactly video and rejects everything else, verified by adding users and watching it fail. Three tests, each verified against a mutation. The rule test resets the permissions and fires only the event, so it fails if the rule is inert. The end-state test fails rather than skips when the rule is missing - skipping made deleting the rule quieter, three passed becoming one passed and two skips, which reads as an improvement in the wrong direction. Also: the check "the controller cannot open the card" was testing root, which bypasses permissions entirely, and reported a working rule as a failure. The account is named now. Then four runs in a row leaked agents, and the cause was upstream of the leak. The victim reused the first agent's credential, and a node is identified by its credential, not by its config's node_name - so it heartbeated as poc-agent-node, was never offered the long job, and sat there for the whole step while a correct liveness check reported that the worker it had killed had survived. It has a third node and its own credential now, so the requeued job is not immediately reclaimed by an agent already stopped for the step. The last leak was a worker asleep in a 600-second job, unable to see a stop file because the check is at the top of its loop rather than inside the work. Four runs left one each. The job is 10 seconds now: the victim exits the instant it claims it and never runs it, so the length was never the point. PoC node names and idempotency keys carry a per-run suffix, so a second run against the same database does not die on a UniqueViolation - and there is no route to delete a node. Three consecutive runs, zero failures, zero leaks. 451 pass with a database, 424 without. Deployment untouched: users=2 orgs=3 projects=0 audit=43, no compute tables. |
||
|
umbra
|
220bc00823 |
An upgrade path for the agent, and the GPU device boundary
Two questions worth answering properly rather than by assertion. How an agent is updated. `setup` has always doubled as an upgrade - it installs the file unconditionally, which is most of what an upgrade needs and the one thing it should not do: a broken file replaces a working agent on every node with no way back. `update` names the version being replaced, keeps the previous file, and verifies that the new agent *starts as umbra-poc* rather than that it parses. A file that imports cleanly under umbra's virtualenv may not be readable by the account that runs it. Verified against a deliberately broken agent: installed, detected, previous restored. The agent does not update itself. An agent that can fetch and execute code has a remote-code-execution channel, and the point of the account is that it holds no credentials and reaches nothing. Updates are an out-of-band deploy per node. Nothing enforces a minimum agent version either - the console can report "behind", but refusing a heartbeat would strand a node mid-job for tidiness. The GPU. /dev/nvidia* is 0666 root:root because the driver creates it that way and its own 71-nvidia.rules sets no MODE or GROUP. Measured, not assumed: nvidia-smi -L succeeds for umbra, ollama and umbra-poc alike - so the controller can open the card today, and the only thing stopping it from loading a model on top of the node's work is the scheduler choosing not to. 72-umbra-nvidia.rules sets 0660 root:video, sorting after the driver's rule so it wins. video already contains ollama, which is right: ollama is the account that opens the GPU. umbra-poc must be added to it, and traced nvidia-smi opens /dev/nvidia0, /dev/nvidiactl and /dev/nvidia-uvm - so without the membership the agent starts, heartbeats, and reports a machine with no GPU. That fails silently, as a scheduling problem rather than a permissions one. It is the agent's only supplementary group and grants a device, not a filesystem, so it is added by the apply script rather than by setup. What it does not buy: separation between two agents. Both are umbra-poc, both in video, both could open the card. On one box with one GPU, placement is the control and the kernel is not. What it buys is that every other account stops being able to touch the card, the controller above all. apply verifies the rule took, that ollama and the agent can still measure the GPU, and that ollama still serves models - and reverts itself if any of those fail. Also fixes a check this session broke: a sed meant for new code rewrote verify's agent-start line to run a command called `x`, so it had been failing on every run and I read past it. `bash -n` cannot see that class of damage; verify did, which is the argument for it. |
||
|
umbra
|
0f5c1e3408 |
Run the agent as the service account it claims to be
The compute proof of concept had never been run with the agent as umbra-poc. Running it that way found three defects, all the same shape: a check that passed because its mechanism did nothing. The demonstration never reached the agent at all. It held the pid it spawned, which under `sudo -n -u` is the sudo wrapper - sudo forks the agent before exec'ing. Signalling that killed the wrapper and left the worker running, renewing its lease and holding the job indefinitely, under a printed note saying "exactly like a power cut". An unprivileged process cannot signal another account's process at all, and kill(2) on a process group succeeds as soon as one member is signalled, so the wrapper's death read as the agent's. Every run leaked an agent. Lease recovery was a diagnosis rather than a mechanism. sweep_expired_leases documented itself as "called by the controller's own timer" and there was no timer; a killed node's job came back only when somebody noticed a stalled job and posted to an administrative route by hand. The proof of concept swept by hand and recorded the result as automatic. app.py now runs the loop for the life of the application, building the compute repository at startup so a restart mid-job still recovers it. The power cut is now the agent's own: os._exit(137) the instant it claims a job, before running it, because a process that does not own the agent cannot signal it. A clean stop goes through a stop_file the agent polls, for the same reason. Liveness is measured by finding the worker's own pid in /proc matched on its config path, sampled repeatedly and accumulated - the wrapper and the worker are not simultaneous, so a single sample finds only the wrapper. The node key travels by `sudo --preserve-env` rather than as VAR=value in argv, because argv is world-readable through /proc/<pid>/cmdline and the environment is not. The demonstration checks whether the key will survive before starting and falls back to a pipe, which node_agent.py --node-key-stdin accepts. Neither a file nor argv ever holds it. Eight tests, each verified to fail against a mutation of the code it covers: three for the sweeper (it must return an expired lease, must survive a failing round, must not invent a repository it has no database for) and five driving a real agent process against a stub controller (stops on the stop file, exits 137 holding an uncompleted job, takes the key from both routes and neither, reports an empty pipe as an empty pipe). 448 pass with a database, 421 without. The deployment database is untouched: users=2 orgs=3 projects=0 audit=43, no compute tables. |
||
|
umbra audio
|
1ab0fa2498 |
A service-account setup and boundary check for the node agent
The agent is stdlib-only - no package imports, no third-party dependencies - so it needs no virtualenv and no access to the controller's home directory. That is the whole reason the setup is short: an account with no group memberships at all, running one file from /opt on the system python. poc/poc_node_user.sh has four subcommands. setup needs root and creates the account, installs the agent, and writes a config that deliberately contains no key. verify asserts the boundary. run does both. status reports what exists and what is reachable, needing no privilege at all. The check that matters: the agent must not be able to read ~/.config/umbratrans/umbratrans.env. It cannot, and there is a second layer behind it - peer authentication maps a unix user to a database role, and umbra-poc has none, so even a leaked DSN would not connect. The two are independent and neither alone is the boundary. Writing verify produced the third instance of a bug class that has come up repeatedly in this work: a check that passes because its mechanism failed. When sudo -u could not run, every positive check correctly failed and every negative check passed - because `test -r` on an unreadable file returns false, which is exactly what "correctly denied" looks like. The script would have reported a boundary as holding while testing nothing. It now confirms the privilege drop once, up front, and abandons the run rather than reporting; status prints "unknown (no privilege drop)" rather than "no". Found by running the checks against an account it could not drop to, which is the only way to see it. Also: the agent's config now travels in a 0750 directory, because mkdtemp gives 0700 and a different user could not have read its own config out of it. The script is dry-run against an absent account, an existing unprivileged account, a missing controller, and a rootless setup invocation. It has not been run end to end, because umbra-poc does not exist yet. |
||
|
umbra audio
|
d13664a607 |
A compute substrate: node inventory, a leased queue, and an agent
Implements §6.16-6.20 and §7 of the specification, and proves it end to end on this box: a controller and an agent, the agent with no database credential at all, ten checks against the properties whose failure modes are silent. Migration 0003 adds compute_nodes, compute_devices, node_credentials, model_registry, node_models, node_heartbeats and job_events, and creates jobs - which §6.8 had specified and 0001 had never migrated. Every column added to jobs is nullable or defaulted, every policy is written against the same app.* helpers the existing eleven tables use, and re-applying is safe: CREATE POLICY has no IF NOT EXISTS, which is how 0001 learned the same lesson for ADD CONSTRAINT. Four properties, and each exists because the alternative fails silently: - **I25** claiming is FOR UPDATE SKIP LOCKED. Two agents claiming at once get different jobs. The proof of purpose counts the 'claimed' events rather than counting successes, because two agents both succeeding is not evidence. - **I26** a lease that expires returns the job to the queue, and the proof of purpose SIGKILLs the holder rather than stopping it, because a clean stop would release the lease and a clean stop is not the failure being tested. - **I24** a node is online because it heartbeated. Creation is not readiness. - **I28** a node credential reaches queue operations and nothing else. The organization comes from the credential, so a forged tenant identifier has nothing to attach to. And the one that is hardest to get right in the cheap direction: **a device that is slow is eligible, and one that is too small is not.** A model that does not fit in VRAM runs on system RAM, slowly, and a job nothing can run is 'blocked' with the reason rather than left looking like it is progressing. The proof of purpose found six defects, listed in §7.8. The two worth naming here: - Mounting one router at two prefixes published the administrator's routes behind node-credential authentication, so a node could create nodes and issue itself more authority. Two routers now, with a test that pins the surfaces disjoint and the node surface exactly. - **The correction I was given about unified memory was in the specification and not in the code.** Eligibility was VRAM-only, so every CPU and every shared-memory machine was ineligible, which would have reduced the fleet to one machine. Writing the implementation is what kept the error alive. Also: the CSRF middleware blocked the whole node API, because a bearer-authenticated agent has no ambient cookie for a cross-site request to ride on. Exempted by prefix, with a test asserting that no cookie-authenticated route joined it. 419 tests pass with the integration suite against a real database. The deployment database is untouched and has no compute tables. |
||
|
umbra audio
|
bfde763f00 |
Fix a 500 on /api/health from an uninterpolated SQL placeholder
The first deploy of the previous commit took the health endpoint down. The
tenant-row predicate had been added to `count_users_with_role` as a literal
`{TENANT_ROW_PREDICATE}` inside a plain triple-quoted string rather than an
f-string, so the braces were sent to PostgreSQL as text and the statement failed
to parse. That query backs `setup_is_required`, which every unauthenticated
health probe calls, so `/api/health` returned 500 and the deployment reported
itself broken.
Nothing caught it before production, and the reasons are worth keeping:
- the unit tests use the in-memory repository, which has no SQL;
- the integration tests did not call `count_users_with_role`;
- and the source-level test I wrote to pin the predicate's presence was
satisfied just as well by the broken version, because the broken version does
contain the text. A test that checks a predicate was *written down* cannot tell
a working predicate from an inert one.
Two tests now, each catching the regression on its own:
- `test_no_sql_string_carries_an_uninterpolated_placeholder` walks the parse tree
rather than the text, so a `{...}` in a non-f-string fails whether or not the
surrounding code mentions the constant. It reports the offending query, not a
line number: `inspect.getsource` returns a class body lifted out of the file, so
its line numbers do not point anywhere a reader could open.
- `test_the_health_probe_queries_run_against_the_real_schema` runs the probe's
own queries against the real schema, and
`test_setup_is_required_ignores_project_assignments` holds the predicate's
purpose: a project row is never mistaken for a platform administrator.
413 tests pass with the integration suite; 388 without a database.
|
||
|
umbra audio
|
ed72f61975 |
Separate organizations, and give members their organization's projects
An organization's work is separate from every other organization's, and the separation is now one module rather than a property each route remembers. `tenancy.py` resolves the caller's scope in a single place. A member is pinned to the organization on their own assignment, expressed as a concrete identifier rather than a flag, so "no filter" is unreachable except for a platform administrator. Every read and write resolves through it, so there is no per-route variation to drift. Within an organization, every account sees every project. The brief is explicit that two people working on different books both see both, and assignment says who is *working* on a project rather than who may *know* it exists. This diverges from `app.can_access_project`, which restricts visibility to assigned members; the divergence, and what it means if row-level security is ever switched on, are recorded in migrations/README.md because that would otherwise be a silent tenant-internal regression no test here would catch. Two rules, both learned by getting them wrong first: - A read naming another organization is ignored, because refusing it would confirm the identifier exists. A write is refused rather than retargeted, because a project landing in an organization the caller did not ask for is somewhere they will not look. - A missing project and a foreign one get the same error, so an identifier cannot be probed across a boundary. Projects: the `projects` table, its statuses, and the project-scoped arm of `user_roles` were already in the schema and unused. No migration. The dashboard is one server-scoped call, `/api/workspace`, rather than several assembled in the browser — pieces from different tenants that nobody notices are not a failure mode worth shipping. Filters narrow the list already in the page, so switching cannot mix two fetches, and an empty list is distinguishable from a failed one. **A sign-in lockout, found by the integration suite.** An assignment is a second active `user_roles` row, and `_resolve_assignment` refused an account with more than one. So assigning anybody to anything made them unable to authenticate — locked out of the page the assignment was meant to show them. `_sync_assignment` had the same defect. Six queries were scope-blind; three were harmless only because the assignment role is named `viewer` and never matched a tenant role name, which is luck rather than a property, so a structural test now pins the predicate on every one. The guard against two *tenant* rows survives, narrowed to the ambiguity that actually matters. Also fixed: `get_project_by_slug` cast `slug::text`, making citext's case-insensitivity a case-*sensitive* comparison that appeared to work everywhere else. An assignment grants the `viewer` project role, since no `member` role exists at project scope; it is a named constant, not a query literal, because the project workflow is not defined yet. 410 tests pass with the integration suite against a real database; 386 without. |
||
|
umbra audio
|
b12c702c66 |
Show whether an account's address is confirmed, and let an admin re-send
The accounts page now has an Email column carrying the address and its verification state. A confirmed address shows a plain marker; an unconfirmed one shows a button that sends another confirmation link. The field was already on the response, and it was lying. AccountResponse.is_email_verified defaults to True and from_view never set it, so every account in the list reported itself as confirmed, invited ones included. A badge that always reads "Confirmed" is worse than no badge, and about to hang a re-send control off it made that worth fixing first. It is passed explicitly now, with a test covering an invited account. The address moves out of the Account cell into its own column, because the verification state is a fact about the address and reads better next to it than stacked under a name. The badge is a button only when unverified. Offering to re-send a confirmation for an address that is already confirmed would imply a pending one might exist, and would put a live control on every row for no reason. The re-send posts to the endpoint the sign-in page already uses, which answers identically whether or not an account is awaiting confirmation. That is deliberate: the console holds a list of accounts, so "is this address verified?" is exactly the question that should not be askable of an arbitrary address, and an administrator-specific endpoint would answer it precisely. A test asserts the two responses are byte-identical. Verified end to end against a scratch database: an invited account renders as "Not confirmed", the control sends a verification link rather than a reset link, and an address with no account produces the same message. |
||
|
umbra audio
|
fa781cf708 |
Stop the reset page calling a dead link when it cannot reach the service
The reset link was reported as "no longer valid" while it was still live.
The token in the store was unconsumed and had not expired, so the link was
fine and the page was wrong.
Two mistakes, and the second is the one that mattered.
The first was a double question mark. passwordResetAvailable did
'/api/auth/reset-password?' + buildQuery(...), and buildQuery already
returns a leading '?', so the request went out as:
GET /api/auth/reset-password??token=... -> 422
The server read an empty parameter name and rejected the request, which is
correct. Now the call site appends buildQuery's output instead.
The second is why it took this long to find. The page had one failure
path, so every outcome became "this link is no longer valid": a malformed
request, a 500, a dropped connection, and a genuinely expired link were
all reported the same way. That is a confident false statement to make to
someone holding a good link, and it is exactly what happened.
There is now a third state, shown when the service could not be asked. It
says the check could not be completed and that this is not the same as the
link having expired, because it is not. Only the server's own "not
available" answer claims the link is dead, which is what keeps the
non-enumeration property intact: the page still cannot be used to learn
whether a link ever existed.
Two tests. One asserts no request site adds a '?' to buildQuery's output,
and that the named method uses the right shape; reverting the fix fails it.
The other asserts the unreachable state exists and is distinct.
The token from the earlier attempt is still valid until 06:57, so the same
link can be used once this is deployed.
|
||
|
umbra audio
|
e034061ec1 |
Add password reset and self-service account settings
An administrator could not reset a password for bdmurph73@gmail.com, and there was no way for a signed-in person to change their own. Both are now in place. Migration 0002 gives single-use tokens a purpose. 0001 reused the control plane's unused bootstrap_tokens for email verification, and the table has no notion of what a token is for, which was invisible while there was only one kind. With a second kind it stops being invisible: a reset link clicked by anyone holding it would confirm an address, and a verification link would set a password. The column defaults to the existing purpose, so no row is rewritten and verification is unchanged. Every token lookup now filters on purpose, and the adapter reads the purpose from the row rather than hardcoding it. An administrator may start a reset but never finish one. The link goes to the address on the account, only its holder can complete it, and the endpoint returns a bool rather than a password. An administrator who can set someone's password can also sign in as them indefinitely. A reset revokes every session, because a reset is what someone reaches for when they think their password is compromised, and revoking only the session that completed it would leave a stolen cookie working. Reset links live one hour against the two days a verification link gets. A verification link confirms an address the person already controls; a reset link is a live credential for as long as it is valid. Three bugs found while building it, all mine: - request_password_reset did not call _authorize_target, so an organization administrator could start a reset on an account in another tenant. Caught by a test written for the scoping rule that revoke_sessions already followed. - The in-memory repository never checked token expiry, only PostgreSQL did. So an expired link was honoured in every test and refused in production - the wrong direction for a divergence between two stores of the same rule. - The PostgreSQL adapter's get_live_verification_token took a purpose but not the now its in-memory counterpart needed, so the first call against a real database raised TypeError and returned a 500. The conformance test only checked that a method existed. It now compares signatures, which immediately caught that the protocol itself was missing the parameter. The reported failure was a deadlock rather than a missing feature. The account was invited: it could not sign in because the address was unverified, and I refused to reset anything not yet active. It is now allowed, and completing the reset also confirms the address, since holding the link proves control of it. Without that the flow would not converge - reset the password, sign in, refused again, using a verification link that was never sent. Suspended and deactivated accounts are still refused, because a reset restores sign-in and would otherwise undo a suspension without going through the status controls. Verified live: the account that could not be reset is now active, verified and able to sign in, with password.reset_requested, password.reset_completed and email.verified all in the audit trail. A reset token is refused by the verification endpoint, an invented token gives the same answer as a used one, and a token in a reset URL is redacted from the access log. 327 tests pass, 337 with the integration suite. |
||
|
umbra audio
|
7696c9d2e0 |
Open the organization editor when the edit button is used
The "Edit name / slug" button on the organizations page did nothing. It calls openOrganizationEditor, which filled the dialog in and returned without displaying it. Every sibling helper in the file - the account status dialog, the revoke dialog, the organization status dialog - opens its modal, so the name was carrying a promise the body did not keep. It went unnoticed because the "New organization" button works, and that one is a static [data-open] control whose handler calls openOrganizationEditor and then opens the modal itself. The row button is built in JavaScript, so the handler bound to [data-open] at startup never saw it, and nothing else displayed the dialog. Both paths share the helper, so one worked and one did not, which is what made it look like a dead button rather than a missing feature. The editor now opens the modal as well as filling it. The generic handler still calls open() afterwards, which is a no-op because the modal guards on being open already. A new test asserts that every helper named open* in the file opens a modal, so the next one cannot break the same way. Verified it fails with the fix reverted and passes with it. 302 tests pass, 311 with the integration suite. |
||
|
umbra audio
|
60aa2c9ba8 |
Keep verification tokens out of the access log
Checking the deployment after first-run setup turned up a leak I had not looked for. Uvicorn's access log records the full request line, so a request to /verify-email.html?token=<secret> wrote the verification token to the journal verbatim. Five of them were sitting there. Anyone able to read the journal could confirm an address they do not control, which is the one thing email verification exists to prevent. The tokens are single-use and expire, so the window is short, but it is a window and it did not need to exist. The redaction filter was installed only on the application's own loggers; uvicorn's is a separate tree. The filter now covers the server's loggers as well, and two things were needed rather than one. Existing loggers are patched directly, and a `_RedactingLogger` is installed as the logger class so loggers created later are covered too, because Python does not apply a parent's filters to a child's records. Without the second, a module imported lazily after startup logs in the clear, silently. The first attempt at the filter broke the log format: rendering the record, redacting it and clearing the arguments scrubbed the token but cost uvicorn's level prefix and trailing status text, so access lines became `Message: '...' 200`. Arguments are now redacted in place and the format string is left alone. One case still collapses the record, when the format string itself names a secret as in "created password=%s", because redacting it would consume the %s and break the substitution; that is confined to application loggers, since the server's own format strings carry nothing secret-shaped. Verified live: a request with a distinctive marker in the query string logs as `?token=[redacted]`, the marker appears nowhere in the journal, and the line keeps its prefix, request and status. Ordinary query strings are untouched, since a log nobody can read is not a log. |
||
|
umbra audio
|
b27915bc68 |
Fix the confirmation link, and stop the test suite eating a database
Two separate problems, one of them mine and serious. The confirmation link did nothing. `request` in api.js refuses a state-changing call with no CSRF token *in the browser*, before fetch is reached, so no request was sent and nothing appeared in the service log. The confirmation page is the only page that arrives without having made a safe request first, because it is opened straight from a link in an email, so it was the only one with no token. Added `ensureCsrfToken`, which fetches one if there is none, and called it before the POST. The alternative was exempting /api/auth/verify-email from CSRF the way login and first-run setup are. Those are exempt because nothing about them relies on a session; this one is state-changing and a session can exist, so the exemption list stays short. Verified against a live token: page loads, no cookie, priming GET, then POST returns 200 and the account becomes active. Replaying the same link then returns 400, so single-use still holds. The second problem is worse. This commit's earlier state truncated users, organizations and audit_events after every test, and I ran it against the deployment database. It destroyed the two accounts and the organization that were in it. My own "leave the database clean" change from two commits ago caused it: TRUNCATE removes the suite's rows and every other row equally, and the check that had been passing only asserted the suite's rows were gone, which is also true of a full database. It passed because I had just emptied that database by hand, so it never tested the case that mattered. Tried fixing it by deleting selectively instead. That cannot work either: audit_events has no ON DELETE CASCADE, and app.prevent_audit_mutation raises on delete, so the rows cannot be removed one at a time at all. So the safety is not in cleaning up carefully, it is in refusing to start. Two guards: the DSN must be accompanied by UMBRATRANS_TEST_DATABASE_IS_DISPOSABLE=1, because a database name is not evidence and an empty database is indistinguishable from an unconfigured deployment; and the suite then refuses to run if the target holds any rows, naming them. Verified both ways, with a planted row that survives. The deployment database is cleared and first-run setup is open again. |
||
|
umbra audio
|
2b3a093e96 |
Configure the domain mail transport and run in production
bdmurphy.me's MX records point at privateemail.com. Its submission host is smtp.privateemail.com:587 with STARTTLS, offering PLAIN and LOGIN over TLS 1.3. Not the MX host: an MX serves inbound mail and its submission ports are closed, so the obvious place to look has nothing listening. A test message to umbra@bdmurphy.me was delivered, and so was a real confirmation message through the provisioning path, leaving the account in `invited` awaiting confirmation. Flipping to production surfaced two guards that had become limitations rather than protections, and both are fixed rather than routed around: - `serve` refused any production configuration. It is now the deployment command as well as the development one, so it takes --allow-production by name. That flag is an acknowledgement, not the control: the configuration gate independently refuses a deployment without a PostgreSQL DSN, Secure cookies, CSRF, an absolute verification base URL, and a mail transport. - `create_app` refused to build a production app unless a repository was injected, with a message saying the PostgreSQL adapter "is not part of this slice". It is. The check is now about which adapter is in use rather than where it came from, stated twice: once against the configured kind before anything is opened, and once against the built object, because an injected repository bypasses the first. Two tests asserted the old behaviour and were rewritten rather than deleted. One claimed a production app must be refused for want of an injected repository, which would have made a production deployment impossible to start; it now covers both routes to a development store. The other built a production app on an in-memory repository to assert a Set-Cookie header, which the corrected rule refuses; it now asserts the cookie flags on settings that ask for them, with the production half covered by the configuration gate that refuses insecure cookies. /api/health now reports environment=production, repository=postgres, mail_deliverable=true, setup_required=false, and the service survives a restart with its data intact. |
||
|
umbra audio
|
17f8e81fb7 |
Verify the mail transport instead of trusting its configuration
Gmail rejects an account password for SMTP and wants an app password instead, which it says in a reply nobody reads. Configured with smtp.gmail.com:587, a port, and a password, the deployment looked healthy: `_mail_is_deliverable` asked whether the transport was smtp and a host was set, both true, so /api/health reported mail_deliverable=true and the control panel allowed account creation. Every account created that way would have been stranded in `invited` with a confirmation link that was never delivered and cannot be re-sent, and the administrator would have seen a success response each time. That is a data problem created by a configuration problem, invisible until somebody tried to sign in. So the transports report whether they can actually deliver: - `Mailer.verify()` on all three, sharing the connection path with `send` so a probe cannot succeed by a route real delivery would not take. - SmtpMailer authenticates and disconnects, and logs the server's own rejection code, because "wrong password" and "server unreachable" are different problems and a bare False throws that away. The reply contains no credential. - health reports the verified state, not the configured one. And enrollment refuses when mail cannot be delivered, in the service rather than the browser. The control panel already disabled its own buttons, but a client-side check is a courtesy, not a control: anything reaching the process would still have created the account. Exempt is the operator path, which attests the address instead of mailing it, because guarding that would deadlock first-run setup behind SMTP configuration, and development, where the logging transport is the point and the flow would otherwise be untestable. Verified against smtp.gmail.com on both 587 (STARTTLS) and 465 (implicit TLS): reachable, TLS established, AUTH offered, and 535 5.7.8 "Username and Password not accepted" on both. The network and the server are right; the credential is not. 286 tests pass without a database, 295 with one. |
||
|
umbra audio
|
c8b8cdb118 |
Deploy the service: dedicated database, unit, and the public hostname
Brings up a working deployment and, in the process, fixes a unit that could not start and a test suite that had been writing into it. - Created the dedicated `umbratrans` database owned by the non-superuser role `umbra`, applied the control-plane schema and the phase-1 migration. The migration is idempotent, verified by applying it twice. The existing `umbracontrol` control plane is untouched, which is the point of a separate database. - The unit could not start: `ProtectHome=read-only` forbids the write that `uv run` makes to ~/.cache, so it exited 2 and restart-looped until systemd's start limit stopped it. Switched to the venv interpreter directly, which is the right shape for a service anyway: a fixed interpreter, nothing to resolve at boot, and no developer tool's cache able to change what runs. - The integration suite leaves the database as it found it. It creates real accounts, sessions, credentials and audit rows and used to leave every one behind, which was only visible because the scratch database was disposable. Running it against the deployment database filled it with ten test accounts, and first-run setup then reported that an administrator already existed — a confusing failure with no visible cause. Teardown truncates the rows the suite's code creates and leaves the schema's reference data alone. - Bound to 192.168.123.14:8442, the private LAN address the reverse proxy reaches, replacing a static file server that had no API behind it. The public hostname now serves the real application. First-run setup is open, and the deployment runs staging with the mail requirement explicitly waived: /api/health reports mail_deliverable=false, the control panel shows a standing banner, and account creation is refused there. Sign-in and setup work; nothing needing a delivered email does, until SMTP credentials arrive. 288 tests pass against this database, and leave it clean. |
||
|
umbra audio
|
ef6bf1e6a3 |
Add first-run setup so a deployment can be finished in the browser
The gap this closes: the operator command that creates the first administrator refuses to run in production, and production refuses to start without SMTP credentials that only the operator has. So a real deployment could not be brought up at all. Everything else already worked. Setup is offered only when both of these hold, checked against the store on every request rather than cached at boot: * no platform administrator exists, and * the token generated at that boot is presented. The token is what keeps this from being first-requester-wins: anyone who reached the URL before the operator would otherwise become the platform administrator. It is written to the journal, so control of the service is still what authorises the first administrator, and it is regenerated on restart. It is spent when it authorises rather than when the account is created, so a password the policy rejects cannot leave a shown token usable. Five failed attempts burn it. Comparison is constant-time. The status endpoint never returns it, and there is no endpoint that would. One deliberate exception to the log filter: the announcement uses no key=value shape, so redact_text leaves it alone, and a test pins both halves — the setup code still gets through, and `setup_token=` still does not. The filter is not weakened, disabled, or special-cased. Also: - a `staging` environment, so a deployment can be brought up before SMTP exists. It holds every production rule except that one, and the waiver that permits it is refused in production outright, reported by /api/health as mail_deliverable, and shown as a standing banner in the control panel, which then refuses to create accounts. A staging deployment cannot become a convenient way to run something production would not allow. - `serve` accepts `--allow-lan-bind` for a named private address, because the reverse proxy runs on another host. It refuses a wildcard and a public address, so the flag cannot open the port to the internet. AF_UNIX was added to the unit's address-family set, without which the PostgreSQL socket connection cannot be made. - `count_users_with_role` on the repository, counting active assignments rather than profile rows, because a suspended administrator is not one. - setup.html and the sign-in link that appears only while setup is outstanding, plus the deployment runbook and a rootless unit. A test of mine was asserting over every credential row in the shared scratch database rather than the one it created, so it passed only while no account had replaced its one-time password. Scoped it, and added the other half of the property. 279 pass without a database, 288 against one. |
||
|
umbra audio
|
ef9785a5da |
Give the confirmation email a page that resolves
The link the service builds pointed at /verify-email?token=..., but the frontend is mounted as static files and that mount resolves verify-email.html and not a bare /verify-email, so the link 404'd. Following the confirmation email did nothing at all, which is the one step of the flow that cannot be worked around. The existing test followed /api/auth/verify-email, which is the API route and not the link that goes in the message. Two different URLs, so the page a recipient clicks was never exercised. The new test builds the app with a static root the way a deployment does and requests the path taken out of the delivered body, so the two can no longer drift apart. web/verify-email.html and web/assets/verify.js are the landing page. It consumes the token from the query or the fragment on arrival, accepts a pasted link or a bare token, and strips the token from the address bar once used so it is not left in history. It reuses the sign-in card's styling and the shared API client, and carries no inline script or style, to stay inside the content security policy. The duplicate verification view that had been added to the sign-in page is removed in favour of this one, and its dead bindings with it. The sign-in page keeps the panel that appears when the API answers email_not_verified, since that is only ever reached once the password has already been proven correct. |
||
|
umbra audio
|
d1b19fbdb3 |
Update the site icon and wire the UI to email login
Icon: the supplied artwork is now UmbraTrans_Icon2_899x920.jpg, gold and
cyan on white, with the wordmark corrected to "UmbraTrans". It is copied
verbatim to web/assets/umbratrans-icon.jpg as the source, and
web/assets/umbratrans-emblem.png is derived from it and is what the pages
load: the book cropped to its ink bounds, the white page made transparent
by flood-filling inward from the corners so the white lettering inside
the book survives, letterboxed square rather than side-trimmed so no part
of the emblem is cut, and emitted at 256px.
A square crop of a 827x695 emblem has to choose between trimming the
sides and letterboxing, and trimming was cutting the book's frame. The
glow also moved from box-shadow to filter, because a box-shadow follows
the border box rather than the artwork's alpha and would have drawn a
glowing rectangle behind a book-shaped emblem.
The UI was still written against username login, so the sign-in form
could not have worked against the API: it posted {username, password} and
LoginRequest requires {email, password}. Fixed, along with the rest of
the email-verification flow that had no UI at all:
- the sign-in form collects an email address, and its copy says so;
- a 403 email_not_verified, which is only ever reached once the password
is already proven correct, reveals a panel offering a new link. The
resend wording is the server's and is identical for real and unknown
addresses;
- a confirmation view accepts a pasted link or token, and one is opened
automatically when the URL carries a token, which is then stripped from
the address bar so it is not left in history;
- the password-policy hint read the sign-in field to tell the user their
password must avoid their "account name", but the rule checks the
account's username. It now uses the signed-in account's handle, and
says nothing rather than the wrong thing when there is no handle;
- the provision form called email optional while the API requires it,
and let a member be created with no organization, which the store
rejects. Both are caught client-side now with wording that names the
field to fix;
- `serve` accepts --repository postgres, so the app can be exercised
against the authoritative store.
Verified against a live server on PostgreSQL: sign-in with the email
address, forced first-password change, provisioning an invited member,
401 for a wrong password and 403 email_not_verified for the right one,
and byte-identical resend answers for a real and an unknown address.
|
||
|
umbra audio
|
64af39eb68 |
Run the PostgreSQL integration suite against the real schema
The integration suite had never been executed. Running it against a PostgreSQL 16 instance carrying an exact copy of the control-plane schema found seven real defects, all of which were invisible without a database. Migration: - `ADD CONSTRAINT ... UNIQUE USING INDEX` failed when the index was already attached, for both `organizations_slug_key` and `users_username_key`. Because the file runs in one transaction, that error rolled back every other change, leaving a half-applied schema rather than an obviously untouched one. Both are now guarded, so the file applies cleanly to the real schema and is idempotent on re-apply. The adapter: - `get_user_by_email` was never implemented. The protocol stub returned `None`, so email login found no account against a real database while working perfectly against the in-memory one. - The four verification-token methods were never implemented either, so email verification links never worked against PostgreSQL. - The RLS identity GUC was wrong. `app.current_user_id()` reads `app.user_id`, not `app.current_user_id`; setting the similarly named GUC left every policy helper resolving to NULL, so every policy denied and every query returned zero rows. Added `verify_rls_contract()`, called at startup, which proves the two agree. It has teeth: pointed at a wrong GUC it refuses. - `SET` takes no bind parameters, so the statement timeout was a syntax error on every transaction. The identity GUC now uses `set_config`, which is parameterisable and keeps the value bound. - `append_audit_event` returned only `id` and then mapped the whole row, and passed a text `target_id` where a `UUID` was required. The domain: - A member with no organization is storable in the development JSON store and rejected by `user_roles_check`, so the same call succeeded in development and failed in production. Enforced in the service now, and 43 tests were relying on the JSON store's leniency. - `create_account` accepted `require_email_verification` and silently dropped it on the operator path, returning an unverified account to a caller that had asked for a verified one. Now refused, with the reason. - All 26 protocol stubs returned `None` instead of raising, so a missing method was indistinguishable from an empty database. They now raise, and a conformance test checks each repository implements every declared method rather than merely inheriting it. The old conformance test used `hasattr`, which inheritance always satisfies; it passed while five methods were missing. - The bootstrap CLI refused an unsatisfiable role only after the operator had typed a one-time password. The migration README now records which review questions are answered and which remain, including that RLS is enabled but not forced, so the adapter's own scoping is currently the only tenant control and the isolation tests below it are still outstanding. 255 pass without a database, 263 against one. |
||
|
umbra audio
|
0461dff9d2 |
Use the email address as the login identifier and require confirmation
Sign-in now takes the email address rather than a username. This needed no migration: users.email is already citext NOT NULL UNIQUE in the control plane, so it is a case-insensitively unique login key on its own. username survives as an optional display handle for audit lines and is never a login path, which removes the redundancy of asking an administrator for two identifiers. An account created by an administrator now starts in the control plane's invited state and cannot sign in until a single-use link is consumed. The token maps onto bootstrap_tokens, which was already shaped for exactly this and had been unused, so that needed no migration either. Modelling invited as a real state also retires the adapter's earlier fail-closed refusal of that value, which existed because invited was previously unrepresentable. Three decisions worth stating plainly. The "email not verified" message is sent only after the password has been verified. Reporting it earlier would turn the login form into an oracle for which addresses have accounts, which is the one thing the uniform authentication error exists to prevent. A test asserts that a wrong password and an unknown account are byte-identical, and it caught a real bug: the check was first written before the password comparison, and the generic "not active" branch swallowed invited accounts. Both are now ordered deliberately. The operator bootstrap path does not require verification. The operator is attesting to the address themselves, which is the standing an administrator already has, and requiring a delivered email there would deadlock first-time setup on SMTP configuration that does not exist yet. The logging mail transport is refused in production. A deployment that logged links instead of sending them would appear to work while every new account sat permanently unverified. The development transport writes the link with the token redacted, because a log is where live credentials end up. That means a human cannot complete the flow by reading the log; the test harness captures the message in memory instead. This is a deliberate limitation rather than an oversight, and it is documented. Also: production now requires a mail transport and an absolute verification base URL alongside the existing DSN gate, because an account that cannot be verified cannot be used at all. 269 tests pass, 3 skipped. Verified end to end against a real uvicorn server: bootstrap login, organization creation, an invited account with its one-time password, and a handle rejected as a login path. |
||
|
umbra audio
|
31fd8720ec |
Add the PostgreSQL adapter for the authoritative store
Implements AccountRepository against the control-plane schema plus the phase-1 identity migration, and wires it in so production can select it. The adapter is not a thin SQL wrapper, because the control plane does not store the two things the application needs on the users row. Role and organization come from user_roles. There is no role column and no organization column on users; a role is assigned through user_roles with a scope, and app.validate_user_role_scope() enforces that the role's scope matches the assignment. Resolving one role per user is therefore a join, and a user holding more than one active assignment is refused rather than resolved to an arbitrary row. The organization on that assignment becomes the scope of every administrative decision, so guessing would hand out the wrong tenant's authority. Reconciling this with project-scoped membership stays an open review question, recorded as such. Email became a required field on account creation. users.email is NOT NULL UNIQUE, so accepting a missing address would let account creation succeed against the development repositories and then fail on the authoritative store. It is now required in the service, the API schema, and the bootstrap command, so it fails identically everywhere. Audit identifiers now belong to the store. audit_events.id is a bigint identity, so the adapter reports that value; the application correlation UUID is preserved in the detail and never displaces the row key. AuditEvent.id is a string for that reason, which removes a type the two stores disagreed about. Three further fail-closed behaviours: a users.status of invited or locked is refused rather than coerced onto active; a row with mfa_required is refused, because this phase cannot satisfy a second factor and authenticating it anyway would grant access to someone who was supposed to present one; and a missing organization blocks access rather than granting it. save_user deliberately does not write username or email, since both are identity keys with no reviewed change path and no audit event behind them. Every operation runs in one transaction on one pooled connection and publishes the caller with SET LOCAL app.current_user_id, so a returned connection cannot carry one request's identity into the next. That is inert until the reviewed RLS policies exist; the explicit WHERE clauses enforce scope today. A statement timeout bounds how long a pooled connection can be held. Production fails closed at two independent gates: the settings layer refuses without a DSN, and the repository boundary refuses to build a PostgreSQL store without one. Verified by running both, including that an unreachable host refuses to start rather than degrading. The migration gains an index on audit_events (target_type, target_id), which the administrator audit filter needs and nothing covered, and the review questions now include the three things that must be checked before provisioning: the roles rows and their scopes, any row with mfa_required set, and any user holding more than one active assignment. Verification, and its limit. 236 tests pass with 3 skipped. The adapter's row mapping, role resolution, conflict mapping, and configuration are unit tested and run everywhere. Its SQL, transactions, and pooled reuse are NOT executed: this environment has no database credentials, because sudo to the postgres user needs a password, there is no umbra database role, and there is no .pgpass. The integration suite is written and selected by UMBRATRANS_TEST_DSN, and its fixture refuses any DSN whose database name contains umbracontrol. A green run here says nothing about whether the SQL is correct. |
||
|
umbra audio
|
08a44dbacd |
Wire the browser client to the real account and organization API
The pages under web/ were a static mockup with hardcoded names and counts. They now call the real backend, and the areas with no backend are explicit empty states rather than invented numbers. Real: sign in, the mandatory first-login password change, sign out, the account table with filters and lifecycle actions, organization create/rename/re-slug/ status, the audit log, and dashboard counts scoped to the caller. The project workspace keeps its Studio identity and its real session wiring, but its project, maintenance, and statistics areas now say which backend work is pending instead of showing sample figures. Removed the mock release-readiness pipeline and the fake production metrics rather than restyling them. The client depends on properties the server already enforces, and states them where a future editor will look: - No secret is sent by a client. Account provisioning has no password field; the service generates the one-time password and returns it once. - That one-time password lives in exactly one variable, is cleared when the reveal panel closes and on pagehide, and is never written to storage, a cookie, the URL, the page title, or the console. - Every unsafe request carries X-CSRF-Token read fresh from the readable cookie, and GET /api/health on page load bootstraps it because the middleware issues none on an unsafe method and login is exempt. - Nothing is trusted from the browser. Controls a role cannot use are hidden, and the service re-checks each one. Rendered markup is built by an el() helper that routes all server-supplied text through textContent, with no innerHTML anywhere. The response CSP is script-src 'self' and style-src 'self', so an inline script or style attribute would be refused by the browser; there is none. Verification, and its limits. The banned-pattern grep for inline script, inline handlers, style attributes, innerHTML, storage, and console output comes back empty. Replaying the exact request sequence the client performs against a real uvicorn server confirmed all 86 contract assertions: every field the JavaScript reads is present, the CSRF cookie is issued by the health probe, the one-time password is absent from every later response, and the error codes the UI branches on are the ones the API returns. The client was NOT executed in a browser. No desktop browser is attached to this session and no JavaScript runtime is installed on this host, so the one-time-password reveal, the modal focus trap, and the role gating are verified by reading and by the contract replay, not by driving a real browser. That gap needs closing before this is trusted. web/README.md replaces the mock description, including the run command and the fact that this is not a production deployment: create_app still refuses to start in production because the PostgreSQL adapter does not exist. |
||
|
umbra audio
|
0b9c4d4220 |
Add organization tenancy and same-origin frontend mounting
The administrator interface needs organizations to manage, and the slice had none: users.organization_id was a bare UUID pointing at nothing, so there was no tenant to create, list, or validate an account against. Tenancy: - Organization record with name, slug, and status active|suspended|archived, matching the control-plane vocabulary rather than reusing the account one. - Reads are open to any administrator; every write is platform-admin only, so the tenant boundary stays under a single authority. An organization admin is pinned to the organization on their own principal. - A derived slug is normalized from the display name, but a slug the operator typed must already be canonical, so nothing is silently rewritten behind their back. archived is terminal. - GET /api/uadmin/summary for the dashboard, recomputed per request from the same scoped queries the list endpoints use, so a summary cannot disclose a number the caller could not reach by listing. Two integrity gaps closed: 1. Account creation accepted any UUID as an organization. It now requires the organization to exist and to be active. 2. Suspending an organization revoked member sessions but did not stop new logins, because each member's own account status stays active. Both authenticate and resolve_session now refuse a member of a non-active organization, and a missing organization record blocks access rather than granting it. A test asserts the member is still active yet locked out. Migration, still unapplied: section 4 creates organizations with CREATE TABLE IF NOT EXISTS so a control-plane table is skipped, never redefined, and the rollback drops it only when this migration actually created it. The membership divergence is recorded as the top open review question rather than decided: user_roles is the control plane's membership source, this phase uses a column, and the migration adds no membership column so the choice stays open. Serving: UMBRATRANS_STATIC_ROOT mounts the browser client at / after the API routes, so the session cookie and CSRF header work without CORS. HTML responses get their own CSP allowing only same-origin script and style, which is why the frontend carries no inline script or style. Verification: 204 tests pass (21 new organization tests plus the existing suite), ruff check and format clean. Also exercised against a real uvicorn server rather than only the test client: bootstrap, forced password change, organization creation, one-time password issue, member lockout on organization suspension, and rejection of an account assigned to a suspended tenant. |
||
|
umbra audio
|
6482921d0b |
Run the static prototype under systemd
The prototype was served by a bare `python3 -m http.server` in a foreground shell. It has no restart policy, so it died silently and the public URL returned a 502 from Caddy with valid DNS and a valid certificate already in place. This unit makes the failure mode visible and recoverable. umbratrans-web-prototype.service: - Restart=on-failure with a 5s delay and a 120s/5-attempt start limit, so a crash loop stops retrying but a clean stop always starts again. - Binds 0.0.0.0:8442 serving web/; Caddy keeps terminating TLS upstream. - Sandboxed: runs unprivileged with an empty capability bounding set, no new privileges, read-only home and system, private tmp and devices, system call filtering, and no access to kernel or control-group interfaces. - Journal output to the user journal for inspection. PrivateUsers=yes is documented as deliberately omitted. It hides /etc/group from the manager, so systemd cannot resolve supplementary groups and the unit fails with status=216/GROUP. A probe unit confirmed the remaining sandbox directives are all valid in user scope. Verified: all three routes and the icon return 200 locally and the public HTTPS URL returns 200. SIGKILL of the main PID recovered automatically within the restart delay. `systemd-analyze verify` is clean. Two follow-ups requiring root are recorded in web/README.md: the rootless install must drop User= because a --user manager cannot setuid, and `loginctl enable-linger umbra` is needed for the service to survive logout and start at boot. |
||
|
umbra audio
|
b474995414 |
Add phase-1 account management backend slice
Implements the account and user-management system behind the new web UI against development repositories only. Nothing here connects to, reads, or modifies the production umbracontrol database, and no account is created in any approved environment. Account service and API: - Local temporary-password and OIDC enrollment, separated by type. - Argon2id credentials, must-change enforcement, server-side sessions with hashed tokens, idle and absolute deadlines, rotation and revocation. - Administrator account management: list, get, create, suspend, reactivate, deactivate, revoke sessions, org-scoped audit view. - Hidden-prompt operator command for the first administrator, refusing production mode and refusing to run without a TTY. Temporary passwords are never accepted from a client. The request schema has no password field and refuses unknown fields; the administrator path generates the one-time password in process, hashes it immediately, and returns it once in the creation response. The account can reach no route, including /api/auth/me, until it replaces that password, after which the one-time value no longer authenticates. Security posture: fail-closed configuration, Secure cookies in production, double-submit CSRF, constant-time token comparison, generic authentication errors, audit detail allow-listing, and logger-tree secret redaction. Reviewed but unapplied: migrations/0001_phase1_identity.sql adds users.username, a separate user_credentials table, and additive session columns, with a rollback and open review questions. The PostgreSQL adapter is deliberately absent, so the application refuses to start in production rather than run against an unreviewed store. Verification: 169 tests pass (148 new, 21 pre-existing); ruff check, ruff format, compileall, and make web-check are clean. |
||
|
umbra audio
|
37b2056808 | Add UmbraTrans UI prototype and integration design | ||
|
umbra audio
|
6a3baacb34 | Harden agent communication validation | ||
|
umbra audio
|
127a048cfe | Restrict package discovery to agent_comm | ||
|
umbra audio
|
8a61d9298b | Add verified NAS backup automation | ||
|
umbra audio
|
fc6c36331b | Bootstrap UmbraTrans agent communication workspace |