0ab1f0f1de
This is the verification I said could not be done until the machine rebooted. The reboot proved the boot path wrong, and uncovering why took three rounds. **The boundary was gone.** /dev/nvidiactl came back 0666 root:root, so the controller could open the card again - the exact thing the udev rule was applied to prevent. **Why: the unit ran before anything existed.** umbra-nvidia-devices ran at 05:38:24, ordered After=systemd-udev-settle, globbed /dev, found no nodes, and exited 0. udev-settle means "udev has finished its queue", not "the proprietary driver has finished initialising". The nodes appeared moments later at 0666 and nothing revisited them. The helper ended in an unconditional `exit 0` - I had written that deliberately, so an early boot would not fail a unit - and it is precisely what made "I fixed the permissions" indistinguishable from "there were nothing to fix". **Why the first repair also failed: nvidia-persistenced resets the mode.** It chmods /dev/nvidiactl back to 0666 on every initialisation, via nvidia-modprobe. That is out of udev's reach entirely, because udev does not manage these nodes. So the fix is no longer an ordering problem but an eventual one: the unit now runs After=nvidia-persistenced.service, and a two-minute timer re-asserts the permissions because the trigger that breaks them can fire at any time. **Why the timer did nothing either.** RemainAfterExit=yes on the oneshot keeps it "active (exited)", so systemd treated every activation as already satisfied: the timer fired on schedule, the service reported Result=success and ExecMainStatus=0, and the permission was still wrong two minutes later. That is the original bug repeated one level out, inside the mechanism built to repair it. Removed, and a test now pins it - by directive name, with comments stripped, since the unit documents the omission by naming it and a substring search failed on the explanation. **I also broke nvidia-persistenced.** It drops to uid 122 with setuid and without initgroups(), so it never receives the video supplementary group and failed with "Insufficient Permissions" once the nodes were 0660 root:video. A systemd drop-in runs it as root - upstream's own default - which works without widening the device permissions. It was running fine before. The helper now waits for the nodes, verifies the modes afterwards, and exits non-zero under --strict rather than reporting a no-op as success. It takes UMBRA_NVIDIA_DEVDIR so its three paths can be tested without root and without a GPU; the first attempt at that test pointed CANARY at a non-existent path and still exited 0, because the fix loop globbed /dev directly and the override only reached the wait. Also: nvidia-device-access.sh `show` reported "cannot measure the GPU" when a cached sudo credential had simply expired - a finding about the machine when the truth was that the tool could not ask. It now distinguishes no from could-not-check, because I was misled by it during this check. **Still outstanding, and I could not finish it.** The repository is correct; the installed state is stale, because clearing the unit's active (exited) state needs root and that authorization was declined. As of now /dev/nvidiactl is 0666 and the controller can open it. To finish: systemctl stop umbra-nvidia-devices.service systemctl start umbra-nvidia-devices.service stat -c '%a %U:%G %n' /dev/nvidiactl and expect 660 root:video. Otherwise the system came through the reboot intact: the deployment was down only because a leftover `python3 -m http.server` from an earlier session was holding port 8442, and the backend mounts the same files itself. Deployment healthy, database unchanged at users=2 orgs=3 projects=0 audit=43 with no compute or datastore tables, ollama serving, agent account intact, zero stray agents. 471 pass with a database, 430 without.
46 lines
2.2 KiB
Desktop File
46 lines
2.2 KiB
Desktop File
# Restrict the NVIDIA device nodes to the video group, and keep them that way.
|
|
#
|
|
# Two facts about the nvidia driver shape this unit, and both were learned from a
|
|
# reboot that went wrong (2026-10-01).
|
|
#
|
|
# 1. The driver creates /dev/nvidia* in the kernel without registering them in
|
|
# sysfs and without letting udev process them. A rule matching the device nodes
|
|
# never runs; the udev side hooks the driver's bind event instead.
|
|
#
|
|
# 2. **nvidia-persistenced resets /dev/nvidiactl to 0666 every time it
|
|
# initialises**, via nvidia-modprobe, and that happens after any boot-time
|
|
# oneshot has run. Ordering `After=systemd-udev-settle` was not enough: on
|
|
# 2026-10-01 the unit ran at 05:38:24, before the driver had created any node,
|
|
# found nothing, exited 0, and the permissions were 0666 from then on.
|
|
#
|
|
# So this unit runs *after* nvidia-persistenced and waits for the nodes rather
|
|
# than assuming them, and umbra-nvidia-devices.timer re-checks periodically
|
|
# because the trigger that resets the mode is outside udev's reach and can fire
|
|
# at any time.
|
|
#
|
|
# --strict turns "the nodes never appeared" into a failed unit. A unit that
|
|
# reports success while having changed nothing is worse than one that fails: the
|
|
# failure is visible, and that is the whole difference between this being caught
|
|
# in a log and this being caught in six days.
|
|
|
|
[Unit]
|
|
Description=Restrict the NVIDIA device nodes to the video group
|
|
After=systemd-udev-settle.service nvidia-persistenced.service
|
|
Wants=nvidia-persistenced.service
|
|
|
|
[Service]
|
|
Type=oneshot
|
|
# Deliberately NOT RemainAfterExit=yes.
|
|
#
|
|
# The timer activates this unit, and a oneshot with RemainAfterExit stays
|
|
# "active (exited)" after it completes - so systemd treats every subsequent
|
|
# activation as already satisfied and runs nothing. The first version of the
|
|
# timer did exactly that: the timer fired on schedule, the service reported
|
|
# Result=success and ExecMainStatus=0, and /dev/nvidiactl was still 0666 two
|
|
# minutes later. A timer that reports success while doing nothing is the same
|
|
# failure as the boot unit that did, one level further out.
|
|
ExecStart=/usr/lib/udev/nvidia-restrict-permissions --wait=60 --strict
|
|
|
|
[Install]
|
|
WantedBy=multi-user.target
|