umbragit/UmbraTrans
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
master
UmbraTrans/umbratrans_web/bootstrap.py
umbra audio e35d7a144f Add Apache-2.0 LICENSE, NOTICE, and SPDX headers
2026-10-18 19:01:05 -07:00

532 lines
22 KiB
Python
Raw Permalink Blame History

# Copyright 2026 Umbra Tools (umbratools.com)
# SPDX-License-Identifier: Apache-2.0
"""The bootstrap: one file, and it decides for itself whether it may act as root.
The alternative this replaces: a person reading setup instructions and running them.
That does not scale past a couple of nodes, it puts a credential in a terminal, and it
makes the controller a courier - every new node needs a human in the loop.
The rule that shapes everything here: **the script works identically with or without
root.** It asks, and branches.
- With root it installs to `/opt/umbra` and registers a system systemd unit, so the
agent runs as a dedicated `umbra-node` account regardless of who ran the bootstrap.
- Without root it installs under the invoking user's home and registers a *user* systemd
unit, so the agent runs as that user.
This is the exclusive/shared distinction encoded rather than assumed. An exclusive node
(238) gets the system install, because it should survive logout and should not run as
an administrator's account. A shared node (the audio machine, which has real production
work on it) gets the user install, and needs no local administrator involvement at all -
which matters because on a shared node the administrator is a person with other duties,
not us.
Nothing here needs a password and nothing is sent back except what pairing already
returns. The script is read over HTTP and executed, so its integrity rests on the
pairing code: the script can only do what the code it holds authorises, and the code is
single-use and node-name-bound.
Deliberately not here:
- No package installation. `apt install` needs root and changes the machine's shape
without being asked. Missing prerequisites are reported instead.
- No GPU device rules. Those need root, are specific to nvidia hardware, and belong to
the agent-update work rather than to first contact.
- No self-updating. The agent has an update path; the bootstrap runs once.
"""
from __future__ import annotations
BOOTSTRAP_VERSION = "1.0.0"
BOOTSTRAP = r'''#!/bin/sh
# UmbraTrans node bootstrap.
#
# Generated by the controller. Read it before running it: it is a script fetched over
# the network, and "trust me" is not a security property.
#
# What it does, in order:
# 1. checks the machine is one this can run on, and says plainly if it is not
# 2. pairs with the controller using the code below, getting a credential
# 3. installs the node agent
# 4. registers it with systemd, as root or as you, depending on what you are
# 5. measures the machine by executing instructions, and reports what it found
#
# What it needs: a POSIX shell, curl, python3, and systemd. No password is used and
# none is asked for. Root is used if it is available and skipped cleanly if not.
set -eu
BOOTSTRAP_VERSION="__BOOTSTRAP_VERSION__"
NODE_NAME="__NODE_NAME__"
PAIRING_CODE="__PAIRING_CODE__"
# A pairing code of 000000 means "nobody filled this in" - which is exactly what the
# template served by GET /api/node/bootstrap carries. So the script asks.
#
# Without this the two flows cannot both exist. `curl ... | sh` downloads a script with
# no code in it, and a script that cannot prompt is a script that can never pair. The
# console's rendered copy has the code substituted and skips the prompt, so an
# operator who already has a rendered script is not asked for a number they do not
# need.
if [ "$PAIRING_CODE" = "000000" ]; then
printf 'Enter the six digit pairing code: ' >&2
read -r PAIRING_CODE
[ -n "$PAIRING_CODE" ] || die "A pairing code is required. Ask an administrator for one."
fi
# The MAC of the first non-loopback interface that has one.
#
# Sent so the controller can make this machine's name unique: several machines can
# legitimately be called VM238, and the controller resolves the real name as
# `VM238-AC4B` using the last four hex digits of this. It is asked for here rather than
# derived from the base name because the controller cannot know a MAC before the machine
# speaks, and the pairing code is bound to the base name.
#
# Every step of this is allowed to fail. A machine that cannot report a MAC still pairs -
# it keeps the base name, and the controller refuses a second node of that name as it
# always did. Guessing here would be worse than not knowing: a wrong MAC produces a name
# that looks unique and is not.
NODE_MAC=""
for _iface in $(ls /sys/class/net 2>/dev/null); do
case "$_iface" in lo|docker*|veth*|br-*|virbr*) continue ;; esac
if [ -r "/sys/class/net/$_iface/address" ]; then
_mac=$(cat "/sys/class/net/$_iface/address" 2>/dev/null || true)
case "$_mac" in
??:??:??:??:??:??) NODE_MAC="$_mac"; break ;;
esac
fi
done
[ -n "$NODE_MAC" ] || say " no MAC address found; pairing with the plain node name"
# The same reasoning for the name. The template carries a placeholder, and the name is
# what the code is bound to, so a template run without asking pairs nothing and the
# failure points at the code - which is the wrong thing to be looking at.
if [ "$NODE_NAME" = "unpaired-node" ]; then
printf 'Node name (as the administrator entered it): ' >&2
read -r NODE_NAME
[ -n "$NODE_NAME" ] || die "A node name is required. It must match the code exactly."
fi
CONTROLLER="${UMBRA_CONTROLLER:-__CONTROLLER__}"
AGENT_DIR="__AGENT_DIR__"
CONFIG_PATH="__CONFIG_PATH__"
UNIT_NAME="__UNIT_NAME__"
SERVICE_USER="__SERVICE_USER__"
INSTALL_ROOT="__INSTALL_ROOT__"
say() { printf '%s\n' "$*"; }
step() { printf '\n== %s\n' "$*"; }
die() { printf '\nERROR: %s\n' "$*" >&2; exit 1; }
# ---------------------------------------------------------------- who am I
# The controller this script was fetched from. Taken from the request rather than baked
# in, because the template is served by every deployment and has to work on all of them.
# The first version hardcoded a placeholder host and the failure was "could not resolve
# host", with an error message that pointed at the pairing code - the wrong thing
# entirely, and the reason this is worth stating plainly.
case "$CONTROLLER" in
*controller.invalid*)
CONTROLLER="__DEFAULT_CONTROLLER__"
say " controller $CONTROLLER (defaulted; override with UMBRA_CONTROLLER=...)"
;;
*) say " controller $CONTROLLER" ;;
esac
step "Checking this machine"
[ "$(uname -s)" = "Linux" ] || die "This bootstrap is for Linux. Found $(uname -s)."
say " os $(. /etc/os-release 2>/dev/null && echo "${PRETTY_NAME:-unknown}" || echo unknown)"
say " kernel $(uname -sr)"
say " arch $(uname -m)"
have() { command -v "$1" >/dev/null 2>&1; }
have curl || die "curl is required and was not found."
have python3 || die "python3 is required and was not found. Install it and re-run; this script will not install it for you."
say " python $(python3 -V 2>&1)"
PY_OK=$(python3 - <<'EOF' 2>/dev/null || echo no
import sys
print("yes" if sys.version_info >= (3, 9) else "no")
EOF
)
[ "$PY_OK" = yes ] || die "python3 3.9 or newer is required; this is $(python3 -V 2>&1)."
# ------------------------------------------------------------- root or not
IS_ROOT=no
if [ "$(id -u)" = "0" ]; then
IS_ROOT=yes
fi
say " privilege $([ "$IS_ROOT" = yes ] && echo 'root (system install, agent runs as a dedicated account)' \
|| echo "user (installs under $HOME, agent runs as you)")"
if [ "$IS_ROOT" = yes ]; then
# A system install that ran the agent as root would defeat the point of the GPU
# boundary and of every other boundary: the agent executes work sent by the
# controller. It runs as its own unprivileged account.
if id -u "$SERVICE_USER" >/dev/null 2>&1; then
say " service existing account $SERVICE_USER"
else
useradd --system --create-home --home-dir "$INSTALL_ROOT/home" \
--shell /usr/sbin/nologin "$SERVICE_USER" \
|| die "Could not create the service account $SERVICE_USER."
say " service created system account $SERVICE_USER"
fi
mkdir -p "$AGENT_DIR"
chown -R "$SERVICE_USER" "$INSTALL_ROOT"
RUN_AS="$SERVICE_USER"
else
mkdir -p "$AGENT_DIR"
RUN_AS=""
say " service running as $(id -un); no system account is created"
fi
say " systemd $(systemctl --version >/dev/null 2>&1 && echo present || echo 'NOT FOUND - the agent will not be started automatically')"
# ------------------------------------------------------------------ pair
step "Pairing with the controller"
say " this will be visible to the controller: node $NODE_NAME"
PAIR_RESPONSE=$(curl -fsS --max-time 30 \
-X POST "$CONTROLLER/api/node/pair" \
-H 'Content-Type: application/json' \
-d "{\"code\":\"$PAIRING_CODE\",\"node_name\":\"$NODE_NAME\",\"mac\":\"$NODE_MAC\",\"agent_version\":\"bootstrap-$BOOTSTRAP_VERSION\"}" \
) || die "Pairing failed. Check the code has not expired (they last 10-30 minutes) and that the node name matches exactly."
NODE_KEY=$(printf '%s' "$PAIR_RESPONSE" | python3 -c 'import json,sys; print(json.load(sys.stdin).get("key",""))')
KEY_PREFIX=$(printf '%s' "$PAIR_RESPONSE" | python3 -c 'import json,sys; print(json.load(sys.stdin).get("key_prefix",""))')
[ -n "$NODE_KEY" ] || die "The controller returned no credential. Response was: $PAIR_RESPONSE"
# The controller may have resolved the name to something unique - `VM238-AC4B` - because
# another machine already holds the base name. From here on the script uses the name it
# was *given*, not the one it asked for: the two differ, and reporting one while
# registering the other is how a node ends up appearing under a name it does not have.
PAIRED_NAME=$(printf '%s' "$PAIR_RESPONSE" | python3 -c 'import json,sys; print(json.load(sys.stdin).get("node_name",""))')
[ -n "$PAIRED_NAME" ] || PAIRED_NAME="$NODE_NAME"
say " paired credential $KEY_PREFIX"
if [ "$PAIRED_NAME" != "$NODE_NAME" ]; then
say " name '$NODE_NAME' was taken, so this node is '$PAIRED_NAME'"
fi
# ---------------------------------------------------------------- config
step "Writing the node configuration"
mkdir -p "$(dirname "$CONFIG_PATH")"
umask 077
cat > "$CONFIG_PATH" <<EOF
{
"controller_url": "$CONTROLLER",
"node_name": "$PAIRED_NAME"
}
EOF
chmod 600 "$CONFIG_PATH"
say " wrote $CONFIG_PATH (mode 600, no key inside it)"
# The key is not in the config file on purpose. A config holding a secret is a secret
# that ends up in a backup, and the agent reads UMBRA_NODE_KEY from the environment
# precisely so it need not.
# ---------------------------------------------------------------- install
step "Fetching the node agent"
# The agent endpoint is credentialed like every other node route, so the key just
# issued is sent with the request. Fetching it anonymously was the first version's
# mistake and it failed exactly as it should: 401, with nothing written.
AGENT_URL="$CONTROLLER/api/node/agent"
curl -fsS --max-time 60 -H "Authorization: Bearer $NODE_KEY" \
"$AGENT_URL" -o "$AGENT_DIR/node_agent.py" \
|| die "Could not fetch the agent from $AGENT_URL (the request is authenticated with the credential just issued)."
chmod 644 "$AGENT_DIR/node_agent.py" # source, not a binary: no +x, no shebang
AGENT_VERSION=$(grep -m1 '^AGENT_VERSION' "$AGENT_DIR/node_agent.py" 2>/dev/null | sed 's/.*"\(.*\)".*/\1/' || echo unknown)
say " installed $AGENT_DIR/node_agent.py (version $AGENT_VERSION)"
# It must at least parse before anything tries to run it as a service. A syntax error
# here would otherwise surface as a unit that fails to start, with the real cause
# buried in a journal.
python3 -c "import ast,sys; ast.parse(open(sys.argv[1]).read())" "$AGENT_DIR/node_agent.py" \
|| die "The fetched agent does not parse as Python. Refusing to install it."
# ---------------------------------------------------------------- systemd
step "Registering the service"
UNIT_PATH="/etc/systemd/system/$UNIT_NAME.service"
# The credential lives in its own 0600 file, referenced by systemd's EnvironmentFile.
# Three alternatives were rejected and the reasons matter:
#
# * In the unit file - unit files are world-readable. The key would be too.
# * In the JSON config - a config holding a secret ends up in a backup, and the
# agent deliberately refuses to read the key from the config when the environment
# can carry it.
# * Piped on stdin - needs shell plumbing that systemd does not expand, and a helper
# script that becomes a second thing to get right. The environment is simpler and is
# what the agent already prefers.
#
# EnvironmentFile is read by systemd as root before dropping privileges, which is what
# lets a unit running as an unprivileged account still get a secret it cannot read
# itself.
ENV_FILE="__CONFIG_PATH__.env"
write_unit() {
target="$1"
cat > "$target" <<UNIT
[Unit]
Description=UmbraTrans node agent ($NODE_NAME)
After=network-online.target
Wants=network-online.target
[Service]
Type=simple
EnvironmentFile=$ENV_FILE
# Invoked through python3 explicitly. The agent file has no shebang, and making it
# executable would leave the kernel trying to run it as a binary - which is what the
# first version did, and produced "Exec format error" with a service in a restart loop
# and nothing on the controller. The interpreter is named explicitly instead.
ExecStart=/usr/bin/env python3 $AGENT_DIR/node_agent.py --config $CONFIG_PATH
Restart=always
RestartSec=5
# The agent executes work the controller sends. It has no reason to be an
# administrator, and every reason not to be.
$(if [ "$IS_ROOT" = yes ]; then echo "User=$SERVICE_USER"; echo "Group=$SERVICE_USER"; fi)
NoNewPrivileges=yes
ProtectSystem=strict
ProtectHome=yes
PrivateTmp=yes
ReadWritePaths=$AGENT_DIR
ProtectKernelTunables=yes
ProtectKernelModules=yes
ProtectControlGroups=yes
RestrictSUIDSGID=yes
[Install]
WantedBy=$WANTED_BY
UNIT
}
step "Registering the service"
# The key file, written under umask 077 so it is never briefly world-readable.
umask 077
printf 'UMBRA_NODE_KEY=%s\n' "$NODE_KEY" > "$ENV_FILE"
chmod 600 "$ENV_FILE"
if [ "$IS_ROOT" = yes ]; then
# Read by systemd as root, so the unprivileged service account can have a secret it
# could not otherwise read. The file stays root-owned.
chown root:root "$ENV_FILE" 2>/dev/null || true
UNIT_PATH="/etc/systemd/system/$UNIT_NAME.service"
WANTED_BY="multi-user.target"
write_unit "$UNIT_PATH"
systemctl daemon-reload
systemctl enable --now "$UNIT_NAME.service" \
|| die "The service was installed but would not start. Check: journalctl -u $UNIT_NAME -n 50"
say " started $UNIT_NAME.service (system unit, runs as $SERVICE_USER)"
else
# A user unit cannot ProtectHome=strict and still read its own config, and it does
# not need to: there is no privilege boundary being defended here, only a process
# running as the person who ran the bootstrap.
UNIT_PATH="$HOME/.config/systemd/user/$UNIT_NAME.service"
WANTED_BY="default.target"
mkdir -p "$(dirname "$UNIT_PATH")"
write_unit "$UNIT_PATH"
# sed rather than a branch inside write_unit: the hardening directives are simply
# wrong for a user unit, and removing them after the fact is clearer than
# conditionals sprinkled through the unit.
# Filtered through a temp file rather than `sed -i`: BSD sed reads the next
# argument as a backup suffix, so `sed -i -e ... file` on macOS writes "file-e"
# and leaves the original alone. The unit would then be installed with hardening
# directives that cannot work, and nothing would say so.
FILTERED="$UNIT_PATH.filtered"
grep -v -e '^ProtectSystem=' -e '^ProtectHome=' -e '^ProtectKernel' \
-e '^ProtectControlGroups=' -e '^RestrictSUIDSGID=' \
"$UNIT_PATH" > "$FILTERED" || true
mv "$FILTERED" "$UNIT_PATH"
systemctl --user daemon-reload \
|| die "systemctl --user is unavailable. Log in and run 'systemctl --user daemon-reload', then start the service."
systemctl --user enable --now "$UNIT_NAME.service" \
|| die "The user service was installed but would not start. Check: systemctl --user status $UNIT_NAME"
say " started $UNIT_NAME.service (user unit, runs as $(id -un))"
say
say " A user service stops when you log out unless lingering is enabled. On a shared"
say " machine a local administrator needs to run, once:"
say " loginctl enable-linger $(id -un)"
fi
# ---------------------------------------------------------------- measure
step "Measuring this machine"
say " The agent measures what it can do by executing instructions, not by reading"
say " /proc/cpuinfo - a hypervisor can present a CPU that does not exist."
say
if [ -x "$AGENT_DIR/node_agent.py" ]; then
python3 - "$AGENT_DIR" <<'PROBE' || say " (capability probe unavailable; the agent will report this)"
import sys, pathlib
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parent))
try:
from umbratrans_web.capability import collect
except ImportError:
sys.exit(0)
profile = collect(node_name="__NODE_NAME__")
for feature in profile.features:
mark = {"present": "yes", "absent": "NO ", "unverified": "??"}[feature.status]
line = f" {mark} {feature.name}"
if feature.detail:
line += f" {feature.detail}"
print(line)
if profile.cpu_declaration_suspect:
print()
print(" the machine's description disagrees with what was measured:")
for note in profile.cpu_declaration_suspect.split("; "):
print(f" ! {note}")
PROBE
fi
# ---------------------------------------------------------------- done
step "Done"
say " node $NODE_NAME"
say " key $KEY_PREFIX (shown once; stored at $CONFIG_PATH.key with mode 600)"
say " agent $AGENT_DIR/node_agent.py"
say
say " The node stays 'provisioning' until it heartbeats, then 'online'."
say " To watch it: journalctl -u $UNIT_NAME -f (system)"
say " journalctl --user -u $UNIT_NAME -f (user)"
say
say " The pairing code has now been spent and cannot be reused."
'''
def render_full(
*,
node_name: str,
pairing_code: str,
controller: str,
agent_dir: str,
config_path: str,
unit_name: str,
service_user: str,
install_root: str,
bootstrap_version: str = BOOTSTRAP_VERSION,
) -> str:
"""Fill the template.
Substitution is explicit and total - every placeholder is checked - because a
bootstrap that renders with an unsubstituted ``__NODE_NAME__`` would pair under a
literal name and fail with a message that points nowhere near the cause.
"""
values = {
"__NODE_NAME__": node_name,
"__PAIRING_CODE__": pairing_code,
"__CONTROLLER__": controller.rstrip("/"),
"__AGENT_DIR__": agent_dir,
"__CONFIG_PATH__": config_path,
"__UNIT_NAME__": unit_name,
"__SERVICE_USER__": service_user,
"__INSTALL_ROOT__": install_root,
"__BOOTSTRAP_VERSION__": bootstrap_version,
}
rendered = BOOTSTRAP
for placeholder, value in values.items():
rendered = rendered.replace(placeholder, value)
leftovers = [
line for line in rendered.splitlines() if "__" in line and line.strip().startswith("#")
]
# Not fatal: a leftover in a comment is harmless. Recorded so it is visible rather
# than silently shipped.
if leftovers:
rendered += "\n# NOTE: unsubstituted placeholders remained: " + ", ".join(leftovers) + "\n"
return rendered
#: Where things go when the bootstrap runs as root, and when it does not. The two sets
#: differ because a system install must live somewhere a service account can read and
#: a user's home is the only place a user install can rely on.
LAYOUT_ROOT = {
"install_root": "/opt/umbra",
"agent_dir": "/opt/umbra/agent",
"config_path": "/etc/umbra/node.json",
"unit_name": "umbra-node",
"service_user": "umbra-node",
}
LAYOUT_USER = {
"install_root": "",
"agent_dir": "",
"config_path": "",
"unit_name": "umbra-node",
"service_user": "",
}
def render(
*,
node_name: str,
pairing_code: str,
controller: str,
as_root: bool = False,
) -> str:
"""Render the bootstrap for a node.
``as_root`` picks the *expected* layout; the script still checks for root when it
runs and takes whichever branch is actually available. That is deliberate. An
administrator rendering this from the web UI does not always know whether the person
at the machine will run it as root or as themselves, and a script that silently did
the wrong thing would be worse than one that adapts. Both layouts are in the output;
the runtime decides.
"""
layout = dict(LAYOUT_ROOT if as_root else LAYOUT_USER)
if not as_root:
# Left as shell expressions so the script resolves them against the invoking
# user at run time rather than against whoever rendered the script.
layout["install_root"] = "$HOME/.local/share/umbra"
layout["agent_dir"] = "$HOME/.local/share/umbra/agent"
layout["config_path"] = "$HOME/.config/umbra/node.json"
return render_template(
node_name=node_name,
pairing_code=pairing_code,
controller=controller,
bootstrap_version=BOOTSTRAP_VERSION,
**layout,
)
#: Kept as a distinct name so the tests can drive the substitution directly without
#: depending on the layout choice.
render_template = render_full
def render_bootstrap_template(default_controller: str = "") -> str:
"""The bootstrap with placeholders and no usable pairing code.
What ``GET /api/node/bootstrap`` serves. Inert: it checks the machine and then
fails to pair, because the code it needs was never filled in. That is the property
that makes ``curl | sh`` reasonable to hand to someone - the downloaded program has
no authority until an operator types a code into it.
"""
return (
BOOTSTRAP.replace("__NODE_NAME__", "unpaired-node")
.replace("__PAIRING_CODE__", "000000")
.replace("__CONTROLLER__", default_controller or "")
.replace("__BOOTSTRAP_VERSION__", BOOTSTRAP_VERSION)
.replace("__AGENT_DIR__", "$HOME/.local/share/umbra/agent")
.replace("__CONFIG_PATH__", "$HOME/.config/umbra/node.json")
.replace("__UNIT_NAME__", "umbra-node")
.replace("__SERVICE_USER__", "umbra-node")
.replace("__INSTALL_ROOT__", "$HOME/.local/share/umbra")
)
def bootstrap_version() -> str:
"""The template version, so a caller can tell whether what it fetched is current."""
return BOOTSTRAP_VERSION
Reference in New Issue View Git Blame Copy Permalink