Commit Graph

29 Commits

Author SHA1 Message Date
Umbra Tools
25605e2cad Record the paragraph-boundary drift findings and the verification stage
Also records the negative result from the independent cross-check attempt:
names are not a drift signal, and the glossary-anchor detector conflates
drift with ordinary glossary non-compliance. Both would otherwise be
tried again.
2026-11-06 21:40:37 -08:00
Umbra Tools
6bf0baa2f8 Record the Chinese-only pipeline finding and the delivery script gate
The "Spanish" production on oldbox is a second Chinese draft, and the
drafter's hard-coded Simplified Chinese instructions are why. Also recorded:
the target-language audio render belongs to UmbraTrans per spec 18, the
Qwen3-TTS cross-language clone probe is still outstanding, and the live UI
was re-measured after the gate.
2026-11-06 20:09:23 -08:00
Umbra Tools
81915b4ec1 Record the translation alignment session, and fix the summary header
The "Current Session Summary" heading had run into the first session's
heading on one line, so the first session read as part of the header and
never appeared in the outline. Split them, and add this session's entry
above.
2026-11-06 19:44:41 -08:00
B.D. Murphy
6bb69729b9 Correct the SSH note: the :2222 connect is intermittently refused, not blocked
The earlier note blamed git-over-HTTP on Local Network privacy and used a
url.insteadOf workaround. The real finding: HTTP genuinely does not
work for git (per-binary entitlement - curl reaches it), while SSH on
2222 works but the connect is intermittently refused with 'Undefined
error: 0'. The Host mini-eq ssh block now sets BatchMode, ConnectTimeout
and ConnectionAttempts 5, so ssh retries the refused connect; plain
git-over-ssh is 5/5 reliable and needs no GIT_SSH_COMMAND.

Also recorded both pinned Mini-EQ host keys - :22 and :2222 keep separate
keys - and that ssh-keyscan without -p 2222 scans the wrong port.
2026-11-06 18:00:16 -08:00
B.D. Murphy
ff5fe0402d Record the git-over-HTTP TCC blocker and the SSH push route
Git cannot reach Mini-EQ over HTTP on this machine - it fails with 'No
route to host' while curl to the same address returns 200, because the
Local Network entitlement is per-binary. The remote URL is unchanged and
now transparently rides SSH via url.insteadOf. Also recorded that the
remote main can move without the local branch.
2026-11-06 17:13:41 -08:00
B.D. Murphy
8ad12534cd Document the login gate and its security properties 2026-10-09 12:39:00 -07:00
B.D. Murphy
976e2bff50 Record distribution spec decisions, m4b quality gap, and the in-flight regen 2026-10-08 12:40:40 -07:00
B.D. Murphy
e48009fbde Record Qwen TTS boot start and retirement of the root qwen relay 2026-10-08 07:45:24 -07:00
B.D. Murphy
000aa99e05 Correct GPU retirement note: controller work continues, GPU returns via controller 2026-10-08 07:39:47 -07:00
B.D. Murphy
0722a85ffd Retire GPU box 192.168.123.14 from TTS and node-report configuration
The box is offline and being reconfigured. Removed the relay helper URL from
.env (qwen_tts_helper_url defaults to empty, so local :8881 is now the only
backend), dropped the 18881 relay entry from MEANINGFUL_PORTS, and made the
node report state plainly that no controller endpoint exists rather than
advertising one on a retired host.

Also recorded that ModelCoordinator.ensure_qwen_server() already spawns the
Qwen server on demand - it checks /health first, so a launchd-owned instance is
reused instead of double-spawned.
2026-10-08 07:33:54 -07:00
B.D. Murphy
a6ed6e86be Correct public-endpoint finding: no public A record, plus NAT hairpin blind spot 2026-10-08 07:16:51 -07:00
B.D. Murphy
f5ce26b6a4 Fix boot race: install web LaunchAgent to /Library/LaunchAgents
launchd scans $HOME/Library/LaunchAgents once at login, and $HOME is the
relocated /Volumes/Dock/worstauthor. On 2026-10-08 the Dock volume mounted at
05:59 but login was 05:58, so the plist was invisible and com.umbraaudio.web
never loaded - the web server stayed down after a reboot.

Installed to /Library/LaunchAgents instead (system-wide, on the boot volume,
scanned regardless of $HOME). Removed the $HOME mirror since two plists with
one Label is a race; backup kept. Verified running from the new path.
2026-10-08 07:07:17 -07:00
B.D. Murphy
753d80d0b3 Remove duplicated heading in AGENTS.md session summary 2026-10-08 06:50:39 -07:00
B.D. Murphy
a85c6aa838 Document voice-maintenance guard, pid-gated marker testing, and boot mount-race 2026-10-08 06:50:32 -07:00
B.D. Murphy
3b8595b405 AGENTS.md: record Mini-EQ Gitea access, private repos, and submodule repoint
Records the access mechanism (osxkeychain credential helper + Keychain-stored PAT,
deliberately not the token value), the private/login-required repo posture, the
push order, the shallow-clone fix before mirroring Qwen3-TTS, the :2222-vs-:22
Gitea SSH trap with its separate host key, the forced-password-change gate that
hid the SSH key settings, the mini-eq .18/.138 addressing quirk, and the LAN
dependency the submodule repoint introduces.
2026-10-06 07:03:45 -07:00
B.D. Murphy
29667d0b62 AGENTS.md: record SSH mini-m4 -> oldbox now working
Supersedes the long-standing 'SSH denied, needs manual provisioning' note, which
was wrong in its diagnosis. Recorded: root cause was authorization (our ed25519
public key missing from umbra's authorized_keys on oldbox), not networking,
TCC, or host keys; the local key + fingerprint; all three pinned host-key
fingerprints verified matching; the Host oldbox config block and its backup; the
remote-broker profile now returning exit 0 (and the oldbox-umbra vs
oldbox-umbra-ssh naming trap); the read-only fingerprint-verification recipe and
the rule not to pin a handed-over fingerprint unchecked; and oldbox's specs
including the NVML permission gap that blocks nvidia-smi for umbra.
2026-10-05 20:38:49 -07:00
B.D. Murphy
9f3a0a4b18 Release comment as a release step; releasing grants implicit voice approval
Two corrections from the operator, and I had both wrong.

1. "I didn't explicitly approve all minor character voices. By releasing the
   audiobook I have given implicit approval those voices are ready and should be
   tagged." The previous session reported "24 of 61 speakers have an approval
   record; 38 do not" as if it were a gap. It is not a gap. Releasing IS the
   approval, and those voices were waiting to be tagged.

   chapter_approved and audiobook_approved now read true, with
   approved_by: "this publication record" - there is no separate audiobook
   approval in this system, and pretending otherwise understated the release.

   The boundary matters, so it is measured rather than assumed: implicit
   approval covers voices that HAVE LINES in the released chapters, computed from
   the database. A character who never speaks is not covered, because releasing a
   book does not approve a voice it never uses. On Deal Hunter edition 1 that is
   24 explicitly approved, 29 implicitly approved, and 10 in-profile voices with
   no lines. The three categories are never summed - an explicitly approved voice
   can also be one that stopped speaking.

   grant_implicit_voice_approval() writes plain status strings, the shape the web
   UI already reads, and keeps the provenance in the immutable publication record
   rather than the mutable ledger, so it cannot be edited away.

2. The m4b inter-chapter gap should be the same 2 s as the lead and trail
   padding. export.py had gap_ms=4000 in one path and a default of 1000 in the
   other, while ar_export.SILENCE_MS = 2000 governed lead and trail: three
   drifting values for one concept. The gap now imports SILENCE_MS as
   CHAPTER_GAP_MS, so all three are one number that changes in one place.

New umbraaudio/release_comments.py - the comment is a gate, not a caption.

  designator  2026-10-03 #2 14:32:05
  slug        release-20261003-143205-02

The sequence is per date, not global, which is the whole point: a problem found
and corrected on the same day is unambiguous about which was last. Two comments
authored in the same second still get distinct slugs, because the sequence is
allocated by scanning the directory. First line is the summary, the rest is the
body, like a commit message. An approved comment is FROZEN - changing an approved
note is a different claim about a different state, so it needs a new comment and
therefore a new publication.

require_approved() blocks --publish: passing a draft exits 2 and says what to do
next. Verified live.

draft_comment_text() composes the comment from the measured build rather than
from memory. Deal Hunter's draft is written and waiting:
release-20261003-133508-01, 32 chapters, the 24/29/10 voice split, 32/32
loudness and peak gates green, and the post-approval padding note.

New invariants R-8 through R-11 in spec 22. Two of my own test premises were
wrong and the implementation was right - recorded in AGENTS.md because the tests
were the weaker artefact.

Full suite 538 passed / 8 failed, the same 8 pre-existing failures. ruff clean.
Nothing was published: the Deal Hunter comment is a draft awaiting review, and
voice_status.json is untouched because publishing never ran.
2026-10-03 06:49:42 -07:00
B.D. Murphy
ed3e249d0b Add the publish flag: pinned release sets, immutability, the Deal Hunter lesson
The operator clarified the release model: the system keeps current and previous
state, there must be an explicit publish flag the user sets, there are approvals
for a speaker voice but none for the audiobook or chapters, the chapters that
make the m4b are the versions to flag, and a fix found after assembly causes a
change rather than editing what shipped.

Deal Hunter is the precedent, and it is measurable rather than theoretical.
Against the real package (built_at 2026-09-09T11:21:11Z), chapter 6:

  ch6_full.wav                    2227.26 s @ 24000 Hz   what a reviewer listens to
  Deal_Hunter_01_Salvage.mp3      2231.26 s @ 44100 Hz   what a listener received
                                  +4.00 s = 2 s lead + 2 s trail

plus a resample and a -20.5 dB RMS normalization. Found on 32 of 32 chapters.
The approved artifact and the published artifact were different files, and
nothing in the system recorded either fact. ar_package.json is rewritten by
every rebuild, so it is a build status, not a release record.

umbraaudio/release.py
  DeliveryProfile.from_ar_package reads the profile the exporter actually used,
  so the record follows the code instead of a declaration that can rot.
  build_publication is read-only and pins every chapter wav, every mp3 and the
  package by full SHA-256, plus a per-chapter source_unit_key_digest so a release
  pins which source revision produced it, not merely which bytes.
  verify_publication returns ok / drift / warnings and never silently accepts a
  changed artifact. write_publication is atomic and refuses to overwrite an
  existing publication id - immutability enforced by the writer, not by
  convention. A later fix is edition n+1 with supersedes pointing at n.

The rule that prevents a repeat: you cannot approve a chapter and then pad it.
An approval must name the artifact a listener receives. Every record states
chapter_approved: false and audiobook_approved: false in its own words, because
that is what would have caught Deal Hunter.

The record also refuses to overstate approval. voice_status.json is an approval
LEDGER - it only holds speakers someone explicitly touched - so it has 24 entries
against 61 speakers in the profile. Reporting "24 approved" as if it meant "all
approved" would claim an approval nobody gave, so the counts are separate and the
38-speaker gap is named.

scripts/publish_release.py: dry run by default; --publish, --verify, --list,
--edition/--supersedes. Exits 2 while blockers exist.

Also found, reported and NOT fixed (specs/22 section 7 gap 1): export_m4b's
_stitch_chapter hard-codes 500 ms/0 ms silence while ChapterBuilder uses
AudioStitcher defaults and AudioConfig holds its own values, so the m4b path can
stitch a chapter with different silence than the approved one. Until that is
closed an m4b release record would pin the wrong bytes, so it is better not to
offer one yet.

And projects.status is never advanced to 'released' in any of the four live
projects - all sit at parsed/analyzed - so the publication record is currently the
only release signal in the system.

tests/test_release.py: 24 tests. Full suite 512 passed / 8 failed, the same 8
pre-existing failures. ruff clean. No export, TTS or project data was modified.
2026-10-03 06:20:22 -07:00
B.D. Murphy
829a03b89f Action the controller alignment proposal: source_unit_key + phase reports
The controller (oldbox) sent a data-model alignment proposal and asked for six
additive changes on this node, the validation protocol, and an honest report.
Six of the seven changes are done; the seventh is declared missing rather than faked.

Stage 0 - reachability verified exactly as documented: health 200, heartbeat 401
(the correct answer - auth is refusing), missing Host 400. Nothing else was sent.

source_unit_key (controller 5.1, the blocking item)
  umbraaudio/source_units.py. sha256("uk1" | chapter_source_text_sha256 |
  occurrence | text_sha256). Nothing positional in the key: "occurrence"
  disambiguates repeated text in document order.
  - 18,181/18,181 units keyed across all four projects, 0 collisions
  - chapter 1 renumbered +100 -> 0 keys changed (the load-bearing proof)
  - backfill 10,015 rows in 0.11s, second run stamps 0 (idempotent)
  - alignment phase 3: 200/200 probe units resolved with used_index: false
  - tolerates the older projects that have no chapters.chapter_index
  Recorded at generation time in chapter_builder, BEFORE the rows are read and
  BEFORE the marker is written, so every path that renders audio also produces
  identity and the clip can name its source unit.

5.7 recorded: utterance_index is KEPT as a display ordinal and made
  non-load-bearing. spec 21 §10.

Blob hashes and effect chain (5.2)
  Markers now carry the full sha256 of the delivered bytes and the effects that
  produced them. Verified the existing order is already correct - write_marker
  runs AFTER apply_effects - and closed the remaining hole: a changed trait list
  changed the hash with no recorded cause. sha256 verification is opt-in, since
  hashing every file on every validation pass would re-read gigabytes.

Vocabularies published (5.4) - umbraaudio/definitions.py, 19 sets, 126 values,
  each value carrying whether it is 'declared' (spec/controller) or 'extracted'
  (read out of code), so a diff shows a difference of meaning, not of wording.

Alignment report tool (6.6) - scripts/alignment_report.py --phase all. Emits the
  controller's envelope with the per-phase evidence shapes, and obeys the two
  rules they set: no evidence means blocked, never pass; a field we cannot produce
  is null PLUS a finding, never omitted. Phase 2 is deliberately 'fail' - the
  three-way store disagreement is real, and a pass there would be a check that
  passed by doing nothing.

Fingerprint discipline (5.5) - the drift detector reported its OWN fingerprint
  change when source_unit_key was added rather than hiding it. Node profile
  updated with the new fingerprints and an honest note that they moved.

Node report extended - per-project source-unit-key coverage, per-voice-asset
  sha256 + object_key, definition_set_version, controller stage reached.

Six gaps found in the proposal (spec 21 §11), including the one they invited plus
five more. The sharpest: "hash at generation" ignores that audio is
post-processed afterwards, and our key is anchored to chapters.source_text - a
second copy of the manuscript - so any change to chapter storage silently
invalidates every key in that chapter.

5.6 deliberately NOT faked: personality has no versioned artifact here, only
  unversioned LLM prose in speakers.json. Reported as phase 6b finding B-03
  instead, because wrapping unversioned inputs in a version field would produce
  artifacts claiming a lineage they do not have.

Answered the controller's two open questions (infra/controller-handoff-2026-10-03.md):
forward-only link for a revealed recurring character, and yes - a new edition may
contradict a frozen state, since this node's correction path IS a regeneration.

Full suite 484 passed / 8 failed - the same 8 pre-existing and unrelated failures.
ruff clean on every file touched. The parallel session's chapter_builder hunk
(helper_url) was filtered out of the commit and left intact in the working tree.
2026-10-02 20:16:06 -07:00
B.D. Murphy
34c221ea85 AGENTS.md: document designation scanner session 2026-10-01 07:05:40 -07:00
B.D. Murphy
442005e51b Correct two false claims in AGENTS.md about the qwen-relay LaunchDaemon
Verified against the live system on 2026-09-27:

1. The settimeout(None) fix was documented as deployed but was ABSENT from
   /Library/LaunchDaemons/qwen_relay.py (mtime Sep 10 06:33), so the bug it
   describes - any TTS call over 10s killed mid-response - was still live.
   Patched (backup qwen_relay.py.bak.20260927) and restarted the daemon with
   launchctl kickstart -k.

2. The relay was never down. launchctl print reports state = running and root
   holds 127.0.0.1:18881 LISTEN. An unprivileged lsof cannot see root processes,
   which makes it look absent, and curl to a relay with a dead upstream returns
   http_code=000 because the relay accepts then drops. 18881 -> 000 means
   "relay up, upstream refused", not "relay down". Confirm with sudo lsof.

Actual current failure is the GPU box 192.168.123.14:8881 refusing connections;
its Qwen server is a manual start and is down. qwen_relay.py has no upstream
fallback, so every call is dropped and the app falls through to silent
Edge-TTS. Keeping the relay cannot fix that; local 127.0.0.1:8881 works today
and loopback is exempt from Local Network TCC.
2026-09-27 05:42:41 -07:00
B.D. Murphy
b224c5f10f AGENTS.md: document speaker designation + discovery rules session 2026-09-25 18:41:20 -07:00
B.D. Murphy
79e1cabe1e AGENTS.md: document Phase 3 name-token UI session 2026-09-25 03:21:14 -07:00
B.D. Murphy
14ee391722 AGENTS.md: document Phase 2 orphan name-token session + concurrent-work warning 2026-09-24 11:27:51 -07:00
B.D. Murphy
65724ed159 AGENTS.md: document utterance clarity signals session (spec 17 Phase 1) 2026-09-23 12:16:14 -07:00
B.D. Murphy
5ed9db7871 Add weekly launchd commit reminder (com.umbraaudio.commitcheck)
Wires the commit-cadence policy to an actual reminder instead of a script
nobody runs.

- scripts/commit_check_notify.py: runs scripts/commit_check.sh (piped to
  bash -s so the script is never opened on the external volume), logs to
  logs/commit_check.log and posts a macOS notification via osascript when the
  tree is dirty. Defaults to --any (week's work) ; --age-only restores the
  >threshold behaviour; --no-notify for testing.
- scripts/commit_check.sh: new --any mode (warn on any uncommitted change,
  ignoring age) for the weekly job.
- LaunchAgent fires Sunday 18:00; launcher
  ~/.umbraaudio/umbra_commit_check.sh mirrors umbraweb_login.sh (cd project,
  export PATH, exec .venv/bin/python). The venv python is the only interpreter
  approved for the external volume, so a shell never reads a project file.
- Verified under launchd: detects dirty tree, notification sent, exit 1;
  clean tree exits 0. Probe artifacts removed.
2026-09-21 07:09:52 -07:00
B.D. Murphy
2ea2b7c065 Add scripts/commit_check.sh weekly commit-cadence guard
Mechanical nudge for the policy in AGENTS.md -> "Commit Cadence": it measures
how long the working tree has been dirty (oldest mtime among staged, unstaged
and untracked files) and exits 1 once that exceeds --days (default 7).

- Ignores submodule working-tree dirt (separate repo); still surfaces a moved
  submodule pointer.
- Reports dirty age, last edit, last commit; prints a reminder block.
- --quiet silences the clean case only; --help documents exit codes.
- Bash 3.2 compatible (macOS /bin/bash); no external deps.
- Verified: clean/dirty/old-dirt/threshold/bad-args/quiet all behave.

Also cross-reference it from AGENTS.md and README.md.
2026-09-21 07:04:52 -07:00
B.D. Murphy
df36dad8aa Weekly commit: attribution QA, dramatic variations, GPU TTS, AR export
Squashes ~3 weeks of uncommitted work (2026-09-03 → 09-21). The repo is the
only durable copy; a single accidental `git checkout` on routes.py nearly lost
a month of work (recovered from an editor snapshot, not git). See AGENTS.md →
"Commit Cadence" — commit every Sunday.

Attribution & QA
- New umbraaudio/analysis/attribution_qa.py: mechanical check_attribution /
  fix_high_confidence / verify_conversation_flow (speech-tag mismatch, narrator
  dialogue, role-tag guessing) plus scripts/qa_attribution.py CLI.
- llm_attributor prompt rules 3e-3h (unnamed roles, pronoun coreference,
  repeated lines) with wider scene/position context.
- Per-utterance QA box in chapter audio detail UI: hover issues, Recheck
  (auto-fix loop) and Accept (persisted qa_accepted* + text/speaker signature,
  auto re-surfaces when either changes). New POST /api/utterance/{id}/recheck
  and /api/utterance/{id}/qa-accept. _ensure_utterance_columns migration fixes
  legacy DBs that lacked the columns.

Generation
- Dramatic utterance variations: 26 emotions + pace + custom instruct,
  composed into every TTS voice mode (clone/icl/ref/embedding) and all
  generation sites; markers record the instruct so edits re-trigger.
- Staleness markers are JSON {speaker,text}; legacy plain-text markers always
  stale.
- Scene-break segmentation (generation/scene_break.py) + resegment merge.
- AR package exporter (assembly/ar_export.py) with tests.
- Voice ref resolution: rglob refs, canonical alias / first-word fallback.
- Edge-TTS fallback visibility: durable voice_engines.json ledger + UI badge.

GPU / TTS server
- qwen_tts_server: instruct + clone/icl support (voice_description →
  instruct_ids prepended to the talker input).
- qwen_tts.py remote path sends voice_description alongside voice_ref/slug.

Config / ops
- .env: UMBRA_TTS__QWEN_TTS_URL=http://localhost:8881.
- .gitignore: book projects, tmp_out, logs, one-off samples A/B demo.
- AGENTS.md + README.md: Commit Cadence (commit at least weekly, on Sunday).

New tests: test_qa_acceptance, test_variations, test_ar_export,
test_scene_break, test_resegment_merge, test_vocals_project. Suite: 327 pass
(6 pre-existing voice-profile accent-wording failures, unrelated).
2026-09-21 06:44:22 -07:00
B.D. Murphy
c86e323d7a Initial commit: UmbraAudio audiobook generation program
Quality-first multi-pass audiobook generation from English manuscripts.

Includes:
- umbraaudio/ core package (pipeline, config, voices, web review UI)
- scripts/ CLI utilities + model download script
- tts_server/ remote TTS server
- specs/ design docs, tests/, pyproject.toml + uv.lock
- Qwen3-TTS git submodule (pinned 022e286) with local demo.py UI patch
  saved at scripts/qwen3_tts_demo_umbra_patch.patch
- .env with runtime config/secrets

Excluded (see .gitignore): models/ (downloaded via scripts/download_models.sh),
output/, logs/, .venv, .umbra runtime data, and book projects
(DealHunterShip_*), which are committed separately.
2026-08-30 07:58:24 -07:00