umbragit/UmbraAudio
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add the publish flag: pinned release sets, immutability, the Deal Hunter lesson

Browse Source 
The operator clarified the release model: the system keeps current and previous
state, there must be an explicit publish flag the user sets, there are approvals
for a speaker voice but none for the audiobook or chapters, the chapters that
make the m4b are the versions to flag, and a fix found after assembly causes a
change rather than editing what shipped.

Deal Hunter is the precedent, and it is measurable rather than theoretical.
Against the real package (built_at 2026-09-09T11:21:11Z), chapter 6:

  ch6_full.wav                    2227.26 s @ 24000 Hz   what a reviewer listens to
  Deal_Hunter_01_Salvage.mp3      2231.26 s @ 44100 Hz   what a listener received
                                  +4.00 s = 2 s lead + 2 s trail

plus a resample and a -20.5 dB RMS normalization. Found on 32 of 32 chapters.
The approved artifact and the published artifact were different files, and
nothing in the system recorded either fact. ar_package.json is rewritten by
every rebuild, so it is a build status, not a release record.

umbraaudio/release.py
  DeliveryProfile.from_ar_package reads the profile the exporter actually used,
  so the record follows the code instead of a declaration that can rot.
  build_publication is read-only and pins every chapter wav, every mp3 and the
  package by full SHA-256, plus a per-chapter source_unit_key_digest so a release
  pins which source revision produced it, not merely which bytes.
  verify_publication returns ok / drift / warnings and never silently accepts a
  changed artifact. write_publication is atomic and refuses to overwrite an
  existing publication id - immutability enforced by the writer, not by
  convention. A later fix is edition n+1 with supersedes pointing at n.

The rule that prevents a repeat: you cannot approve a chapter and then pad it.
An approval must name the artifact a listener receives. Every record states
chapter_approved: false and audiobook_approved: false in its own words, because
that is what would have caught Deal Hunter.

The record also refuses to overstate approval. voice_status.json is an approval
LEDGER - it only holds speakers someone explicitly touched - so it has 24 entries
against 61 speakers in the profile. Reporting "24 approved" as if it meant "all
approved" would claim an approval nobody gave, so the counts are separate and the
38-speaker gap is named.

scripts/publish_release.py: dry run by default; --publish, --verify, --list,
--edition/--supersedes. Exits 2 while blockers exist.

Also found, reported and NOT fixed (specs/22 section 7 gap 1): export_m4b's
_stitch_chapter hard-codes 500 ms/0 ms silence while ChapterBuilder uses
AudioStitcher defaults and AudioConfig holds its own values, so the m4b path can
stitch a chapter with different silence than the approved one. Until that is
closed an m4b release record would pin the wrong bytes, so it is better not to
offer one yet.

And projects.status is never advanced to 'released' in any of the four live
projects - all sit at parsed/analyzed - so the publication record is currently the
only release signal in the system.

tests/test_release.py: 24 tests. Full suite 512 passed / 8 failed, the same 8
pre-existing failures. ruff clean. No export, TTS or project data was modified.
This commit is contained in:
B.D. Murphy
2026-10-03 06:20:22 -07:00
parent 829a03b89f
commit ed3e249d0b
5 changed files with 1256 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

199
specs/22-releases-and-the-publish-flag.md Normal file
View File

@@ -0,0 +1,199 @@
# Spec 22 — Releases: the Publish Flag, Pinned Artifact Sets, and Post-Publication Change
- **Status:** implemented on the UmbraAudio node (backend + CLI + tests). Awaiting
ratification by the controller.
- **Date:** 2026-10-03
- **Origin:** an operator clarification of the release model, prompted by Deal Hunter.
- **Companions:** `specs/21` (nodes, tasks, changes), `specs/20` §F (`editions`,
`edition_artifacts`, invariant I-H), controller decisions D9/D12/D14/D15.
---
## 1. The clarification, stated back
> The system, with time markers, acts as a git repository: it stores the current state and
> keeps the previous ones. There should be an explicit flag. The user tells the system *this
> is the version that was published*, making explicit all the items and approvals that make
> up the final state. There are approvals for items like a speaker voice, but **not** for the
> full audiobook and chapters. **The chapters that make an m4b are the versions to flag.**
> After assembly into final chapters and m4b, other issues may need fixing; the publishing
> requirements must then be met, and that **causes a change**. Deal Hunter experienced this
> when 2 seconds of dead sound had to be added to the beginning and end of each chapter,
> after approval.
Five consequences, each implemented:
1. **A release is a pinned set, not a build directory.** Every chapter artifact and the
package carry a full SHA-256 in the release record.
2. **The publish flag is the release record.** Writing it *is* the approval — because no
other artifact-level audiobook approval exists.
3. **The pinned set is the chapters that make the m4b**, not the loose utterances and not the
pre-encode intermediate.
4. **Everything applied after a chapter is approved is part of the release, and is recorded
in it.** Delivery profile: sample rate, channels, bitrate, CBR, leading/trailing silence,
loudness target, gain limit, peak ceiling.
5. **A change after publication is a successor, never an edit.** A published record is
immutable; a fix produces edition *n+1* with `supersedes` pointing at *n*.
## 2. What was already wrong, measured
Not hypothetical. On the published Deal Hunter package (`ar_package.json`,
`built_at 2026-09-09T11:21:11Z`), chapter 6:
| artifact | duration | rate | what it is |
|---|---|---|---|
| `chapter_audio/ch6_full.wav` | **2227.26 s** | 24 000 Hz | what a reviewer listens to |
| `export/ar_package/Deal_Hunter_01_Salvage.mp3` | **2231.26 s** | 44 100 Hz | what a listener received |
**+4.00 s = 2 s leading + 2 s trailing digital silence**, plus a resample and a loudness
normalization to −20.5 dB RMS. Applied to **32 of 32 chapters**.
The approved artifact and the published artifact were different files, and the system
recorded neither fact. `ar_package.json` is rewritten by every rebuild, so it is a build
status — not a release record.
A second, independent instance of the same class, still latent: `export_m4b._stitch_chapter`
hard-codes `silence_between_utterances_ms=500, silence_between_chapters_ms=0` while
`ChapterBuilder` uses `AudioStitcher()` defaults and `AudioConfig` holds its own values. The
m4b path can therefore stitch a chapter with different silence than the approved one. Reported,
not fixed — see §7.
## 3. The rule that actually prevents a repeat
> **You cannot approve a chapter and then pad it.**
An approval must name the artifact a listener receives. Every transform applied between
"chapter finished" and "bytes shipped" is part of the release and is recorded in it. Where a
transform necessarily changes the bytes — padding, resampling, loudness normalization — the
approval is taken on the *post-transform* artifact, or it is not an approval of what ships.
## 4. What is implemented
### 4.1 `umbraaudio/release.py`
| function | purpose |
|---|---|
| `build_publication(...)` | Captures the pinned set + approvals + delivery profile. **Read-only** — hashes files, writes nothing. |
| `DeliveryProfile.from_ar_package(...)` | Reads the profile the exporter *actually used* from its own manifest. Measured, not declared. |
| `verify_publication(...)` | Re-checks a record against the filesystem → `ok` / `drift` / `warnings`. |
| `write_publication(...)` | Atomic (tmp + `os.replace`), and **refuses to overwrite an existing publication id**. |
| `list_publications(...)` | Every published edition. |
A publication record:
```json
{
"publication_id": "pub-001-2026-09-09",
"work": "...", "edition_number": 1, "supersedes": null,
"published_at": "...", "published_by": "...", "git_commit": "...",
"delivery_profile": {
"sample_rate": 44100, "channels": 1, "bit_rate_kbps": 192, "cbr": true,
"lead_silence_s": 2.0, "trail_silence_s": 2.0,
"target_rms_db": -20.5, "max_gain_db": 12.0, "peak_ceiling_dbfs": -3.0
},
"package": { "object_key": "export/Deal_Hunter.m4b", "sha256": "...", "bytes": 325875309 },
"approvals": {
"voices_in_profile": 61, "voices_approved": 24, "voices_not_approved": 38,
"voice_status_digest": "...",
"chapter_approved": false, "audiobook_approved": false
},
"chapters": [
{ "chapter_id": 6, "seq": 6, "title": "1 Salvage",
"chapter_wav": { "object_key": "chapter_audio/ch6_full.wav",
"sha256": "...", "duration_s": 2227.26, "sample_rate": 24000 },
"mp3": { "object_key": "export/ar_package/Deal_Hunter_01_Salvage.mp3",
"sha256": "...", "duration_s": 2231.26,
"rms_db": -20.5, "peak_db": -3.0,
"loudness_ok": true, "peak_ok": true },
"source_unit_key_digest": "...", "utterances": 364,
"warnings": ["shipped track is 4.0s longer than the approved chapter wav ..."] }
],
"blockers": [], "publishable": true
}
```
Two fields carry most of the weight and are worth calling out:
- **`source_unit_key_digest`** — a digest of every `source_unit_key` in the chapter
(`specs/21` §10). A release therefore pins *which source revision* produced it, not just
which bytes. Audio and source are traceable to each other after the fact.
- **`voices_not_approved: 38`** — `voice_status.json` is an *approval ledger*: it only holds
speakers someone explicitly touched, so it has 24 entries against 61 speakers. Reporting
"24 approved" as if it meant "all approved" would claim an approval that was never given,
so the two counts are separate and the gap is named.
### 4.2 `scripts/publish_release.py`
```bash
python scripts/publish_release.py --project "DealHunterShip_4.0e_whole-to-audio" # dry run
python scripts/publish_release.py --project "..." --publish --by worstauthor # write
python scripts/publish_release.py --project "..." --verify pub-001-2026-09-09 # re-check
python scripts/publish_release.py --project "..." --list # editions
```
Dry run is the default and prints the pinned set, the delivery profile, the approval gap, and
**every post-approval delta it found**. `--publish` refuses to run while `blockers` is
non-empty. A second `--publish` against the same id raises `FileExistsError` — immutability
is enforced by the writer, not by convention.
## 5. Invariants
| ID | invariant | Enforced by |
|---|---|---|
| **R-1** | A published record is immutable; a later change is a successor edition with `supersedes` | `write_publication` refuses to overwrite |
| **R-2** | A release pins the chapters that make the package, each by SHA-256 | `build_publication`, `verify_publication` |
| **R-3** | No publication occurs while a blocker exists | `publish_release.py --publish` exits 2 |
| **R-4** | Any post-approval transform is recorded in the release | `delivery_profile` + per-chapter `warnings` |
| **R-5** | A changed artifact after publication is reported as drift, never silently accepted | `verify_publication` → `drift` |
| **R-6** | An approval is never inferred from the absence of a complaint | `approvals.chapter_approved = false` is recorded explicitly |
| **R-7** | The approval ledger's coverage is stated, not assumed | `voices_in_profile` vs `voices_approved` |
R-6 is the one that would have prevented Deal Hunter: the release says, in its own words,
that no chapter and no audiobook was approved.
## 6. Relationship to the controller's model
This is the local, immutable half of what the controller calls `editions` + `edition_artifacts`:
| local field | controller entity |
|---|---|
| `publication_id`, `edition_number` | `editions` |
| `supersedes` | `editions.supersedes_edition` |
| `chapters[].chapter_wav.sha256`, `mp3.sha256` | `edition_artifacts` (immutable member set, I-H) |
| `delivery_profile` | `derivation` on each artifact |
| `chapters[].source_unit_key_digest` | linkage back to `source_revisions` / `source_units` |
| `approvals` | the acceptance record behind a release decision |
| `published_at` / `published_by` | `released_at` / actor |
The record is shaped so ingest is a **copy**, not a transformation. Nothing the controller
would need is expressed in a local-only dialect.
## 7. Known gaps, stated rather than hidden
1. **The m4b path stitches differently from the approved chapter.**
`export_m4b._stitch_chapter` hard-codes 500 ms / 0 ms silence; `ChapterBuilder` uses
`AudioStitcher()` defaults (500/2000/4000) and `AudioConfig` carries its own values. Until
the m4b path consumes the *same* stitched chapter artifact the reviewer approved, a
chapter approval still does not cover the m4b. **This is the last real hole in R-2 and it
is not yet closed.** Proposed fix: make `export_m4b` consume `ch{N}_full.wav` directly
instead of re-stitching from utterances.
2. **No release-level approval in the web UI.** The flag exists as a CLI and a file. A
reviewer cannot yet press "publish" in the review interface, which is where they will
expect it.
3. **`projects.status` is never advanced to `released`.** All four live projects sit at
`parsed`/`analyzed`. The publication record is currently the only release signal.
4. **Not idempotent against a rebuild.** Re-running the exporter overwrites
`ar_package.json`; the publication record keeps the old hashes, so verification correctly
reports drift — but there is no automatic "this needs a new edition" workflow yet.
5. **Scope.** Only the AR MP3 package path is covered. A single-workbook `m4b` built by
`export_m4b` is not yet represented, for the reason in gap 1.
## 8. What to do next
- Close gap 1 first: make the m4b path consume the approved `ch{N}_full.wav`. Until then the
release record for an m4b would be pinning the wrong bytes, and it is better not to offer
one.
- Put the publish flag behind the review UI, gated on `verify_publication` returning `ok`.
- Set `projects.status = 'released'` from the same act that writes the record, so the two
cannot disagree.
- Send this shape to the controller as the ingest payload for `editions` / `edition_artifacts`.