umbragit/UmbraAudio
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge branch 'main' into chore/untrack-env

Browse Source 
# Conflicts:
#	.env
This commit is contained in:
umbra audio
2026-10-19 04:36:00 -07:00
parent 9f403cb873 03f8a858dd
commit 19ee5db43a
307 changed files with 10198 additions and 229 deletions
Download Patch File Download Diff File Expand all files Collapse all files

174
AGENTS.md
View File

@@ -209,7 +209,179 @@ Each pass has quality gates. Quality is the primary concern; generation speed is
- Never add a dependency if stdlib suffices
- Prefer: stdlib → third-party → custom implementation
## Current Session Summary
## Current Session Summary### Session: Login gate on the review UI (replaces the open surface)
- **The review UI is now authenticated.** Until 2026-10-09 it had **no auth at all**:
`active_project` is an unsigned cookie and the server binds `0.0.0.0`, so every page -
including mutation endpoints and audio downloads - was reachable from the LAN and,
through the Caddy proxy, the internet. Spec 21/22 called this an unauthenticated
mutation surface; it is now closed locally.
- **Gate is middleware, not per-route dependencies** (`umbraaudio/web/__init__.py`), so a
newly added route is protected by default and forgetting a decorator cannot expose the
UI. **Fail-CLOSED**: an exception while checking a session DENIES the request. Falling
through to "no session required" would turn a bug into an open door.
- `umbraaudio/auth.py` - PBKDF2-HMAC-SHA256 (stdlib: the project rule is stdlib before
third-party) + random per-user salt. Sessions are HMAC-signed, carry **no secret** (just
username + expiry), compared with `hmac.compare_digest`. Store writes are atomic
(tmp + `os.replace`) so an interrupted save cannot lock the user out. Both `auth.json`
and `session_secret` are chmod 0600 under `/.umbra/`, which is **gitignored** (line 42).
- **First use**: username `bdumbra`, one-time password `First321`, stored **hashed** and
marked `must_change`, so the bootstrap password never lands on disk in plaintext and is
dead after the first login. Password rule: **more than six chars, >=1 letter and >=1
special**. Sessions 14 days, sliding.
- Login failures are deliberately vague - wrong username and wrong password give the SAME
message, so the form cannot enumerate accounts.
- **19 tests** in `tests/test_auth.py`, incl. tampered/forged/expired tokens, plaintext
absence, 0600 mode, corrupt-store recovery. Full suite **649 passed** (8 pre-existing).
- **Deliberately separate module**: `auth.py` + `web/auth_routes.py` mirror the
`name_token_routes` pattern, so the hot `routes.py` stays untouched and controller-delegated
identity can replace this without touching route code.
- End state is the **controller on oldbox** authenticating centrally (spec 21/22). Until
then this is the local stand-in - see `oldbox` for that plan.
### Session: Distribution spec (AR + Spotify); peak gate fixed; Deal Hunter regen in flight
- **Distribution target: Author's Republic + Spotify for Authors.** INaudio is ruled out -
DVN must be an unmodified LPF from Google Play Books / ElevenLabs / Spoken Press only, and
submitting TTS as "Human Narrated" risks removal/account action. Umbra Audio is neither.
- **Spotify route is `Spotify for Authors` directly**, not INaudio: Spotify accepts DVN with a
mandatory disclosure checkbox, and its DVN titles are **not shared to referral partners** -
so Spotify is the entire reach of that submission. Non-exclusive, free, no title limit.
- **The two specs agree on everything except head silence.** INaudio wants 0.5-1 s, AR wants
1-5 s; the only overlap is **exactly 1.0 s** (tail 2.0 s satisfies both). Aligning to AR at
head 1.0 s keeps the masters INaudio-ready if that ever changes.
- **Sample Track is intentional and stays**: a 93 s multi-voice capability demo, not book text.
My earlier flag (item 6) was INaudio's "no credits in a sample" rule, which no longer applies.
Spotify only requires <10 min, no spoilers.
- **Narrator rule: first name `Umbra`, last name `Audio`, sole narrator** - all Umbra Audio
books. Credit lines read `<Title>, Written by <Author>, Narrated by Umbra Audio`.
- **Peak gate fixed** (commit `0962d82`) - see below. **Closing credit ordering fixed** too.
- **Pending decision (operator chose "fix later"):** `export_m4b` encodes `-c:a aac -b:a 128k`
with **no `-ar`**, so it emits **24 kHz / ~96 kbps** audio while the MP3 masters are
44.1 kHz / 192 kbps CBR mono. For a file sold directly this is measurably worse than the MP3
set. Re-export the m4b as-is now; fixing it to 44.1 kHz is a separate follow-up. Also open:
spec 22 SS7 gap 1 - `export_m4b._stitch_chapter` hard-codes
`silence_between_utterances_ms=500, silence_between_chapters_ms=0` while `ChapterBuilder`
uses `AudioStitcher()` defaults (500/2000/4000).
- **IN FLIGHT: full Deal Hunter audio regeneration** (PID from `logs/regen_full.log`).
2,924 of 4,319 utterances are stale due to a **pre-existing index drift**: the audio holds a
leading chapter-heading utterance that the DB no longer has, so in ch6 every marker stores
`DB[i-1]` and none store `DB[i]`. Deliberately NOT "fixed" by shifting markers - that would
make the staleness check pass while leaving `utt_*.wav` misaligned with utterance ids, the
same declared-vs-measured class of bug just fixed in the audio gate. DB backup:
`/tmp/project.PRE_END.20261008_111927.db`.
- **`The End.`** now replaces a bare `END` as ch32's final utterance (DB + `source_text`); its
marker was invalidated so it re-renders. Not yet in audio.
- **Website was already in the closing track** - `ch35/utt_0001.wav.txt` renders "My Website is
authorbdmurphy.com". Note `ch35.source_text` (a 616-char author bio) **disagrees** with its
three utterances; the utterances are the good closing credit. A re-segmentation would
overwrite them - worth fixing before anything re-segments.
- **NEXT: after regen finishes, re-export BOTH** the AR MP3 package (which applies the peak fix
and closing-last ordering) and the m4b. **Never export mid-run** - `export-mp3` stitches
whatever `ch*_full.wav` exist, and has no guard against a concurrent batch.
### Session: GPU box retired; Qwen TTS boot start; public DNS findings
- **GPU box `192.168.123.14` is RETIRED (2026-10-08)** - offline and being reconfigured.
Removed from configuration: `.env` no longer sets `UMBRA_TTS__QWEN_TTS_HELPER_URL`
(default is `""`, so this is a no-op fallback), `MEANINGFUL_PORTS` lost the `18881`
relay entry, and `umbra_node_report.py` now reports `controller_endpoint: None` with a
`controller_status` explaining why. `config.py` comment updated. **Local `:8881` is the
only TTS backend.** The root relay `/Library/LaunchDaemons/com.umba.qwen-relay.plist` +
`qwen_relay.py` still need removing with sudo.
- **NOT a blocker for the controller.** The UmbraTrans controller is **still in
development**; specs 21/22 proceed unchanged. GPU capacity will return through the
**controller system** by a different route, not via the removed direct relay. Do not
reintroduce `qwen_tts_helper_url`/`:18881` speculatively - re-add a TTS backend only
when the controller exposes one.
- **Qwen TTS now auto-starts at boot** (`com.umbraaudio.qwentts`, installed to
`/Library/LaunchAgents` for the same mount-race reason as the web server). Launcher is
`/Users/worstauthor/.umbraaudio/umbra_tts_login.sh`, kept on the **boot** volume because a
launchd shell cannot open anything on the external volume; it only `cd`s and execs the venv
python, which does all I/O including the stdout/stderr `dup2` into
`logs/qwen_tts_server.log`. `KeepAlive` verified by `kill -9`: respawned in ~4s. LM Studio is
deliberately **manual** - an LLM model and the TTS model cannot be resident together in 16 GB.
- **Do not point launchd stdio at `/Volumes/1TB_APL_Storage`** - the job dies with
`EX_CONFIG 78`. Both plists therefore have **no** `StandardOutPath`/`StandardErrorPath`.
- **The root qwen relay is retired** (`/Library/LaunchDaemons/com.umba.qwen-relay.plist` +
`qwen_relay.py`, was running as root PID 850 and therefore **invisible to unprivileged
`lsof`/`ps`**). Booted out and moved to
`/Users/worstauthor/.umbraaudio/retired/*.retired.20261008` - deliberately moved **out of**
`/Library/LaunchDaemons` so nothing can re-scan them. `:18881` is now closed.
- **The web UI already self-heals TTS:** `ModelCoordinator.ensure_qwen_server()`
(`model_coordinator.py:238`) spawns `scripts/qwen_tts_server.py` via
`Popen(start_new_session=True)` when `/health` fails. It is called only from
`switch_to()` (line 293) - i.e. **on demand during pipeline work, NOT at boot or web
startup**. It checks `qwen_health()` first, so a launchd-owned instance is reused
rather than double-spawned: the two mechanisms are complementary, not conflicting.
- **`umbraaudio.bdmurphy.me` TLS: Caddy fails with `internal error` (alert 80)** on the
vhost while `bdmurphy.me` itself presents a valid cert. Cause was the missing public A
record - with no DNS entry an ACME HTTP-01 challenge cannot validate, so on-demand TLS
aborts the handshake. Caddy is at **`192.168.123.5`** (NOT the public IP: port-forwarded
from 97.113.1.212). Diagnosing this via the public IP is misleading: the LAN host
answers :80 and :443, the public IP does not hairpin from inside the LAN.
- Useful negative result: Caddy answers `308 -> https://<Host>/` for **any** hostname,
including invented ones, so a redirect Location cannot be used to infer vhost config.
Only an SNI handshake (`openssl s_client -servername ...`) distinguishes them.
### Session: Voice-maintenance guard (409) + boot auto-start failure diagnosed
- **The banner lied by omission.** `voice_maintenance` was wired into `routes.py` **only**
as a template global (`_voice_maintenance_for_template`, ~line 65) — a *display* hook, not
a gate. Nothing stopped generation. Clicking Generate during an LLM run would retry Qwen,
exhaust attempts, and write a **degraded Edge-TTS ref to disk**. Commit `eb37777` adds
`_voice_maintenance_block()` returning **409** and wires it into the **six** endpoints that
synthesize audio: `generate_utterance_audio`, `delete_utterance`, `create_utterance`,
`regenerate_speaker_voice`, `generate_all_speaker_voices`, `generate_full_audiobook`.
- **Deliberately NOT blocked:** `speakers_list_partial` (a Qwen *health check* — that IS the
banner the reviewer needs) and `set_voice_status` (reads an existing `_ref.wav` to capture
a vmatrix — part of the approval workflow). Blocking either would break review while paused.
- Verified live: all 6 return 409 with an explanatory body; the 2 allowed ones return 400.
- **`active_voice_maintenance()` is pid-gated** (`os.kill(pid, 0)`), so a crashed run
self-heals. **This will bite you when testing:** writing the marker from a short-lived
process returns `None` and the guard looks broken. Test with a *live* holder process, e.g.
`python -c "from umbraaudio.voice_maintenance import set_voice_maintenance; import time;
set_voice_maintenance('TEST'); time.sleep(240)" &`. Verified self-heal: kill the holder →
marker goes inactive → endpoints unblock.
- **The web server did NOT restart on boot — root cause was a mount race, and it is FIXED.**
The plist was always sound (`RunAtLoad` + `KeepAlive`, no volume stdio paths, so no
`EX_CONFIG 78`). The real fault: **`$HOME` is `/Volumes/Dock/worstauthor`** (relocated) and
launchd scans `$HOME/Library/LaunchAgents` **ONCE at login**. Measured 2026-10-08: boot
**05:58**, Dock mounted **05:59** — the scan ran before the volume existed, so the plist was
invisible and the job never loaded. The `/Users/worstauthor/Library/LaunchAgents` copy was
never auto-scanned either, because it is not `$HOME`.
- **Fix installed:** the plist now lives in **`/Library/LaunchAgents/com.umbraaudio.web.plist`**
(system-wide, on the boot volume, scanned for every user regardless of `$HOME`). The
`$HOME` mirror was **removed** — two plists sharing one `Label` is a race, since launchd
loads only the first and errors on the other. Backup at
`/Users/worstauthor/.umbraaudio/com.umbraaudio.web.plist.dock.bak`; the identical
`/Users/worstauthor/...` copy is retained as a manual-bootstrap/rollback source.
- Verified running from the new path (`launchctl print` → `path = /Library/LaunchAgents/
com.umbraaudio.web.plist`, `state = running`, PID 9612 on *:8420). Corroborated empirically:
`com.carriez.RustDesk_server` is loaded from `/Library/LaunchAgents`, proving launchd does
scan that directory in this session.
- Manual recovery (either works):
`launchctl bootstrap gui/$(id -u) /Library/LaunchAgents/com.umbraaudio.web.plist` or
`... /Users/worstauthor/Library/LaunchAgents/com.umbraaudio.web.plist`.
- **launchd footgun learned the hard way:** `launchctl bootstrap` against an
already-loaded Label fails with **125 "Domain does not support specified action"**, and
`bootout` + `bootstrap` in one scripted step can hit it if the job is still loaded. Never
suppress stderr on `bootout`, and **confirm the port is actually down before bootstrapping**
— otherwise you get a false "bootstrap failed" and think you broke something. Also note
`KeepAlive` means `kickstart -k` is the right way to restart, not bootout/bootstrap.
- Still worth a reboot check when convenient — the durable proof is a real login.
- **The `edit` tool and bare shells cannot write `/Volumes/1TB_APL_Storage`** (`EPERM`), and
`/Volumes/Dock` flapped independently over minutes — sometimes readable, sometimes not, while
`ls`/`df` kept working. Use **`.venv/bin/python`** for edits on the volume (it IS
volume-approved), and re-check access before assuming a file is writable.
- **`umbraaudio.bdmurphy.me` cannot be verified from this Mac — two independent reasons.**
(a) **No public A record exists.** Queried the *authoritative* nameservers
(`dns1/dns2.registrar-servers.com`) directly, bypassing every cache: they return
`97.113.1.212` for the apex `bdmurphy.me` but **no record at all** for the `umbraaudio`
subdomain. The LAN resolver (192.168.123.254, nginx) also says NXDOMAIN. So this is DNS-side,
not a cache artifact. (b) **NAT hairpin failure** — from inside the LAN, TCP to 443 on the
public IP fails the TLS handshake even for the apex, so the app behind Caddy cannot be
probed locally. Caddy on `97.113.1.212` answers :80 with a blanket `308 -> https://<Host>/`
for *any* hostname (verified with an invented one), so vhost presence cannot be inferred.
The operator states the site **is running** — treat (a) as "no public DNS record yet", not
"down". The app itself is verified healthy on LAN `192.168.123.12:8420`.
### Session: Gitea on Mini-EQ wired up; UmbraAudio + Qwen3-TTS pushed
- **UmbraAudio now has an origin.** It had **zero remotes** for its whole life; first

202
LICENSE Normal file
View File

@@ -0,0 +1,202 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright 2026 Umbra Tools (umbratools.com)
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

2
NOTICE Normal file
View File

@@ -0,0 +1,2 @@
Umbra Tools (https://umbratools.com)
Copyright 2026 Umbra Tools (umbratools.com)

BIN
UmbraAudio_icon.jpeg
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 661 KiB

3
backup/Starthere.sh
View File

@@ -1,4 +1,7 @@
#!/usr/bin/env bash
# Copyright 2026 Umbra Tools (umbratools.com)
# SPDX-License-Identifier: Apache-2.0
# Restore an UmbraAudio full backup and rebuild a portable runtime.
set -Eeuo pipefail

215
backup/author_voice_lab/FINDINGS.md Normal file
View File

@@ -0,0 +1,215 @@
# Author voice — matrix backup, matrix→WAV reproducibility, and voice-range lab
Date: 2026-10-09
Base: `author.pt` vmatrix (SHA-256 `d38a2dc8…`), the Author's own voice.
Artifacts: `backup/author_voice_matrix/`, `backup/author_voice_lab/`
---
## 1. The matrix is backed up and its provenance is now proven
Two locations, checksummed, with a manifest and a restore sheet:
- `backup/author_voice_matrix/20261009T191636Z/` (1 TB volume)
- `~/.umbra/library/author_voice_matrix/20261009T191636Z/` (boot volume)
Contains `author_v2.pt` (live), `author_v1.pt` (superseded), the full reference
chain, `manifest.json` (SHA-256 + provenance per file) and `RESTORE.md`.
**This mattered.** `voice_prompts/` is git-ignored project cache, and the *only*
previous copy of v1 sat in an OS temp directory that macOS is free to purge. The
v1/v2 account in the round-2 notes checks out: both matrices are 61,853 bytes with
`ref_code` (429, 16), but different SHA-256 — same duration, different content,
consistent with the `noisered 0.4` → `0.15` change.
## 2. The matrix can reproduce the WAV — proven, not assumed
`ref_code` is the codec-token matrix of the reference speech, and the codec's
decoder is the exact inverse of the encoder that produced it:
```
Qwen3TTSTokenizer.from_pretrained(
models/Qwen3-TTS-12Hz-1.7B-Base/speech_tokenizer
).decode({"audio_codes": ref_code})
```
Result: **34.32 s @ 24 kHz, median F0 129.4 Hz** — the reference's own documented
value is 129.4 Hz.
Fidelity, measured phase-insensitively (a 12 Hz codec does not preserve phase, so
waveform correlation is meaningless here):
| Measure | Decoded from matrix | Original capture | Delta |
|---|---|---|---|
| median F0 | 129.4 Hz | 128.6 Hz | +0.8 Hz |
| voiced frames | 1145 | 1132 | +1.2 % |
| LTAS band deviation (80 Hz–12 kHz, 8 bands) | — | — | **0.55 dB RMS**, worst band +1.10 dB |
So the matrix alone is sufficient. The WAV is a derived artifact, not a
prerequisite. `author_ref.wav` is now installed in the project **decoded from the
matrix**, with a sidecar JSON recording the matrix's SHA-256 and the exact
regeneration command.
### Provenance proven by re-encoding
Rather than trusting the notes, each candidate source was re-encoded and its
`audio_codes` compared to the stored `ref_code`:
| Candidate | shape | exact match | frame agreement |
|---|---|---|---|
| `BDMurphy_Voice_NOBG_v2.wav` (documented v2 source) | (429, 16) | no | **0.432** |
| `BDMurphy_Voice_NOBG.wav` (v1 source) | (429, 16) | no | 0.103 |
| `BDMurphy_Voice.wav` (raw recording) | (430, 16) | no | shape mismatch |
43.2 % vs 10.3 % — 4.2× — so the live matrix provably came from the v2 file. (Not
100 % because the codec encode is not bit-deterministic under MPS/bfloat16.)
**Note:** embedding cosine similarity is useless for speaker ID here — author vs
nanny scores 0.933 and Kainda vs Xylia 0.976. All 2048-d embeddings sit in one
tight cluster. Do not use it to test "is this the same person".
## 3. Voice-range lab — 44 samples on the Author voice
All samples use one fixed text so comparisons are valid. Every sample recorded F0,
spectral-envelope centroid and its semitone offset, brightness, flatness, level and
whisper WER.
### 3a. The neural instruct axis is nearly inert on a real human reference
| Instruct | ΔF0 | Δenvelope | WER |
|---|---|---|---|
| "a young woman's voice, bright and light" | **+1.3 st** | −2.7 st | 0.08 |
| "a mature woman's voice, low and calm" | +0.9 st | −2.0 st | 0.00 |
| "a young boy's voice, light and eager" | −0.3 st | −0.9 st | 0.08 |
| "an elderly man's voice, deep, gravelly" | +0.6 st | −0.5 st | 0.00 |
| "a very large man's voice, heavy and booming" | +1.0 st | −0.9 st | 0.00 |
| "a small woman's voice, petite and light" | +0.8 st | +2.5 st | 0.16 |
**This is the headline and it is counterintuitive.** The identical instruct run
against the *Narrator* TTS reference moved F0 by +6.9 st. Against the Author's own
recorded voice it moves 1.3 st. A real human capture anchors the clone far harder
than a synthesised reference, so the instruct has almost no leverage left.
Practical consequence: **the neural instruct is not a usable morph axis for the
Author voice. DSP is the only axis that works.** That is good for brand
consistency and bad for variety, and it is not fixable by better prompt wording.
### 3b. The DSP pitch axis is effectively one-directional
| shift | F0 | Δenvelope | WER |
|---|---|---|---|
| +12 st | 266.3 Hz | +6.1 st | **0.00** |
| +10 st | 238.6 Hz | +4.8 st | **0.00** |
| +8 st | 211.4 Hz | +3.7 st | **0.00** |
| +6 st | 188.3 Hz | +2.3 st | 0.00 |
| +4 st | 167.8 Hz | +1.5 st | 0.00 |
| +2 st | 150.3 Hz | +0.4 st | 0.04 |
| 0 | 133.9 Hz | −1.6 st | 0.04 |
| −2 st | 118.6 Hz | −3.1 st | **0.64** |
| −4 st | 104.5 Hz | −3.8 st | **0.80** |
| −6 st | 93.6 Hz | −5.4 st | **1.00** |
Upward is free to +12 st (266 Hz — squarely female range, WER 0.00).
**Downward collapses after 2 semitones.**
Re-tested on a 58-word text with 2 seeds, because a 25-word sample is too coarse
to stake a claim on:
| shift | WER (seed A / seed B) |
|---|---|
| +2 st | 0.069 / 0.086 |
| 0 | 0.069 / 0.069 |
| **−2 st** | **0.724 / 0.845** |
| −4 st | 0.983 / 0.948 |
| −6 st | 1.000 / 0.897 |
Confirmed and reproducible. **Caveat worth stating plainly: WER measures machine
recognition, not human comprehension.** A human adapts to a low, growl-like voice
far better than whisper does. `wav_cliff/cliff-06_s7.wav` and `cliff-04_s7.wav`
should be listened to before concluding the monster band is unusable for *speaking*
monsters rather than roaring ones.
### 3c. How many distinct voices does one recording actually support?
15 samples of the *same* voice across 5 seeds × 3 temperatures, no DSP:
- F0 123.5–141.9 Hz → **+2.4 st total spread**, sd 0.76 st
- envelope 763–930 Hz → 3.4 st spread, sd 0.78 st
So **re-rolling the seed alone moves the voice ~2.4 semitones.** Any variant closer
than ~1.5 st (2 sd) to another is not reliably distinguishable from a re-roll — it
is the same voice.
Applying both gates (separation > 1.5 st **and** WER ≤ 0.20) to the measured axis:
> **~7 reliably distinct, intelligible voices from this one recording, spanning
> +2.6 to +12.5 semitones — every one of them above the Author's natural register.**
There is no room below. Adding the unintelligible-creature band the operator
sanctioned takes the span to +0.6 … +12.5 st, still ~8 voices, because downward is
the same one axis.
**This is the number that matters and it is the real constraint.** One source
recording buys ~7–8 audibly distinct voices, all higher than the Author. To go
further you need a genuinely independent second dimension — which brings us to the
missing primitive below.
## 4. The missing primitive, re-scoped
There is **no independent formant shifter** in the toolchain. The obvious
implementation — resample then time-stretch — was written and measured: **it is an
identity** (ΔF0 +0.1 st, Δenvelope +0.1 st), because the two operations exactly
cancel. Recorded here so nobody repeats it.
`pitch_shift_pv` is asymmetric, and measured on the Author voice:
- upward: formants move at **0.28** of the pitch shift
- downward: formants move at **0.85** of it (overshoot)
The upward 0.28 ratio happens to match the real male→female vocal-tract ratio
(0.29), which is why +8 to +12 st produces a convincing female register with no
extra machinery. That is luck, not design.
**Revised recommendation.** My earlier advice was "don't build the formant shifter,
the phase vocoder already covers it." That was right about male→female and wrong
about cast size. With only one usable axis, the ceiling is ~7–8 voices. An
independent formant shift adds a **second** dimension (tract × pitch), which is the
only way to break that ceiling — e.g. keeping pitch fixed while changing vocal
tract gives a family sharing one throat, or the reverse for unrelated characters at
the same pitch. Options: `rubberband` (has a real `formantScale`),
`pyworld` + envelope warping (pure Python, ~40 lines, mediocre quality on speech).
Worth doing only if a >8-voice cast is actually wanted. It is not needed for
female voices or for the monster band.
## 5. Bug found and reported, not fixed
**32 of 97 reference WAVs in the DealHunter project are 3–4 second stubs**, not
30–58 s Rainbow Passages — including `b.d._murphy_ref.wav` (3.48 s) and
`kainda_ref.wav` (3.00 s), all dated Jul 31 22:37–22:39. `kainda_walters_foster_
murphy_ref.wav` (the real one) exists separately, so these look like truncated
writes that were never cleaned up. Any speaker resolving to a stub gets a 3-second
reference. Not fixed — reported only, as it does not block the Author lab.
## 6. Reproducing
```bash
cd /Volumes/1TB_APL_Storage/UmbraAudio
.venv/bin/python backup/author_voice_lab/scripts/matrix_roundtrip.py # matrix -> wav
.venv/bin/python backup/author_voice_lab/scripts/voice_lab.py # 44-sample sweep
.venv/bin/python backup/author_voice_lab/scripts/verify_cliff.py # cliff re-test
```
`voice_lab.py` reads `author_ref.wav` and writes only into this lab directory; it
does not touch any project's audio.
### Samples worth listening to first
| File | What it is |
|---|---|
| `wav/A_pitch+08.wav` | +8.5 st, 211 Hz, WER 0.00 — convincing female register |
| `wav/A_pitch+12.wav` | +12.5 st, 266 Hz, WER 0.00 — upper edge |
| `wav/C_monster_grit_full.wav` | dragon instruct + sat/ring/chorus/jitter, WER 0.12 |
| `wav/B_female_young.wav` | the instruct that barely moved — the key negative result |
| `wav_cliff/cliff-04_s7.wav` | where downward shifting breaks |
| `wav_cliff/cliff+02_s7.wav` | control, free upward |

85
backup/author_voice_lab/cliff_verification.json Normal file
View File

@@ -0,0 +1,85 @@
{
"text_words": 58,
"rows": [
{
"shift_st": 0,
"seed": 7,
"wer": 0.069,
"f0": 130.1,
"hyp": "e set the lantern on the stone ledge and listened he cavern breathed around them slow and patient and some were far below the water was moving against rock ary nn kept one hand on the rail and waited for him to speak first because she had learned that what he said next would decide which of them went forward",
"sec": 32.0
},
{
"shift_st": 0,
"seed": 8,
"wer": 0.069,
"f0": 127.9,
"hyp": "e set the lantern on the stone ledge and listened he cavern breathed around them slow and patient and some were far below the water was moving against rock ary nn kept one hand on the rail and waited for him to speak first because she had learned that what he said next would decide which of them went forward",
"sec": 27.6
},
{
"shift_st": -2,
"seed": 7,
"wer": 0.724,
"f0": 116.6,
"hyp": "e said who turn on the stone why should listen he cabin breaks and ll move so in person and so will the fabule of water was moving and swung am the one who am all unworthy for and to speak first because you are the one who sent me to decide which is the one for",
"sec": 31.1
},
{
"shift_st": -2,
"seed": 8,
"wer": 0.845,
"f0": 114.6,
"hyp": "es a goodass honest oldass man listen es having great trouble slow impression and some of the fog in the water was moving as well ow he ends up working on the water while he finds his big face es a shamed man that wont be so nascult to side but should go more forward",
"sec": 28.6
},
{
"shift_st": -4,
"seed": 7,
"wer": 0.983,
"f0": 103.3,
"hyp": "he circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of the circle of",
"sec": 33.5
},
{
"shift_st": -4,
"seed": 8,
"wer": 0.948,
"f0": 102.1,
"hyp": "dont know if was going to have a solution dont know if could have a solution dont know if could have a solution dont know how would have a solution dont know if could have a solution",
"sec": 27.8
},
{
"shift_st": -6,
"seed": 7,
"wer": 1.0,
"f0": 92.0,
"hyp": "hantana chokyusha naka shing aba kawa kawa koshia naka shing hantana chokyusha naka shing awa kawa kawa koshia naka shing hantana chokyusha naka shing",
"sec": 31.9
},
{
"shift_st": -6,
"seed": 8,
"wer": 0.897,
"f0": 90.9,
"hyp": "have a lot of work to do with the rest of my life have a lot of work to do with the rest of my life have a lot of work to do with the rest of my life have a lot of work to do with the rest of my life have a lot of work to do with the rest of my life",
"sec": 27.4
},
{
"shift_st": 2,
"seed": 7,
"wer": 0.069,
"f0": 146.0,
"hyp": "e said the lantern on the stone ledge and listened he cavern breathed around them slow and patient and somewhere far below the water was moving against rock ary nn kept one hand on the rail and waited for him to speak first because she had learned that what he said next would decide which of them went forward",
"sec": 32.2
},
{
"shift_st": 2,
"seed": 8,
"wer": 0.086,
"f0": 144.4,
"hyp": "e set the lantern on the stern ledge and listened he cavern breathed around them slow and patient and somewhere far below the water was moving against rock ary and kept one hand on the rail and waited for him to speak first because she had learned that what he said next would decide which of them d forward",
"sec": 27.7
}
]
}

873
backup/author_voice_lab/results.json Normal file
View File

@@ -0,0 +1,873 @@
{
"reference": {
"f0_median": 129.4,
"envelope_hz": 894,
"path": "/Volumes/1TB_APL_Storage/UmbraAudio/DealHunterShip_4.0e_whole-to-audio/.umbra/voices/author_ref.wav"
},
"rows": [
{
"f0_median": 133.9,
"f0_sd": 15.1,
"envelope_hz": 814,
"centroid_hz": 1597,
"flatness": 0.04796,
"dur_s": 8.0,
"rms_db": -23.0,
"peak_db": -7.5,
"d_f0_st": 0.6,
"label": "A00_baseline",
"gen_s": 21.0,
"instruct": null,
"temperature": 0.7,
"seed": 1234,
"dsp": null,
"d_env_st": -1.6,
"wer": 0.04
},
{
"f0_median": 65.8,
"f0_sd": 6.4,
"envelope_hz": 488,
"centroid_hz": 718,
"flatness": 0.00187,
"dur_s": 8.0,
"rms_db": -26.9,
"peak_db": -10.5,
"d_f0_st": -11.7,
"label": "A_pitch-12",
"gen_s": 12.8,
"instruct": null,
"temperature": 0.7,
"seed": 1234,
"dsp": [
"pitch_shift_pv"
],
"d_env_st": -10.5,
"wer": 1.0
},
{
"f0_median": 75.2,
"f0_sd": 7.6,
"envelope_hz": 528,
"centroid_hz": 809,
"flatness": 0.00264,
"dur_s": 8.0,
"rms_db": -26.7,
"peak_db": -9.6,
"d_f0_st": -9.4,
"label": "A_pitch-10",
"gen_s": 12.8,
"instruct": null,
"temperature": 0.7,
"seed": 1234,
"dsp": [
"pitch_shift_pv"
],
"d_env_st": -9.1,
"wer": 1.0
},
{
"f0_median": 82.7,
"f0_sd": 45.6,
"envelope_hz": 578,
"centroid_hz": 920,
"flatness": 0.00423,
"dur_s": 8.0,
"rms_db": -26.3,
"peak_db": -10.8,
"d_f0_st": -7.7,
"label": "A_pitch-08",
"gen_s": 12.7,
"instruct": null,
"temperature": 0.7,
"seed": 1234,
"dsp": [
"pitch_shift_pv"
],
"d_env_st": -7.6,
"wer": 1.0
},
{
"f0_median": 93.6,
"f0_sd": 10.8,
"envelope_hz": 654,
"centroid_hz": 1049,
"flatness": 0.00618,
"dur_s": 8.0,
"rms_db": -27.1,
"peak_db": -10.2,
"d_f0_st": -5.6,
"label": "A_pitch-06",
"gen_s": 12.8,
"instruct": null,
"temperature": 0.7,
"seed": 1234,
"dsp": [
"pitch_shift_pv"
],
"d_env_st": -5.4,
"wer": 1.0
},
{
"f0_median": 104.5,
"f0_sd": 58.7,
"envelope_hz": 719,
"centroid_hz": 1182,
"flatness": 0.01017,
"dur_s": 8.0,
"rms_db": -26.6,
"peak_db": -9.1,
"d_f0_st": -3.7,
"label": "A_pitch-04",
"gen_s": 12.8,
"instruct": null,
"temperature": 0.7,
"seed": 1234,
"dsp": [
"pitch_shift_pv"
],
"d_env_st": -3.8,
"wer": 0.8
},
{
"f0_median": 118.6,
"f0_sd": 12.4,
"envelope_hz": 749,
"centroid_hz": 1327,
"flatness": 0.01774,
"dur_s": 8.0,
"rms_db": -26.6,
"peak_db": -9.9,
"d_f0_st": -1.5,
"label": "A_pitch-02",
"gen_s": 12.9,
"instruct": null,
"temperature": 0.7,
"seed": 1234,
"dsp": [
"pitch_shift_pv"
],
"d_env_st": -3.1,
"wer": 0.64
},
{
"f0_median": 133.9,
"f0_sd": 15.1,
"envelope_hz": 814,
"centroid_hz": 1597,
"flatness": 0.04796,
"dur_s": 8.0,
"rms_db": -23.0,
"peak_db": -7.5,
"d_f0_st": 0.6,
"label": "A_pitch+00",
"gen_s": 13.4,
"instruct": null,
"temperature": 0.7,
"seed": 1234,
"dsp": [
"pitch_shift_pv"
],
"d_env_st": -1.6,
"wer": 0.04
},
{
"f0_median": 150.3,
"f0_sd": 16.0,
"envelope_hz": 913,
"centroid_hz": 1606,
"flatness": 0.03015,
"dur_s": 8.0,
"rms_db": -26.9,
"peak_db": -10.5,
"d_f0_st": 2.6,
"label": "A_pitch+02",
"gen_s": 12.8,
"instruct": null,
"temperature": 0.7,
"seed": 1234,
"dsp": [
"pitch_shift_pv"
],
"d_env_st": 0.4,
"wer": 0.04
},
{
"f0_median": 167.8,
"f0_sd": 18.1,
"envelope_hz": 977,
"centroid_hz": 1718,
"flatness": 0.03189,
"dur_s": 8.0,
"rms_db": -26.1,
"peak_db": -9.8,
"d_f0_st": 4.5,
"label": "A_pitch+04",
"gen_s": 12.9,
"instruct": null,
"temperature": 0.7,
"seed": 1234,
"dsp": [
"pitch_shift_pv"
],
"d_env_st": 1.5,
"wer": 0.0
},
{
"f0_median": 188.3,
"f0_sd": 21.8,
"envelope_hz": 1021,
"centroid_hz": 1837,
"flatness": 0.02948,
"dur_s": 8.0,
"rms_db": -26.6,
"peak_db": -9.4,
"d_f0_st": 6.5,
"label": "A_pitch+06",
"gen_s": 13.0,
"instruct": null,
"temperature": 0.7,
"seed": 1234,
"dsp": [
"pitch_shift_pv"
],
"d_env_st": 2.3,
"wer": 0.0
},
{
"f0_median": 211.4,
"f0_sd": 23.4,
"envelope_hz": 1109,
"centroid_hz": 1946,
"flatness": 0.02749,
"dur_s": 8.0,
"rms_db": -26.4,
"peak_db": -9.8,
"d_f0_st": 8.5,
"label": "A_pitch+08",
"gen_s": 13.1,
"instruct": null,
"temperature": 0.7,
"seed": 1234,
"dsp": [
"pitch_shift_pv"
],
"d_env_st": 3.7,
"wer": 0.0
},
{
"f0_median": 238.6,
"f0_sd": 27.5,
"envelope_hz": 1178,
"centroid_hz": 2086,
"flatness": 0.0298,
"dur_s": 8.0,
"rms_db": -25.7,
"peak_db": -7.3,
"d_f0_st": 10.6,
"label": "A_pitch+10",
"gen_s": 13.1,
"instruct": null,
"temperature": 0.7,
"seed": 1234,
"dsp": [
"pitch_shift_pv"
],
"d_env_st": 4.8,
"wer": 0.0
},
{
"f0_median": 266.3,
"f0_sd": 31.5,
"envelope_hz": 1269,
"centroid_hz": 2258,
"flatness": 0.02758,
"dur_s": 8.0,
"rms_db": -25.3,
"peak_db": -9.0,
"d_f0_st": 12.5,
"label": "A_pitch+12",
"gen_s": 13.0,
"instruct": null,
"temperature": 0.7,
"seed": 1234,
"dsp": [
"pitch_shift_pv"
],
"d_env_st": 6.1,
"wer": 0.0
},
{
"f0_median": 139.8,
"f0_sd": 17.7,
"envelope_hz": 766,
"centroid_hz": 1527,
"flatness": 0.0408,
"dur_s": 8.24,
"rms_db": -23.8,
"peak_db": -8.1,
"d_f0_st": 1.3,
"label": "B_female_young",
"gen_s": 13.4,
"instruct": "A young woman's voice, bright and light, with a higher and clearer tone.",
"temperature": 0.7,
"seed": 1234,
"dsp": null,
"d_env_st": -2.7,
"wer": 0.08
},
{
"f0_median": 136.3,
"f0_sd": 18.9,
"envelope_hz": 796,
"centroid_hz": 1505,
"flatness": 0.03024,
"dur_s": 7.76,
"rms_db": -22.1,
"peak_db": -4.8,
"d_f0_st": 0.9,
"label": "B_female_mature",
"gen_s": 12.9,
"instruct": "A mature woman's voice, low and calm for a woman, unhurried and steady.",
"temperature": 0.7,
"seed": 1234,
"dsp": null,
"d_env_st": -2.0,
"wer": 0.0
},
{
"f0_median": 127.1,
"f0_sd": 12.4,
"envelope_hz": 849,
"centroid_hz": 1583,
"flatness": 0.03735,
"dur_s": 7.44,
"rms_db": -24.8,
"peak_db": -7.1,
"d_f0_st": -0.3,
"label": "B_boy",
"gen_s": 11.7,
"instruct": "A young boy's voice, light and eager, not yet broken.",
"temperature": 0.7,
"seed": 1234,
"dsp": null,
"d_env_st": -0.9,
"wer": 0.08
},
{
"f0_median": 133.9,
"f0_sd": 28.1,
"envelope_hz": 868,
"centroid_hz": 1592,
"flatness": 0.04646,
"dur_s": 7.04,
"rms_db": -22.3,
"peak_db": -6.1,
"d_f0_st": 0.6,
"label": "B_elderly_m",
"gen_s": 11.1,
"instruct": "An elderly man's voice, very deep, gravelly and slow.",
"temperature": 0.7,
"seed": 1234,
"dsp": null,
"d_env_st": -0.5,
"wer": 0.0
},
{
"f0_median": 133.9,
"f0_sd": 13.9,
"envelope_hz": 766,
"centroid_hz": 1621,
"flatness": 0.04052,
"dur_s": 8.08,
"rms_db": -22.0,
"peak_db": -4.9,
"d_f0_st": 0.6,
"label": "B_young_m",
"gen_s": 12.3,
"instruct": "A young man's voice, light, energetic and clear.",
"temperature": 0.7,
"seed": 1234,
"dsp": null,
"d_env_st": -2.7,
"wer": 0.0
},
{
"f0_median": 137.0,
"f0_sd": 17.7,
"envelope_hz": 851,
"centroid_hz": 1748,
"flatness": 0.05657,
"dur_s": 8.08,
"rms_db": -25.4,
"peak_db": -8.4,
"d_f0_st": 1.0,
"label": "B_large_m",
"gen_s": 12.2,
"instruct": "A very large man's voice, heavy and booming, with a deep chest.",
"temperature": 0.7,
"seed": 1234,
"dsp": null,
"d_env_st": -0.9,
"wer": 0.0
},
{
"f0_median": 135.5,
"f0_sd": 19.4,
"envelope_hz": 1034,
"centroid_hz": 1614,
"flatness": 0.03173,
"dur_s": 7.52,
"rms_db": -23.0,
"peak_db": -5.4,
"d_f0_st": 0.8,
"label": "B_small_w",
"gen_s": 11.3,
"instruct": "A small woman's voice, petite and light, with a narrow bright tone.",
"temperature": 0.7,
"seed": 1234,
"dsp": null,
"d_env_st": 2.5,
"wer": 0.16
},
{
"f0_median": 131.6,
"f0_sd": 28.1,
"envelope_hz": 779,
"centroid_hz": 1464,
"flatness": 0.03531,
"dur_s": 7.92,
"rms_db": -22.5,
"peak_db": -5.3,
"d_f0_st": 0.3,
"label": "B_androgynous",
"gen_s": 11.9,
"instruct": "A neutral, androgynous voice, neither distinctly male nor female.",
"temperature": 0.7,
"seed": 1234,
"dsp": null,
"d_env_st": -2.4,
"wer": 0.04
},
{
"f0_median": 177.7,
"f0_sd": 22.7,
"envelope_hz": 926,
"centroid_hz": 1635,
"flatness": 0.02632,
"dur_s": 8.24,
"rms_db": -27.2,
"peak_db": -11.7,
"d_f0_st": 5.5,
"label": "C_female+4st",
"gen_s": 12.3,
"instruct": "A young woman's voice, bright and light, with a higher and clearer tone.",
"temperature": 0.7,
"seed": 1234,
"dsp": [
"pitch_shift_pv"
],
"d_env_st": 0.6,
"wer": 0.04
},
{
"f0_median": 226.5,
"f0_sd": 27.6,
"envelope_hz": 1106,
"centroid_hz": 1864,
"flatness": 0.02673,
"dur_s": 8.24,
"rms_db": -27.2,
"peak_db": -11.3,
"d_f0_st": 9.7,
"label": "C_female+8st",
"gen_s": 12.2,
"instruct": "A young woman's voice, bright and light, with a higher and clearer tone.",
"temperature": 0.7,
"seed": 1234,
"dsp": [
"pitch_shift_pv"
],
"d_env_st": 3.7,
"wer": 0.0
},
{
"f0_median": 75.2,
"f0_sd": 7.6,
"envelope_hz": 528,
"centroid_hz": 809,
"flatness": 0.00264,
"dur_s": 8.0,
"rms_db": -26.7,
"peak_db": -9.6,
"d_f0_st": -9.4,
"label": "C_monster_pitch-10",
"gen_s": 12.0,
"instruct": null,
"temperature": 0.7,
"seed": 1234,
"dsp": [
"pitch_shift_pv"
],
"d_env_st": -9.1,
"wer": 1.0
},
{
"f0_median": 132.0,
"f0_sd": 16.9,
"envelope_hz": 837,
"centroid_hz": 1525,
"flatness": 0.0437,
"dur_s": 8.4,
"rms_db": -13.6,
"peak_db": -0.5,
"d_f0_st": 0.4,
"label": "C_monster_grit_sat",
"gen_s": 12.6,
"instruct": "A huge ancient dragon: deep, rumbling, growling, with a metallic edge and a resonant echoing chest tone.",
"temperature": 0.7,
"seed": 1234,
"dsp": [
"tube_sat"
],
"d_env_st": -1.1,
"wer": 0.12
},
{
"f0_median": 74.3,
"f0_sd": 6.9,
"envelope_hz": 758,
"centroid_hz": 1463,
"flatness": 0.04517,
"dur_s": 8.4,
"rms_db": -27.7,
"peak_db": -7.4,
"d_f0_st": -9.6,
"label": "C_monster_grit_ring",
"gen_s": 12.4,
"instruct": "A huge ancient dragon: deep, rumbling, growling, with a metallic edge and a resonant echoing chest tone.",
"temperature": 0.7,
"seed": 1234,
"dsp": [
"ring_modulate"
],
"d_env_st": -2.9,
"wer": 0.12
},
{
"f0_median": 132.4,
"f0_sd": 17.8,
"envelope_hz": 827,
"centroid_hz": 1476,
"flatness": 0.03293,
"dur_s": 8.4,
"rms_db": -28.3,
"peak_db": -9.7,
"d_f0_st": 0.4,
"label": "C_monster_grit_chorus",
"gen_s": 12.4,
"instruct": "A huge ancient dragon: deep, rumbling, growling, with a metallic edge and a resonant echoing chest tone.",
"temperature": 0.7,
"seed": 1234,
"dsp": [
"chorus"
],
"d_env_st": -1.4,
"wer": 0.12
},
{
"f0_median": 69.7,
"f0_sd": 3.3,
"envelope_hz": 817,
"centroid_hz": 1575,
"flatness": 0.04212,
"dur_s": 8.4,
"rms_db": -20.1,
"peak_db": -3.2,
"d_f0_st": -10.7,
"label": "C_monster_grit_full",
"gen_s": 12.6,
"instruct": "A huge ancient dragon: deep, rumbling, growling, with a metallic edge and a resonant echoing chest tone.",
"temperature": 0.7,
"seed": 1234,
"dsp": [
"tube_sat",
"ring_modulate",
"chorus",
"pitch_jitter"
],
"d_env_st": -1.6,
"wer": 0.04
},
{
"f0_median": 127.1,
"f0_sd": 13.3,
"envelope_hz": 859,
"centroid_hz": 1568,
"flatness": 0.03092,
"dur_s": 7.92,
"rms_db": -23.1,
"peak_db": -6.0,
"d_f0_st": -0.3,
"label": "D_seed11_t0.3",
"gen_s": 11.8,
"instruct": null,
"temperature": 0.3,
"seed": 11,
"dsp": null,
"d_env_st": -0.7
},
{
"f0_median": 133.9,
"f0_sd": 54.6,
"envelope_hz": 930,
"centroid_hz": 1445,
"flatness": 0.01365,
"dur_s": 7.92,
"rms_db": -22.9,
"peak_db": -5.8,
"d_f0_st": 0.6,
"label": "D_seed22_t0.3",
"gen_s": 11.7,
"instruct": null,
"temperature": 0.3,
"seed": 22,
"dsp": null,
"d_env_st": 0.7
},
{
"f0_median": 137.8,
"f0_sd": 14.1,
"envelope_hz": 776,
"centroid_hz": 1431,
"flatness": 0.02802,
"dur_s": 7.92,
"rms_db": -22.2,
"peak_db": -5.8,
"d_f0_st": 1.1,
"label": "D_seed33_t0.3",
"gen_s": 11.8,
"instruct": null,
"temperature": 0.3,
"seed": 33,
"dsp": null,
"d_env_st": -2.5
},
{
"f0_median": 123.5,
"f0_sd": 17.4,
"envelope_hz": 847,
"centroid_hz": 1590,
"flatness": 0.03914,
"dur_s": 8.16,
"rms_db": -24.9,
"peak_db": -5.6,
"d_f0_st": -0.8,
"label": "D_seed44_t0.3",
"gen_s": 12.3,
"instruct": null,
"temperature": 0.3,
"seed": 44,
"dsp": null,
"d_env_st": -0.9
},
{
"f0_median": 134.7,
"f0_sd": 21.3,
"envelope_hz": 836,
"centroid_hz": 1599,
"flatness": 0.04181,
"dur_s": 8.16,
"rms_db": -22.9,
"peak_db": -5.4,
"d_f0_st": 0.7,
"label": "D_seed55_t0.3",
"gen_s": 12.1,
"instruct": null,
"temperature": 0.3,
"seed": 55,
"dsp": null,
"d_env_st": -1.2
},
{
"f0_median": 127.1,
"f0_sd": 15.7,
"envelope_hz": 832,
"centroid_hz": 1519,
"flatness": 0.02939,
"dur_s": 7.84,
"rms_db": -22.7,
"peak_db": -5.2,
"d_f0_st": -0.3,
"label": "D_seed11_t0.7",
"gen_s": 11.7,
"instruct": null,
"temperature": 0.7,
"seed": 11,
"dsp": null,
"d_env_st": -1.2
},
{
"f0_median": 133.1,
"f0_sd": 56.2,
"envelope_hz": 836,
"centroid_hz": 1366,
"flatness": 0.00919,
"dur_s": 7.68,
"rms_db": -22.9,
"peak_db": -6.7,
"d_f0_st": 0.5,
"label": "D_seed22_t0.7",
"gen_s": 11.6,
"instruct": null,
"temperature": 0.7,
"seed": 22,
"dsp": null,
"d_env_st": -1.2
},
{
"f0_median": 123.5,
"f0_sd": 11.8,
"envelope_hz": 763,
"centroid_hz": 1541,
"flatness": 0.04872,
"dur_s": 7.36,
"rms_db": -23.7,
"peak_db": -7.0,
"d_f0_st": -0.8,
"label": "D_seed33_t0.7",
"gen_s": 12.0,
"instruct": null,
"temperature": 0.7,
"seed": 33,
"dsp": null,
"d_env_st": -2.7
},
{
"f0_median": 125.0,
"f0_sd": 18.9,
"envelope_hz": 818,
"centroid_hz": 1370,
"flatness": 0.02523,
"dur_s": 7.04,
"rms_db": -26.0,
"peak_db": -8.1,
"d_f0_st": -0.6,
"label": "D_seed44_t0.7",
"gen_s": 11.8,
"instruct": null,
"temperature": 0.7,
"seed": 44,
"dsp": null,
"d_env_st": -1.5
},
{
"f0_median": 133.1,
"f0_sd": 17.9,
"envelope_hz": 788,
"centroid_hz": 1491,
"flatness": 0.0285,
"dur_s": 7.84,
"rms_db": -20.9,
"peak_db": -3.9,
"d_f0_st": 0.5,
"label": "D_seed55_t0.7",
"gen_s": 13.0,
"instruct": null,
"temperature": 0.7,
"seed": 55,
"dsp": null,
"d_env_st": -2.2
},
{
"f0_median": 123.5,
"f0_sd": 12.3,
"envelope_hz": 801,
"centroid_hz": 1500,
"flatness": 0.03422,
"dur_s": 7.84,
"rms_db": -22.7,
"peak_db": -5.9,
"d_f0_st": -0.8,
"label": "D_seed11_t0.9",
"gen_s": 13.2,
"instruct": null,
"temperature": 0.9,
"seed": 11,
"dsp": null,
"d_env_st": -1.9
},
{
"f0_median": 141.9,
"f0_sd": 56.3,
"envelope_hz": 822,
"centroid_hz": 1390,
"flatness": 0.00808,
"dur_s": 7.92,
"rms_db": -21.9,
"peak_db": -3.9,
"d_f0_st": 1.6,
"label": "D_seed22_t0.9",
"gen_s": 12.8,
"instruct": null,
"temperature": 0.9,
"seed": 22,
"dsp": null,
"d_env_st": -1.5
},
{
"f0_median": 136.7,
"f0_sd": 22.6,
"envelope_hz": 779,
"centroid_hz": 1452,
"flatness": 0.03224,
"dur_s": 6.88,
"rms_db": -22.4,
"peak_db": -6.5,
"d_f0_st": 1.0,
"label": "D_seed33_t0.9",
"gen_s": 10.4,
"instruct": null,
"temperature": 0.9,
"seed": 33,
"dsp": null,
"d_env_st": -2.4
},
{
"f0_median": 135.5,
"f0_sd": 23.9,
"envelope_hz": 846,
"centroid_hz": 1564,
"flatness": 0.03484,
"dur_s": 9.12,
"rms_db": -22.2,
"peak_db": -3.7,
"d_f0_st": 0.8,
"label": "D_seed44_t0.9",
"gen_s": 13.5,
"instruct": null,
"temperature": 0.9,
"seed": 44,
"dsp": null,
"d_env_st": -1.0
},
{
"f0_median": 128.6,
"f0_sd": 18.5,
"envelope_hz": 801,
"centroid_hz": 1598,
"flatness": 0.03799,
"dur_s": 7.68,
"rms_db": -22.7,
"peak_db": -5.8,
"d_f0_st": -0.1,
"label": "D_seed55_t0.9",
"gen_s": 11.3,
"instruct": null,
"temperature": 0.9,
"seed": 55,
"dsp": null,
"d_env_st": -1.9
}
]
}

94
backup/author_voice_lab/scripts/analyse_lab.py Normal file
View File

@@ -0,0 +1,94 @@
# Copyright 2026 Umbra Tools (umbratools.com)
# SPDX-License-Identifier: Apache-2.0
"""Analyse the lab: variance vs separation, and verify a suspicious F0 reading."""
from __future__ import annotations
import json
from pathlib import Path
import librosa
import numpy as np
import soundfile as sf
LAB = Path("/private/var/folders/y8/cbvhds297918lrxt2wd3btsr0000gn/T/opencode/author_lab")
res = json.loads((LAB / "results.json").read_text())
rows = res["rows"]
REF_F0 = res["reference"]["f0_median"]
REF_ENV = res["reference"]["envelope_hz"]
print(f"reference: F0 {REF_F0} Hz envelope {REF_ENV} Hz\n")
def st(v, base):
return 12 * np.log2(v / base) if v and base else float("nan")
# ---------------------------------------------------------------- 1. variance
print("=== INTRA-VOICE VARIANCE (same voice, different seed/temperature) ===")
D = [r for r in rows if r["label"].startswith("D_")]
f0s = np.array([r["f0_median"] for r in D], dtype=float)
envs = np.array([r["envelope_hz"] for r in D], dtype=float)
print(f"n={len(D)} seeds 11-55 x temp 0.3/0.7/0.9")
print(f" F0 {f0s.min():.1f} - {f0s.max():.1f} Hz spread {f0s.max()-f0s.min():.1f} Hz "
f"= {st(f0s.max(), REF_F0)-st(f0s.min(), REF_F0):+.1f} st sd {f0s.std():.1f} Hz")
print(f" envelope {envs.min()} - {envs.max()} Hz spread "
f"{st(envs.max(), REF_ENV)-st(envs.min(), REF_ENV):+.1f} st sd {envs.std():.0f} Hz")
F0_SD_ST = f0s.std() / REF_F0 * 12 * np.log2(np.e) # semitone sd
ENV_SD_ST = envs.std() / REF_ENV * 12 * np.log2(np.e)
print(f" => 1-sd noise floor: F0 {F0_SD_ST:.2f} st, envelope {ENV_SD_ST:.2f} st")
print(f" => 2-sd (where a variant becomes reliably 'not just a re-roll'): "
f"F0 {2*F0_SD_ST:.2f} st")
# ---------------------------------------------------------------- 2. separation
print("\n=== AXIS B: did the neural instruct move anything on a REAL HUMAN reference? ===")
print(f"{'variant':20} {'d_F0_st':>9} {'d_env_st':>10} {'WER':>6}")
for r in rows:
if r["label"].startswith("B_") or r["label"].startswith("A00"):
print(f"{r['label']:20} {r.get('d_f0_st'):>9} {r.get('d_env_st'):>10} {r.get('wer'):>6}")
# ---------------------------------------------------------------- 3. verify F0 artifact
print("\n=== VERIFYING C_monster_grit_full (F0 69.7 Hz but WER 0.04) ===")
def true_f0(y, sr):
"""Autocorrelation F0 over 60-400 Hz - independent of pyin's tracking."""
y = y.astype(np.float32)
f0, _, _ = librosa.pyin(y, fmin=60.0, fmax=400.0, sr=sr, frame_length=1024)
v = f0[~np.isnan(f0)]
ac = librosa.autocorrelate(y, max_size=int(sr / 60))
lo, hi = int(sr / 400), int(sr / 60)
peak = lo + int(np.argmax(ac[lo:hi]))
return float(np.median(v)), sr / peak
for label in ("A00_baseline", "C_monster_grit_ring", "C_monster_grit_full",
"C_monster_pitch-10"):
p = LAB / "wav" / f"{label}.wav"
if not p.exists():
continue
y, sr = sf.read(str(p), dtype="float32")
if y.ndim > 1:
y = y.mean(axis=1)
pyin_f0, ac_f0 = true_f0(y, sr)
# strong spectral peaks
S = np.abs(librosa.stft(y, n_fft=4096, hop_length=1024))
f = librosa.fft_frequencies(sr=sr, n_fft=4096)
m = (f > 40) & (f < 500)
amp = S[m].mean(axis=1)
peaks = f[m][np.argsort(amp)[-4:]][::-1]
print(f" {label:24} pyin={pyin_f0:6.1f} autocorr={ac_f0:6.1f} Hz "
f"low-freq peaks={[round(float(x)) for x in peaks]}")
# ---------------------------------------------------------------- 4. cast headroom
print("\n=== CAST HEADROOM on the pitch axis ===")
A = [r for r in rows if r["label"].startswith("A_pitch") and r.get("f0_median")]
sep = np.array([abs(st(r["f0_median"], REF_F0)) for r in A])
print(f" reliable separation needs > 2-sd = {2*F0_SD_ST:.1f} st")
print(f" usable DSP variants found (|dF0| > 2sd): "
f"{sorted(round(float(x),1) for x in sep if x > 2*F0_SD_ST)}")
usable = [r for r in A if abs(st(r["f0_median"], REF_F0)) > 2 * F0_SD_ST]
lo = min(st(r["f0_median"], REF_F0) for r in usable)
hi = max(st(r["f0_median"], REF_F0) for r in usable)
step = 2 * F0_SD_ST
print(f" span {lo:+.1f}..{hi:+.1f} st / {step:.1f} st per voice => "
f"~{int((hi-lo)//step)+1} reliably distinct voices on ONE axis")

242
backup/author_voice_lab/scripts/backup_author_matrix.py Normal file
View File

@@ -0,0 +1,242 @@
# Copyright 2026 Umbra Tools (umbratools.com)
# SPDX-License-Identifier: Apache-2.0
"""Durable backup of the Author voice matrix (vmatrix) + its reference chain.
The live matrix lives in a git-ignored project cache and the previous version's
only copy sits in an OS temp directory that macOS may purge. This writes a
self-contained, checksummed, timestamped backup in two places on two physical
disks, plus a manifest recording exactly which source WAV each matrix came from.
Read-only with respect to every original file.
"""
from __future__ import annotations
import hashlib
import json
import os
import platform
import shutil
import socket
from datetime import datetime, timezone
from pathlib import Path
PROJECT = Path("/Volumes/1TB_APL_Storage/UmbraAudio")
LIVE_MATRIX = PROJECT / "DealHunterShip_4.0e_whole-to-audio/.umbra/voice_prompts/author.pt"
V1_BACKUP = Path("/private/var/folders/y8/cbvhds297918lrxt2wd3btsr0000gn/T/opencode/author_v1_backup.pt")
DOCS = Path.home() / "Documents"
A4 = DOCS / "Audacity4"
STAMP = datetime.now(timezone.utc).strftime("%Y%m%dT%H%M%SZ")
# What each matrix was captured from, per BDMurphy_Voice_analysis.md round 1 / round 2.
PROVENANCE = {
"author_v2.pt": {
"generation": "v2 (current live matrix)",
"captured_from": "BDMurphy_Voice_NOBG_v2.wav",
"processing": "noisered 0.15 + highpass 40 Hz, resampled 44100->24000, gain -3 dB",
"source_recording": "BDMurphy_Voice.wav (raw, 34.33 s, 44100 Hz)",
"reason": "round 2 fix for the reported 'hollow / sounds like a building' quality",
},
"author_v1.pt": {
"generation": "v1 (superseded)",
"captured_from": "BDMurphy_Voice_NOBG.wav",
"processing": "noisered 0.4 + highpass 75 Hz, resampled 44100->24000, gain -3 dB",
"reason": "first capture; replaced by v2",
},
}
# Reference-chain audio, so the backup is self-contained even if Documents is lost.
CHAIN = [
A4 / "BDMurphy_Voice.wav",
A4 / "BDMurphy_Voice_NOBG.wav",
A4 / "BDMurphy_Voice_NOBG_v2.wav",
A4 / "BDMurphy_Voice_Author_Rainbow_v2.wav",
A4 / "BDMurphy_Voice_Author_Rainbow_v2_air.wav",
DOCS / "BDMurphy_Voice_Author_Rainbow_v2_air2.wav",
A4 / "BDMurphy_Voice_analysis.md",
]
DESTS = [
PROJECT / "backup" / "author_voice_matrix" / STAMP,
Path.home() / ".umbra" / "library" / "author_voice_matrix" / STAMP,
]
def sha256(p: Path) -> str:
h = hashlib.sha256()
with open(p, "rb") as f:
for chunk in iter(lambda: f.read(1 << 20), b""):
h.update(chunk)
return h.hexdigest()
def describe(p: Path) -> dict:
d = {"bytes": p.stat().st_size, "sha256": sha256(p)}
try:
import soundfile as sf
i = sf.info(str(p))
d |= {"duration_s": round(i.duration, 3), "sample_rate": i.samplerate,
"channels": i.channels, "subtype": i.subtype}
except Exception:
pass
return d
def main() -> None:
manifest: dict = {
"created_utc": datetime.now(timezone.utc).isoformat(),
"host": socket.gethostname(),
"platform": platform.platform(),
"purpose": ("Durable backup of the Author voice matrix (vmatrix) and its reference "
"chain. The vmatrix is the source of truth for the Author voice; the WAV "
"is reproducible from it."),
"matrix_format": {
"file": "voice_prompts/<slug>.pt",
"structure": "torch.save({'items': [{'ref_code','ref_spk_embedding',"
"'x_vector_only_mode','icl_mode','ref_text'}]})",
"ref_code": "(T,16) int64 codec tokens @12.5 Hz = re-encoded reference speech",
"ref_spk_embedding": "(2048,) bfloat16 speaker x-vector",
"full_vmatrix": "x_vector_only_mode=False, icl_mode=True",
},
"capture_endpoint": "POST http://127.0.0.1:8881/v1/voice-prompt "
"slug=author x_vector_only=0 force=1",
"matrices": {},
"reference_chain": [],
"notes": [
"author.pt is git-ignored project cache; this backup is the durable copy.",
"The superseded v1 matrix's only prior copy was in an OS temp dir (purgeable).",
"verdict: full vmatrix (ref_code + embedding), NOT x_vector_only.",
],
}
# --- matrices ---
matrix_srcs = [("author_v2.pt", LIVE_MATRIX), ("author_v1.pt", V1_BACKUP)]
for name, src in matrix_srcs:
if not src.exists():
manifest["matrices"][name] = {"status": "MISSING", "source": str(src)}
continue
info = describe(src)
info.update(PROVENANCE[name])
info["source"] = str(src)
try:
import torch
it = torch.load(str(src), weights_only=False)["items"][0]
rc = it["ref_code"]
info["ref_code_shape"] = list(rc.shape)
info["ref_code_seconds"] = round(rc.shape[0] / 12.5, 3)
info["icl_mode"] = bool(it["icl_mode"])
info["x_vector_only_mode"] = bool(it["x_vector_only_mode"])
info["ref_text_is_rainbow"] = str(it["ref_text"]).startswith(
"When the sunlight strikes raindrops")
except Exception as e: # pragma: no cover
info["inspect_error"] = str(e)
manifest["matrices"][name] = info
# --- reference chain ---
for src in CHAIN:
if src.exists():
e = describe(src)
e["source"] = str(src)
manifest["reference_chain"].append(e)
else:
manifest["reference_chain"].append({"source": str(src), "status": "MISSING"})
# --- write ---
written = []
for dest in DESTS:
dest.mkdir(parents=True, exist_ok=True)
for name, src in matrix_srcs:
if src.exists():
shutil.copy2(src, dest / name)
for src in CHAIN:
if src.exists():
shutil.copy2(src, dest / src.name)
(dest / "manifest.json").write_text(json.dumps(manifest, indent=2) + "\n")
(dest / "RESTORE.md").write_text(RESTORE_MD)
written.append(str(dest))
# Verify every copied byte against the manifest.
problems = []
for dest in DESTS:
for name, src in matrix_srcs:
if not src.exists():
continue
d = dest / name
if not d.exists():
problems.append(f"MISSING COPY {d}")
elif sha256(d) != manifest["matrices"][name]["sha256"]:
problems.append(f"HASH MISMATCH {d}")
manifest["written_to"] = written
manifest["verification"] = "ok" if not problems else problems
for dest in DESTS:
(dest / "manifest.json").write_text(json.dumps(manifest, indent=2) + "\n")
print(json.dumps({
"stamp": STAMP,
"written_to": written,
"verification": manifest["verification"],
"matrices": {k: {"sha256": v.get("sha256", "")[:16], "bytes": v.get("bytes"),
"from": v.get("captured_from"), "ref_code": v.get("ref_code_shape")}
for k, v in manifest["matrices"].items()},
"chain_files": len(manifest["reference_chain"]),
}, indent=2))
RESTORE_MD = """# Author voice matrix — restore
## What this is
The **Author** voice identity, stored as a Qwen3-TTS *vmatrix* (the numbers that
ARE the voice). Captured once; reuses those exact numbers for every utterance, so
the voice never drifts between takes.
## Files
| File | What it is |
|---|---|
| `author_v2.pt` | **The live matrix.** Use this one. |
| `author_v1.pt` | Superseded first capture, kept for rollback. |
| `BDMurphy_Voice*.wav` | The reference chain the matrix was captured from. |
| `manifest.json` | SHA-256, provenance, ref_code shape for every file. |
## Restore
```bash
# Matrix -> the server's prompt cache (PROMPT_DIR for the local server):
cp author_v2.pt <project>/.umbra/voice_prompts/author.pt
# Reference WAV -> a project's approved-reference dir:
cp BDMurphy_Voice_NOBG_v2.wav <project>/.umbra/voices/author_ref.wav
```
The capture call that produced this matrix (re-run ONLY if you are deliberately
changing the voice — it is not needed to use it):
```
POST http://127.0.0.1:8881/v1/voice-prompt
slug=author x_vector_only=0 force=1
ref_text=<Rainbow Passage> ref_audio=BDMurphy_Voice_NOBG_v2.wav
```
`x_vector_only=0` is the full vmatrix (`ref_code` + speaker embedding) and is the
only mode that reproduces this voice faithfully. `x_vector_only=1` captures just
the coarse embedding and is what historically caused voices to drift.
## Verify
```bash
shasum -a 256 author_v2.pt # must equal manifest.json -> matrices.author_v2.sha256
```
To confirm a matrix still renders the right speaker, generate from
`prompt_slug=author` and compare the median F0 against the reference
(reference is 129.4 Hz; a correct render lands ~130-133 Hz).
"""
if __name__ == "__main__":
main()

77
backup/author_voice_lab/scripts/fidelity.py Normal file
View File

@@ -0,0 +1,77 @@
# Copyright 2026 Umbra Tools (umbratools.com)
# SPDX-License-Identifier: Apache-2.0
"""Phase-insensitive fidelity: how faithfully does the matrix reproduce the reference?
Frame-wise waveform correlation and LSD are dominated by phase, which a 12 Hz
neural codec does not preserve. Identity lives in the long-term average spectrum
(tonal balance), the pitch contour, and the mel envelope — all phase-insensitive.
"""
from __future__ import annotations
from pathlib import Path
import librosa
import numpy as np
import soundfile as sf
OUT = Path("/private/var/folders/y8/cbvhds297918lrxt2wd3btsr0000gn/T/opencode/probe_out")
A4 = Path.home() / "Documents/Audacity4"
dec, sr = sf.read(str(OUT / "author_ref_DECODED_FROM_MATRIX.wav"), dtype="float32")
src, _ = sf.read(str(A4 / "BDMurphy_Voice_NOBG_v2.wav"), dtype="float32")
if src.ndim > 1:
src = src.mean(axis=1)
BANDS = [(80, 160), (160, 315), (315, 630), (630, 1250), (1250, 2500),
(2500, 5000), (5000, 10000), (10000, 12000)]
def ltas(y):
S = np.abs(librosa.stft(y.astype(np.float32), n_fft=2048, hop_length=512))
f = librosa.fft_frequencies(sr=sr, n_fft=2048)
out = []
for lo, hi in BANDS:
m = (f >= lo) & (f < hi)
out.append(10 * np.log10((S[m] ** 2).sum() + 1e-12))
return np.array(out)
def mel_cep(y):
m = librosa.feature.melspectrogram(y=y.astype(np.float32), sr=sr, n_mels=40,
n_fft=1024, hop_length=256)
return np.log10(m + 1e-10)
def f0_track(y):
f0, _, _ = librosa.pyin(y.astype(np.float32), fmin=60.0, fmax=400.0, sr=sr)
return f0
print(f"{'':28} {'decoded':>9} {'reference':>10} {'delta':>8}")
print("--- long-term average spectrum, per band (dB, energy-normalised) ---")
a, b = ltas(dec), ltas(src)
a_n = a - a.mean()
b_n = b - b.mean()
for (lo, hi), x, y in zip(BANDS, a_n, b_n):
print(f"{lo:>6}-{hi:<6} Hz{'':10} {x:9.2f} {y:10.2f} {x - y:+8.2f}")
print(f"{'RMS deviation':28} {'':9} {'':10} {float(np.sqrt((a_n - b_n) ** 2).mean()):+8.2f} dB")
print("\n--- mel envelope (40 bands, log) ---")
ma, mb = mel_cep(dec), mel_cep(src)
n = min(ma.shape[1], mb.shape[1])
ma_n = ma[:, :n] - ma[:, :n].mean(axis=0, keepdims=True)
mb_n = mb[:, :n] - mb[:, :n].mean(axis=0, keepdims=True)
print(f"mean |log-mel diff| = {np.abs(ma_n - mb_n).mean():.4f} (log10 units)")
print("\n--- pitch contour ---")
fa, fb = f0_track(dec), f0_track(src)
va, vb = fa[~np.isnan(fa)], fb[~np.isnan(fb)]
print(f"median F0: decoded {np.median(va):.1f} Hz reference {np.median(vb):.1f} Hz")
print(f"voiced frames: decoded {len(va)} reference {len(vb)}")
# The real fidelity test: re-encode the DECODED audio. If the codec round-trip is
# stable, decoding twice converges, i.e. matrix -> wav -> codes -> same codes.
import sys
sys.path.insert(0, "/Volumes/1TB_APL_Storage/UmbraAudio")
print("\n(note: full re-encode check runs in matrix_roundtrip.py)")

50
backup/author_voice_lab/scripts/install_ref.py Normal file
View File

@@ -0,0 +1,50 @@
# Copyright 2026 Umbra Tools (umbratools.com)
# SPDX-License-Identifier: Apache-2.0
"""Install the matrix-derived Author reference into the project.
The matrix is the source of truth; this writes the WAV *decoded from that matrix*
so the project's reference and the matrix provably agree (LTAS 0.55 dB RMS,
median F0 within 0.8 Hz of the original capture).
"""
from __future__ import annotations
import hashlib
import json
import shutil
from datetime import datetime, timezone
from pathlib import Path
PROJECT = Path("/Volumes/1TB_APL_Storage/UmbraAudio")
DECODED = Path("/private/var/folders/y8/cbvhds297918lrxt2wd3btsr0000gn/T/opencode/probe_out/"
"author_ref_DECODED_FROM_MATRIX.wav")
VOICES = PROJECT / "DealHunterShip_4.0e_whole-to-audio/.umbra/voices"
MATRIX = PROJECT / "DealHunterShip_4.0e_whole-to-audio/.umbra/voice_prompts/author.pt"
def sha256(p: Path) -> str:
h = hashlib.sha256()
h.update(Path(p).read_bytes())
return h.hexdigest()
dest = VOICES / "author_ref.wav"
if dest.exists():
backup = VOICES / f"author_ref.PRE_MATRIX_DECODE.{datetime.now():%Y%m%d_%H%M%S}.wav"
shutil.copy2(dest, backup)
print(f"existing author_ref.wav -> {backup.name}")
shutil.copy2(DECODED, dest)
info = {
"installed": str(dest),
"decoded_from_matrix": str(MATRIX),
"matrix_sha256": sha256(MATRIX),
"ref_sha256": sha256(dest),
"provenance": "decoded from author.pt ref_code via Qwen3TTSTokenizer.decode()",
"note": ("Regenerate with the matrix only: "
"Qwen3TTSTokenizer.from_pretrained(models/Qwen3-TTS-12Hz-1.7B-Base/"
"speech_tokenizer).decode({'audio_codes': ref_code})"),
}
side = dest.with_suffix(".json")
side.write_text(json.dumps(info, indent=2) + "\n")
print(json.dumps(info, indent=2))

132
backup/author_voice_lab/scripts/matrix_roundtrip.py Normal file
View File

@@ -0,0 +1,132 @@
# Copyright 2026 Umbra Tools (umbratools.com)
# SPDX-License-Identifier: Apache-2.0
"""Prove: the vmatrix alone can reproduce the reference WAV — and verify its provenance.
1. DECODE author.pt ref_code -> waveform (matrix -> audio; the user's requirement)
2. VERIFY that decoded audio matches the WAV the matrix was captured from
3. PROVE provenance by re-encoding each candidate source WAV and comparing its
audio_codes to the stored ref_code. Identical codes => that file is
provably where this matrix came from (no trusting the notes).
Runs on CPU with only the 682 MB speech tokenizer, so the TTS server keeps its
MPS model resident and nothing else contends for memory.
"""
from __future__ import annotations
import json
import sys
from pathlib import Path
import librosa
import numpy as np
import soundfile as sf
import torch
sys.path.insert(0, "/Volumes/1TB_APL_Storage/UmbraAudio")
from qwen_tts import Qwen3TTSTokenizer # noqa: E402
PROJECT = Path("/Volumes/1TB_APL_Storage/UmbraAudio")
MATRIX = PROJECT / "DealHunterShip_4.0e_whole-to-audio/.umbra/voice_prompts/author.pt"
TOKENIZER = PROJECT / "models/Qwen3-TTS-12Hz-1.7B-Base/speech_tokenizer"
A4 = Path.home() / "Documents/Audacity4"
OUT = Path("/private/var/folders/y8/cbvhds297918lrxt2wd3btsr0000gn/T/opencode/probe_out")
OUT.mkdir(parents=True, exist_ok=True)
CANDIDATES = {
"NOBG_v2 (documented v2 source)": A4 / "BDMurphy_Voice_NOBG_v2.wav",
"NOBG (documented v1 source)": A4 / "BDMurphy_Voice_NOBG.wav",
"raw recording": A4 / "BDMurphy_Voice.wav",
}
def f0_median(y, sr):
f0, _, _ = librosa.pyin(y.astype(np.float32), fmin=60.0, fmax=400.0, sr=sr)
v = f0[~np.isnan(f0)]
return float(np.median(v)) if len(v) else float("nan")
def lsd(a, b, sr):
"""Log-spectral distance over the speech band (dB)."""
n = min(len(a), len(b))
A = np.abs(librosa.stft(a[:n].astype(np.float32), n_fft=1024, hop_length=256))
B = np.abs(librosa.stft(b[:n].astype(np.float32), n_fft=1024, hop_length=256))
m = min(A.shape[1], B.shape[1])
A, B = A[:, :m], B[:, :m]
f = librosa.fft_frequencies(sr=sr, n_fft=1024)
band = (f >= 80) & (f <= 8000)
la = 20 * np.log10(A[band] + 1e-8)
lb = 20 * np.log10(B[band] + 1e-8)
return float(np.sqrt(np.mean((la - lb) ** 2)))
def main() -> None:
item = torch.load(str(MATRIX), weights_only=False)["items"][0]
ref_code = item["ref_code"].cpu()
print(f"matrix ref_code: {tuple(ref_code.shape)} (~{ref_code.shape[0] / 12.5:.2f}s)")
print(f"icl_mode={item['icl_mode']} x_vector_only_mode={item['x_vector_only_mode']}")
tok = Qwen3TTSTokenizer.from_pretrained(str(TOKENIZER), device_map="cpu")
print(f"tokenizer loaded, output sr = {tok.feature_extractor.sampling_rate}")
# ---------- 1. DECODE: matrix -> wav ----------
wavs, sr = tok.decode({"audio_codes": ref_code})
decoded = wavs[0].astype(np.float32)
dec_path = OUT / "author_ref_DECODED_FROM_MATRIX.wav"
sf.write(str(dec_path), decoded, sr, subtype="PCM_16")
print(f"\nDECODED from matrix: {len(decoded)/sr:.2f}s @ {sr} Hz -> {dec_path.name}")
print(f" median F0 = {f0_median(decoded, sr):.1f} Hz")
# ---------- 2. VERIFY against each candidate ----------
print("\n--- decoded vs candidate source WAVs ---")
print(f"{'candidate':36} {'dur':>7} {'F0':>7} {'LSD_dB':>8} {'corr':>7}")
results = {}
for label, p in CANDIDATES.items():
if not p.exists():
print(f"{label:36} MISSING")
continue
y, ysr = sf.read(str(p), dtype="float32")
if y.ndim > 1:
y = y.mean(axis=1)
if ysr != sr:
y = librosa.resample(y, orig_sr=ysr, target_sr=sr)
n = min(len(y), len(decoded))
corr = float(np.corrcoef(y[:n], decoded[:n])[0, 1])
d = lsd(y, decoded, sr)
results[label] = {"dur": round(len(y) / sr, 2), "f0": round(f0_median(y, sr), 1),
"lsd_db": round(d, 2), "corr": round(corr, 4)}
print(f"{label:36} {len(y)/sr:7.2f} {f0_median(y, sr):7.1f} {d:8.2f} {corr:7.4f}")
# ---------- 3. PROVE provenance: re-encode and compare codes ----------
print("\n--- provenance: re-encode candidate, compare audio_codes to stored ref_code ---")
print(f"{'candidate':36} {'shape':>12} {'exact_match':>13} {'frame_agree':>13}")
prov = {}
for label, p in CANDIDATES.items():
if not p.exists():
continue
enc = tok.encode([str(p)])
codes = enc.audio_codes[0].cpu()
same_shape = tuple(codes.shape) == tuple(ref_code.shape)
if same_shape:
exact = bool(torch.equal(codes, ref_code))
agree = float((codes == ref_code).float().mean())
else:
exact, agree = False, float("nan")
prov[label] = {"shape": list(codes.shape), "exact_match": exact,
"frame_agreement": round(agree, 6)}
print(f"{label:36} {str(tuple(codes.shape)):>12} {str(exact):>13} {agree:>13.6f}")
out = {
"matrix": str(MATRIX),
"ref_code_shape": list(ref_code.shape),
"decoded": {"path": str(dec_path), "duration_s": round(len(decoded) / sr, 3),
"sample_rate": sr, "median_f0_hz": round(f0_median(decoded, sr), 1)},
"verified_against": results,
"provenance": prov,
}
(OUT / "matrix_roundtrip.json").write_text(json.dumps(out, indent=2) + "\n")
print("\nwrote", OUT / "matrix_roundtrip.json")
if __name__ == "__main__":
main()

59
backup/author_voice_lab/scripts/probe_dsp.py Normal file
View File

@@ -0,0 +1,59 @@
# Copyright 2026 Umbra Tools (umbratools.com)
# SPDX-License-Identifier: Apache-2.0
"""Probe 2: DSP axis limits. What does pitch_shift_pv actually move?"""
from __future__ import annotations
import sys
from pathlib import Path
sys.path.insert(0, "/Volumes/1TB_APL_Storage/UmbraAudio")
import librosa
import numpy as np
import soundfile as sf
from umbraaudio.generation.audio_effects import EFFECT_FN
# NOTE: apply_effects_by_name() only accepts effect NAMES and uses hardcoded
# defaults, so it cannot dial a parameter. A sweep needs EFFECT_FN directly.
_pv = EFFECT_FN["pitch_shift_pv"]
SRC = Path("/private/var/folders/y8/cbvhds297918lrxt2wd3btsr0000gn/T/opencode/probe_out/A_baseline.wav")
def measure(y, sr):
f0, _, _ = librosa.pyin(y.astype(np.float32), fmin=60.0, fmax=400.0, sr=sr)
v = f0[~np.isnan(f0)]
S = np.abs(librosa.stft(y.astype(np.float32), n_fft=2048, hop_length=512))
cen = float(librosa.feature.spectral_centroid(S=S, sr=sr).mean())
# Formant-ish proxies: energy-weighted centroid in low band vs overall
return float(np.median(v)), cen
def formant_band_ratio(y, sr):
"""Crude vocal-tract length proxy: spectral centroid of the voiced spectrum."""
S = np.abs(librosa.stft(y.astype(np.float32), n_fft=2048, hop_length=512))
freqs = librosa.fft_frequencies(sr=sr, n_fft=2048)
m = freqs <= 4000
return float((S[m] * freqs[m][:, None]).sum() / S[m].sum())
y, sr = sf.read(str(SRC), dtype="float32")
base_f0, base_cen = measure(y, sr)
base_ftr = formant_band_ratio(y, sr)
print(f"BASE F0med={base_f0:7.1f}Hz centroid={base_cen:7.0f}Hz LTAS4k={base_ftr:7.0f}Hz")
print("\n--- pitch_shift_pv sweep (semitones) ---")
print(f"{'semi':>6} {'F0med':>8} {'d_F0_st':>8} {'centroid':>9} {'d_cen_st':>8} {'LTAS4k':>8} {'d_ftr_st':>9}")
for st in (-12, -8, -4, 0, 4, 8, 12):
out = _pv(y, sr, semitones=float(st))
f0, cen = measure(out, sr)
ftr = formant_band_ratio(out, sr)
d_f0 = 12 * np.log2(f0 / base_f0)
d_cen = 12 * np.log2(cen / base_cen)
d_ftr = 12 * np.log2(ftr / base_ftr)
print(f"{st:>6} {f0:>8.1f} {d_f0:>8.1f} {cen:>9.0f} {d_cen:>8.1f} {ftr:>8.0f} {d_ftr:>9.1f}")
print("\nInterpretation: d_cen ~= d_f0 => full-spectrum shift => formants move WITH pitch.")
print("A true male->female conversion needs d_cen/d_F0 ~= +2.5 to +3 st, d_F0 ~= +9 to +12 st.")

72
backup/author_voice_lab/scripts/probe_formant.py Normal file
View File

@@ -0,0 +1,72 @@
# Copyright 2026 Umbra Tools (umbratools.com)
# SPDX-License-Identifier: Apache-2.0
"""Probe 3: prove the missing primitive is an INDEPENDENT FORMANT SHIFTER.
pitch_shift_pv is a phase vocoder -> it moves F0 and the spectral envelope by
DIFFERENT amounts (measured: formants lag pitch going up, overshoot going down).
A vocal-tract (formant) shift is the missing primitive and is achievable with
resample + time_stretch, which decouples the two.
"""
from __future__ import annotations
import sys
from pathlib import Path
sys.path.insert(0, "/Volumes/1TB_APL_Storage/UmbraAudio")
import librosa
import numpy as np
import soundfile as sf
from umbraaudio.generation.audio_effects import EFFECT_FN
SRC = Path("/private/var/folders/y8/cbvhds297918lrxt2wd3btsr0000gn/T/opencode/probe_out/A_baseline.wav")
OUT = Path("/private/var/folders/y8/cbvhds297918lrxt2wd3btsr0000gn/T/opencode/probe_out")
_pv = EFFECT_FN["pitch_shift_pv"]
def m(y, sr):
y = y.astype(np.float32)
f0, _, _ = librosa.pyin(y, fmin=60.0, fmax=400.0, sr=sr)
v = f0[~np.isnan(f0)]
S = np.abs(librosa.stft(y, n_fft=2048, hop_length=512))
freqs = librosa.fft_frequencies(sr=sr, n_fft=2048)
msk = freqs <= 4000
ftr = float((S[msk] * freqs[msk][:, None]).sum() / S[msk].sum())
return float(np.median(v)), ftr
def formant_shift(y, sr, semitones: float) -> np.ndarray:
"""Shift ONLY the vocal-tract envelope, leaving F0 and duration untouched."""
r = 2.0 ** (semitones / 12.0)
up = librosa.resample(y.astype(np.float32), orig_sr=sr, target_sr=int(round(sr * r)))
back = librosa.resample(up, orig_sr=int(round(sr * r)), target_sr=sr)
return librosa.effects.time_stretch(back.astype(np.float32), rate=r).astype(np.float32)
y, sr = sf.read(str(SRC), dtype="float32")
b_f0, b_ftr = m(y, sr)
print(f"BASE F0={b_f0:6.1f}Hz LTAS4k={b_ftr:6.0f}Hz\n")
print("Formant shift ALONE (F0 must stay put, LTAS4k must move):")
print(f"{'f_st':>6} {'F0':>8} {'dF0_st':>8} {'LTAS4k':>8} {'dftr_st':>8}")
for fs in (-6, -3, 3, 6):
o = formant_shift(y, sr, fs)
f0, ftr = m(o, sr)
print(f"{fs:>6} {f0:>8.1f} {12*np.log2(f0/b_f0):>8.1f} {ftr:>8.0f} {12*np.log2(ftr/b_ftr):>8.1f}")
sf.write(OUT / f"formant_{fs:+d}.wav", o, sr)
print("\nCOMBINED male->female recipe: pitch +10 st THEN formant +3 st")
o = formant_shift(_pv(y, sr, semitones=10.0), sr, 3.0)
f0, ftr = m(o, sr)
print(f" F0={f0:.1f}Hz ({12*np.log2(f0/b_f0):+.1f} st) LTAS4k={ftr:.0f}Hz "
f"({12*np.log2(ftr/b_ftr):+.1f} st)")
sf.write(OUT / "female_recipe.wav", o, sr)
print("\nCOMBINED male->dragon recipe: pitch -10 st THEN formant -8 st (vocal-tract growth)")
o = formant_shift(_pv(y, sr, semitones=-10.0), sr, -8.0)
f0, ftr = m(o, sr)
print(f" F0={f0:.1f}Hz ({12*np.log2(f0/b_f0):+.1f} st) LTAS4k={ftr:.0f}Hz "
f"({12*np.log2(ftr/b_ftr):+.1f} st)")
sf.write(OUT / "dragon_recipe.wav", o, sr)

94
backup/author_voice_lab/scripts/probe_morph.py Normal file
View File

@@ -0,0 +1,94 @@
# Copyright 2026 Umbra Tools (umbratools.com)
# SPDX-License-Identifier: Apache-2.0
"""Probe: can the narrator vmatrix be morphed (gender / creature) via instruct?
Read-only against the project; writes WAVs + measurements to an off-volume temp dir.
Uses the cached narrator vmatrix (prompt_slug) so voice identity is pinned.
"""
from __future__ import annotations
import io
import json
import time
from pathlib import Path
import librosa
import numpy as np
import soundfile as sf
import httpx
URL = "http://127.0.0.1:8881"
OUT = Path("/private/var/folders/y8/cbvhds297918lrxt2wd3btsr0000gn/T/opencode/probe_out")
OUT.mkdir(parents=True, exist_ok=True)
TEXT = (
"The lantern guttered as she pushed the door open, and the cold air of the "
"cavern rolled over her like the breath of something waiting."
)
def measure(wav: np.ndarray, sr: int) -> dict:
y = wav.astype(np.float32)
f0, _, _ = librosa.pyin(y, fmin=60.0, fmax=400.0, sr=sr)
v = f0[~np.isnan(f0)]
S = np.abs(librosa.stft(y, n_fft=2048, hop_length=512))
cen = float(librosa.feature.spectral_centroid(S=S, sr=sr).mean())
flat = float(librosa.feature.spectral_flatness(S=S).mean())
# long-term average spectrum slope: brightness/tilth
rms = float(20 * np.log10(np.sqrt((y**2).mean()) + 1e-12))
if len(v):
out = {
"f0_median": round(float(np.median(v)), 1),
"f0_sd": round(float(v.std()), 1),
"f0_range": [round(float(v.min())), round(float(v.max()))],
"semitones_vs_132hz": round(12 * np.log2(np.median(v) / 132.4), 1),
}
else:
out = {"f0_median": None, "f0_sd": None, "f0_range": None, "semitones_vs_132hz": None}
out.update(
centroid_hz=round(cen),
flatness=round(flat, 5),
rms_db=round(rms, 1),
dur_s=round(len(y) / sr, 2),
)
return out
REF = Path("/Volumes/1TB_APL_Storage/UmbraAudio/.umbra/voices/narrator_ref.wav")
def gen(label: str, *, instruct: str | None, seed: int = 1234) -> dict:
data = {"text": TEXT, "sample_rate": 24000, "temperature": 0.7, "do_sample": True,
"seed": seed}
files = {"ref_audio": ("ref.wav", REF.read_bytes(), "audio/wav")}
if instruct:
data["voice_description"] = instruct
t0 = time.time()
r = httpx.post(f"{URL}/v1/tts", data=data, files=files, timeout=900)
dt = time.time() - t0
if r.status_code != 200:
return {"label": label, "error": f"HTTP {r.status_code}: {r.text[:200]}"}
wav, sr = sf.read(io.BytesIO(r.content), dtype="float32")
if wav.ndim > 1:
wav = wav.mean(axis=1)
sf.write(OUT / f"{label}.wav", wav, sr)
m = measure(wav, sr)
m.update(label=label, seconds=round(dt, 1), instruct=instruct)
print(json.dumps(m), flush=True)
return m
CASES = [
("A_baseline", None),
("B_female", "A young woman's voice, bright and light, with a higher and clearer tone."),
("C_elderly_m", "An elderly man's voice, very deep, gravelly and slow."),
("D_dragon", "A huge ancient dragon's voice: a deep, rumbling, growling monster with "
"a metallic edge and a resonant, echoing chest tone."),
]
if __name__ == "__main__":
results = [gen(label, instruct=instr) for label, instr in CASES]
(OUT / "probe_results.json").write_text(json.dumps(results, indent=2))
print("WROTE", OUT / "probe_results.json")

89
backup/author_voice_lab/scripts/verify_cliff.py Normal file
View File

@@ -0,0 +1,89 @@
# Copyright 2026 Umbra Tools (umbratools.com)
# SPDX-License-Identifier: Apache-2.0
"""Verify the downward-pitch intelligibility cliff on a LONGER text, 2 seeds.
The 25-word probe suggested WER collapses below ~0 st. WER on 25 words is coarse
(4% per word) and the earlier narrator probe was much gentler, so this re-tests
with ~3x the text and repeats each point.
"""
from __future__ import annotations
import io
import json
import re
import sys
import time
from pathlib import Path
import librosa
import numpy as np
import soundfile as sf
sys.path.insert(0, "/Volumes/1TB_APL_Storage/UmbraAudio")
from umbraaudio.generation.audio_effects import EFFECT_FN # noqa: E402
import httpx # noqa: E402
import whisper # noqa: E402
PROJECT = Path("/Volumes/1TB_APL_Storage/UmbraAudio")
REF = PROJECT / "DealHunterShip_4.0e_whole-to-audio/.umbra/voices/author_ref.wav"
OUT = Path("/private/var/folders/y8/cbvhds297918lrxt2wd3btsr0000gn/T/opencode/author_lab")
(OUT / "wav_cliff").mkdir(exist_ok=True)
LONG = (
"He set the lantern on the stone ledge and listened. The cavern breathed around "
"them, slow and patient, and somewhere far below the water was moving against rock. "
"MaryAnn kept one hand on the rail and waited for him to speak first, because she "
"had learned that what he said next would decide which of them went forward."
)
REF_WORDS = re.sub(r"[^a-z ]", "", LONG.lower()).split()
print(f"long text: {len(REF_WORDS)} words")
_pv = EFFECT_FN["pitch_shift_pv"]
wm = whisper.load_model("base", device="cpu")
def run(st, seed):
r = httpx.post("http://127.0.0.1:8881/v1/tts",
data={"text": LONG, "sample_rate": 24000, "temperature": 0.7,
"do_sample": True, "seed": seed},
files={"ref_audio": ("ref.wav", REF.read_bytes(), "audio/wav")},
timeout=900)
y, sr = sf.read(io.BytesIO(r.content), dtype="float32")
if y.ndim > 1:
y = y.mean(axis=1)
if st:
y = _pv(y, sr, semitones=float(st))
p = float(np.max(np.abs(y)))
if p > 1.0:
y = y / p
import difflib
hyp = re.sub(r"[^a-z ]", "",
wm.transcribe(y, language="en", fp16=False, temperature=0.0)["text"]
).lower().split()
n = sum(b.size for b in difflib.SequenceMatcher(None, REF_WORDS, hyp).get_matching_blocks())
f0, _, _ = librosa.pyin(y.astype(np.float32), fmin=60.0, fmax=400.0, sr=sr)
v = f0[~np.isnan(f0)]
sf.write(OUT / "wav_cliff" / f"cliff{st:+03d}_s{seed}.wav", y, sr, subtype="PCM_16")
return {"shift_st": st, "seed": seed, "wer": round((len(REF_WORDS) - n) / len(REF_WORDS), 3),
"f0": round(float(np.median(v)), 1), "hyp": " ".join(hyp)}
rows = []
for st in (0, -2, -4, -6, +2):
for seed in (7, 8):
t0 = time.time()
m = run(st, seed)
m["sec"] = round(time.time() - t0, 1)
rows.append(m)
print(f" shift {st:+3d} st seed {seed} WER {m['wer']:.3f} F0 {m['f0']:6.1f} Hz "
f"{m['sec']}s", flush=True)
(OUT / "cliff_verification.json").write_text(json.dumps(
{"text_words": len(REF_WORDS), "rows": rows}, indent=2) + "\n")
print("\n--- mean WER by shift ---")
for st in (0, -2, -4, -6, +2):
ws = [r["wer"] for r in rows if r["shift_st"] == st]
print(f" {st:+3d} st: mean WER {sum(ws)/len(ws):.3f} {ws}")

223
backup/author_voice_lab/scripts/voice_lab.py Normal file
View File

@@ -0,0 +1,223 @@
# Copyright 2026 Umbra Tools (umbratools.com)
# SPDX-License-Identifier: Apache-2.0
"""voice_lab — measure what the Author voice matrix can become.
One fixed text for every sample (comparisons are only valid at constant text).
Every sample records: F0 median/sd/range, spectral-envelope centroid and its
semitone offset (the "vocal tract" proxy), brightness, flatness, level, whisper
WER, and elapsed time.
Axes:
A pitch pitch_shift_pv, -12..+12 st -> where is the intelligibility cliff
B tract neural instruct, 8 specs -> how far will identity move
C hybrid instruct + DSP trim -> close the gap instruct leaves
D variance seeds x temperatures -> intra-voice spread (cast-size limit)
Outputs WAVs + results.json + results.md under OUT.
Nothing in the project tree is read except the reference; nothing is written to it.
"""
from __future__ import annotations
import io
import json
import re
import sys
import time
from pathlib import Path
import librosa
import numpy as np
import soundfile as sf
sys.path.insert(0, "/Volumes/1TB_APL_Storage/UmbraAudio")
from umbraaudio.generation.audio_effects import EFFECT_FN # noqa: E402
import httpx # noqa: E402
PROJECT = Path("/Volumes/1TB_APL_Storage/UmbraAudio")
REF = PROJECT / "DealHunterShip_4.0e_whole-to-audio/.umbra/voices/author_ref.wav"
OUT = Path("/private/var/folders/y8/cbvhds297918lrxt2wd3btsr0000gn/T/opencode/author_lab")
URL = "http://127.0.0.1:8881"
OUT.mkdir(parents=True, exist_ok=True)
(OUT / "wav").mkdir(exist_ok=True)
# ONE text for every sample.
TEXT = ("The lantern guttered as she pushed the door open, and the cold air of the "
"cavern rolled over her like the breath of something waiting.")
REF_F0 = None # set from the reference wav at startup
_pv = EFFECT_FN["pitch_shift_pv"]
_sat = EFFECT_FN["tube_sat"]
_ring = EFFECT_FN["ring_modulate"]
_chorus = EFFECT_FN["chorus"]
_jitter = EFFECT_FN["pitch_jitter"]
# ---------------------------------------------------------------- measurement
def measure(y: np.ndarray, sr: int) -> dict:
y = y.astype(np.float32)
f0, _, _ = librosa.pyin(y, fmin=60.0, fmax=400.0, sr=sr)
v = f0[~np.isnan(f0)]
S = np.abs(librosa.stft(y, n_fft=2048, hop_length=512))
f = librosa.fft_frequencies(sr=sr, n_fft=2048)
band = f <= 4000
ftr = float((S[band] * f[band][:, None]).sum() / S[band].sum())
m = {
"f0_median": round(float(np.median(v)), 1) if len(v) else None,
"f0_sd": round(float(v.std()), 1) if len(v) else None,
"envelope_hz": round(ftr),
"centroid_hz": round(float(librosa.feature.spectral_centroid(S=S, sr=sr).mean())),
"flatness": round(float(librosa.feature.spectral_flatness(S=S).mean()), 5),
"dur_s": round(len(y) / sr, 2),
"rms_db": round(float(20 * np.log10(np.sqrt((y**2).mean()) + 1e-12)), 1),
"peak_db": round(float(20 * np.log10(np.max(np.abs(y)) + 1e-12)), 1),
}
if m["f0_median"] and REF_F0:
m["d_f0_st"] = round(12 * np.log2(m["f0_median"] / REF_F0), 1)
return m
def wer(model, ref_words: list[str], y: np.ndarray) -> float | None:
import difflib
try:
txt = model.transcribe(y.astype(np.float32), language="en", fp16=False,
temperature=0.0)["text"]
except Exception:
return None
hyp = re.sub(r"[^a-z ]", "", txt.lower()).split()
n = sum(b.size for b in difflib.SequenceMatcher(None, ref_words, hyp).get_matching_blocks())
return round((len(ref_words) - n) / max(1, len(ref_words)), 3)
# ---------------------------------------------------------------- generation
def synth(label: str, *, instruct: str | None = None, temperature: float = 0.7,
seed: int = 1234, dsp: list | None = None, want_wer: bool = True,
whisper_model=None) -> dict:
data = {"text": TEXT, "sample_rate": 24000, "temperature": temperature,
"do_sample": True, "seed": seed}
if instruct:
data["voice_description"] = instruct
t0 = time.time()
r = httpx.post(f"{URL}/v1/tts", data=data,
files={"ref_audio": ("ref.wav", REF.read_bytes(), "audio/wav")},
timeout=900)
if r.status_code != 200:
return {"label": label, "error": f"HTTP {r.status_code}: {r.text[:160]}"}
y, sr = sf.read(io.BytesIO(r.content), dtype="float32")
if y.ndim > 1:
y = y.mean(axis=1)
gen_s = round(time.time() - t0, 1)
chain_desc = None
if dsp:
for name, params in dsp:
y = EFFECT_FN[name](y, sr, **params)
p = float(np.max(np.abs(y)))
if p > 1.0:
y = y / p
chain_desc = [n for n, _ in dsp]
m = measure(y, sr)
m.update(label=label, gen_s=gen_s, instruct=instruct, temperature=temperature,
seed=seed, dsp=chain_desc,
d_env_st=(round(12 * np.log2(m["envelope_hz"] / REF_ENV), 1)
if REF_ENV else None))
if want_wer and whisper_model is not None:
m["wer"] = wer(whisper_model, REF_WORDS, y)
sf.write(OUT / "wav" / f"{label}.wav", y, sr, subtype="PCM_16")
return m
# ---------------------------------------------------------------- axes
TRACTS = {
"B_female_young": "A young woman's voice, bright and light, with a higher and clearer tone.",
"B_female_mature": "A mature woman's voice, low and calm for a woman, unhurried and steady.",
"B_boy": "A young boy's voice, light and eager, not yet broken.",
"B_elderly_m": "An elderly man's voice, very deep, gravelly and slow.",
"B_young_m": "A young man's voice, light, energetic and clear.",
"B_large_m": "A very large man's voice, heavy and booming, with a deep chest.",
"B_small_w": "A small woman's voice, petite and light, with a narrow bright tone.",
"B_androgynous": "A neutral, androgynous voice, neither distinctly male nor female.",
}
GRIT = {
"grit_sat": [("tube_sat", {"drive_db": 12.0, "mix": 1.0})],
"grit_ring": [("ring_modulate", {"freq": 38.0})],
"grit_chorus": [("chorus", {"rate_hz": 1.6, "depth": 0.3, "centre_delay_ms": 8.0,
"feedback": 0.2, "mix": 0.4})],
"grit_full": [("tube_sat", {"drive_db": 12.0, "mix": 1.0}),
("ring_modulate", {"freq": 38.0}),
("chorus", {"rate_hz": 1.6, "depth": 0.3, "centre_delay_ms": 8.0,
"feedback": 0.2, "mix": 0.4}),
("pitch_jitter", {"jitter_hz": 6.0, "depth": 0.03})],
}
def main() -> None:
global REF_F0, REF_ENV, REF_WORDS
import whisper
ref_y, ref_sr = sf.read(str(REF), dtype="float32")
if ref_y.ndim > 1:
ref_y = ref_y.mean(axis=1)
S = np.abs(librosa.stft(ref_y.astype(np.float32), n_fft=2048, hop_length=512))
f = librosa.fft_frequencies(sr=ref_sr, n_fft=2048)
band = f <= 4000
REF_ENV = float((S[band] * f[band][:, None]).sum() / S[band].sum())
f0r, _, _ = librosa.pyin(ref_y.astype(np.float32), fmin=60.0, fmax=400.0, sr=ref_sr)
REF_F0 = float(np.median(f0r[~np.isnan(f0r)]))
REF_WORDS = re.sub(r"[^a-z ]", "", TEXT.lower()).split()
print(f"REFERENCE (author_ref.wav): F0 {REF_F0:.1f} Hz envelope {REF_ENV:.0f} Hz "
f"{len(ref_y)/ref_sr:.1f}s")
wm = whisper.load_model("base", device="cpu")
rows: list[dict] = []
def run(label, **kw):
m = synth(label, whisper_model=wm, **kw)
rows.append(m)
print(f" {label:26} F0={m.get('f0_median')} "
f"({m.get('d_f0_st')}st) env={m.get('d_env_st')}st "
f"WER={m.get('wer')} {m.get('gen_s')}s", flush=True)
return m
print("\n[baseline]")
run("A00_baseline", temperature=0.7, seed=1234)
print("\n[A] pitch axis -12..+12 step 2")
for st in range(-12, 13, 2):
run(f"A_pitch{st:+03d}", dsp=[("pitch_shift_pv", {"semitones": float(st)})])
print("\n[B] neural tract morphs")
for label, instr in TRACTS.items():
run(label, instruct=instr)
print("\n[C] hybrids")
for trim in (4, 8):
run(f"C_female+{trim}st", instruct=TRACTS["B_female_young"],
dsp=[("pitch_shift_pv", {"semitones": float(trim)})])
run("C_monster_pitch-10", dsp=[("pitch_shift_pv", {"semitones": -10.0})])
for gname, gchain in GRIT.items():
run(f"C_monster_{gname}",
instruct="A huge ancient dragon: deep, rumbling, growling, with a metallic "
"edge and a resonant echoing chest tone.",
dsp=gchain)
print("\n[D] intra-voice variance (seeds x temperature)")
for temp in (0.3, 0.7, 0.9):
for seed in (11, 22, 33, 44, 55):
run(f"D_seed{seed}_t{temp}", temperature=temp, seed=seed, want_wer=False)
(OUT / "results.json").write_text(json.dumps(
{"reference": {"f0_median": round(REF_F0, 1), "envelope_hz": round(REF_ENV),
"path": str(REF)}, "rows": rows}, indent=2) + "\n")
print(f"\nwrote {OUT/'results.json'} ({len(rows)} samples)")
REF_ENV = None
REF_WORDS: list[str] = []
if __name__ == "__main__":
main()

BIN
backup/author_voice_lab/wav/A00_baseline.wav Normal file
View File

Binary file not shown.

BIN
backup/author_voice_lab/wav/A_pitch+00.wav Normal file
View File

Binary file not shown.

BIN
backup/author_voice_lab/wav/A_pitch+02.wav Normal file
View File

Binary file not shown.

BIN
backup/author_voice_lab/wav/A_pitch+04.wav Normal file
View File

Binary file not shown.

BIN
backup/author_voice_lab/wav/A_pitch+06.wav Normal file
View File

Binary file not shown.

BIN
backup/author_voice_lab/wav/A_pitch+08.wav Normal file
View File

Binary file not shown.

BIN
backup/author_voice_lab/wav/A_pitch+10.wav Normal file
View File

Binary file not shown.

BIN
backup/author_voice_lab/wav/A_pitch+12.wav Normal file
View File

Binary file not shown.

BIN
backup/author_voice_lab/wav/A_pitch-02.wav Normal file
View File

Binary file not shown.

BIN
backup/author_voice_lab/wav/A_pitch-04.wav Normal file
View File

Binary file not shown.

BIN
backup/author_voice_lab/wav/A_pitch-06.wav Normal file
View File

Binary file not shown.

BIN
backup/author_voice_lab/wav/A_pitch-08.wav Normal file
View File

Binary file not shown.

BIN
backup/author_voice_lab/wav/A_pitch-10.wav Normal file
View File

Binary file not shown.

BIN
backup/author_voice_lab/wav/A_pitch-12.wav Normal file
View File

Binary file not shown.

BIN
backup/author_voice_lab/wav/B_androgynous.wav Normal file
View File

Binary file not shown.

BIN
backup/author_voice_lab/wav/B_boy.wav Normal file
View File

Binary file not shown.

BIN
backup/author_voice_lab/wav/B_elderly_m.wav Normal file
View File

Binary file not shown.

BIN
backup/author_voice_lab/wav/B_female_mature.wav Normal file
View File

Binary file not shown.

BIN
backup/author_voice_lab/wav/B_female_young.wav Normal file
View File

Binary file not shown.

BIN
backup/author_voice_lab/wav/B_large_m.wav Normal file
View File

Binary file not shown.

BIN
backup/author_voice_lab/wav/B_small_w.wav Normal file
View File

Binary file not shown.

BIN
backup/author_voice_lab/wav/B_young_m.wav Normal file
View File

Binary file not shown.

BIN
backup/author_voice_lab/wav/C_female+4st.wav Normal file
View File

Binary file not shown.

BIN
backup/author_voice_lab/wav/C_female+8st.wav Normal file
View File

Binary file not shown.

BIN
backup/author_voice_lab/wav/C_monster_grit_chorus.wav Normal file
View File

Binary file not shown.

BIN
backup/author_voice_lab/wav/C_monster_grit_full.wav Normal file
View File

Binary file not shown.

BIN
backup/author_voice_lab/wav/C_monster_grit_ring.wav Normal file
View File

Binary file not shown.

BIN
backup/author_voice_lab/wav/C_monster_grit_sat.wav Normal file
View File

Binary file not shown.

BIN
backup/author_voice_lab/wav/C_monster_pitch-10.wav Normal file
View File

Binary file not shown.

BIN
backup/author_voice_lab/wav/D_seed11_t0.3.wav Normal file
View File

Binary file not shown.

BIN
backup/author_voice_lab/wav/D_seed11_t0.7.wav Normal file
View File

Binary file not shown.

BIN
backup/author_voice_lab/wav/D_seed11_t0.9.wav Normal file
View File

Binary file not shown.

BIN
backup/author_voice_lab/wav/D_seed22_t0.3.wav Normal file
View File

Binary file not shown.

BIN
backup/author_voice_lab/wav/D_seed22_t0.7.wav Normal file
View File

Binary file not shown.

BIN
backup/author_voice_lab/wav/D_seed22_t0.9.wav Normal file
View File

Binary file not shown.

BIN
backup/author_voice_lab/wav/D_seed33_t0.3.wav Normal file
View File

Binary file not shown.

BIN
backup/author_voice_lab/wav/D_seed33_t0.7.wav Normal file
View File

Binary file not shown.

BIN
backup/author_voice_lab/wav/D_seed33_t0.9.wav Normal file
View File

Binary file not shown.

BIN
backup/author_voice_lab/wav/D_seed44_t0.3.wav Normal file
View File

Binary file not shown.

BIN
backup/author_voice_lab/wav/D_seed44_t0.7.wav Normal file
View File

Binary file not shown.

BIN
backup/author_voice_lab/wav/D_seed44_t0.9.wav Normal file
View File

Binary file not shown.

BIN
backup/author_voice_lab/wav/D_seed55_t0.3.wav Normal file
View File

Binary file not shown.

BIN
backup/author_voice_lab/wav/D_seed55_t0.7.wav Normal file
View File

Binary file not shown.

BIN
backup/author_voice_lab/wav/D_seed55_t0.9.wav Normal file
View File

Binary file not shown.

BIN
backup/author_voice_lab/wav_cliff/cliff+00_s7.wav Normal file
View File

Binary file not shown.

BIN
backup/author_voice_lab/wav_cliff/cliff+00_s8.wav Normal file
View File

Binary file not shown.

BIN
backup/author_voice_lab/wav_cliff/cliff+02_s7.wav Normal file
View File

Binary file not shown.

BIN
backup/author_voice_lab/wav_cliff/cliff+02_s8.wav Normal file
View File

Binary file not shown.

BIN
backup/author_voice_lab/wav_cliff/cliff-02_s7.wav Normal file
View File

Binary file not shown.

BIN
backup/author_voice_lab/wav_cliff/cliff-02_s8.wav Normal file
View File

Binary file not shown.

BIN
backup/author_voice_lab/wav_cliff/cliff-04_s7.wav Normal file
View File

Binary file not shown.

BIN
backup/author_voice_lab/wav_cliff/cliff-04_s8.wav Normal file
View File

Binary file not shown.

BIN
backup/author_voice_lab/wav_cliff/cliff-06_s7.wav Normal file
View File

Binary file not shown.

BIN
backup/author_voice_lab/wav_cliff/cliff-06_s8.wav Normal file
View File

Binary file not shown.

BIN
backup/author_voice_matrix/20261009T191636Z/BDMurphy_Voice.wav Normal file
View File

Binary file not shown.

BIN
backup/author_voice_matrix/20261009T191636Z/BDMurphy_Voice_Author_Rainbow_v2.wav Normal file
View File

Binary file not shown.

BIN
backup/author_voice_matrix/20261009T191636Z/BDMurphy_Voice_Author_Rainbow_v2_air.wav Normal file
View File

Binary file not shown.

BIN
backup/author_voice_matrix/20261009T191636Z/BDMurphy_Voice_Author_Rainbow_v2_air2.wav Executable file
View File

Binary file not shown.

BIN
backup/author_voice_matrix/20261009T191636Z/BDMurphy_Voice_NOBG.wav Normal file
View File

Binary file not shown.

BIN
backup/author_voice_matrix/20261009T191636Z/BDMurphy_Voice_NOBG_v2.wav Normal file
View File

Binary file not shown.

241
backup/author_voice_matrix/20261009T191636Z/BDMurphy_Voice_analysis.md Normal file
View File

@@ -0,0 +1,241 @@
# BDMurphy_Voice — Noise Cleanup & UmbraAudio Suitability Analysis
Date: 2026-10-08
Files: `BDMurphy_Voice.wav` (input) → `BDMurphy_Voice_NOBG.wav` (cleaned)
---
## 1. Input analysis (`BDMurphy_Voice.wav`)
| Property | Value |
|---|---|
| Duration | 34.328 s |
| Format | WAV PCM_16, **44,100 Hz**, mono, 3,027,810 bytes |
| Peak | −22.04 dBFS (20 dB of unused headroom) |
| Overall RMS | −41.56 dBFS |
| Speech level (90th-pct frame) | −37.0 dBFS |
| Background in pauses (5 measured) | −64.6 / −65.9 / −66.7 / −66.6 / −68.1 dBFS |
| SNR (90th vs 10th pct frame energy) | 28.4 dB |
| Clipping | 0 samples at full scale |
| DC offset | −3e-06 (none) |
| Median F0 | 130.1 Hz, voiced ratio 0.558 → adult male speaker |
**Noise character** (measured, not guessed):
- The noise is **stationary** — spectrum correlation between the first and second
half of the file's quiet frames: **0.828**.
- It is **broadband with low-frequency dominance**: in pauses the 20–100 Hz band
is the loudest component (rumble/mains hum), then 100–300 Hz, with a
broadband hiss floor across 1–12 kHz.
- Pause-to-speech ratio only **28–31 dB**, i.e. the background is audible
between sentences; house-generated references sit at 50–60 dB.
- The sample is **~20 dB quieter than every reference in the system**
(house refs: RMS −19.8…−26.5 dBFS, peak −1.1…−7.3; this file: RMS −41.6,
peak −22.0).
## 2. Suitability check #1 — **FAIL**
Checked against the gates the UmbraAudio system itself enforces
(`umbraaudio/analysis/voice_analysis.py`: `MIN_REF_DURATION_S=20`,
`TARGET_SAMPLE_RATE=24000`, `SILENCE_RMS=0.002`) plus standard voice-capture
quality gates (noise floor ≤ −45 dBFS, SNR ≥ 20 dB, clipping ≤ 0.1 %,
|DC| ≤ 0.01):
| Gate | Value | Threshold | Result |
|---|---|---|---|
| duration | 34.33 s | ≥ 20 s | PASS |
| **sample_rate** | **44,100 Hz** | **24,000 Hz** | **FAIL** |
| energy | 0.00836 | > 0.002 | PASS |
| noise_floor | −65.4 dBFS | ≤ −45 dBFS | PASS |
| snr | 28.4 dB | ≥ 20 dB | PASS |
| clipping | 0.0 | ≤ 0.001 | PASS |
| dc_offset | −3e-06 | \|x\| ≤ 0.01 | PASS |
Confirmed by running the **system's own** `VoiceAnalysisEngine` on the raw file
as `author_ref.wav`: `passed: false` — only `sample_rate` fails (44100 ≠ 24000).
All 212 existing `*_ref.wav` files in UmbraAudio are 24 kHz mono PCM_16.
Verdict: not usable as-is — wrong sample rate for the system, background noise
only ~30 dB below speech, and 20 dB below the house reference level.
## 3. Cleanup performed → `BDMurphy_Voice_NOBG.wav`
Pipeline (SoX, native rate first so the profile matches the original noise):
1. **Noise profile** from the five *measured* pauses (7.92, 11.92, 14.88, 16.40,
33.12 s) — `sox … trim … noiseprof`. (An earlier profile taken from the
mathematically-quietest 1 s window was rejected: it contained speech leakage
and removed 4× less noise.)
2. **Spectral subtraction** — `noisered profile 0.4`.
3. **High-pass 75 Hz** (`highpass -2 75`) — removes rumble/mains hum; the
speaker's median F0 is 130 Hz, so nothing of the voice is touched.
4. **Resample 44.1 → 24 kHz** (`rate -v`) — meets `TARGET_SAMPLE_RATE`, matching
the system's reference format exactly.
5. **Level match** (`gain -n -3`) → peak −3.0 dBFS, which lands the file inside
the house reference envelope (RMS −23.5 vs −19.8…−26.5; speech −19.0 vs
−15.2…−21.6; peak −3.0 vs −1.1…−7.3).
**Denoise strength was selected by measurement**, not by ear — four candidate
settings were built from the same pause profile and scored on noise removal vs
speech damage:
| Variant | Pause noise (dBFS) | Speech spectral delta | Envelope corr | F0 delta |
|---|---|---|---|---|
| high-pass only | −68.4 … −72.5 | 1.47 dB | 0.799 | 0.0 Hz |
| **noisered 0.4 + HP — CHOSEN** | −75.1 … −91.6 | 5.03 dB | 0.755 | **0.0 Hz** |
| noisered 0.6 + HP | −79.0 … −96.3 | 8.59 dB | 0.681 | 0.0 Hz |
| noisered 0.8 + HP | −84.1 … −96.3 | 15.93 dB | 0.588 | 0.71 Hz (pitch drift) |
The chosen setting removes **~24 dB of background noise** while leaving the
pitch track bit-identical; the next step up doubles speech distortion for 5 dB
more noise removal, and 0.8 starts corrupting the F0 track.
## 4. Result (`BDMurphy_Voice_NOBG.wav`)
| Property | Before | After |
|---|---|---|
| Sample rate | 44,100 Hz | **24,000 Hz** |
| Duration | 34.328 s | 34.305 s (−23 ms resampler edge) |
| Noise floor (10th-pct frame) | −65.4 dBFS | **−76.4 dBFS** |
| Pauses | −64.6 … −68.1 dBFS | −70.0 … −72.5 dBFS (worst: −55.9, a 0.28 s breath) |
| SNR | 28.4 dB | **57.3 dB** |
| Peak / RMS | −22.0 / −41.6 dBFS | −3.0 / −23.5 dBFS |
| Speech level | −37.0 dBFS | −19.0 dBFS |
| Median F0 | 130.1 Hz | 129.4 Hz (preserved) |
| Clipping / DC | none / none | none / none |
Integrity checks against the gain-matched original:
- **Time alignment:** cross-correlation lag **0 samples (0.00 ms)** — no shift.
- **Polarity:** band-limited correlation **0.9755** — same polarity.
- **Dropouts:** 0 near-silent holes punched into speech frames (longest 0 ms).
- **Speech energy:** −0.76 dB on active frames (both peak-normalised).
- Residual pause noise sits **~53 dB below speech**.
## 5. Suitability check #2 — **PASS**
My 7-gate check on `BDMurphy_Voice_NOBG.wav`:
| Gate | Value | Threshold | Result |
|---|---|---|---|
| duration | 34.31 s | ≥ 20 s | PASS |
| sample_rate | 24,000 Hz | 24,000 Hz | PASS |
| energy | 0.0665 | > 0.002 | PASS |
| noise_floor | −76.4 dBFS | ≤ −45 dBFS | PASS |
| snr | 57.3 dB | ≥ 20 dB | PASS |
| clipping | 0.0 | ≤ 0.001 | PASS |
| dc_offset | 0.0 | \|x\| ≤ 0.01 | PASS |
And the **system's own** `VoiceAnalysisEngine` (run against a copy named
`author_ref.wav`): all four requirements pass —
`duration 34.3 ≥ 20`, `sample_rate 24000 = 24000`, `energy 0.0665 > 0.002`,
`uniqueness 0.0 < 0.9` → **"All 1 voices meet requirements", `passed: true`**.
## 6. Voice matrix created — voice name **"Author"**
Because the check passed, the cleaned sample was used to capture the vmatrix
through the system's canonical full-vmatrix path:
```
POST http://localhost:8881/v1/voice-prompt
slug = author
x_vector_only = 0 # FULL vmatrix (ref_code + embedding), the canonical mode
force = 1
ref_text = Rainbow Passage (verified byte-identical to the server's
VOICE_REFERENCE_TEXT)
ref_audio = BDMurphy_Voice_NOBG.wav
→ {"ok": true, "slug": "author", ...}
```
**Artifact:** `/Volumes/1TB_APL_Storage/UmbraAudio/DealHunterShip_4.0e_whole-to-audio/.umbra/voice_prompts/author.pt` (61,853 bytes)
Contents verified by loading the file — structurally identical to the production
prompts (e.g. `agnarr.pt`):
| Field | `author.pt` | production example |
|---|---|---|
| `ref_code` | **(429, 16)** int64 | (453, 16) int64 |
| `ref_spk_embedding` | **(2048,)** bfloat16 | (2048,) bfloat16 |
| `x_vector_only_mode` | False (full vmatrix) | False |
| `icl_mode` | True | True |
| `ref_text` | Rainbow Passage | Rainbow Passage |
429 codec frames ÷ 34.31 s ≈ 12.5 Hz — exactly the Qwen3-TTS 12 Hz codec rate,
i.e. the whole reference was re-spoken through the codec as intended.
**End-to-end verification** — `POST /v1/tts` with `prompt_slug=author`
(HTTP 200, 113 s on MPS):
- Generated 31.4 s of 24 kHz audio, RMS −23.2 dBFS, peak −4.7 dBFS (healthy).
- Generated median F0 **133.1 Hz** vs reference **129.4 Hz** (Δ 3.8 Hz).
- **ECAPA-TDNN speaker similarity: generated audio vs the cleaned reference = 0.887**,
against a control (an unrelated speaker's reference) of **−0.004** — the
vmatrix reproduces this speaker, not some other voice.
The server was returned to its prior state afterwards (`POST /unload {"model_type":"all"}`
— no model resident, as it was before the capture).
## Notes / follow-ups
- The `.pt` lives in the DealHunterShip project's `.umbra/voice_prompts/`
because that is `QWEN_PROMPT_DIR` for the running local server (the system's
canonical prompt cache; the directory is git-ignored, so the repo stays clean).
- The cleaned WAV stays in this folder. To use "Author" as a speaker inside a
project, its reference would normally also live at
`<project>/.umbra/voices/author_ref.wav` — not done here (not requested).
- Observation while testing: ICL generation with `prompt_slug` kept speaking
well past the 12-word test sentence (25 s of speech). That is server-side ICL
stopping behaviour, **not** a defect in this vmatrix — worth a look separately.
---
## 7. Round 2 — the "hollow / sounds like a building" report
Feedback after listening: clean, but hollow — "like the noise removal put the
sound in a building". Measured diagnosis (raw vs cleaned vs generated, all
compared on speech-active frames):
| Hypothesis | Test | Result |
|---|---|---|
| Room reverb unmasked by denoising | decay inside the silent gaps (≥150 ms, absolute dBFS) | **dry**: cleaned falls 33 dB in 25 ms, reaches its floor in 75 ms (raw: 40 ms). A real room is still at −5 dB there. |
| Discrete reflection / comb filter | per-frame autocorrelation 1–40 ms | only peak is the **pitch period** (7.0–7.5 ms = 130 Hz F0), identical in raw, cleaned and generated |
| Tonal imbalance introduced | energy-normalised LTAS band shape | **preserved within 0.2 dB** from 80 Hz to 12 kHz; only 40–80 Hz changed (−12.3 dB, my 75 Hz high-pass) and 2.5–5 kHz (−2.5 dB) |
| Over-reduction artifacts | frame-to-frame spectral jump, band modulation | up 1.90 → 3.00 dB/frame; decomposition: rate→2.61, high-pass→2.74, `noisered 0.15`→2.87, `noisered 0.4`→2.98 (the jump metric itself is also driven by residual-noise content, confirmed by scale-invariance test) |
| Render dullness (TTS side) | band shape of the generated Rainbow vs its reference | boxy-low (+1.7 dB at 80–160, +1.2 at 160–315) and **dull-high (−4.4 dB at 5–10 kHz, −6.6 dB at 10–12 kHz)** |
Two changes follow from the measurements, applied on both sides of the chain:
**Reference — `BDMurphy_Voice_NOBG_v2.wav`** (replaces the v2 capture source)
- `noisered 0.4 → 0.15` (spectral distortion vs the original halves:
LSD 5.02 → 2.71 dB) and `highpass 75 → 40 Hz` (restores 12 dB of body in
the 40–75 Hz band).
- Still comfortably inside every gate: floor −63.3 dBFS (≤ −45), SNR 42.4 dB
(≥ 20), peak −3.0 dBFS, 0 clipped samples, |DC| 0.0, 34.31 s, 24 kHz.
- Alignment vs the original: lag 0 samples, polarity r = 0.893, 0 dropouts.
- The system's own `VoiceAnalysisEngine`: **PASS** (all four requirements).
**vmatrix re-captured** from that v2 file (`slug=author, x_vector_only=0,
force=1`, ref_text = server default `VOICE_REFERENCE_TEXT`). The previous
vmatrix is backed up at
`…/T/opencode/author_v1_backup.pt` (61,853 bytes).
**Renders** — both level-matched to the first sample (active-RMS −20.3 dBFS,
i.e. the −23.9 dBFS overall RMS of the original render):
| File | What it is |
|---|---|
| `BDMurphy_Voice_Author_Rainbow_v2.wav` | Rainbow through the new vmatrix — no extra EQ |
| `BDMurphy_Voice_Author_Rainbow_v2_air.wav` | same render + smooth 1/6-octave tilt correction toward the reference voice (cuts the boxy 315–630 Hz excess, restores the 5–12 kHz air) |
Verification of `…_v2.wav`: 85.6 s @ 24 kHz, RMS −23.8 / peak −5.2 dBFS, 0
clipped samples, median F0 130.9 Hz vs reference 129.4 Hz (Δ 1.5 Hz),
**ECAPA similarity 0.8833 vs the Author reference, −0.0213 vs a control
speaker** → same voice. Tonal gap to the reference above 80 Hz: 8.1 dB (first
render) → 6.6 dB (v2) → **1.5 dB** with the tilt correction, which leaves
identity unchanged (ECAPA 0.8835).
Server returned to its prior state (`POST /unload {"model_type":"all"}`);
UmbraAudio repo still clean. Compare the three renders A/B — if the hollow
quality survives both fixes it is in the model's ICL synthesis, not in the
recording chain.

49
backup/author_voice_matrix/20261009T191636Z/RESTORE.md Normal file
View File

@@ -0,0 +1,49 @@
# Author voice matrix — restore
## What this is
The **Author** voice identity, stored as a Qwen3-TTS *vmatrix* (the numbers that
ARE the voice). Captured once; reuses those exact numbers for every utterance, so
the voice never drifts between takes.
## Files
| File | What it is |
|---|---|
| `author_v2.pt` | **The live matrix.** Use this one. |
| `author_v1.pt` | Superseded first capture, kept for rollback. |
| `BDMurphy_Voice*.wav` | The reference chain the matrix was captured from. |
| `manifest.json` | SHA-256, provenance, ref_code shape for every file. |
## Restore
```bash
# Matrix -> the server's prompt cache (PROMPT_DIR for the local server):
cp author_v2.pt <project>/.umbra/voice_prompts/author.pt
# Reference WAV -> a project's approved-reference dir:
cp BDMurphy_Voice_NOBG_v2.wav <project>/.umbra/voices/author_ref.wav
```
The capture call that produced this matrix (re-run ONLY if you are deliberately
changing the voice — it is not needed to use it):
```
POST http://127.0.0.1:8881/v1/voice-prompt
slug=author x_vector_only=0 force=1
ref_text=<Rainbow Passage> ref_audio=BDMurphy_Voice_NOBG_v2.wav
```
`x_vector_only=0` is the full vmatrix (`ref_code` + speaker embedding) and is the
only mode that reproduces this voice faithfully. `x_vector_only=1` captures just
the coarse embedding and is what historically caused voices to drift.
## Verify
```bash
shasum -a 256 author_v2.pt # must equal manifest.json -> matrices.author_v2.sha256
```
To confirm a matrix still renders the right speaker, generate from
`prompt_slug=author` and compare the median F0 against the reference
(reference is 129.4 Hz; a correct render lands ~130-133 Hz).

BIN
backup/author_voice_matrix/20261009T191636Z/author_v1.pt Normal file
View File

Binary file not shown.

BIN
backup/author_voice_matrix/20261009T191636Z/author_v2.pt Normal file
View File

Binary file not shown.

122
backup/author_voice_matrix/20261009T191636Z/manifest.json Normal file
View File

@@ -0,0 +1,122 @@
{
"created_utc": "2026-10-09T19:16:36.048124+00:00",
"host": "mini.bdmurphy.me",
"platform": "macOS-27.0.1-arm64-arm-64bit",
"purpose": "Durable backup of the Author voice matrix (vmatrix) and its reference chain. The vmatrix is the source of truth for the Author voice; the WAV is reproducible from it.",
"matrix_format": {
"file": "voice_prompts/<slug>.pt",
"structure": "torch.save({'items': [{'ref_code','ref_spk_embedding','x_vector_only_mode','icl_mode','ref_text'}]})",
"ref_code": "(T,16) int64 codec tokens @12.5 Hz = re-encoded reference speech",
"ref_spk_embedding": "(2048,) bfloat16 speaker x-vector",
"full_vmatrix": "x_vector_only_mode=False, icl_mode=True"
},
"capture_endpoint": "POST http://127.0.0.1:8881/v1/voice-prompt slug=author x_vector_only=0 force=1",
"matrices": {
"author_v2.pt": {
"bytes": 61853,
"sha256": "d38a2dc88a9d353cafbf310710d7b5cf8e1293889c3e1987537b888a55c6a83f",
"generation": "v2 (current live matrix)",
"captured_from": "BDMurphy_Voice_NOBG_v2.wav",
"processing": "noisered 0.15 + highpass 40 Hz, resampled 44100->24000, gain -3 dB",
"source_recording": "BDMurphy_Voice.wav (raw, 34.33 s, 44100 Hz)",
"reason": "round 2 fix for the reported 'hollow / sounds like a building' quality",
"source": "/Volumes/1TB_APL_Storage/UmbraAudio/DealHunterShip_4.0e_whole-to-audio/.umbra/voice_prompts/author.pt",
"ref_code_shape": [
429,
16
],
"ref_code_seconds": 34.32,
"icl_mode": true,
"x_vector_only_mode": false,
"ref_text_is_rainbow": true
},
"author_v1.pt": {
"bytes": 61853,
"sha256": "c683975346f9d35ecc14a8d3e64e5cf17c77225b5852075a8ecdba69d2996cb5",
"generation": "v1 (superseded)",
"captured_from": "BDMurphy_Voice_NOBG.wav",
"processing": "noisered 0.4 + highpass 75 Hz, resampled 44100->24000, gain -3 dB",
"reason": "first capture; replaced by v2",
"source": "/private/var/folders/y8/cbvhds297918lrxt2wd3btsr0000gn/T/opencode/author_v1_backup.pt",
"ref_code_shape": [
429,
16
],
"ref_code_seconds": 34.32,
"icl_mode": true,
"x_vector_only_mode": false,
"ref_text_is_rainbow": true
}
},
"reference_chain": [
{
"bytes": 3027810,
"sha256": "a552759369da74fd39fe0d98d67af3b6b4b25359a66fdc858a9c0a79fa076cfe",
"duration_s": 34.328,
"sample_rate": 44100,
"channels": 1,
"subtype": "PCM_16",
"source": "/Volumes/Dock/worstauthor/Documents/Audacity4/BDMurphy_Voice.wav"
},
{
"bytes": 1646694,
"sha256": "229eb18851ceef5d306ab9a807be763a223665cc8361b9e8957d26c86236054c",
"duration_s": 34.305,
"sample_rate": 24000,
"channels": 1,
"subtype": "PCM_16",
"source": "/Volumes/Dock/worstauthor/Documents/Audacity4/BDMurphy_Voice_NOBG.wav"
},
{
"bytes": 1646694,
"sha256": "dadb973adbd6feb1b3a6e667ec20c5236202a749e436a9132de49cabd7e05df0",
"duration_s": 34.305,
"sample_rate": 24000,
"channels": 1,
"subtype": "PCM_16",
"source": "/Volumes/Dock/worstauthor/Documents/Audacity4/BDMurphy_Voice_NOBG_v2.wav"
},
{
"bytes": 4108844,
"sha256": "f2ba2b99b9c863a138924cf02af3f05775d41861fc3fdbe897336b95d063752a",
"duration_s": 85.6,
"sample_rate": 24000,
"channels": 1,
"subtype": "PCM_16",
"source": "/Volumes/Dock/worstauthor/Documents/Audacity4/BDMurphy_Voice_Author_Rainbow_v2.wav"
},
{
"bytes": 4108844,
"sha256": "161615c3133ba5f95bf2b7fa71a3d4cc0cf5a4f0b474848ad7419c7adc3ffe50",
"duration_s": 85.6,
"sample_rate": 24000,
"channels": 1,
"subtype": "PCM_16",
"source": "/Volumes/Dock/worstauthor/Documents/Audacity4/BDMurphy_Voice_Author_Rainbow_v2_air.wav"
},
{
"bytes": 1866910,
"sha256": "2f3b6c161593aea335f42fd01159f93c422e4c07e4806739c24fe366c920e7fc",
"duration_s": 29.17,
"sample_rate": 32000,
"channels": 1,
"subtype": "PCM_16",
"source": "/Volumes/Dock/worstauthor/Documents/BDMurphy_Voice_Author_Rainbow_v2_air2.wav"
},
{
"bytes": 12102,
"sha256": "55d84420ddbcbc0e40af9e2e103d4c37a6611d9dfa5a0bdf3ad52801684926a1",
"source": "/Volumes/Dock/worstauthor/Documents/Audacity4/BDMurphy_Voice_analysis.md"
}
],
"notes": [
"author.pt is git-ignored project cache; this backup is the durable copy.",
"The superseded v1 matrix's only prior copy was in an OS temp dir (purgeable).",
"verdict: full vmatrix (ref_code + embedding), NOT x_vector_only."
],
"written_to": [
"/Volumes/1TB_APL_Storage/UmbraAudio/backup/author_voice_matrix/20261009T191636Z",
"/Volumes/Dock/worstauthor/.umbra/library/author_voice_matrix/20261009T191636Z"
],
"verification": "ok"
}

3
backup/create_full_backup.sh
View File

@@ -1,4 +1,7 @@
#!/usr/bin/env bash
# Copyright 2026 Umbra Tools (umbratools.com)
# SPDX-License-Identifier: Apache-2.0
# Create a complete, restorable UmbraAudio snapshot on a mounted filesystem/NAS.
set -Eeuo pipefail

3
backup/restore_support.py
View File

@@ -1,4 +1,7 @@
#!/usr/bin/env python3
# Copyright 2026 Umbra Tools (umbratools.com)
# SPDX-License-Identifier: Apache-2.0
"""Portable backup/restore support helpers for UmbraAudio.
This module uses only the Python standard library. It snapshots global Umbra

3
infra/agent_bridge/__init__.py
View File

@@ -1,3 +1,6 @@
# Copyright 2026 Umbra Tools (umbratools.com)
# SPDX-License-Identifier: Apache-2.0
"""Temporary browser-to-agent bridge for UmbraTrans."""
__all__: list[str] = []

3
infra/agent_bridge/__main__.py
View File

@@ -1,3 +1,6 @@
# Copyright 2026 Umbra Tools (umbratools.com)
# SPDX-License-Identifier: Apache-2.0
from .server import main
raise SystemExit(main())

3
infra/agent_bridge/client.py
View File

@@ -1,3 +1,6 @@
# Copyright 2026 Umbra Tools (umbratools.com)
# SPDX-License-Identifier: Apache-2.0
from __future__ import annotations
import argparse

3
infra/agent_bridge/server.py
View File

@@ -1,3 +1,6 @@
# Copyright 2026 Umbra Tools (umbratools.com)
# SPDX-License-Identifier: Apache-2.0
from __future__ import annotations
import argparse

3
infra/agent_bridge/tests/test_bridge.py
View File

@@ -1,3 +1,6 @@
# Copyright 2026 Umbra Tools (umbratools.com)
# SPDX-License-Identifier: Apache-2.0
from __future__ import annotations
import json

3
infra/bootstrap_enterprise_mint.sh
View File

@@ -1,4 +1,7 @@
#!/usr/bin/env bash
# Copyright 2026 Umbra Tools (umbratools.com)
# SPDX-License-Identifier: Apache-2.0
# Phase-one enterprise control-plane bootstrap for Linux Mint 22.x.
# This provisions PostgreSQL, database roles, core tenancy/identity tables,
# and least-privilege service accounts. It does not install application code,

3
infra/bootstrap_umbracontrol_database.sh
View File

@@ -1,4 +1,7 @@
#!/usr/bin/env bash
# Copyright 2026 Umbra Tools (umbratools.com)
# SPDX-License-Identifier: Apache-2.0
# Database-only bootstrap for the UmbraTrans control plane.
# This intentionally does not install packages, create application directories,
# install application services, or modify Qwen/Docker/firewall settings.

3
infra/remote_credential_broker/__init__.py
View File

@@ -1,3 +1,6 @@
# Copyright 2026 Umbra Tools (umbratools.com)
# SPDX-License-Identifier: Apache-2.0
"""Local, stdio MCP broker for controlled remote operations.
The broker keeps passwords in the macOS Keychain and exposes only operations,

3
infra/remote_credential_broker/__main__.py
View File

@@ -1,3 +1,6 @@
# Copyright 2026 Umbra Tools (umbratools.com)
# SPDX-License-Identifier: Apache-2.0
from .broker import main
raise SystemExit(main())

3
infra/remote_credential_broker/askpass_helper.py
View File

@@ -1,4 +1,7 @@
#!/usr/bin/env python3
# Copyright 2026 Umbra Tools (umbratools.com)
# SPDX-License-Identifier: Apache-2.0
from __future__ import annotations
import os

3
infra/remote_credential_broker/broker.py
View File

@@ -1,3 +1,6 @@
# Copyright 2026 Umbra Tools (umbratools.com)
# SPDX-License-Identifier: Apache-2.0
from __future__ import annotations
import argparse

3
infra/remote_credential_broker/config.py
View File

@@ -1,3 +1,6 @@
# Copyright 2026 Umbra Tools (umbratools.com)
# SPDX-License-Identifier: Apache-2.0
from __future__ import annotations
import json

3
infra/remote_credential_broker/executor.py
View File

@@ -1,3 +1,6 @@
# Copyright 2026 Umbra Tools (umbratools.com)
# SPDX-License-Identifier: Apache-2.0
from __future__ import annotations
import os

3
infra/remote_credential_broker/keychain.py
View File

@@ -1,3 +1,6 @@
# Copyright 2026 Umbra Tools (umbratools.com)
# SPDX-License-Identifier: Apache-2.0
from __future__ import annotations
import ctypes

3
infra/remote_credential_broker/popup.py
View File

@@ -1,3 +1,6 @@
# Copyright 2026 Umbra Tools (umbratools.com)
# SPDX-License-Identifier: Apache-2.0
from __future__ import annotations
import subprocess

Some files were not shown because too many files have changed in this diff Show More