Add voice library, embedding-mode generation, and webUI improvements
- Add cross-project voice/name library (library.py + webUI library tabs) - chapter_builder: embedding-mode generation + auto-load TTS base model in preflight - qwen_tts: vmatrix capture, embedding/voice-prompt refinements - registry: vmatrix_path column + migration for approved voices - webUI: speakers list, chapter audio detail, proper names updates - docs: update voice generation and tts_server README
This commit is contained in:
B.D. Murphy
@@ -3,55 +3,75 @@
|
|||||||
## Overview
|
## Overview
|
||||||
|
|
||||||
UmbraAudio generates multi-voice audiobooks from English manuscripts. Each character
|
UmbraAudio generates multi-voice audiobooks from English manuscripts. Each character
|
||||||
gets a unique, consistent voice. The voice identity is stored as a **speaker embedding** —
|
gets a unique, consistent voice. The voice identity is stored as a **vmatrix** — the
|
||||||
a 2048-dimensional float32 vector extracted from Qwen3-TTS's internal speaker encoder.
|
set of numbers inside Qwen3-TTS that *is* the voice:
|
||||||
This embedding is the single source of truth for a character's voice.
|
|
||||||
|
1. the **ICL "8k matrix"** (`ref_code`) — a `(T, 16)` tensor of codec tokens rebuilt by
|
||||||
|
re-speaking the reference WAV through the model's codec encoder, AND
|
||||||
|
2. the **speaker embedding** (`ref_spk_embedding`) — a 2048-dim float32/bfloat16 vector
|
||||||
|
extracted from Qwen3-TTS's speaker encoder.
|
||||||
|
|
||||||
|
The vmatrix is captured ONCE per approved voice and cached server-side as a `.pt` prompt
|
||||||
|
(in `voice_prompts/{slug}.pt`). Every utterance reuses those exact cached numbers, so the
|
||||||
|
voice is bit-identical in identity across every utterance and chapter. **An approved voice
|
||||||
|
is therefore two artifacts: a WAV to listen to + the captured vmatrix for consistency.**
|
||||||
|
|
||||||
|
> Historical note: an earlier pipeline stored ONLY the 2048-dim speaker embedding
|
||||||
|
> (`x_vector_only=True`) as the single source of truth. That was the "nearly identical /
|
||||||
|
> drift-prone" coarse capture. The canonical capture is the **full vmatrix** (ref_code +
|
||||||
|
> embedding, `x_vector_only=False`).
|
||||||
|
|
||||||
## Architecture
|
## Architecture
|
||||||
|
|
||||||
### Voice Identity Pipeline
|
### Voice Identity Pipeline
|
||||||
|
|
||||||
```
|
```
|
||||||
Edge-TTS WAV ref ──or── VoiceDesign instruct text
|
Approved reference WAV (per speaker: the author-approved "*_ref.wav")
|
||||||
│ │
|
│
|
||||||
▼ ▼
|
▼
|
||||||
Base model VoiceDesign model
|
POST /v1/voice-prompt (x_vector_only=0)
|
||||||
extract_speaker_ generate_voice_design()
|
│ model re-speaks the reference: codec encoder + speaker encoder
|
||||||
embedding() │
|
▼
|
||||||
│ ▼
|
vmatrix { ref_code: (T,16) int64 ICL "8k matrix"
|
||||||
▼ Base model
|
ref_spk_embedding: (2048,) speaker x-vector
|
||||||
Speaker embedding extract_speaker_embedding()
|
x_vector_only_mode: False, icl_mode: True }
|
||||||
(2048-dim float32) │
|
│
|
||||||
│ ▼
|
▼
|
||||||
▼ Speaker embedding
|
cached as .pt under voice_prompts/{slug}.pt (PROMPT_DIR)
|
||||||
Save as .npy │
|
│
|
||||||
(8KB per speaker) ▼
|
▼
|
||||||
Save as .npy
|
every utterance → generate(prompt_slug={slug}) reuses the SAME cached vmatrix
|
||||||
```
|
```
|
||||||
|
|
||||||
### Key Insight
|
### Key Insight
|
||||||
|
|
||||||
The old approach (WAV ref → extract x-vector at generation time) produced **nearly
|
The previous approach (WAV ref → extract a bare x-vector at generation time, or use the
|
||||||
identical voices** (MFCC cosine similarity 0.99+) because x-vector extraction is lossy.
|
embedding-only `x_vector_only=True` mode) produced **drift-prone / coarse** voices,
|
||||||
The new approach stores the **exact embedding** that the model uses, bypassing extraction
|
because neither kept the exact numbers the model conditions on. The **vmatrix** captures
|
||||||
entirely. This gives:
|
the exact ICL codec-token matrix (`ref_code`) **plus** the speaker embedding the model
|
||||||
|
uses, caches it once, and reuses it for every utterance. This gives:
|
||||||
|
|
||||||
- **Perfect determinism**: same embedding + same seed = identical waveform
|
- **Perfect consistency**: every utterance of a character uses the exact same cached
|
||||||
- **Distinct voices**: narrator 275 Hz, nanny 176 Hz, kainda 227 Hz (measured F0)
|
numbers → the voice can never drift or "change" between chapters
|
||||||
- **Tiny storage**: 8KB per speaker vs megabytes of WAV refs
|
- **Faithful identity**: regeneration re-speaks the reference, so the voice is the
|
||||||
- **No drift**: embedding is fixed once saved, never re-extracted
|
reference voice, not a lossy summary of it
|
||||||
|
- **One-time capture**: the `.pt` is built once per approved voice (on Approve) and at
|
||||||
|
the start of each generation run if missing
|
||||||
|
- **Distinct voices**: narrator / nanny / kainda measured F0s remain distinct
|
||||||
|
|
||||||
## File Structure
|
## File Structure
|
||||||
|
|
||||||
```
|
```
|
||||||
.umbra/
|
.umbra/
|
||||||
├── voice_embeddings/ # Speaker identity cache
|
├── voice_prompts/ # VMXATRIX cache (.pt): the canonical voice identity
|
||||||
|
│ ├── kainda_walters_foster_murphy.pt # ref_code (T,16) + ref_spk_embedding (2048,)
|
||||||
|
│ ├── nanny.pt
|
||||||
|
│ ├── narrator.pt
|
||||||
|
│ └── ... # one .pt per approved voice
|
||||||
|
├── voice_embeddings/ # LEGACY speaker-embedding cache (x_vector_only mode)
|
||||||
│ ├── narrator.npy # 2048-dim float32 (8KB)
|
│ ├── narrator.npy # 2048-dim float32 (8KB)
|
||||||
│ ├── nanny.npy
|
│ └── ... # kept only for fallback/experiments
|
||||||
│ ├── kainda_walters_foster_murphy.npy
|
├── voices/ # Approved reference WAVs (the "listen to" artifact)
|
||||||
│ ├── deal_hunter.npy
|
|
||||||
│ └── ... # 94 speakers total
|
|
||||||
├── voices/ # Legacy WAV refs (kept for fallback)
|
|
||||||
│ ├── narrator_ref.wav
|
│ ├── narrator_ref.wav
|
||||||
│ ├── nanny_ref.wav
|
│ ├── nanny_ref.wav
|
||||||
│ └── ...
|
│ └── ...
|
||||||
@@ -63,40 +83,49 @@ entirely. This gives:
|
|||||||
|
|
||||||
## Generation Modes
|
## Generation Modes
|
||||||
|
|
||||||
### Mode 1: Cached Embedding (Preferred)
|
# Mode 1 is the CANONICAL mode (since the vmatrix fix):
|
||||||
|
# client captures the FULL vmatrix once, then generates via prompt_slug.
|
||||||
|
|
||||||
Fastest and most consistent. Used when a `.npy` embedding exists for the speaker.
|
### Mode 1: Cached Vmatrix via prompt_slug (Preferred — Canonical)
|
||||||
|
|
||||||
|
Full voice identity (ICL "8k matrix" ref_code + speaker embedding) captured once and
|
||||||
|
cached as `.pt`. Every utterance reuses the same cached numbers.
|
||||||
|
|
||||||
```python
|
```python
|
||||||
# Client loads cached embedding
|
# Client captures the vmatrix once per approved voice (equivalent to Approve):
|
||||||
emb = np.load("voice_embeddings/narrator.npy") # 2048-dim float32
|
prompt_slug = engine.build_voice_prompt(
|
||||||
|
ref_path, # the approved *_ref.wav
|
||||||
# Sends to server as JSON (8KB payload)
|
ref_text=VOICE_REFERENCE_TEXT, # Rainbow Passage
|
||||||
POST /v1/tts
|
x_vector_only=False, # FULL vmatrix (ref_code + embedding)
|
||||||
speaker_embedding: "[0.123, -0.456, ...]" # JSON float list
|
force=False, # True only if the ref wav changed
|
||||||
text: "The captain nodded."
|
|
||||||
temperature: 0.5
|
|
||||||
seed: 42
|
|
||||||
|
|
||||||
# Server passes embedding directly to model
|
|
||||||
model.generate_voice_clone(
|
|
||||||
text=text,
|
|
||||||
voice_clone_prompt={
|
|
||||||
"ref_code": [None],
|
|
||||||
"ref_spk_embedding": [spk_tensor],
|
|
||||||
"x_vector_only_mode": [True],
|
|
||||||
"icl_mode": [False],
|
|
||||||
},
|
|
||||||
language="english",
|
|
||||||
temperature=0.5,
|
|
||||||
)
|
)
|
||||||
|
|
||||||
|
# Server: POST /v1/voice-prompt → cached voice_prompts/{prompt_slug}.pt
|
||||||
|
# containing { ref_code: (T,16), ref_spk_embedding: (2048,),
|
||||||
|
# x_vector_only_mode: False, icl_mode: True }
|
||||||
|
|
||||||
|
# Every utterance then references the same identity:
|
||||||
|
engine.generate(text, output_path, prompt_slug=prompt_slug)
|
||||||
```
|
```
|
||||||
|
|
||||||
**Determinism**: Same embedding + same seed = **identical waveform** (verified: mean abs diff = 0.000000).
|
**Consistency**: every utterance of a character uses the exact same cached vmatrix → the
|
||||||
|
voice is identical across all chapters and can never silently change.
|
||||||
|
|
||||||
### Mode 2: WAV Reference (Fallback)
|
### Mode 2: Embedding-only (x_vector_only=1)
|
||||||
|
|
||||||
Used when no cached embedding exists. Extracts x-vector at generation time.
|
Captures ONLY `ref_spk_embedding` (`ref_code=None`). Coarse, drift-prone; this was the
|
||||||
|
pre-fix default and caused the "voice changed / inconsistent" problem. Keep for
|
||||||
|
experiments or machines without ICL VRAM headroom.
|
||||||
|
|
||||||
|
```python
|
||||||
|
POST /v1/voice-prompt x_vector_only=1
|
||||||
|
# → cached voice_prompts/{slug}.pt with ref_code=None, icl_mode=False
|
||||||
|
```
|
||||||
|
|
||||||
|
### Mode 3: WAV Reference (fallback)
|
||||||
|
|
||||||
|
Used only when no cached prompt exists and no slug is passed — server extracts the
|
||||||
|
x-vector at generation time.
|
||||||
|
|
||||||
```python
|
```python
|
||||||
POST /v1/tts
|
POST /v1/tts
|
||||||
@@ -105,9 +134,10 @@ POST /v1/tts
|
|||||||
temperature: 0.5
|
temperature: 0.5
|
||||||
```
|
```
|
||||||
|
|
||||||
### Mode 3: VoiceDesign (New Voice Creation)
|
### Mode 4: VoiceDesign (New Voice Creation)
|
||||||
|
|
||||||
Used to create new character voices from text descriptions.
|
Used to create new character voices from text descriptions (also used to generate the
|
||||||
|
initial `*_ref.wav` for each character).
|
||||||
|
|
||||||
```python
|
```python
|
||||||
POST /v1/tts
|
POST /v1/tts
|
||||||
@@ -201,33 +231,38 @@ Returns server status and loaded models.
|
|||||||
|
|
||||||
### Per-Utterance Consistency
|
### Per-Utterance Consistency
|
||||||
|
|
||||||
Each utterance uses the same embedding + same seed → identical voice identity.
|
Every utterance of a character passes the **same cached vmatrix** (`prompt_slug` → the
|
||||||
Variation comes only from:
|
`.pt` with ref_code + embedding) → identical voice identity. Variation comes only from:
|
||||||
- Different text (phoneme sequence)
|
- Different text (codec-token sequence)
|
||||||
- Temperature (controlled per-chapter)
|
- Temperature (controlled per-chapter)
|
||||||
|
|
||||||
### Cross-Chapter Consistency
|
### Cross-Chapter Consistency
|
||||||
|
|
||||||
Same speaker embedding used across all chapters → same voice identity.
|
Same cached vmatrix used across all chapters → same voice identity.
|
||||||
The narrator always sounds like the narrator. Kainda always sounds like Kainda.
|
The narrator always sounds like the narrator. Kainda always sounds like Kainda —
|
||||||
|
the voice can never silently change between chapters.
|
||||||
|
|
||||||
### Determinism
|
### Capture Discipline
|
||||||
|
|
||||||
With `seed=42` and the same embedding:
|
A voice is only the SAME voice while its cached `.pt` matches its approved reference WAV.
|
||||||
- Run 1 and Run 2 produce **identical waveforms** (mean abs diff = 0.000000)
|
When a `*_ref.wav` is regenerated/approved, the `.pt` MUST be rebuilt (the Approve flow
|
||||||
- Different seeds produce different phrasings but same voice identity
|
does this; `build_voice_prompt(..., force=True)` forces it) — otherwise chapter
|
||||||
|
generation silently uses the stale old voice. This is why "Kainda still had the old
|
||||||
|
voice": the `.pt` was built Aug 27, before the Aug 31 reference replacements.
|
||||||
|
|
||||||
## Chapter Generation Flow
|
## Chapter Generation Flow
|
||||||
|
|
||||||
1. **Preflight**: Verify TTS server is running, Base model loaded
|
1. **Preflight**: Verify TTS server is running, Base model loaded
|
||||||
2. **Validate**: Check which utterances need (re)generation
|
2. **Validate**: Check which utterances need (re)generation
|
||||||
3. **Generate**: For each utterance:
|
3. **Capture vmatrix**: For each distinct speaker with a missing/stale `.pt`,
|
||||||
- Load speaker embedding from `.npy` cache
|
`build_voice_prompt(vref, x_vector_only=False)` → cached `voice_prompts/{slug}.pt`
|
||||||
- Send to server with text + temperature + seed
|
4. **Generate**: For each utterance:
|
||||||
|
- Resolve speaker → prompt_slug (the cached vmatrix)
|
||||||
|
- Send to server with text + temperature + seed + prompt_slug
|
||||||
- Save WAV to `.umbra/chapter_audio/ch{N}/utt{M}.wav`
|
- Save WAV to `.umbra/chapter_audio/ch{N}/utt{M}.wav`
|
||||||
4. **Stitch**: Concatenate utterance WAVs into chapter WAV
|
5. **Stitch**: Concatenate utterance WAVs into chapter WAV
|
||||||
5. **Export**: Convert chapter WAV to MP3 via ffmpeg
|
6. **Export**: Convert chapter WAV to MP3 via ffmpeg
|
||||||
6. **Verify**: Run quality checks
|
7. **Verify**: Run quality checks
|
||||||
|
|
||||||
## Hardware Constraints
|
## Hardware Constraints
|
||||||
|
|
||||||
@@ -281,11 +316,14 @@ Maps character names to Edge-TTS voice identifiers (used for WAV ref generation)
|
|||||||
|
|
||||||
## Troubleshooting
|
## Troubleshooting
|
||||||
|
|
||||||
### Voice sounds wrong
|
### Voice sounds wrong / "it's still the old voice"
|
||||||
|
|
||||||
1. Check if embedding exists: `ls .umbra/voice_embeddings/{slug}.npy`
|
1. Check the voice is approved and has a vmatrix:
|
||||||
2. Regenerate embedding: `python scripts/extract_embeddings.py --speaker {name} --force`
|
`ls .umbra/voice_prompts/{slug}.pt` (slug = ref filename minus `_ref.wav`)
|
||||||
3. Verify with Rainbow Passage sample
|
2. If the `.pt` is missing or older than the ref WAV, rebuild it:
|
||||||
|
approve again via the web UI, or force: `build_voice_prompt(ref, ref_text=Rainbow, x_vector_only=False, force=True)`
|
||||||
|
3. Verify the approved reference WAV (`.umbra/voices/{slug}_ref.wav` — listen to it)
|
||||||
|
4. Only then regenerate the character's chapter utterances
|
||||||
|
|
||||||
### TTS server not responding
|
### TTS server not responding
|
||||||
|
|
||||||
@@ -302,6 +340,7 @@ curl -X POST http://localhost:8881/unload -d '{"model_type": "all"}'
|
|||||||
|
|
||||||
### Inconsistent voices across utterances
|
### Inconsistent voices across utterances
|
||||||
|
|
||||||
This was the old x-vector extraction problem. Now solved by cached embeddings.
|
Solved by the vmatrix: every utterance passes `prompt_slug` → the same cached `.pt`
|
||||||
If still occurring, ensure the caller is using cached embeddings (check logs for
|
(ref_code + embedding). If a character's voice still changes, the `.pt` is being
|
||||||
"Using cached embedding for ...").
|
rebuilt per utterance or is stale vs the approved ref — force-rebuild it once and
|
||||||
|
ensure generation uses `x_vector_only=False` (check logs for cache hits of the slug).
|
||||||
|
|||||||
@@ -257,22 +257,23 @@ async def build_voice_prompt(
|
|||||||
x_vector_only: int = Form(0),
|
x_vector_only: int = Form(0),
|
||||||
force: int = Form(0),
|
force: int = Form(0),
|
||||||
):
|
):
|
||||||
"""Build and cache a speaker's digital voice (a set of numbers) once.
|
"""Build and cache a speaker's vmatrix (digital voice) once.
|
||||||
|
|
||||||
Two capture modes (pick per pipeline; a speaker's cached prompt is used as-is
|
The vmatrix is the set of numbers inside Qwen3-TTS that IS the voice. Two capture
|
||||||
on every subsequent utterance):
|
modes (a speaker's cached prompt is reused on every subsequent utterance via
|
||||||
|
prompt_slug):
|
||||||
|
|
||||||
* x_vector_only=0 (default, ICL/"8k matrix"): extracts the FULL codec-token
|
* x_vector_only=0 (DEFAULT, canonical, "full vmatrix"): extracts the FULL
|
||||||
matrix (ref_code) + speaker embedding. Together those let the generator
|
ICL codec-token matrix (ref_code, the "8k matrix") + speaker embedding.
|
||||||
re-create the exact reference recording -- and every utterance therefore
|
Generation conditions on the exact reference identity, so every utterance
|
||||||
re-speaks the reference first, then trims it (slow + leaky).
|
is faithful AND identical in identity across the whole pipeline.
|
||||||
* x_vector_only=1: captures ONLY the speaker embedding (the digital voice
|
* x_vector_only=1: captures ONLY the coarse speaker embedding (ref_code=None).
|
||||||
variables). ref_code is None. Generation then feeds just those numbers +
|
Legacy/experimental - caused the historical "voice changed between
|
||||||
text into the speech generator -- no reference re-hash, no clone re-run,
|
chapters" inconsistency. Not used for production consistency.
|
||||||
fast and output is exactly the text.
|
|
||||||
|
|
||||||
Idempotent: if `<slug>.pt` is already cached it returns without rebuilding,
|
Idempotent: if `<slug>.pt` is already cached it returns without rebuilding,
|
||||||
unless `force=1` is passed (use force to switch an existing prompt mode).
|
unless `force=1` is passed (use force to switch an existing prompt mode, e.g.
|
||||||
|
after an approved reference WAV is replaced).
|
||||||
"""
|
"""
|
||||||
path = _prompt_path(slug)
|
path = _prompt_path(slug)
|
||||||
if path.exists() and not force:
|
if path.exists() and not force:
|
||||||
|
|||||||
@@ -248,6 +248,32 @@ This table is what later lets a Pass 3 "Always" promotion (`04-mapping.md`, "Spe
|
|||||||
|
|
||||||
Once approved, this is the "normal" voice referenced by everything downstream: Pass 3's punctuation-driven prosody hints modulate *around* it, and human-reviewed per-line descriptor overrides modulate around it too. Neither layer ever resets the baseline itself without going back through this same approval loop.
|
Once approved, this is the "normal" voice referenced by everything downstream: Pass 3's punctuation-driven prosody hints modulate *around* it, and human-reviewed per-line descriptor overrides modulate around it too. Neither layer ever resets the baseline itself without going back through this same approval loop.
|
||||||
|
|
||||||
|
### Approved Voice = Reference WAV + Captured Vmatrix
|
||||||
|
|
||||||
|
An approval now records **two artifacts** for the character (this is the requirement: a wav to *listen* to + the numbers inside the TTS that *are* the voice):
|
||||||
|
|
||||||
|
| Artifact | Path | Purpose |
|
||||||
|
|---|---|---|
|
||||||
|
| Reference WAV | `.umbra/voices/{slug}_ref.wav` | Listenable sample of the approved voice |
|
||||||
|
| **Vmatrix** (.pt) | `.umbra/voice_prompts/{slug}.pt` | The captured voice identity used for consistency |
|
||||||
|
|
||||||
|
The **vmatrix** is what Qwen3-TTS actually conditions on to reproduce a voice:
|
||||||
|
|
||||||
|
- `ref_code` — the ICL **"8k matrix"**: a `(T, 16)` tensor of codec tokens produced by re-speaking the reference WAV through the model's codec encoder.
|
||||||
|
- `ref_spk_embedding` — the 2048-dim extracted speaker x-vector.
|
||||||
|
|
||||||
|
It is captured once (server-side `POST /v1/voice-prompt`, `x_vector_only=0`) and cached
|
||||||
|
under `voice_prompts/{slug}.pt`. `POST /api/speakers/voice-status` **approve** calls
|
||||||
|
`build_voice_prompt(...)` and records `vmatrix_path` in the voice registry next to
|
||||||
|
`voice_ref_path`. Every subsequent utterance passes `prompt_slug={slug}` so it reuses
|
||||||
|
the exact same numbers — a character's voice can never drift or "change" between
|
||||||
|
chapters. If the reference WAV is ever replaced, the `.pt` must be rebuilt (approve
|
||||||
|
again, or `force=True`) or generation will silently use the stale voice.
|
||||||
|
|
||||||
|
Note on modes: `x_vector_only=True` (embedding-only) is the legacy-coarse capture and
|
||||||
|
must NOT be used for production consistency. The full vmatrix (`x_vector_only=False`) is
|
||||||
|
the canonical mode and the engine default.
|
||||||
|
|
||||||
### Skip Logic
|
### Skip Logic
|
||||||
|
|
||||||
Voice references are not regenerated if:
|
Voice references are not regenerated if:
|
||||||
|
|||||||
@@ -12,7 +12,8 @@ Pass 3 generates audio for every utterance in every chapter using **Qwen3-TTS**
|
|||||||
class QwenTTSEngine:
|
class QwenTTSEngine:
|
||||||
def __init__(self, device="cuda", sample_rate=24000, qwen_tts_url: str) -> None
|
def __init__(self, device="cuda", sample_rate=24000, qwen_tts_url: str) -> None
|
||||||
def load_remote_model(self) -> None
|
def load_remote_model(self) -> None
|
||||||
def generate(self, text: str, output_path: Path, voice_ref: Path | None = None, instruct: str = "") -> Path
|
def generate(self, text: str, output_path: Path, voice_ref: Path | None = None, instruct: str = "", prompt_slug: str | None = None) -> Path
|
||||||
|
def build_voice_prompt(self, voice_ref: Path, ref_text: str | None = None, slug: str | None = None, x_vector_only: bool = False, force: bool = False) -> str
|
||||||
def unload_all(self) -> None
|
def unload_all(self) -> None
|
||||||
```
|
```
|
||||||
|
|
||||||
@@ -20,8 +21,28 @@ class QwenTTSEngine:
|
|||||||
- Remote/on-prem synthesis via `qwen_tts_url` (LM Studio / Qwen server)
|
- Remote/on-prem synthesis via `qwen_tts_url` (LM Studio / Qwen server)
|
||||||
- Voice cloning from a reference WAV per character
|
- Voice cloning from a reference WAV per character
|
||||||
- Optional `instruct` text for prosody/emotion direction (e.g. rising question intonation)
|
- Optional `instruct` text for prosody/emotion direction (e.g. rising question intonation)
|
||||||
|
- **Vmatrix capture** (`build_voice_prompt`): computes the voice's "8k matrix" (ICL `ref_code` codec-token matrix) + speaker embedding, caches it server-side as `voice_prompts/{slug}.pt`, returns the `prompt_slug` to pass to `generate()`.
|
||||||
|
- **`prompt_slug` generation** (the consistency path): every utterance of a character passes the same slug, so the exact same cached vmatrix is reused — voice identity can never drift between utterances or chapters.
|
||||||
- WAV output at 24kHz sample rate
|
- WAV output at 24kHz sample rate
|
||||||
|
|
||||||
|
> **`x_vector_only` default is `False` (full vmatrix).** The full capture (ref_code + embedding) is the canonical mode; the embed-only mode (`True`) is legacy/experimental and caused the historical "voice changed between chapters" problem.
|
||||||
|
|
||||||
|
### Vmatrix Capture in the Generation Flow
|
||||||
|
|
||||||
|
Because `build_voice_prompt` defaults to `/v1/voice-prompt` server semantics, the bulk
|
||||||
|
generator captures each distinct speaker's vmatrix once per run (before any utterance is
|
||||||
|
generated) and reuses the slug for all of that speaker's utterances:
|
||||||
|
|
||||||
|
```
|
||||||
|
for each distinct speaker with a missing .pt:
|
||||||
|
prompt_slug = build_voice_prompt(vref, ref_text=VOICE_REFERENCE_TEXT, x_vector_only=False)
|
||||||
|
# server writes/updates voice_prompts/{slug}.pt = {ref_code (T,16), ref_spk_embedding (2048,), icl_mode: True}
|
||||||
|
```
|
||||||
|
|
||||||
|
The `.pt` cache is the same filesystem location the server writes to (`PROMPT_DIR`,
|
||||||
|
default `.umbra/voice_prompts`), so it persists across runs and is shipped to the GPU
|
||||||
|
host via `tts_server/sync_voice_data.sh`.
|
||||||
|
|
||||||
### Edge-TTS (`voices/edge_tts.py`) — Test Samples & Fallback
|
### Edge-TTS (`voices/edge_tts.py`) — Test Samples & Fallback
|
||||||
|
|
||||||
```python
|
```python
|
||||||
@@ -58,6 +79,11 @@ def resolve_reference(
|
|||||||
|
|
||||||
```
|
```
|
||||||
For each chapter:
|
For each chapter:
|
||||||
|
│
|
||||||
|
├── 0. Capture vmatrix for each distinct speaker (once per run)
|
||||||
|
│ prompt_slug = engine.build_voice_prompt(vref, ref_text=VOICE_REFERENCE_TEXT,
|
||||||
|
│ x_vector_only=False) # full "8k matrix" + embedding
|
||||||
|
│ # → voice_prompts/{slug}.pt (cached; missing only when the ref changed)
|
||||||
│
|
│
|
||||||
├── 1. Load utterances from DB
|
├── 1. Load utterances from DB
|
||||||
│ SELECT id, speaker_name, text, utterance_index
|
│ SELECT id, speaker_name, text, utterance_index
|
||||||
@@ -72,11 +98,12 @@ For each chapter:
|
|||||||
├── Skip if file exists
|
├── Skip if file exists
|
||||||
│ if output_path.exists(): continue
|
│ if output_path.exists(): continue
|
||||||
│
|
│
|
||||||
├── Resolve voice reference
|
├── Resolve voice reference → prompt_slug
|
||||||
│ ref_path = resolve_reference(speaker, voice_refs, alias_map, narrator_ref)
|
│ ref_path = resolve_reference(speaker, voice_refs, alias_map, narrator_ref)
|
||||||
|
│ slug = engine.slug_for_ref(ref_path)
|
||||||
│
|
│
|
||||||
├── Generate audio (Qwen3-TTS)
|
├── Generate audio (Qwen3-TTS) — identical cached vmatrix for the speaker
|
||||||
│ engine.generate(text, output_path, voice_ref=ref_path)
|
│ engine.generate(text, output_path, prompt_slug=slug)
|
||||||
│ # no ref → engine.generate(text, output_path, instruct=neutral)
|
│ # no ref → engine.generate(text, output_path, instruct=neutral)
|
||||||
│
|
│
|
||||||
├── Verify output file exists (else count as failed)
|
├── Verify output file exists (else count as failed)
|
||||||
@@ -85,6 +112,10 @@ For each chapter:
|
|||||||
Log error, continue to next utterance
|
Log error, continue to next utterance
|
||||||
```
|
```
|
||||||
|
|
||||||
|
Note the two-layer design: step 0 captures the vmatrix (a set of numbers) **once** per
|
||||||
|
speaker; step 3 reuses that identical capture for every utterance via `prompt_slug`.
|
||||||
|
This is what guarantees a character's voice never drifts across a chapter or the book.
|
||||||
|
|
||||||
**Engine mode:** the engine uses remote synthesis when `cfg.tts.qwen_tts_url` is set (the default deployment). The remote server holds the VoiceDesign and Base models loaded and answers `/v1/tts`; the local `QwenTTSEngine` only loads models itself when no URL is configured.
|
**Engine mode:** the engine uses remote synthesis when `cfg.tts.qwen_tts_url` is set (the default deployment). The remote server holds the VoiceDesign and Base models loaded and answers `/v1/tts`; the local `QwenTTSEngine` only loads models itself when no URL is configured.
|
||||||
|
|
||||||
### Output Structure
|
### Output Structure
|
||||||
@@ -178,7 +209,7 @@ Success rate is reported per run as `generated / failed`; failed utterances can
|
|||||||
|
|
||||||
1. **Qwen3-TTS over Edge-TTS**: cloning from per-character reference samples gives stable, character-accurate voices; Edge-TTS is retained only for test samples and the `_generate_edge_fallback` path in the voice designer
|
1. **Qwen3-TTS over Edge-TTS**: cloning from per-character reference samples gives stable, character-accurate voices; Edge-TTS is retained only for test samples and the `_generate_edge_fallback` path in the voice designer
|
||||||
2. **Remote synthesis**: the Qwen server (`qwen_tts_url`) hosts the models, so bulk generation doesn't fight the LLM (LM Studio) for local GPU/VRAM
|
2. **Remote synthesis**: the Qwen server (`qwen_tts_url`) hosts the models, so bulk generation doesn't fight the LLM (LM Studio) for local GPU/VRAM
|
||||||
3. **Voice references over voice strings**: identity comes from `*_ref.wav` cloning instead of `voice_assignments.json` voice strings with pitch shifts
|
3. **Vmatrix identity from references**: identity comes from the **captured vmatrix** of each `*_ref.wav` (ICL codec-token "8k matrix" + speaker embedding, cached as `voice_prompts/{slug}.pt`), not from `voice_assignments.json` voice strings with pitch shifts, and not from a coarse embedding-only capture. Every utterance reuses the same `.pt` via `prompt_slug`.
|
||||||
4. **Sequential generation**: avoids overloading the remote server and simplifies resumability
|
4. **Sequential generation**: avoids overloading the remote server and simplifies resumability
|
||||||
5. **File-based caching**: skip already-generated files; regeneration after edits is file-targeted
|
5. **File-based caching**: skip already-generated files; regeneration after edits is file-targeted
|
||||||
6. **Error resilience**: individual utterance failures never abort a chapter run; a final summary counts successes and failures
|
6. **Error resilience**: individual utterance failures never abort a chapter run; a final summary counts successes and failures
|
||||||
|
|||||||
113
tests/test_library.py
Normal file
113
tests/test_library.py
Normal file
@@ -0,0 +1,113 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
from umbraaudio import library
|
||||||
|
|
||||||
|
|
||||||
|
def _speaker(name: str = "Distorted Voice", **overrides) -> dict:
|
||||||
|
sp = {
|
||||||
|
"primary_name": name,
|
||||||
|
"aliases": [],
|
||||||
|
"gender": "Female",
|
||||||
|
"accent": "canadian",
|
||||||
|
"vocal_traits": ["raspy"],
|
||||||
|
"distortion_effects": ["hard_clip"],
|
||||||
|
"is_narrator": False,
|
||||||
|
"is_group": False,
|
||||||
|
"description": "test speaker",
|
||||||
|
}
|
||||||
|
sp.update(overrides)
|
||||||
|
return sp
|
||||||
|
|
||||||
|
|
||||||
|
class TestAddSpeakers:
|
||||||
|
def test_adds_and_copies_voice_ref(self, temp_dir: Path):
|
||||||
|
db = temp_dir / "voice_registry.db"
|
||||||
|
voices = temp_dir / "libvoices"
|
||||||
|
voices.mkdir(parents=True)
|
||||||
|
|
||||||
|
src = temp_dir / "src_distorted_voice_ref.wav"
|
||||||
|
src.write_bytes(b"RIFFDATAFORVOICE")
|
||||||
|
|
||||||
|
res = library.add_speakers(
|
||||||
|
[_speaker()],
|
||||||
|
source_project="TestProject",
|
||||||
|
voice_refs={"Distorted Voice": src},
|
||||||
|
db_path=db,
|
||||||
|
voices_dir=voices,
|
||||||
|
)
|
||||||
|
assert res["added"] == ["Distorted Voice"]
|
||||||
|
assert (voices / "distorted_voice_ref.wav").exists()
|
||||||
|
assert (voices / "distorted_voice_ref.wav").read_bytes() == b"RIFFDATAFORVOICE"
|
||||||
|
|
||||||
|
rows = library.list_speakers(db_path=db)
|
||||||
|
assert len(rows) == 1
|
||||||
|
assert rows[0]["primary_name"] == "Distorted Voice"
|
||||||
|
assert "distorted_voice_ref.wav" in rows[0]["voice_ref_path"]
|
||||||
|
|
||||||
|
def test_idempotent_duplicate(self, temp_dir: Path):
|
||||||
|
db = temp_dir / "voice_registry.db"
|
||||||
|
voices = temp_dir / "lv"
|
||||||
|
voices.mkdir(parents=True)
|
||||||
|
src = temp_dir / "narrator_ref.wav"
|
||||||
|
src.write_bytes(b"REF")
|
||||||
|
|
||||||
|
library.add_speakers(
|
||||||
|
[_speaker("Narrator", is_narrator=True)],
|
||||||
|
"P", {"Narrator": src}, db_path=db, voices_dir=voices,
|
||||||
|
)
|
||||||
|
res = library.add_speakers(
|
||||||
|
[_speaker("Narrator", is_narrator=True)],
|
||||||
|
"P", {"Narrator": src}, db_path=db, voices_dir=voices,
|
||||||
|
)
|
||||||
|
assert res["already"] == ["Narrator"]
|
||||||
|
assert len(library.list_speakers(db_path=db)) == 1
|
||||||
|
|
||||||
|
def test_skips_speaker_without_ref(self, temp_dir: Path):
|
||||||
|
db = temp_dir / "voice_registry.db"
|
||||||
|
res = library.add_speakers(
|
||||||
|
[_speaker()], "P", {}, db_path=db, voices_dir=temp_dir,
|
||||||
|
)
|
||||||
|
assert res["added"] == ["Distorted Voice"]
|
||||||
|
assert library.list_speakers(db_path=db)[0]["voice_ref_path"] is None
|
||||||
|
|
||||||
|
|
||||||
|
class TestAddNames:
|
||||||
|
def test_adds_records(self, temp_dir: Path):
|
||||||
|
db = temp_dir / "voice_registry.db"
|
||||||
|
res = library.add_names(
|
||||||
|
[{"name": "Kainda", "phonetic": "KAYN-dah", "source": "user",
|
||||||
|
"common": False, "confirmed": True}],
|
||||||
|
source_project="P",
|
||||||
|
db_path=db,
|
||||||
|
)
|
||||||
|
assert res["added"] == ["Kainda"]
|
||||||
|
rows = library.list_names(db_path=db)
|
||||||
|
assert rows[0]["name"] == "Kainda"
|
||||||
|
assert rows[0]["phonetic"] == "KAYN-dah"
|
||||||
|
assert rows[0]["confirmed"] == 1
|
||||||
|
|
||||||
|
def test_idempotent_duplicate(self, temp_dir: Path):
|
||||||
|
db = temp_dir / "voice_registry.db"
|
||||||
|
rec = {"name": "Kainda", "phonetic": "KAYN-dah"}
|
||||||
|
library.add_names([rec], "P", db_path=db)
|
||||||
|
res = library.add_names([rec], "P", db_path=db)
|
||||||
|
assert res["already"] == ["Kainda"]
|
||||||
|
assert len(library.list_names(db_path=db)) == 1
|
||||||
|
|
||||||
|
|
||||||
|
class TestDelete:
|
||||||
|
def test_delete_speaker(self, temp_dir: Path):
|
||||||
|
db = temp_dir / "voice_registry.db"
|
||||||
|
library.add_speakers([_speaker()], "P", {}, db_path=db, voices_dir=temp_dir)
|
||||||
|
sid = library.list_speakers(db_path=db)[0]["id"]
|
||||||
|
assert library.delete_speaker(sid, db_path=db) is True
|
||||||
|
assert library.list_speakers(db_path=db) == []
|
||||||
|
|
||||||
|
def test_delete_name(self, temp_dir: Path):
|
||||||
|
db = temp_dir / "voice_registry.db"
|
||||||
|
library.add_names([{"name": "Kainda"}], "P", db_path=db)
|
||||||
|
nid = library.list_names(db_path=db)[0]["id"]
|
||||||
|
assert library.delete_name(nid, db_path=db) is True
|
||||||
|
assert library.list_names(db_path=db) == []
|
||||||
@@ -144,3 +144,67 @@ async def test_link_to_project(registry_db_path: Path, character_profile: Charac
|
|||||||
await registry.link_to_project(reg_id, 1, role_in_book="Protagonist")
|
await registry.link_to_project(reg_id, 1, role_in_book="Protagonist")
|
||||||
await registry.link_to_project(reg_id, 2)
|
await registry.link_to_project(reg_id, 2)
|
||||||
await registry.close()
|
await registry.close()
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_set_vmatrix(registry_db_path: Path, character_profile: CharacterProfile):
|
||||||
|
registry = VoiceRegistry(registry_db_path)
|
||||||
|
await registry.connect()
|
||||||
|
reg_id, _ = await registry.find_or_create(character_profile)
|
||||||
|
await registry.set_vmatrix(reg_id, "/umbra/voice_prompts/nanny.pt")
|
||||||
|
found = await registry.find_by_name("John Smith")
|
||||||
|
assert found is not None
|
||||||
|
assert found["vmatrix_path"] == "/umbra/voice_prompts/nanny.pt"
|
||||||
|
await registry.close()
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_migrate_adds_vmatrix_column(
|
||||||
|
registry_db_path: Path, character_profile: CharacterProfile
|
||||||
|
):
|
||||||
|
"""A DB created by the previous schema (no vmatrix_path) gains the column."""
|
||||||
|
|
||||||
|
legacy_schema = """
|
||||||
|
CREATE TABLE voice_registry (
|
||||||
|
id INTEGER PRIMARY KEY,
|
||||||
|
character_name TEXT NOT NULL UNIQUE,
|
||||||
|
aliases TEXT NOT NULL DEFAULT '[]',
|
||||||
|
gender TEXT, age_range TEXT,
|
||||||
|
pitch_register TEXT DEFAULT 'mid',
|
||||||
|
pace_baseline TEXT DEFAULT 'moderate',
|
||||||
|
energy TEXT DEFAULT 'measured',
|
||||||
|
timbre TEXT DEFAULT 'clear',
|
||||||
|
texture TEXT DEFAULT 'smooth',
|
||||||
|
formality TEXT DEFAULT 'conversational',
|
||||||
|
emotional_baseline TEXT DEFAULT 'neutral_calm',
|
||||||
|
distinguishing_trait TEXT,
|
||||||
|
accent_marker TEXT DEFAULT 'none',
|
||||||
|
voice_description TEXT,
|
||||||
|
voice_ref_path TEXT,
|
||||||
|
sample_audio_path TEXT,
|
||||||
|
speaker_embedding BLOB,
|
||||||
|
source_project TEXT,
|
||||||
|
is_narrator INTEGER NOT NULL DEFAULT 0,
|
||||||
|
is_general INTEGER NOT NULL DEFAULT 0,
|
||||||
|
status TEXT NOT NULL DEFAULT 'pending',
|
||||||
|
created_at TEXT NOT NULL DEFAULT (datetime('now')),
|
||||||
|
updated_at TEXT NOT NULL DEFAULT (datetime('now'))
|
||||||
|
);
|
||||||
|
"""
|
||||||
|
import aiosqlite
|
||||||
|
|
||||||
|
conn = await aiosqlite.connect(str(registry_db_path))
|
||||||
|
await conn.executescript(legacy_schema)
|
||||||
|
await conn.commit()
|
||||||
|
await conn.close()
|
||||||
|
|
||||||
|
registry = VoiceRegistry(registry_db_path)
|
||||||
|
await registry.connect()
|
||||||
|
cursor = await registry.conn.execute("PRAGMA table_info(voice_registry)")
|
||||||
|
cols = {row["name"] for row in await cursor.fetchall()}
|
||||||
|
assert "vmatrix_path" in cols
|
||||||
|
reg_id, _ = await registry.find_or_create(character_profile)
|
||||||
|
await registry.set_vmatrix(reg_id, "/umbra/voice_prompts/nanny.pt")
|
||||||
|
found = await registry.find_by_name("John Smith")
|
||||||
|
assert found["vmatrix_path"] == "/umbra/voice_prompts/nanny.pt"
|
||||||
|
await registry.close()
|
||||||
|
|||||||
@@ -7,6 +7,22 @@ Mac (where MPS inference is prohibitively slow).
|
|||||||
All the pipeline code (`QwenTTSEngine`, `ChapterBuilder`) talks to this server
|
All the pipeline code (`QwenTTSEngine`, `ChapterBuilder`) talks to this server
|
||||||
**unchanged** — it exposes the exact same endpoints as the local deployment.
|
**unchanged** — it exposes the exact same endpoints as the local deployment.
|
||||||
|
|
||||||
|
## The vmatrix (voice identity)
|
||||||
|
|
||||||
|
A character's voice = a reference WAV (**what you listen to**) + a **vmatrix** (the
|
||||||
|
set of numbers inside the TTS that *is* the voice, used for consistency). The vmatrix
|
||||||
|
is captured by `POST /v1/voice-prompt` and cached as `<slug>.pt` in `QWEN_PROMPT_DIR`:
|
||||||
|
|
||||||
|
- `ref_code` — the ICL **"8k matrix"**: a `(T, 16)` tensor of codec tokens produced by
|
||||||
|
re-speaking the reference WAV through the model's codec encoder
|
||||||
|
- `ref_spk_embedding` — the 2048-dim extracted speaker x-vector
|
||||||
|
|
||||||
|
**`x_vector_only=0` (the default) captures the FULL vmatrix** (ref_code + embedding,
|
||||||
|
`icl_mode=True`) — faithful and perfectly consistent. `x_vector_only=1` captures only
|
||||||
|
the sparse embedding (`ref_code=None`) and is legacy/experimental; the pipeline client
|
||||||
|
(`QwenTTSEngine.build_voice_prompt`) defaults to the full vmatrix and passes it on every
|
||||||
|
utterance as `prompt_slug`, so a character's voice can never drift between chapters.
|
||||||
|
|
||||||
## Contents
|
## Contents
|
||||||
|
|
||||||
- `qwen_tts_server.py` — the FastAPI server (port 8881, `/health`,
|
- `qwen_tts_server.py` — the FastAPI server (port 8881, `/health`,
|
||||||
@@ -47,6 +63,20 @@ curl -X POST http://<gpu-ip>:8881/v1/voice-prompt ... # build a prompt
|
|||||||
curl -X POST http://<gpu-ip>:8881/v1/tts ... # generate (prompt_slug)
|
curl -X POST http://<gpu-ip>:8881/v1/tts ... # generate (prompt_slug)
|
||||||
```
|
```
|
||||||
|
|
||||||
|
## `POST /v1/voice-prompt` — capture the vmatrix
|
||||||
|
|
||||||
|
| form field | default | meaning |
|
||||||
|
|---|---|---|
|
||||||
|
| `slug` | required | cache key; stored as `{QWEN_PROMPT_DIR}/{slug}.pt` (same key the client passes as `prompt_slug`) |
|
||||||
|
| `ref_audio` | required (file) | the approved reference WAV for the speaker |
|
||||||
|
| `ref_text` | Rainbow Passage | reference transcript used to rebuild the `ref_code` "8k matrix" |
|
||||||
|
| `x_vector_only` | `0` | `0` = FULL vmatrix (`ref_code` + `ref_spk_embedding`, `icl_mode=True`); `1` = embedding-only (`ref_code=None`) |
|
||||||
|
| `force` | `0` | `1` = rebuild even if `<slug>.pt` already cached |
|
||||||
|
|
||||||
|
Idempotent: if `<slug>.pt` exists and `force` is unset, it returns the cached prompt
|
||||||
|
without re-extracting. The client's `build_voice_prompt` defaults to the full vmatrix
|
||||||
|
(`x_vector_only=False`) and calls this once per distinct speaker per generation run.
|
||||||
|
|
||||||
## Point the pipeline at it
|
## Point the pipeline at it
|
||||||
|
|
||||||
The engine already reads the TTS URL from config (`qwen_tts_url`, default
|
The engine already reads the TTS URL from config (`qwen_tts_url`, default
|
||||||
@@ -64,12 +94,17 @@ QWEN_DEVICE=cuda ./run_server.sh # on GPU host, binds 0.0.0.0:8881
|
|||||||
| `QWEN_DEVICE` | `cuda` | `cuda`/`mps`/`cpu` |
|
| `QWEN_DEVICE` | `cuda` | `cuda`/`mps`/`cpu` |
|
||||||
| `QWEN_BASE_MODEL` | `models/...-1.7B-Base` | voice-clone / ICL model |
|
| `QWEN_BASE_MODEL` | `models/...-1.7B-Base` | voice-clone / ICL model |
|
||||||
| `QWEN_VOICEDESIGN_MODEL` | `models/...-1.7B-VoiceDesign` | voice-design model |
|
| `QWEN_VOICEDESIGN_MODEL` | `models/...-1.7B-VoiceDesign` | voice-design model |
|
||||||
| `QWEN_PROMPT_DIR` | `./voice_prompts` | cached ICL `.pt` prompts |
|
| `QWEN_PROMPT_DIR` | `./voice_prompts` | cached vmatrix `.pt` prompts |
|
||||||
| `QWEN_ATTN` | `sdpa` | `sdpa` (fast, built-in) / `eager` (slow) / `flash_attention_2` |
|
| `QWEN_ATTN` | `sdpa` | `sdpa` (fast, built-in) / `eager` (slow) / `flash_attention_2` |
|
||||||
| `QWEN_MAX_GEN_S` | `1800` | watchdog: if a generation hangs this long, the server auto-restarts itself |
|
| `QWEN_MAX_GEN_S` | `1800` | watchdog: if a generation hangs this long, the server auto-restarts itself |
|
||||||
| `QWEN_PORT` | `8881` | listen port |
|
| `QWEN_PORT` | `8881` | listen port |
|
||||||
| `QWEN_HOST` | `0.0.0.0` | bind address |
|
| `QWEN_HOST` | `0.0.0.0` | bind address |
|
||||||
| `QWEN_PROMPT_DIR` | `./voice_prompts` | cached prompts |
|
|
||||||
|
> **Deployment note:** in the live deployment (`192.168.123.14:8881`) `QWEN_PROMPT_DIR`
|
||||||
|
> points at the **project's** `.umbra/voice_prompts` on the shared volume, and
|
||||||
|
> `sync_voice_data.sh` ships both `voices/*_ref.wav` and `voice_prompts/*.pt` to the GPU
|
||||||
|
> box. That shared location is why Approve/build_voice_prompt immediately makes the
|
||||||
|
> vmatrix available to generation without extra steps.
|
||||||
|
|
||||||
> Flash-attn is recommended but not required. With no flash-attn the server sets
|
> Flash-attn is recommended but not required. With no flash-attn the server sets
|
||||||
> `attn_implementation="sdpa"` (the default) for generation, which is fast and
|
> `attn_implementation="sdpa"` (the default) for generation, which is fast and
|
||||||
|
|||||||
@@ -257,22 +257,23 @@ async def build_voice_prompt(
|
|||||||
x_vector_only: int = Form(0),
|
x_vector_only: int = Form(0),
|
||||||
force: int = Form(0),
|
force: int = Form(0),
|
||||||
):
|
):
|
||||||
"""Build and cache a speaker's digital voice (a set of numbers) once.
|
"""Build and cache a speaker's vmatrix (digital voice) once.
|
||||||
|
|
||||||
Two capture modes (pick per pipeline; a speaker's cached prompt is used as-is
|
The vmatrix is the set of numbers inside Qwen3-TTS that IS the voice. Two capture
|
||||||
on every subsequent utterance):
|
modes (a speaker's cached prompt is reused on every subsequent utterance via
|
||||||
|
prompt_slug):
|
||||||
|
|
||||||
* x_vector_only=0 (default, ICL/"8k matrix"): extracts the FULL codec-token
|
* x_vector_only=0 (DEFAULT, canonical, "full vmatrix"): extracts the FULL
|
||||||
matrix (ref_code) + speaker embedding. Together those let the generator
|
ICL codec-token matrix (ref_code, the "8k matrix") + speaker embedding.
|
||||||
re-create the exact reference recording -- and every utterance therefore
|
Generation conditions on the exact reference identity, so every utterance
|
||||||
re-speaks the reference first, then trims it (slow + leaky).
|
is faithful AND identical in identity across the whole pipeline.
|
||||||
* x_vector_only=1: captures ONLY the speaker embedding (the digital voice
|
* x_vector_only=1: captures ONLY the coarse speaker embedding (ref_code=None).
|
||||||
variables). ref_code is None. Generation then feeds just those numbers +
|
Legacy/experimental - caused the historical "voice changed between
|
||||||
text into the speech generator -- no reference re-hash, no clone re-run,
|
chapters" inconsistency. Not used for production consistency.
|
||||||
fast and output is exactly the text.
|
|
||||||
|
|
||||||
Idempotent: if `<slug>.pt` is already cached it returns without rebuilding,
|
Idempotent: if `<slug>.pt` is already cached it returns without rebuilding,
|
||||||
unless `force=1` is passed (use force to switch an existing prompt mode).
|
unless `force=1` is passed (use force to switch an existing prompt mode, e.g.
|
||||||
|
after an approved reference WAV is replaced).
|
||||||
"""
|
"""
|
||||||
path = _prompt_path(slug)
|
path = _prompt_path(slug)
|
||||||
if path.exists() and not force:
|
if path.exists() and not force:
|
||||||
|
|||||||
@@ -16,6 +16,7 @@ from __future__ import annotations
|
|||||||
|
|
||||||
import json
|
import json
|
||||||
import logging
|
import logging
|
||||||
|
import os
|
||||||
import sqlite3
|
import sqlite3
|
||||||
import subprocess
|
import subprocess
|
||||||
import threading
|
import threading
|
||||||
@@ -153,6 +154,12 @@ class ChapterBuilder:
|
|||||||
self.use_seed = use_seed
|
self.use_seed = use_seed
|
||||||
self.max_retries = max_retries
|
self.max_retries = max_retries
|
||||||
self.tts_health_interval = tts_health_interval
|
self.tts_health_interval = tts_health_interval
|
||||||
|
# Generation mode for cached voice prompts. Full-vmatrix ICL generation
|
||||||
|
# (prompt_slug -> cached ref_code) is measured at >15 min/utterance on
|
||||||
|
# the GPU box; embedding-mode generation (speaker embedding extracted
|
||||||
|
# from the current approved reference wav) is ~17s/utterance with the
|
||||||
|
# same voice identity. Default ON; disable via env to force slug mode.
|
||||||
|
self.use_embedding_mode = os.environ.get("UMBRA_TTS_EMBEDDING_MODE", "1") != "0"
|
||||||
|
|
||||||
self.data_dir: Path = config.data_dir
|
self.data_dir: Path = config.data_dir
|
||||||
self.status_path: Path = (
|
self.status_path: Path = (
|
||||||
@@ -248,17 +255,37 @@ class ChapterBuilder:
|
|||||||
# TTS server reachable
|
# TTS server reachable
|
||||||
import httpx
|
import httpx
|
||||||
|
|
||||||
|
url = f"{self.config.tts.qwen_tts_url}/health"
|
||||||
|
health = None
|
||||||
try:
|
try:
|
||||||
r = httpx.get(f"{self.config.tts.qwen_tts_url}/health", timeout=10)
|
r = httpx.get(url, timeout=10)
|
||||||
if r.status_code >= 500:
|
if r.status_code >= 500:
|
||||||
errors.append(f"TTS server returned HTTP {r.status_code}")
|
errors.append(f"TTS server returned HTTP {r.status_code}")
|
||||||
else:
|
else:
|
||||||
health = r.json()
|
health = r.json()
|
||||||
if not health.get("base_loaded"):
|
|
||||||
errors.append("TTS base model not loaded")
|
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
errors.append(f"TTS server unreachable: {e}")
|
errors.append(f"TTS server unreachable: {e}")
|
||||||
|
|
||||||
|
# Auto-load the TTS base model if it isn't already loaded so that the
|
||||||
|
# webUI chapter build is self-sufficient (no manual pre-load step).
|
||||||
|
if not errors and health is not None and not health.get("base_loaded"):
|
||||||
|
load_url = f"{self.config.tts.qwen_tts_url}/load"
|
||||||
|
try:
|
||||||
|
self._set_phase("preflight", "Loading TTS base model...", 3)
|
||||||
|
rr = httpx.post(load_url, json={"model_type": "base"}, timeout=600)
|
||||||
|
if rr.status_code >= 500:
|
||||||
|
errors.append(f"TTS server failed to load base model (HTTP {rr.status_code})")
|
||||||
|
else:
|
||||||
|
hr = httpx.get(url, timeout=10)
|
||||||
|
health = hr.json() if hr.status_code < 500 else {}
|
||||||
|
except Exception as e:
|
||||||
|
errors.append(f"Failed to load TTS base model: {e}")
|
||||||
|
if not errors and not health.get("base_loaded"):
|
||||||
|
errors.append("TTS base model not loaded")
|
||||||
|
|
||||||
|
if not errors and not health.get("base_loaded"):
|
||||||
|
errors.append("TTS base model not loaded")
|
||||||
|
|
||||||
# Chapter has utterances
|
# Chapter has utterances
|
||||||
db = sqlite3.connect(str(self.data_dir / "project.db"))
|
db = sqlite3.connect(str(self.data_dir / "project.db"))
|
||||||
db.row_factory = sqlite3.Row
|
db.row_factory = sqlite3.Row
|
||||||
@@ -405,9 +432,13 @@ class ChapterBuilder:
|
|||||||
|
|
||||||
engine = QwenTTSEngine(qwen_tts_url=self.config.tts.qwen_tts_url)
|
engine = QwenTTSEngine(qwen_tts_url=self.config.tts.qwen_tts_url)
|
||||||
|
|
||||||
# Build the ICL voice-clone prompt (full ref_code "8k matrix" + embedding)
|
# Capture the full vmatrix (the "8k matrix" ref_code + speaker embedding)
|
||||||
# once per distinct speaker, so every utterance reuses the identical voice
|
# once per distinct speaker, cached server-side as .pt under PROMPT_DIR.
|
||||||
# identity derived from the speaker-page reference wav.
|
# x_vector_only=False (explicit, and the engine default) captures the FULL
|
||||||
|
# vmatrix so the approved voice's exact numbers are persisted as the
|
||||||
|
# canonical record. Generation reuses that identity via embedding-mode
|
||||||
|
# (~17s/utt); full-vmatrix ICL generation is >15 min/utt, see
|
||||||
|
# use_embedding_mode.
|
||||||
prompt_map: dict[str, str] = {}
|
prompt_map: dict[str, str] = {}
|
||||||
for u in missing:
|
for u in missing:
|
||||||
spk = (u.get("speaker_name") or "narrator").strip()
|
spk = (u.get("speaker_name") or "narrator").strip()
|
||||||
@@ -419,6 +450,7 @@ class ChapterBuilder:
|
|||||||
prompt_map[spk] = engine.build_voice_prompt(
|
prompt_map[spk] = engine.build_voice_prompt(
|
||||||
vref,
|
vref,
|
||||||
ref_text=VOICE_REFERENCE_TEXT,
|
ref_text=VOICE_REFERENCE_TEXT,
|
||||||
|
x_vector_only=False,
|
||||||
)
|
)
|
||||||
|
|
||||||
self._start_health_monitor()
|
self._start_health_monitor()
|
||||||
@@ -472,7 +504,18 @@ class ChapterBuilder:
|
|||||||
|
|
||||||
try:
|
try:
|
||||||
prompt_slug = prompt_map.get(speaker)
|
prompt_slug = prompt_map.get(speaker)
|
||||||
if prompt_slug:
|
voice_mode = None
|
||||||
|
if voice_ref and self.use_embedding_mode:
|
||||||
|
engine.generate(
|
||||||
|
text=spoken_text,
|
||||||
|
output_path=out,
|
||||||
|
voice_ref=voice_ref,
|
||||||
|
ref_text=VOICE_REFERENCE_TEXT,
|
||||||
|
temperature=self.temperature,
|
||||||
|
seed=seed,
|
||||||
|
)
|
||||||
|
voice_mode = "embedding"
|
||||||
|
elif prompt_slug:
|
||||||
engine.generate(
|
engine.generate(
|
||||||
text=spoken_text,
|
text=spoken_text,
|
||||||
output_path=out,
|
output_path=out,
|
||||||
@@ -480,6 +523,7 @@ class ChapterBuilder:
|
|||||||
temperature=self.temperature,
|
temperature=self.temperature,
|
||||||
seed=seed,
|
seed=seed,
|
||||||
)
|
)
|
||||||
|
voice_mode = "icl"
|
||||||
elif voice_ref:
|
elif voice_ref:
|
||||||
engine.generate(
|
engine.generate(
|
||||||
text=spoken_text,
|
text=spoken_text,
|
||||||
@@ -489,6 +533,7 @@ class ChapterBuilder:
|
|||||||
temperature=self.temperature,
|
temperature=self.temperature,
|
||||||
seed=seed,
|
seed=seed,
|
||||||
)
|
)
|
||||||
|
voice_mode = "embedding"
|
||||||
else:
|
else:
|
||||||
engine.generate(
|
engine.generate(
|
||||||
text=spoken_text,
|
text=spoken_text,
|
||||||
@@ -510,7 +555,7 @@ class ChapterBuilder:
|
|||||||
write_marker(
|
write_marker(
|
||||||
spoken_text, out,
|
spoken_text, out,
|
||||||
speaker=speaker, voice_ref=voice_ref,
|
speaker=speaker, voice_ref=voice_ref,
|
||||||
voice_mode="icl" if prompt_map.get(speaker) else None,
|
voice_mode=voice_mode,
|
||||||
)
|
)
|
||||||
success = True
|
success = True
|
||||||
break
|
break
|
||||||
|
|||||||
253
umbraaudio/library.py
Normal file
253
umbraaudio/library.py
Normal file
@@ -0,0 +1,253 @@
|
|||||||
|
"""Cross-project voice/name library.
|
||||||
|
|
||||||
|
The Library stores speakers (with their voice reference audio), confirmed
|
||||||
|
proper-name pronunciations, and (future) reusable utterances so they can be
|
||||||
|
used on other audiobook projects. Data lives in the shared
|
||||||
|
``~/.umbra/voice_registry.db`` (same database as the Voice Registry) and the
|
||||||
|
copied voice samples live in ``~/.umbra/library/voices/``.
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
import shutil
|
||||||
|
import sqlite3
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
DEFAULT_LIBRARY_DB = Path.home() / ".umbra" / "voice_registry.db"
|
||||||
|
DEFAULT_LIBRARY_VOICES = Path.home() / ".umbra" / "library" / "voices"
|
||||||
|
|
||||||
|
LIBRARY_SCHEMA = """
|
||||||
|
CREATE TABLE IF NOT EXISTS library_speakers (
|
||||||
|
id INTEGER PRIMARY KEY,
|
||||||
|
primary_name TEXT NOT NULL,
|
||||||
|
aliases TEXT NOT NULL DEFAULT '[]',
|
||||||
|
gender TEXT,
|
||||||
|
accent TEXT,
|
||||||
|
vocal_traits TEXT NOT NULL DEFAULT '[]',
|
||||||
|
distortion_effects TEXT NOT NULL DEFAULT '[]',
|
||||||
|
is_narrator INTEGER NOT NULL DEFAULT 0,
|
||||||
|
is_group INTEGER NOT NULL DEFAULT 0,
|
||||||
|
description TEXT,
|
||||||
|
source_project TEXT,
|
||||||
|
voice_ref_path TEXT,
|
||||||
|
created_at TEXT NOT NULL DEFAULT (datetime('now')),
|
||||||
|
UNIQUE(primary_name)
|
||||||
|
);
|
||||||
|
CREATE TABLE IF NOT EXISTS library_names (
|
||||||
|
id INTEGER PRIMARY KEY,
|
||||||
|
name TEXT NOT NULL,
|
||||||
|
phonetic TEXT,
|
||||||
|
source TEXT,
|
||||||
|
common INTEGER NOT NULL DEFAULT 0,
|
||||||
|
confirmed INTEGER NOT NULL DEFAULT 0,
|
||||||
|
source_project TEXT,
|
||||||
|
created_at TEXT NOT NULL DEFAULT (datetime('now')),
|
||||||
|
UNIQUE(name)
|
||||||
|
);
|
||||||
|
CREATE TABLE IF NOT EXISTS library_utterances (
|
||||||
|
id INTEGER PRIMARY KEY,
|
||||||
|
text TEXT NOT NULL,
|
||||||
|
speaker_name TEXT,
|
||||||
|
voice_ref_path TEXT,
|
||||||
|
note TEXT,
|
||||||
|
source_project TEXT,
|
||||||
|
created_at TEXT NOT NULL DEFAULT (datetime('now'))
|
||||||
|
);
|
||||||
|
"""
|
||||||
|
|
||||||
|
|
||||||
|
def get_library_db_path() -> Path:
|
||||||
|
return DEFAULT_LIBRARY_DB
|
||||||
|
|
||||||
|
|
||||||
|
def get_library_voices_dir() -> Path:
|
||||||
|
return DEFAULT_LIBRARY_VOICES
|
||||||
|
|
||||||
|
|
||||||
|
def ensure_library_schema(db_path: Path | None = None) -> sqlite3.Connection:
|
||||||
|
path = db_path or get_library_db_path()
|
||||||
|
path.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
conn = sqlite3.connect(str(path))
|
||||||
|
conn.row_factory = sqlite3.Row
|
||||||
|
conn.executescript(LIBRARY_SCHEMA)
|
||||||
|
conn.commit()
|
||||||
|
return conn
|
||||||
|
|
||||||
|
|
||||||
|
def _json_str(value: Any) -> str:
|
||||||
|
if isinstance(value, str):
|
||||||
|
try:
|
||||||
|
json.loads(value)
|
||||||
|
return value
|
||||||
|
except Exception:
|
||||||
|
return json.dumps([value])
|
||||||
|
if value is None:
|
||||||
|
return "[]"
|
||||||
|
return json.dumps(value)
|
||||||
|
|
||||||
|
|
||||||
|
def add_speakers(
|
||||||
|
speakers: list[dict],
|
||||||
|
source_project: str | None,
|
||||||
|
voice_refs: dict[str, Path],
|
||||||
|
db_path: Path | None = None,
|
||||||
|
voices_dir: Path | None = None,
|
||||||
|
) -> dict:
|
||||||
|
"""Add speakers to the library, copying their voice reference into the
|
||||||
|
library voices directory.
|
||||||
|
|
||||||
|
``speakers`` are the speaker dicts (from speakers.json). ``voice_refs``
|
||||||
|
maps speaker primary_name -> path to that speaker's ``*_ref.wav``.
|
||||||
|
Returns ``{"added": [...], "already": [...], "skipped": [...]}``.
|
||||||
|
"""
|
||||||
|
conn = ensure_library_schema(db_path)
|
||||||
|
vdir = voices_dir or get_library_voices_dir()
|
||||||
|
vdir.mkdir(parents=True, exist_ok=True)
|
||||||
|
|
||||||
|
added, already, skipped = [], [], []
|
||||||
|
inserted = 0
|
||||||
|
for sp in speakers:
|
||||||
|
name = (sp.get("primary_name") or "").strip()
|
||||||
|
if not name:
|
||||||
|
continue
|
||||||
|
exists = conn.execute(
|
||||||
|
"SELECT id FROM library_speakers WHERE primary_name = ?", (name,)
|
||||||
|
).fetchone()
|
||||||
|
if exists:
|
||||||
|
already.append(name)
|
||||||
|
continue
|
||||||
|
|
||||||
|
ref_src = voice_refs.get(name)
|
||||||
|
copied_ref = None
|
||||||
|
if ref_src and Path(ref_src).exists():
|
||||||
|
dest = vdir / f"{name.lower().replace(' ', '_')}_ref.wav"
|
||||||
|
try:
|
||||||
|
shutil.copyfile(str(ref_src), str(dest))
|
||||||
|
copied_ref = str(dest)
|
||||||
|
except Exception:
|
||||||
|
copied_ref = None
|
||||||
|
|
||||||
|
conn.execute(
|
||||||
|
"""INSERT INTO library_speakers
|
||||||
|
(primary_name, aliases, gender, accent, vocal_traits,
|
||||||
|
distortion_effects, is_narrator, is_group, description,
|
||||||
|
source_project, voice_ref_path)
|
||||||
|
VALUES (?,?,?,?,?,?,?,?,?,?,?)""",
|
||||||
|
(
|
||||||
|
name,
|
||||||
|
_json_str(sp.get("aliases", [])),
|
||||||
|
sp.get("gender") or sp.get("demographics", {}).get("gender"),
|
||||||
|
sp.get("accent") or sp.get("demographics", {}).get("accent"),
|
||||||
|
_json_str(sp.get("vocal_traits", [])),
|
||||||
|
_json_str(sp.get("distortion_effects", [])),
|
||||||
|
1 if sp.get("is_narrator") else 0,
|
||||||
|
1 if sp.get("is_group") else 0,
|
||||||
|
sp.get("description"),
|
||||||
|
source_project,
|
||||||
|
copied_ref,
|
||||||
|
),
|
||||||
|
)
|
||||||
|
inserted += 1
|
||||||
|
added.append(name)
|
||||||
|
|
||||||
|
conn.commit()
|
||||||
|
conn.close()
|
||||||
|
return {"added": added, "already": already, "skipped": skipped, "inserted": inserted}
|
||||||
|
|
||||||
|
|
||||||
|
def add_names(
|
||||||
|
names: list[dict],
|
||||||
|
source_project: str | None,
|
||||||
|
db_path: Path | None = None,
|
||||||
|
) -> dict:
|
||||||
|
"""Add proper-name pronunciation records to the library.
|
||||||
|
|
||||||
|
``names`` are records with at least ``name`` and optionally ``phonetic``,
|
||||||
|
``source``, ``common``, ``confirmed``.
|
||||||
|
"""
|
||||||
|
conn = ensure_library_schema(db_path)
|
||||||
|
added, already = [], []
|
||||||
|
for rec in names:
|
||||||
|
name = (rec.get("name") or "").strip()
|
||||||
|
if not name:
|
||||||
|
continue
|
||||||
|
exists = conn.execute(
|
||||||
|
"SELECT id FROM library_names WHERE name = ?", (name,)
|
||||||
|
).fetchone()
|
||||||
|
if exists:
|
||||||
|
already.append(name)
|
||||||
|
continue
|
||||||
|
conn.execute(
|
||||||
|
"""INSERT INTO library_names (name, phonetic, source, common, confirmed, source_project)
|
||||||
|
VALUES (?,?,?,?,?,?)""",
|
||||||
|
(
|
||||||
|
name,
|
||||||
|
rec.get("phonetic"),
|
||||||
|
rec.get("source"),
|
||||||
|
1 if rec.get("common") else 0,
|
||||||
|
1 if rec.get("confirmed") else 0,
|
||||||
|
source_project,
|
||||||
|
),
|
||||||
|
)
|
||||||
|
added.append(name)
|
||||||
|
conn.commit()
|
||||||
|
conn.close()
|
||||||
|
return {"added": added, "already": already}
|
||||||
|
|
||||||
|
|
||||||
|
def list_speakers(db_path: Path | None = None) -> list[dict]:
|
||||||
|
conn = ensure_library_schema(db_path)
|
||||||
|
rows = conn.execute(
|
||||||
|
"SELECT * FROM library_speakers ORDER BY is_narrator DESC, primary_name"
|
||||||
|
).fetchall()
|
||||||
|
data = [dict(r) for r in rows]
|
||||||
|
conn.close()
|
||||||
|
return data
|
||||||
|
|
||||||
|
|
||||||
|
def list_names(db_path: Path | None = None) -> list[dict]:
|
||||||
|
conn = ensure_library_schema(db_path)
|
||||||
|
rows = conn.execute(
|
||||||
|
"SELECT * FROM library_names ORDER BY name"
|
||||||
|
).fetchall()
|
||||||
|
data = [dict(r) for r in rows]
|
||||||
|
conn.close()
|
||||||
|
return data
|
||||||
|
|
||||||
|
|
||||||
|
def list_utterances(db_path: Path | None = None) -> list[dict]:
|
||||||
|
conn = ensure_library_schema(db_path)
|
||||||
|
rows = conn.execute(
|
||||||
|
"SELECT * FROM library_utterances ORDER BY id"
|
||||||
|
).fetchall()
|
||||||
|
data = [dict(r) for r in rows]
|
||||||
|
conn.close()
|
||||||
|
return data
|
||||||
|
|
||||||
|
|
||||||
|
def delete_speaker(speaker_id: int, db_path: Path | None = None) -> bool:
|
||||||
|
conn = ensure_library_schema(db_path)
|
||||||
|
cur = conn.execute("DELETE FROM library_speakers WHERE id = ?", (speaker_id,))
|
||||||
|
conn.commit()
|
||||||
|
conn.close()
|
||||||
|
return cur.rowcount > 0
|
||||||
|
|
||||||
|
|
||||||
|
def delete_name(name_id: int, db_path: Path | None = None) -> bool:
|
||||||
|
conn = ensure_library_schema(db_path)
|
||||||
|
cur = conn.execute("DELETE FROM library_names WHERE id = ?", (name_id,))
|
||||||
|
conn.commit()
|
||||||
|
conn.close()
|
||||||
|
return cur.rowcount > 0
|
||||||
|
|
||||||
|
|
||||||
|
def speaker_voice_ref_path(speaker_id: int, db_path: Path | None = None) -> Path | None:
|
||||||
|
conn = ensure_library_schema(db_path)
|
||||||
|
row = conn.execute(
|
||||||
|
"SELECT voice_ref_path FROM library_speakers WHERE id = ?", (speaker_id,)
|
||||||
|
).fetchone()
|
||||||
|
conn.close()
|
||||||
|
if row and row["voice_ref_path"]:
|
||||||
|
return Path(row["voice_ref_path"])
|
||||||
|
return None
|
||||||
@@ -237,21 +237,29 @@ class QwenTTSEngine:
|
|||||||
voice_ref: Path,
|
voice_ref: Path,
|
||||||
ref_text: str | None = None,
|
ref_text: str | None = None,
|
||||||
slug: str | None = None,
|
slug: str | None = None,
|
||||||
x_vector_only: bool = True,
|
x_vector_only: bool = False,
|
||||||
force: bool = False,
|
force: bool = False,
|
||||||
) -> str:
|
) -> str:
|
||||||
"""Capture & cache a speaker's digital voice (a set of numbers) once.
|
"""Capture & cache a speaker's **vmatrix** (the digital voice) once.
|
||||||
|
|
||||||
Uploads the reference audio to the server's /v1/voice-prompt endpoint,
|
The vmatrix is the set of numbers inside Qwen3-TTS that *is* the voice:
|
||||||
which extracts the speaker embedding ("digital voice"), caches it under
|
- the ICL codec-token "8k matrix" (``ref_code`` — the re-encoded
|
||||||
the slug, and returns the slug to pass as prompt_slug to generate().
|
reference speech tokens the model conditions on), AND
|
||||||
|
- the ``ref_spk_embedding`` (the extracted speaker x-vector).
|
||||||
|
|
||||||
With x_vector_only=True (default) it captures ONLY the speaker embedding
|
Uploads the reference WAV to the server's /v1/voice-prompt endpoint,
|
||||||
(ref_code=None). Every utterance then feeds those cached numbers + text
|
which computes the full vmatrix, caches it as a ``.pt`` under PROMPT_DIR
|
||||||
straight into the speech generator -- no reference re-hash, no clone
|
keyed by the slug, and returns the slug. Callers then pass it as
|
||||||
re-run, no leaked reference audio. Set x_vector_only=False only if you
|
``prompt_slug`` to :meth:`generate`; every utterance reuses the exact
|
||||||
explicitly want the full ICL "8k matrix" (which re-speaks the reference
|
same cached numbers, so the voice is identical across all utterances and
|
||||||
each utterance).
|
chapters (this is the requirement: an approved voice = WAV to listen to
|
||||||
|
+ the captured vmatrix for consistency).
|
||||||
|
|
||||||
|
x_vector_only=False is the DEFAULT (and intended pipeline mode): capture
|
||||||
|
the FULL vmatrix (ref_code + embedding). x_vector_only=True captures only
|
||||||
|
the coarse speaker embedding (ref_code=None) — historically the default,
|
||||||
|
which is what produced the "inconsistent / changed voice" problem; keep
|
||||||
|
it only for experiments or where the extra VRAM of ICL is unavailable.
|
||||||
"""
|
"""
|
||||||
if not self._use_remote:
|
if not self._use_remote:
|
||||||
raise ValueError("build_voice_prompt only works with the remote TTS server")
|
raise ValueError("build_voice_prompt only works with the remote TTS server")
|
||||||
|
|||||||
@@ -29,6 +29,7 @@ CREATE TABLE IF NOT EXISTS voice_registry (
|
|||||||
accent_marker TEXT DEFAULT 'none',
|
accent_marker TEXT DEFAULT 'none',
|
||||||
voice_description TEXT,
|
voice_description TEXT,
|
||||||
voice_ref_path TEXT,
|
voice_ref_path TEXT,
|
||||||
|
vmatrix_path TEXT,
|
||||||
sample_audio_path TEXT,
|
sample_audio_path TEXT,
|
||||||
speaker_embedding BLOB,
|
speaker_embedding BLOB,
|
||||||
source_project TEXT,
|
source_project TEXT,
|
||||||
@@ -60,8 +61,19 @@ class VoiceRegistry:
|
|||||||
self._conn = await aiosqlite.connect(str(self.db_path))
|
self._conn = await aiosqlite.connect(str(self.db_path))
|
||||||
self._conn.row_factory = aiosqlite.Row
|
self._conn.row_factory = aiosqlite.Row
|
||||||
await self._conn.executescript(REGISTRY_SCHEMA)
|
await self._conn.executescript(REGISTRY_SCHEMA)
|
||||||
|
await self._migrate()
|
||||||
await self._conn.commit()
|
await self._conn.commit()
|
||||||
|
|
||||||
|
async def _migrate(self) -> None:
|
||||||
|
"""Add columns introduced after the table was first created."""
|
||||||
|
cursor = await self._conn.execute("PRAGMA table_info(voice_registry)")
|
||||||
|
cols = {row["name"] for row in await cursor.fetchall()}
|
||||||
|
if "vmatrix_path" not in cols:
|
||||||
|
# vmatrix_path: path to the captured vmatrix (.pt) for the approved voice
|
||||||
|
await self._conn.execute(
|
||||||
|
"ALTER TABLE voice_registry ADD COLUMN vmatrix_path TEXT"
|
||||||
|
)
|
||||||
|
|
||||||
async def close(self) -> None:
|
async def close(self) -> None:
|
||||||
if self._conn:
|
if self._conn:
|
||||||
await self._conn.close()
|
await self._conn.close()
|
||||||
@@ -149,6 +161,24 @@ class VoiceRegistry:
|
|||||||
)
|
)
|
||||||
await self.conn.commit()
|
await self.conn.commit()
|
||||||
|
|
||||||
|
async def set_vmatrix(
|
||||||
|
self, registry_id: int, vmatrix_path: str | None
|
||||||
|
) -> None:
|
||||||
|
"""Record the captured vmatrix (.pt) for the approved voice.
|
||||||
|
|
||||||
|
The vmatrix is the set of numbers inside Qwen3-TTS that IS the voice
|
||||||
|
(ICL "8k matrix" ref_code + speaker embedding). An approved speaker is
|
||||||
|
therefore: the reference WAV (voice_ref_path, what you listen to) plus
|
||||||
|
this cached .pt (ponder the consistency across every utterance).
|
||||||
|
"""
|
||||||
|
await self.conn.execute(
|
||||||
|
"""UPDATE voice_registry
|
||||||
|
SET vmatrix_path = ?, updated_at = datetime('now')
|
||||||
|
WHERE id = ?""",
|
||||||
|
(vmatrix_path, registry_id),
|
||||||
|
)
|
||||||
|
await self.conn.commit()
|
||||||
|
|
||||||
async def update_profile(self, registry_id: int, profile: CharacterProfile) -> None:
|
async def update_profile(self, registry_id: int, profile: CharacterProfile) -> None:
|
||||||
await self.conn.execute(
|
await self.conn.execute(
|
||||||
"""UPDATE voice_registry
|
"""UPDATE voice_registry
|
||||||
|
|||||||
@@ -547,7 +547,12 @@ def build_profile_from_speaker(speaker: dict[str, Any]) -> CharacterProfile:
|
|||||||
if unmatched:
|
if unmatched:
|
||||||
desc_parts.append(f"Vocal characteristics: {', '.join(unmatched[:3])}.")
|
desc_parts.append(f"Vocal characteristics: {', '.join(unmatched[:3])}.")
|
||||||
|
|
||||||
if description and not unmatched:
|
# Description always wins: the human/author-supplied description MUST shape the
|
||||||
|
# voice, even when vocal_traits contain unmatched keywords. (Previously gated on
|
||||||
|
# `not unmatched`, which silently dropped the description for any character with
|
||||||
|
# rich vocal_traits -- e.g. Nanny -- so editing the description produced no
|
||||||
|
# audible change and the user "couldn't get a different voice".)
|
||||||
|
if description:
|
||||||
desc_parts.append(description[:200].rstrip(".") + ".")
|
desc_parts.append(description[:200].rstrip(".") + ".")
|
||||||
|
|
||||||
voice_description = " ".join(desc_parts) + "."
|
voice_description = " ".join(desc_parts) + "."
|
||||||
|
|||||||
@@ -2689,21 +2689,54 @@ async def set_voice_status(request: Request):
|
|||||||
ref_path = voices_dir / f"{name.lower().replace(' ', '_')}_ref.wav"
|
ref_path = voices_dir / f"{name.lower().replace(' ', '_')}_ref.wav"
|
||||||
if registry_path.exists() and ref_path.exists():
|
if registry_path.exists() and ref_path.exists():
|
||||||
try:
|
try:
|
||||||
|
# Capture the approved voice's vmatrix so every utterance reuses
|
||||||
|
# the identical voice identity, not just a coarse x-vector.
|
||||||
|
# The vmatrix (.pt: ICL "8k matrix" ref_code + speaker embedding)
|
||||||
|
# is the set of numbers inside Qwen3-TTS that IS the voice; it is
|
||||||
|
# captured server-side under data_dir/voice_prompts/{slug}.pt.
|
||||||
|
vmatrix_path = None
|
||||||
|
try:
|
||||||
|
from umbraaudio.voices.qwen_tts import (
|
||||||
|
VOICE_REFERENCE_TEXT,
|
||||||
|
QwenTTSEngine,
|
||||||
|
)
|
||||||
|
|
||||||
|
engine = QwenTTSEngine(qwen_tts_url=config.tts.qwen_tts_url)
|
||||||
|
slug = engine.build_voice_prompt(
|
||||||
|
ref_path,
|
||||||
|
ref_text=VOICE_REFERENCE_TEXT,
|
||||||
|
x_vector_only=False,
|
||||||
|
)
|
||||||
|
candidate = config.data_dir / "voice_prompts" / f"{slug}.pt"
|
||||||
|
if candidate.exists():
|
||||||
|
vmatrix_path = str(candidate)
|
||||||
|
except Exception as e:
|
||||||
|
logger.warning(
|
||||||
|
"Failed to capture vmatrix for approved voice %s: %s", name, e
|
||||||
|
)
|
||||||
|
|
||||||
rdb = sqlite3.connect(str(registry_path))
|
rdb = sqlite3.connect(str(registry_path))
|
||||||
rdb.row_factory = sqlite3.Row
|
rdb.row_factory = sqlite3.Row
|
||||||
|
# ensure vmatrix_path column exists on legacy databases
|
||||||
|
has_vmatrix = any(
|
||||||
|
r["name"] == "vmatrix_path"
|
||||||
|
for r in rdb.execute("PRAGMA table_info(voice_registry)").fetchall()
|
||||||
|
)
|
||||||
|
if not has_vmatrix:
|
||||||
|
rdb.execute("ALTER TABLE voice_registry ADD COLUMN vmatrix_path TEXT")
|
||||||
existing = rdb.execute(
|
existing = rdb.execute(
|
||||||
"SELECT id FROM voice_registry WHERE LOWER(character_name) = LOWER(?)",
|
"SELECT id FROM voice_registry WHERE LOWER(character_name) = LOWER(?)",
|
||||||
(name,),
|
(name,),
|
||||||
).fetchone()
|
).fetchone()
|
||||||
if existing:
|
if existing:
|
||||||
rdb.execute(
|
rdb.execute(
|
||||||
"UPDATE voice_registry SET status='voiced', voice_ref_path=?, updated_at=datetime('now') WHERE id=?",
|
"UPDATE voice_registry SET status='voiced', voice_ref_path=?, vmatrix_path=?, updated_at=datetime('now') WHERE id=?",
|
||||||
(str(ref_path), existing["id"]),
|
(str(ref_path), vmatrix_path, existing["id"]),
|
||||||
)
|
)
|
||||||
else:
|
else:
|
||||||
rdb.execute(
|
rdb.execute(
|
||||||
"INSERT INTO voice_registry (character_name, status, voice_ref_path, source_project) VALUES (?, 'voiced', ?, ?)",
|
"INSERT INTO voice_registry (character_name, status, voice_ref_path, vmatrix_path, source_project) VALUES (?, 'voiced', ?, ?, ?)",
|
||||||
(name, str(ref_path), config.project_dir.name),
|
(name, str(ref_path), vmatrix_path, config.project_dir.name),
|
||||||
)
|
)
|
||||||
rdb.commit()
|
rdb.commit()
|
||||||
rdb.close()
|
rdb.close()
|
||||||
@@ -4385,3 +4418,133 @@ async def quality_gate_check(request: Request):
|
|||||||
"message": "Quality gate passed" if passed else f"Quality gate FAILED: {exception_rate:.2f}% exceptions exceeds 1% threshold"
|
"message": "Quality gate passed" if passed else f"Quality gate FAILED: {exception_rate:.2f}% exceptions exceeds 1% threshold"
|
||||||
})
|
})
|
||||||
return JSONResponse({"models": []})
|
return JSONResponse({"models": []})
|
||||||
|
|
||||||
|
|
||||||
|
# ==============================================================================
|
||||||
|
# Library (cross-project voices/names/utterances)
|
||||||
|
# ==============================================================================
|
||||||
|
|
||||||
|
@router.get("/library", response_class=HTMLResponse)
|
||||||
|
async def library_index(request: Request):
|
||||||
|
config = _get_active_config(request)
|
||||||
|
return templates.TemplateResponse(request, "library.html", {"active_tab": "speakers"})
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/library/speakers", response_class=HTMLResponse)
|
||||||
|
async def library_speakers(request: Request):
|
||||||
|
from umbraaudio.library import list_speakers
|
||||||
|
|
||||||
|
speakers = list_speakers()
|
||||||
|
return templates.TemplateResponse(
|
||||||
|
request, "_library_speakers.html", {"speakers": speakers}
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/library/names", response_class=HTMLResponse)
|
||||||
|
async def library_names(request: Request):
|
||||||
|
from umbraaudio.library import list_names
|
||||||
|
|
||||||
|
names = list_names()
|
||||||
|
return templates.TemplateResponse(
|
||||||
|
request, "_library_names.html", {"names": names}
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/library/utterances", response_class=HTMLResponse)
|
||||||
|
async def library_utterances(request: Request):
|
||||||
|
from umbraaudio.library import list_utterances
|
||||||
|
|
||||||
|
utterances = list_utterances()
|
||||||
|
return templates.TemplateResponse(
|
||||||
|
request, "_library_utterances.html", {"utterances": utterances}
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@router.post("/api/library/speakers")
|
||||||
|
async def library_add_speakers(request: Request):
|
||||||
|
from umbraaudio.library import add_speakers
|
||||||
|
|
||||||
|
body = await request.json()
|
||||||
|
names = body.get("speaker_names") or body.get("names") or []
|
||||||
|
if not names:
|
||||||
|
return JSONResponse({"error": "speaker_names required"}, status_code=400)
|
||||||
|
|
||||||
|
config = _get_active_config(request)
|
||||||
|
speakers = _load_speakers(request)
|
||||||
|
by_name = {s["primary_name"]: s for s in speakers}
|
||||||
|
|
||||||
|
# Resolve voice ref for each selected speaker (mirrors voice_ref resolution).
|
||||||
|
from umbraaudio.voices.voice_designer import GENERAL_SPEAKER_NAMES
|
||||||
|
|
||||||
|
voices_dir = config.data_dir / "voices"
|
||||||
|
voice_refs: dict[str, Path] = {}
|
||||||
|
for name in names:
|
||||||
|
if name not in by_name:
|
||||||
|
continue
|
||||||
|
sp = by_name[name]
|
||||||
|
if sp.get("is_narrator"):
|
||||||
|
ref_name = "narrator_ref.wav"
|
||||||
|
elif name.lower() in GENERAL_SPEAKER_NAMES:
|
||||||
|
ref_name = "general_ref.wav"
|
||||||
|
else:
|
||||||
|
ref_name = name.lower().replace(" ", "_") + "_ref.wav"
|
||||||
|
ref_path = voices_dir / ref_name
|
||||||
|
if ref_path.exists():
|
||||||
|
voice_refs[name] = ref_path
|
||||||
|
|
||||||
|
to_add = [by_name[n] for n in names if n in by_name]
|
||||||
|
result = add_speakers(to_add, source_project=config.project_dir.name, voice_refs=voice_refs)
|
||||||
|
return JSONResponse({"ok": True, **result})
|
||||||
|
|
||||||
|
|
||||||
|
@router.post("/api/library/names")
|
||||||
|
async def library_add_names(request: Request):
|
||||||
|
from umbraaudio.library import add_names
|
||||||
|
|
||||||
|
body = await request.json()
|
||||||
|
names = body.get("names") or []
|
||||||
|
if not names:
|
||||||
|
return JSONResponse({"error": "names required"}, status_code=400)
|
||||||
|
|
||||||
|
config = _get_active_config(request)
|
||||||
|
phonetic = _load_pronunciation_records(config.data_dir)
|
||||||
|
records = []
|
||||||
|
for n in names:
|
||||||
|
rec = phonetic.get(n, {})
|
||||||
|
records.append(
|
||||||
|
{
|
||||||
|
"name": n,
|
||||||
|
"phonetic": rec.get("phonetic"),
|
||||||
|
"source": rec.get("source"),
|
||||||
|
"common": rec.get("common", False),
|
||||||
|
"confirmed": rec.get("confirmed", False),
|
||||||
|
}
|
||||||
|
)
|
||||||
|
result = add_names(records, source_project=config.project_dir.name)
|
||||||
|
return JSONResponse({"ok": True, **result})
|
||||||
|
|
||||||
|
|
||||||
|
@router.delete("/api/library/speakers/{speaker_id}")
|
||||||
|
async def library_delete_speaker(request: Request, speaker_id: int):
|
||||||
|
from umbraaudio.library import delete_speaker
|
||||||
|
|
||||||
|
ok = delete_speaker(speaker_id)
|
||||||
|
return JSONResponse({"ok": ok})
|
||||||
|
|
||||||
|
|
||||||
|
@router.delete("/api/library/names/{name_id}")
|
||||||
|
async def library_delete_name(request: Request, name_id: int):
|
||||||
|
from umbraaudio.library import delete_name
|
||||||
|
|
||||||
|
ok = delete_name(name_id)
|
||||||
|
return JSONResponse({"ok": ok})
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/library-audio/{speaker_id}/{filename}")
|
||||||
|
async def serve_library_audio(request: Request, speaker_id: int, filename: str):
|
||||||
|
from umbraaudio.library import speaker_voice_ref_path
|
||||||
|
|
||||||
|
audio_path = speaker_voice_ref_path(speaker_id)
|
||||||
|
if not audio_path or Path(audio_path).name != filename or not Path(audio_path).exists():
|
||||||
|
return JSONResponse({"error": "not found"}, status_code=404)
|
||||||
|
return FileResponse(str(audio_path), media_type="audio/wav")
|
||||||
|
|||||||
48
umbraaudio/web/templates/_library_names.html
Normal file
48
umbraaudio/web/templates/_library_names.html
Normal file
@@ -0,0 +1,48 @@
|
|||||||
|
<div class="card">
|
||||||
|
<div style="display:flex; justify-content:space-between; align-items:center; margin-bottom:1rem">
|
||||||
|
<h3 style="margin:0">Library Names ({{ names|length }})</h3>
|
||||||
|
<span style="color:#64748b; font-size:0.85rem">Confirmed proper-name pronunciations added from the project Names page.</span>
|
||||||
|
</div>
|
||||||
|
{% if not names %}
|
||||||
|
<p style="color:#64748b">No names in the library yet.</p>
|
||||||
|
{% else %}
|
||||||
|
<table>
|
||||||
|
<thead>
|
||||||
|
<tr>
|
||||||
|
<th>Name</th>
|
||||||
|
<th>Phonetic</th>
|
||||||
|
<th>Status</th>
|
||||||
|
<th>Source</th>
|
||||||
|
<th></th>
|
||||||
|
</tr>
|
||||||
|
</thead>
|
||||||
|
<tbody>
|
||||||
|
{% for n in names %}
|
||||||
|
<tr id="lib-name-{{ n.id }}">
|
||||||
|
<td><strong>{{ n.name }}</strong></td>
|
||||||
|
<td><code style="color:#f59e0b">{{ n.phonetic or '—' }}</code></td>
|
||||||
|
<td>
|
||||||
|
{% if n.confirmed %}<span class="badge badge-green">Confirmed</span>
|
||||||
|
{% elif n.common %}<span class="badge badge-blue">Common</span>
|
||||||
|
{% else %}<span class="badge badge-yellow">Pending</span>{% endif %}
|
||||||
|
</td>
|
||||||
|
<td style="font-size:0.8rem; color:#94a3b8">{{ n.source_project or 'manual' }}</td>
|
||||||
|
<td>
|
||||||
|
<button class="btn btn-reject" style="font-size:0.7rem; padding:0.2rem 0.5rem"
|
||||||
|
onclick="removeLibName({{ n.id }}, '{{ n.name|e }}')">✕</button>
|
||||||
|
</td>
|
||||||
|
</tr>
|
||||||
|
{% endfor %}
|
||||||
|
</tbody>
|
||||||
|
</table>
|
||||||
|
{% endif %}
|
||||||
|
</div>
|
||||||
|
<script>
|
||||||
|
function removeLibName(id, name) {
|
||||||
|
if (!confirm('Remove "' + name + '" from the library?')) return;
|
||||||
|
fetch('/api/library/names/' + id, {method: 'DELETE'})
|
||||||
|
.then(r => r.json()).then(d => {
|
||||||
|
htmx.ajax('GET', '/library/names', {target: '#lib-tab', swap: 'innerHTML'});
|
||||||
|
});
|
||||||
|
}
|
||||||
|
</script>
|
||||||
71
umbraaudio/web/templates/_library_speakers.html
Normal file
71
umbraaudio/web/templates/_library_speakers.html
Normal file
@@ -0,0 +1,71 @@
|
|||||||
|
<div class="card">
|
||||||
|
<div style="display:flex; justify-content:space-between; align-items:center; margin-bottom:1rem">
|
||||||
|
<h3 style="margin:0">Library Speakers ({{ speakers|length }})</h3>
|
||||||
|
<span style="color:#64748b; font-size:0.85rem">Add speakers from the project Speakers page using the "Lib" button.</span>
|
||||||
|
</div>
|
||||||
|
{% if not speakers %}
|
||||||
|
<p style="color:#64748b">No speakers in the library yet.</p>
|
||||||
|
{% else %}
|
||||||
|
<table>
|
||||||
|
<thead>
|
||||||
|
<tr>
|
||||||
|
<th>Name</th>
|
||||||
|
<th>Demographics</th>
|
||||||
|
<th>Voice</th>
|
||||||
|
<th>Source Project</th>
|
||||||
|
<th>Reference</th>
|
||||||
|
<th></th>
|
||||||
|
</tr>
|
||||||
|
</thead>
|
||||||
|
<tbody>
|
||||||
|
{% for s in speakers %}
|
||||||
|
<tr id="lib-speaker-{{ s.id }}">
|
||||||
|
<td>
|
||||||
|
<strong>{{ s.primary_name }}</strong>
|
||||||
|
{% if s.is_narrator %}<span class="badge badge-blue" style="margin-left:0.3rem">Narrator</span>{% endif %}
|
||||||
|
{% if s.is_group %}<span class="badge badge-yellow" style="margin-left:0.3rem">Group</span>{% endif %}
|
||||||
|
{% if s.aliases and s.aliases not in ('[]','') %}
|
||||||
|
<br><small style="color:#94a3b8">aliases: {{ s.aliases }}</small>
|
||||||
|
{% endif %}
|
||||||
|
</td>
|
||||||
|
<td style="font-size:0.8rem; color:#94a3b8">
|
||||||
|
{{ s.gender or '—' }}{% if s.accent %} · {{ s.accent }}{% endif %}
|
||||||
|
{% if s.vocal_traits and s.vocal_traits not in ('[]','') %}
|
||||||
|
<br><small>{{ s.vocal_traits }}</small>
|
||||||
|
{% endif %}
|
||||||
|
</td>
|
||||||
|
<td style="font-size:0.8rem; color:#94a3b8">
|
||||||
|
{% if s.distortion_effects and s.distortion_effects not in ('[]','') %}
|
||||||
|
<span style="color:#f59e0b">{{ s.distortion_effects }}</span>
|
||||||
|
{% else %}standard{% endif %}
|
||||||
|
</td>
|
||||||
|
<td style="font-size:0.8rem; color:#94a3b8">{{ s.source_project or 'manual' }}</td>
|
||||||
|
<td>
|
||||||
|
{% if s.voice_ref_path %}
|
||||||
|
{% set ref_name = s.voice_ref_path.split('/')[-1] %}
|
||||||
|
<audio controls preload="none" style="height:32px">
|
||||||
|
<source src="/library-audio/{{ s.id }}/{{ ref_name }}" type="audio/wav">
|
||||||
|
</audio>
|
||||||
|
{% else %}
|
||||||
|
<span style="color:#64748b">No ref</span>
|
||||||
|
{% endif %}
|
||||||
|
</td>
|
||||||
|
<td>
|
||||||
|
<button class="btn btn-reject" style="font-size:0.7rem; padding:0.2rem 0.5rem"
|
||||||
|
onclick="removeLibSpeaker({{ s.id }}, '{{ s.primary_name|e }}')">✕</button>
|
||||||
|
</td>
|
||||||
|
</tr>
|
||||||
|
{% endfor %}
|
||||||
|
</tbody>
|
||||||
|
</table>
|
||||||
|
{% endif %}
|
||||||
|
</div>
|
||||||
|
<script>
|
||||||
|
function removeLibSpeaker(id, name) {
|
||||||
|
if (!confirm('Remove "' + name + '" from the library?')) return;
|
||||||
|
fetch('/api/library/speakers/' + id, {method: 'DELETE'})
|
||||||
|
.then(r => r.json()).then(d => {
|
||||||
|
htmx.ajax('GET', '/library/speakers', {target: '#lib-tab', swap: 'innerHTML'});
|
||||||
|
});
|
||||||
|
}
|
||||||
|
</script>
|
||||||
28
umbraaudio/web/templates/_library_utterances.html
Normal file
28
umbraaudio/web/templates/_library_utterances.html
Normal file
@@ -0,0 +1,28 @@
|
|||||||
|
<div class="card">
|
||||||
|
<div style="display:flex; justify-content:space-between; align-items:center; margin-bottom:1rem">
|
||||||
|
<h3 style="margin:0">Library Utterances ({{ utterances|length }})</h3>
|
||||||
|
<span class="badge badge-blue">Coming soon</span>
|
||||||
|
</div>
|
||||||
|
{% if not utterances %}
|
||||||
|
<p style="color:#64748b">
|
||||||
|
Captured, reusable utterances (e.g. the Umbra Audio system introduction) will appear here.
|
||||||
|
The ability to capture and add utterances is not yet available.
|
||||||
|
</p>
|
||||||
|
{% else %}
|
||||||
|
<table>
|
||||||
|
<thead>
|
||||||
|
<tr><th>#</th><th>Text</th><th>Speaker</th><th>Source Project</th></tr>
|
||||||
|
</thead>
|
||||||
|
<tbody>
|
||||||
|
{% for u in utterances %}
|
||||||
|
<tr>
|
||||||
|
<td>{{ u.id }}</td>
|
||||||
|
<td>{{ u.text }}</td>
|
||||||
|
<td>{{ u.speaker_name or '—' }}</td>
|
||||||
|
<td style="font-size:0.8rem; color:#94a3b8">{{ u.source_project or '' }}</td>
|
||||||
|
</tr>
|
||||||
|
{% endfor %}
|
||||||
|
</tbody>
|
||||||
|
</table>
|
||||||
|
{% endif %}
|
||||||
|
</div>
|
||||||
@@ -13,6 +13,7 @@
|
|||||||
</a>
|
</a>
|
||||||
<button class="btn btn-generate" id="gen-all-voices-btn" onclick="generateAllVoices()">🎙 Generate All Voices</button>
|
<button class="btn btn-generate" id="gen-all-voices-btn" onclick="generateAllVoices()">🎙 Generate All Voices</button>
|
||||||
<button class="btn btn-generate" onclick="toggleAddForm()">+ Add Speaker</button>
|
<button class="btn btn-generate" onclick="toggleAddForm()">+ Add Speaker</button>
|
||||||
|
<button class="btn" style="background:#0f3460; color:#7c3aed; border:1px solid #7c3aed" onclick="openLibSpeakersModal()">🕮 Lib</button>
|
||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
@@ -160,6 +161,35 @@
|
|||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
|
<div id="lib-speakers-modal" style="display:none; position:fixed; top:0; left:0; width:100%; height:100%; background:rgba(0,0,0,0.7); z-index:1001; justify-content:center; align-items:center">
|
||||||
|
<div style="background:#16213e; border-radius:8px; padding:1.5rem; max-width:560px; width:92%; border:1px solid #7c3aed">
|
||||||
|
<h3 style="color:#a78bfa; margin-bottom:0.25rem">Add Speakers to Library</h3>
|
||||||
|
<p style="color:#94a3b8; font-size:0.85rem; margin-bottom:1rem">
|
||||||
|
Selected speakers (and their voice reference samples) will be saved to the shared library for reuse on other projects.
|
||||||
|
</p>
|
||||||
|
<div style="display:flex; justify-content:space-between; align-items:center; margin-bottom:0.5rem">
|
||||||
|
<label style="color:#94a3b8; font-size:0.85rem"><input type="checkbox" id="lib-check-all-speakers" onchange="toggleAllLibSpeakers(this)"> Select all</label>
|
||||||
|
<span style="color:#64748b; font-size:0.8rem" id="lib-speakers-count">0 selected</span>
|
||||||
|
</div>
|
||||||
|
<div id="lib-speakers-list" style="max-height:320px; overflow-y:auto; border:1px solid #333; border-radius:4px; padding:0.5rem">
|
||||||
|
{% for speaker in speakers %}
|
||||||
|
<label style="display:flex; align-items:center; gap:0.5rem; padding:0.35rem 0.25rem; border-bottom:1px solid #24263f; cursor:pointer">
|
||||||
|
<input type="checkbox" class="lib-speaker-cb" value="{{ speaker.primary_name }}" onchange="updateLibSpeakersCount()">
|
||||||
|
<strong>{{ speaker.primary_name }}</strong>
|
||||||
|
{% if speaker.is_narrator %}<span class="badge badge-blue" style="font-size:0.65rem">Narrator</span>{% endif %}
|
||||||
|
{% if speaker.has_voice_ref %}<span class="badge badge-green" style="font-size:0.65rem">voice</span>
|
||||||
|
{% else %}<span class="badge badge-yellow" style="font-size:0.65rem">no voice</span>{% endif %}
|
||||||
|
</label>
|
||||||
|
{% endfor %}
|
||||||
|
</div>
|
||||||
|
<div style="margin-top:1rem; display:flex; gap:0.5rem; justify-content:flex-end">
|
||||||
|
<button class="btn" style="background:#374151; color:#9ca3af" onclick="closeLibSpeakersModal()">Cancel</button>
|
||||||
|
<button class="btn btn-generate" id="lib-add-speakers-btn" onclick="addSelectedToLib()">Add to Library</button>
|
||||||
|
</div>
|
||||||
|
<p style="color:#10b981; font-size:0.85rem; margin-top:0.5rem" id="lib-speakers-status"></p>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
<script>
|
<script>
|
||||||
function showAliasModal(speakerIdx, alias) {
|
function showAliasModal(speakerIdx, alias) {
|
||||||
window._aliasModal = { speakerIdx, alias };
|
window._aliasModal = { speakerIdx, alias };
|
||||||
@@ -233,4 +263,66 @@ function generateAllVoices() {
|
|||||||
alert('Failed: ' + e);
|
alert('Failed: ' + e);
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// --- Library (multi-select add) ---
|
||||||
|
function openLibSpeakersModal() {
|
||||||
|
var modal = document.getElementById('lib-speakers-modal');
|
||||||
|
modal.style.display = 'flex';
|
||||||
|
updateLibSpeakersCount();
|
||||||
|
}
|
||||||
|
|
||||||
|
function closeLibSpeakersModal() {
|
||||||
|
document.getElementById('lib-speakers-modal').style.display = 'none';
|
||||||
|
document.getElementById('lib-speakers-status').textContent = '';
|
||||||
|
}
|
||||||
|
|
||||||
|
function toggleAllLibSpeakers(el) {
|
||||||
|
var boxes = document.querySelectorAll('.lib-speaker-cb');
|
||||||
|
boxes.forEach(function(b) { b.checked = el.checked; });
|
||||||
|
updateLibSpeakersCount();
|
||||||
|
}
|
||||||
|
|
||||||
|
function updateLibSpeakersCount() {
|
||||||
|
var boxes = document.querySelectorAll('.lib-speaker-cb');
|
||||||
|
var n = 0;
|
||||||
|
boxes.forEach(function(b) { if (b.checked) n++; });
|
||||||
|
document.getElementById('lib-speakers-count').textContent = n + ' selected';
|
||||||
|
}
|
||||||
|
|
||||||
|
function addSelectedToLib() {
|
||||||
|
var boxes = document.querySelectorAll('.lib-speaker-cb');
|
||||||
|
var names = [];
|
||||||
|
boxes.forEach(function(b) { if (b.checked) names.push(b.value); });
|
||||||
|
if (names.length === 0) { alert('Select at least one speaker.'); return; }
|
||||||
|
var status = document.getElementById('lib-speakers-status');
|
||||||
|
var btn = document.getElementById('lib-add-speakers-btn');
|
||||||
|
btn.disabled = true;
|
||||||
|
btn.textContent = 'Adding...';
|
||||||
|
status.style.color = '#94a3b8';
|
||||||
|
status.textContent = 'Adding to library...';
|
||||||
|
fetch('/api/library/speakers', {
|
||||||
|
method: 'POST',
|
||||||
|
headers: {'Content-Type': 'application/json'},
|
||||||
|
body: JSON.stringify({speaker_names: names})
|
||||||
|
}).then(r => r.json()).then(d => {
|
||||||
|
btn.disabled = false;
|
||||||
|
btn.textContent = 'Add to Library';
|
||||||
|
if (d.ok) {
|
||||||
|
var parts = [];
|
||||||
|
if (d.added && d.added.length) parts.push('Added: ' + d.added.join(', '));
|
||||||
|
if (d.already && d.already.length) parts.push('Already in library: ' + d.already.join(', '));
|
||||||
|
if (d.skipped && d.skipped.length) parts.push('Skipped: ' + d.skipped.join(', '));
|
||||||
|
status.style.color = '#10b981';
|
||||||
|
status.textContent = parts.join(' · ') || 'Done';
|
||||||
|
} else {
|
||||||
|
status.style.color = '#ef4444';
|
||||||
|
status.textContent = 'Failed: ' + (d.error || 'unknown');
|
||||||
|
}
|
||||||
|
}).catch(e => {
|
||||||
|
btn.disabled = false;
|
||||||
|
btn.textContent = 'Add to Library';
|
||||||
|
status.style.color = '#ef4444';
|
||||||
|
status.textContent = 'Error: ' + e;
|
||||||
|
});
|
||||||
|
}
|
||||||
</script>
|
</script>
|
||||||
@@ -70,6 +70,7 @@
|
|||||||
<a href="/chapters-audio">Chapters</a>
|
<a href="/chapters-audio">Chapters</a>
|
||||||
<a href="/quality">Quality</a>
|
<a href="/quality">Quality</a>
|
||||||
<a href="/registry">Voice Registry</a>
|
<a href="/registry">Voice Registry</a>
|
||||||
|
<a href="/library">Lib</a>
|
||||||
<a href="/projects" style="margin-left: auto; border-color: #a78bfa;">Projects</a>
|
<a href="/projects" style="margin-left: auto; border-color: #a78bfa;">Projects</a>
|
||||||
</nav>
|
</nav>
|
||||||
{% block content %}{% endblock %}
|
{% block content %}{% endblock %}
|
||||||
|
|||||||
@@ -229,14 +229,64 @@ function saveVariation(uttId, index) {
|
|||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
|
||||||
|
function showGenPopup(message) {
|
||||||
|
let el = document.getElementById('gen-popup');
|
||||||
|
if (!el) {
|
||||||
|
el = document.createElement('div');
|
||||||
|
el.id = 'gen-popup';
|
||||||
|
el.style.cssText = 'position:fixed;top:0;left:0;right:0;bottom:0;background:rgba(0,0,0,0.7);z-index:9999;display:flex;align-items:center;justify-content:center;';
|
||||||
|
el.innerHTML = '<div style="background:#1a1a2e;border:1px solid #7c3aed;border-radius:8px;padding:2rem;text-align:center;color:#e0e0e0;min-width:260px">'
|
||||||
|
+ '<div id="gen-popup-spinner" style="border:4px solid #333;border-top:4px solid #7c3aed;border-radius:50%;width:40px;height:40px;margin:0 auto 1rem;animation:spin 1s linear infinite"></div>'
|
||||||
|
+ '<p id="gen-popup-text" style="margin:0;font-size:1rem;color:#fff"></p>'
|
||||||
|
+ '<small style="color:#94a3b8;display:block;margin-top:0.5rem">You can listen once this closes</small>'
|
||||||
|
+ '</div>';
|
||||||
|
if (!document.getElementById('gen-popup-style')) {
|
||||||
|
const st = document.createElement('style');
|
||||||
|
st.id = 'gen-popup-style';
|
||||||
|
st.textContent = '@keyframes spin { to { transform: rotate(360deg); } }';
|
||||||
|
document.head.appendChild(st);
|
||||||
|
}
|
||||||
|
document.body.appendChild(el);
|
||||||
|
}
|
||||||
|
document.getElementById('gen-popup-text').textContent = message || 'Regenerating...';
|
||||||
|
el.style.display = 'flex';
|
||||||
|
}
|
||||||
|
|
||||||
|
function hideGenPopup() {
|
||||||
|
const el = document.getElementById('gen-popup');
|
||||||
|
if (el) { el.style.display = 'none'; }
|
||||||
|
}
|
||||||
|
|
||||||
function generateUtterance(uttId, index) {
|
function generateUtterance(uttId, index) {
|
||||||
const btn = document.getElementById('gen-btn-' + index);
|
const btn = document.getElementById('gen-btn-' + index);
|
||||||
const origText = btn.textContent;
|
const origText = btn.textContent;
|
||||||
btn.disabled = true;
|
btn.disabled = true;
|
||||||
btn.textContent = '...';
|
btn.textContent = '...';
|
||||||
|
showGenPopup('Regenerating...');
|
||||||
|
|
||||||
fetch('/api/utterance/' + uttId + '/generate', { method: 'POST' })
|
// Persist any pending text edit before generating so the wording change
|
||||||
.then(r => r.json()).then(data => {
|
// is actually used. If the edit box is open, its value may differ from
|
||||||
|
// what's stored in the DB — flush it first.
|
||||||
|
const textEdit = document.getElementById('text-edit-' + index);
|
||||||
|
const textInput = document.getElementById('text-input-' + index);
|
||||||
|
|
||||||
|
const maybeSaveText = () => {
|
||||||
|
if (textEdit && textEdit.style.display !== 'none' && textInput) {
|
||||||
|
const newText = textInput.value;
|
||||||
|
return fetch('/api/utterance/' + uttId + '/text', {
|
||||||
|
method: 'PUT',
|
||||||
|
headers: {'Content-Type': 'application/json'},
|
||||||
|
body: JSON.stringify({text: newText})
|
||||||
|
}).then(r => r.json()).then(data => {
|
||||||
|
if (!data.ok) { throw new Error('Failed to save text: ' + (data.error || 'unknown')); }
|
||||||
|
});
|
||||||
|
}
|
||||||
|
return Promise.resolve();
|
||||||
|
};
|
||||||
|
|
||||||
|
maybeSaveText().then(() => {
|
||||||
|
return fetch('/api/utterance/' + uttId + '/generate', { method: 'POST' });
|
||||||
|
}).then(r => r.json()).then(data => {
|
||||||
if (data.ok) {
|
if (data.ok) {
|
||||||
btn.textContent = '⏳';
|
btn.textContent = '⏳';
|
||||||
btn.title = 'Generating...';
|
btn.title = 'Generating...';
|
||||||
@@ -244,17 +294,19 @@ function generateUtterance(uttId, index) {
|
|||||||
} else {
|
} else {
|
||||||
btn.disabled = false;
|
btn.disabled = false;
|
||||||
btn.textContent = origText;
|
btn.textContent = origText;
|
||||||
|
hideGenPopup();
|
||||||
alert('Failed: ' + (data.error || 'unknown'));
|
alert('Failed: ' + (data.error || 'unknown'));
|
||||||
}
|
}
|
||||||
}).catch(() => {
|
}).catch((err) => {
|
||||||
btn.disabled = false;
|
btn.disabled = false;
|
||||||
btn.textContent = origText;
|
btn.textContent = origText;
|
||||||
|
hideGenPopup();
|
||||||
|
alert('Error: ' + (err.message || 'unknown'));
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
|
||||||
function pollGeneration(uttId, index, filename) {
|
function pollGeneration(uttId, index, filename) {
|
||||||
const btn = document.getElementById('gen-btn-' + index);
|
const btn = document.getElementById('gen-btn-' + index);
|
||||||
const row = document.getElementById('utterance-' + index);
|
|
||||||
let elapsed = 0;
|
let elapsed = 0;
|
||||||
const checkInterval = setInterval(() => {
|
const checkInterval = setInterval(() => {
|
||||||
elapsed += 2000;
|
elapsed += 2000;
|
||||||
@@ -262,6 +314,7 @@ function pollGeneration(uttId, index, filename) {
|
|||||||
.then(r => {
|
.then(r => {
|
||||||
if (r.ok) {
|
if (r.ok) {
|
||||||
clearInterval(checkInterval);
|
clearInterval(checkInterval);
|
||||||
|
hideGenPopup();
|
||||||
location.reload();
|
location.reload();
|
||||||
}
|
}
|
||||||
}).catch(() => {});
|
}).catch(() => {});
|
||||||
@@ -273,6 +326,7 @@ function pollGeneration(uttId, index, filename) {
|
|||||||
btn.disabled = false;
|
btn.disabled = false;
|
||||||
btn.textContent = '⟳';
|
btn.textContent = '⟳';
|
||||||
btn.title = 'Generation may still be in progress';
|
btn.title = 'Generation may still be in progress';
|
||||||
|
hideGenPopup();
|
||||||
}
|
}
|
||||||
}, 180000);
|
}, 180000);
|
||||||
}
|
}
|
||||||
|
|||||||
36
umbraaudio/web/templates/library.html
Normal file
36
umbraaudio/web/templates/library.html
Normal file
@@ -0,0 +1,36 @@
|
|||||||
|
{% extends "base.html" %}
|
||||||
|
{% block content %}
|
||||||
|
|
||||||
|
<div class="card">
|
||||||
|
<h2 style="margin:0">Library</h2>
|
||||||
|
<p style="color:#94a3b8; margin:0.25rem 0 0">
|
||||||
|
Reusable voices, proper-name pronunciations, and captured utterances shared across projects.
|
||||||
|
</p>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div style="display:flex; gap:0.5rem; margin-bottom:1rem; flex-wrap:wrap">
|
||||||
|
<button class="btn" style="border:1px solid #7c3aed; color:#7c3aed" onclick="setLibTab('speakers', this)">Speakers</button>
|
||||||
|
<button class="btn" style="border:1px solid #7c3aed; color:#7c3aed" onclick="setLibTab('names', this)">Names</button>
|
||||||
|
<button class="btn" style="border:1px solid #7c3aed; color:#7c3aed" onclick="setLibTab('utterances', this)">Utterances</button>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div id="lib-content">
|
||||||
|
<div id="lib-tab" hx-get="/library/speakers" hx-trigger="load" hx-swap="innerHTML">
|
||||||
|
<p>Loading...</p>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<script>
|
||||||
|
function setLibTab(tab, btn) {
|
||||||
|
document.querySelectorAll('#lib-content .lib-tab-swap').forEach(function() {});
|
||||||
|
var el = document.getElementById('lib-tab');
|
||||||
|
var urls = {speakers: '/library/speakers', names: '/library/names', utterances: '/library/utterances'};
|
||||||
|
el.setAttribute('hx-get', urls[tab]);
|
||||||
|
htmx.process(el);
|
||||||
|
htmx.ajax('GET', urls[tab], {target: '#lib-tab', swap: 'innerHTML'});
|
||||||
|
document.querySelectorAll('button').forEach(function(b) { b.style.background = ''; });
|
||||||
|
if (btn) { btn.style.background = '#7c3aed'; btn.style.color = '#fff'; }
|
||||||
|
}
|
||||||
|
</script>
|
||||||
|
|
||||||
|
{% endblock %}
|
||||||
@@ -26,6 +26,7 @@
|
|||||||
<option value="none">No pronunciation</option>
|
<option value="none">No pronunciation</option>
|
||||||
</select>
|
</select>
|
||||||
<span style="color:#94a3b8; font-size:0.85rem" id="name-count">{{ names|length }} names found</span>
|
<span style="color:#94a3b8; font-size:0.85rem" id="name-count">{{ names|length }} names found</span>
|
||||||
|
<button class="btn" style="background:#0f3460; color:#7c3aed; border:1px solid #7c3aed" onclick="openLibNamesModal()">🕮 Lib</button>
|
||||||
<button class="btn btn-generate" id="auto-lookup-btn" onclick="autoLookupAll()" style="font-size:0.8rem; margin-left:auto">
|
<button class="btn btn-generate" id="auto-lookup-btn" onclick="autoLookupAll()" style="font-size:0.8rem; margin-left:auto">
|
||||||
Look Up All Pronunciations
|
Look Up All Pronunciations
|
||||||
</button>
|
</button>
|
||||||
@@ -106,6 +107,36 @@
|
|||||||
</tbody>
|
</tbody>
|
||||||
</table>
|
</table>
|
||||||
|
|
||||||
|
<div id="lib-names-modal" style="display:none; position:fixed; top:0; left:0; width:100%; height:100%; background:rgba(0,0,0,0.7); z-index:1001; justify-content:center; align-items:center">
|
||||||
|
<div style="background:#16213e; border-radius:8px; padding:1.5rem; max-width:560px; width:92%; border:1px solid #7c3aed">
|
||||||
|
<h3 style="color:#a78bfa; margin-bottom:0.25rem">Add Names to Library</h3>
|
||||||
|
<p style="color:#94a3b8; font-size:0.85rem; margin-bottom:1rem">
|
||||||
|
Selected proper-name pronunciations will be saved to the shared library for reuse on other projects.
|
||||||
|
</p>
|
||||||
|
<div style="display:flex; justify-content:space-between; align-items:center; margin-bottom:0.5rem">
|
||||||
|
<label style="color:#94a3b8; font-size:0.85rem"><input type="checkbox" id="lib-check-all-names" onchange="toggleAllLibNames(this)"> Select all</label>
|
||||||
|
<span style="color:#64748b; font-size:0.8rem" id="lib-names-count">0 selected</span>
|
||||||
|
</div>
|
||||||
|
<div id="lib-names-list" style="max-height:320px; overflow-y:auto; border:1px solid #333; border-radius:4px; padding:0.5rem">
|
||||||
|
{% for entry in names %}
|
||||||
|
<label style="display:flex; align-items:center; gap:0.5rem; padding:0.35rem 0.25rem; border-bottom:1px solid #24263f; cursor:pointer">
|
||||||
|
<input type="checkbox" class="lib-name-cb" value="{{ entry.name }}" onchange="updateLibNamesCount()">
|
||||||
|
<strong>{{ entry.name }}</strong>
|
||||||
|
{% if entry.phonetic %}<code style="color:#f59e0b; font-size:0.8rem">{{ entry.phonetic }}</code>{% endif %}
|
||||||
|
{% if entry.common %}<span class="badge badge-blue" style="font-size:0.65rem">common</span>
|
||||||
|
{% elif entry.confirmed %}<span class="badge badge-green" style="font-size:0.65rem">confirmed</span>
|
||||||
|
{% else %}<span class="badge badge-yellow" style="font-size:0.65rem">pending</span>{% endif %}
|
||||||
|
</label>
|
||||||
|
{% endfor %}
|
||||||
|
</div>
|
||||||
|
<div style="margin-top:1rem; display:flex; gap:0.5rem; justify-content:flex-end">
|
||||||
|
<button class="btn" style="background:#374151; color:#9ca3af" onclick="closeLibNamesModal()">Cancel</button>
|
||||||
|
<button class="btn btn-generate" id="lib-add-names-btn" onclick="addNamesToLib()">Add to Library</button>
|
||||||
|
</div>
|
||||||
|
<p style="color:#10b981; font-size:0.85rem; margin-top:0.5rem" id="lib-names-status"></p>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
<script>
|
<script>
|
||||||
function rowOf(el) {
|
function rowOf(el) {
|
||||||
return el.closest('tr');
|
return el.closest('tr');
|
||||||
@@ -327,5 +358,66 @@ function autoLookupAll() {
|
|||||||
renderIntoRow(row, rec);
|
renderIntoRow(row, rec);
|
||||||
});
|
});
|
||||||
})();
|
})();
|
||||||
|
|
||||||
|
// --- Library (multi-select add names) ---
|
||||||
|
function openLibNamesModal() {
|
||||||
|
var modal = document.getElementById('lib-names-modal');
|
||||||
|
modal.style.display = 'flex';
|
||||||
|
updateLibNamesCount();
|
||||||
|
}
|
||||||
|
|
||||||
|
function closeLibNamesModal() {
|
||||||
|
document.getElementById('lib-names-modal').style.display = 'none';
|
||||||
|
document.getElementById('lib-names-status').textContent = '';
|
||||||
|
}
|
||||||
|
|
||||||
|
function toggleAllLibNames(el) {
|
||||||
|
var boxes = document.querySelectorAll('.lib-name-cb');
|
||||||
|
boxes.forEach(function(b) { b.checked = el.checked; });
|
||||||
|
updateLibNamesCount();
|
||||||
|
}
|
||||||
|
|
||||||
|
function updateLibNamesCount() {
|
||||||
|
var boxes = document.querySelectorAll('.lib-name-cb');
|
||||||
|
var n = 0;
|
||||||
|
boxes.forEach(function(b) { if (b.checked) n++; });
|
||||||
|
document.getElementById('lib-names-count').textContent = n + ' selected';
|
||||||
|
}
|
||||||
|
|
||||||
|
function addNamesToLib() {
|
||||||
|
var boxes = document.querySelectorAll('.lib-name-cb');
|
||||||
|
var names = [];
|
||||||
|
boxes.forEach(function(b) { if (b.checked) names.push(b.value); });
|
||||||
|
if (names.length === 0) { alert('Select at least one name.'); return; }
|
||||||
|
var status = document.getElementById('lib-names-status');
|
||||||
|
var btn = document.getElementById('lib-add-names-btn');
|
||||||
|
btn.disabled = true;
|
||||||
|
btn.textContent = 'Adding...';
|
||||||
|
status.style.color = '#94a3b8';
|
||||||
|
status.textContent = 'Adding to library...';
|
||||||
|
fetch('/api/library/names', {
|
||||||
|
method: 'POST',
|
||||||
|
headers: {'Content-Type': 'application/json'},
|
||||||
|
body: JSON.stringify({names: names})
|
||||||
|
}).then(r => r.json()).then(d => {
|
||||||
|
btn.disabled = false;
|
||||||
|
btn.textContent = 'Add to Library';
|
||||||
|
if (d.ok) {
|
||||||
|
var parts = [];
|
||||||
|
if (d.added && d.added.length) parts.push('Added: ' + d.added.join(', '));
|
||||||
|
if (d.already && d.already.length) parts.push('Already in library: ' + d.already.join(', '));
|
||||||
|
status.style.color = '#10b981';
|
||||||
|
status.textContent = parts.join(' · ') || 'Done';
|
||||||
|
} else {
|
||||||
|
status.style.color = '#ef4444';
|
||||||
|
status.textContent = 'Failed: ' + (d.error || 'unknown');
|
||||||
|
}
|
||||||
|
}).catch(e => {
|
||||||
|
btn.disabled = false;
|
||||||
|
btn.textContent = 'Add to Library';
|
||||||
|
status.style.color = '#ef4444';
|
||||||
|
status.textContent = 'Error: ' + e;
|
||||||
|
});
|
||||||
|
}
|
||||||
</script>
|
</script>
|
||||||
{% endblock %}
|
{% endblock %}
|
||||||
|
|||||||
Reference in New Issue
Block a user