Quality-first multi-pass audiobook generation from English manuscripts. Includes: - umbraaudio/ core package (pipeline, config, voices, web review UI) - scripts/ CLI utilities + model download script - tts_server/ remote TTS server - specs/ design docs, tests/, pyproject.toml + uv.lock - Qwen3-TTS git submodule (pinned 022e286) with local demo.py UI patch saved at scripts/qwen3_tts_demo_umbra_patch.patch - .env with runtime config/secrets Excluded (see .gitignore): models/ (downloaded via scripts/download_models.sh), output/, logs/, .venv, .umbra runtime data, and book projects (DealHunterShip_*), which are committed separately.
4.2 KiB
4.2 KiB
Spec: Assembly & Export
Overview
The assembly module stitches normalized audio files into final audiobook formats (M4B, MP3, WAV) using FFmpeg and Python audio libraries.
Components
1. Audio Stitcher (assembly/__init__.py)
class AudioStitcher:
def __init__(
self,
silence_between_utterances_ms: int = 500,
silence_between_paragraphs_ms: int = 2000,
silence_between_chapters_ms: int = 4000,
sample_rate: int = 24000,
) -> None
def stitch_utterances(self, audio_files: list[Path], output_path: Path) -> Path
def stitch_chapters(self, chapter_files: list[Path], output_path: Path) -> Path
Silence Generation
def _make_silence(self, duration_ms: int) -> np.ndarray:
num_samples = int(self.sample_rate * duration_ms / 1000)
return np.zeros(num_samples, dtype=np.float32)
Utterance Stitching
Concatenates audio files with configurable silence between them:
- Each utterance followed by
silence_between_utterances_ms(500ms default) - Mono downmix if any file is stereo
- Output: single WAV file per chapter
Chapter Stitching
Concatenates chapter files with longer pauses:
- Each chapter followed by
silence_between_chapters_ms(4000ms default) - Used for final audiobook assembly
2. Audiobook Exporter (assembly/export.py)
class AudiobookExporter:
def export_m4b(
self,
chapter_files: list[Path],
output_path: Path,
metadata: dict | None = None,
) -> Path
def export_mp3(
self,
chapter_files: list[Path],
output_dir: Path,
metadata: dict | None = None,
) -> list[Path]
def export_wav(
self,
chapter_files: list[Path],
output_dir: Path,
) -> list[Path]
M4B Export
def export_m4b(self, chapter_files, output_path, metadata=None):
# Create FFmpeg concat file
with open(concat_file, "w") as f:
for chapter in chapter_files:
f.write(f"file '{chapter.absolute()}'\n")
# Run FFmpeg
subprocess.run([
"ffmpeg", "-f", "concat", "-safe", "0",
"-i", str(concat_file),
"-c", "copy",
str(output_path),
])
Requirements: FFmpeg must be installed on the system.
MP3 Export
def export_mp3(self, chapter_files, output_dir, metadata=None):
for i, chapter in enumerate(chapter_files):
output_path = output_dir / f"chapter_{i+1:03d}.mp3"
subprocess.run([
"ffmpeg", "-i", str(chapter),
"-codec:a", "libmp3lame",
"-b:a", "192k",
str(output_path),
])
Settings: 192kbps CBR MP3 encoding.
WAV Export
def export_wav(self, chapter_files, output_dir):
for i, chapter in enumerate(chapter_files):
output_path = output_dir / f"chapter_{i+1:03d}.wav"
shutil.copy2(chapter, output_path) # No re-encoding
3. Metadata (assembly/metadata.py)
Currently a stub (empty file). Planned: ID3 tag writing for MP3/M4B metadata (title, author, chapter markers).
Export Flow
Normalized audio files
│
├── Stitch per chapter → chapter WAVs
│
├── Export formats:
│ ├── M4B: FFmpeg concat copy
│ ├── MP3: FFmpeg 192kbps per chapter
│ └── WAV: Direct file copy
│
└── Output: .umbra/export/
├── *.m4b
├── *.mp3
└── *.wav
CLI Integration
@main.command()
@click.option("--format", "-f", "formats", multiple=True, default=["m4b", "mp3"])
def export(ctx, formats):
# Source: prefers audio_normalized/, falls back to audio/
# Output: .umbra/export/
Dependencies
- FFmpeg: Required for M4B and MP3 export
- soundfile: WAV reading/writing
- numpy: Audio array manipulation
Default Timing Parameters
| Parameter | Default | Description |
|---|---|---|
silence_between_utterances_ms |
500ms | Gap between sentences |
silence_between_paragraphs_ms |
2000ms | Gap between paragraphs |
silence_between_chapters_ms |
4000ms | Gap between chapters |
sample_rate |
24000 Hz | Audio sample rate |
mp3_bitrate |
192k | MP3 encoding bitrate |