UmbraAudio/specs/01-parsing.md

Spec: EPUB Parsing (Pass 1a)

Overview

The parser converts manuscripts in EPUB, PDF, TXT, or MD format into a list of chapter objects with title and source text. All parsing logic lives in umbraaudio/parsers/__init__.py.

Entry Point

def parse_manuscript(path: Path, fmt: str) -> list[dict[str, str | None]]

Returns: [{"title": str | None, "text": str}, ...]

Format Detection

def detect_format(path: Path) -> str

Extension	Format	Status
.txt	"txt"	Implemented
.md, .markdown	"md"	Implemented
.epub	"epub"	Implemented
.pdf	"pdf"	Implemented

Raises ValueError for unsupported formats.

EPUB Parser

def _parse_epub(path: Path) -> list[dict]

• Uses ebooklib to iterate over ITEM_DOCUMENT items
• Extracts text via BeautifulSoup.get_text()
• Title detection: first h1, h2, or h3 tag in each document
• Groups consecutive items under the same heading into one chapter
• Falls back to item index if no heading found

Dependencies

• ebooklib >= 0.18
• beautifulsoup4

TXT Parser

def _parse_txt(path: Path) -> list[dict]

• Reads file as UTF-8 text
• Splits on newlines
• Chapter detection: Lines that are < 100 characters AND entirely UPPERCASE
• Groups text under each detected header into a chapter
• Title = the uppercase header line

MD Parser

def _parse_md(path: Path) -> list[dict]

• Reads file as UTF-8 text
• Splits on lines starting with # (any heading level)
• Title = text after stripping leading # characters and whitespace
• Groups content between headings into chapter text

PDF Parser

def _parse_pdf(path: Path) -> list[dict]

• Uses PyMuPDF (fitz) for text extraction
• Same ALL-UPPERCASE header detection as TXT parser
• Additional constraint: header must have ≤ 8 words
• Groups text between headers into chapters

Dependencies

• PyMuPDF >= 1.23

Integration with CLI

The init command calls parse_manuscript():

@main.command()
@click.argument("manuscript", type=click.Path(exists=True))
@click.option("--name", "-n", help="Project name")
def init(ctx, manuscript, name):
    fmt = detect_format(path)
    chapters = parse_manuscript(path, fmt)
    # Insert into DB: chapters table

Output Schema

Each chapter dict:

{
    "title": str | None,   # Chapter title (from heading)
    "text": str,           # Full chapter text
}

Edge Cases

• EPUBs with no headings: falls back to document item index as title
• TXT/PDF with no uppercase headers: entire file treated as one chapter
• Empty chapters: filtered out (no empty source_text)
• Encoding: UTF-8 assumed; no fallback for other encodings
• Nested headings (MD): only first # level triggers chapter break