# Multi-Project Readiness — Audit & Action Plan

> **Status: HOLD.** Do **NOT** implement these changes until the **Deal Hunter**
> audiobook is **fully built** (including the stale-voice rebuild batch and m4b
> publish). These are code-only changes that touch shared pipeline code and must
> not risk the in-flight Deal Hunter build.
>
> **Created:** 2026-09-02
> **Related spec:** `specs/15-multi-project.md` (target/design). This doc is the
> **current-state audit** plus a **staged execution plan** to reconcile reality
> with that spec. Where the two differ, this audit reflects what is actually
> deployed today.

---

## 1. Why this matters (user requirement)

Starting a new book after Deal Hunter finishes. Requirements:

- **Separate in the UI** — add projects and switch between them easily.
- **Separate in sync/backup NAS** — each book stays isolated in backup.
- **Each project processed correctly** — no cross-contamination of data/voices/audio.
- **Answer the DB question** — does the database need to become a multi-user server?

---

## 2. Current-State Audit (verified 2026-09-02)

### What is ALREADY correct

1. **Per-project disk isolation.** Each book lives in its own directory
   `ProjectBase/<slug>/` with its own `.umbra/` containing `project.db`,
   `voices/`, `chapter_audio/`, `speakers.json`, `manifests/`, `voice_tests/`,
   `export/`, etc. (config.py `list_projects()` only counts dirs with `.umbra/project.db`).

2. **Per-project SQLite DB.** Each project has its own `project.db` (10 tables;
   all project tables carry `project_id` FKs). Chapters/utterances/audio are
   structurally isolated by project directory.
   - Global shared **voice registry** at `~/.umbra/voice_registry.db` (multi-project,
     tracks `source_project`) is shared by design — the cross-book voice library.

3. **Project switching (Web).** `/projects` page; clicking a project sets an
   `active_project` cookie; `_get_active_config()` in `web/routes.py:223` routes
   all requests to that project dir.

4. **Project switching (CLI).** Every CLI command accepts `-p <slug>`.

5. **NAS sync is multi-project by nature.** `scripts/nas_lifecycle.py` uses
   `rsync -a --delete` mirroring the **whole tree**, so each project folder
   syncs independently and stays separate on `/Volumes/Umbra/UmbraAudio`.
   (NAS is a file mirror, **not** a git clone.)

### Issues found (must fix before 2nd book)

| # | Severity | Issue | Where |
|---|----------|-------|-------|
| 1 | **CRITICAL** | Hard-coded `project_id=1` in pipeline code — works only because a fresh project DB has a single row `id=1`; breaks if two sources ever land in one DB | `cli.py`, `orchestrator.py`, `scripts/run_llm_attribution.py` |
| 2 | **HIGH** | No Web UI to **create a new project**. `POST /api/upload-manuscript` writes into the *currently active* project's existing DB (reuses `id=1`). Must use CLI `init` to bootstrap a new book | `web/routes.py:259` |
| 3 | **MEDIUM** | Stale legacy root project `.umbra/project.db` (named `...docx`) shows in `/projects`, duplicates the real project name, pollutes switching | top-level `.umbra/` |
| 4 | **MEDIUM** | No SQLite concurrency protection (no WAL, no locks); web opens fresh `sqlite3.connect()` per request while background TTS threads also write → `SQLITE_BUSY` if UI + CLI run on same project simultaneously | `database.py`, `web/routes.py` |
| 5 | **LOW** | Hardcoded project paths in sync/NAS helper scripts (single-project assumption) | `tts_server/sync_voice_data.sh`, `~/.umbra/backup_project_to_nas.sh` |

---

## 3. Does the database need to become a multi-user server?

**No.** Current DB is **SQLite, local, single-user** (no postgres/mysql anywhere).
For a single operator working on one book at a time this is correct and
appropriate — keep SQLite.

Recommended minimal hardening (no server migration needed):

1. **Enable WAL mode** on project + registry DB connections (cheap, prevents
   `SQLITE_BUSY` when web UI and a CLI batch touch the same project).
2. Keep **one project per DB file** (already the case) — this is the real
   isolation guarantee.
3. Only revisit the "server DB" question if you ever need **multiple concurrent
   editors** on the *same* book (not your workflow).

---

## 4. Recommended Fixes (staged plan)

### Phase A — Correctness (do not block Deal Hunter; apply on next branch)

- [ ] **A1. Kill hard-coded `project_id=1`.** In `cli.py`, `orchestrator.py`,
      `scripts/run_llm_attribution.py`, replace `get_project(1)`/`get_chapters(1)`/
      `project_id=1` with a dynamic lookup of the active project's id from
      `config.data_dir`. Add a small helper `active_project_id(db)`.
- [ ] **A2. Enable SQLite WAL** for project DB and `~/.umbra/voice_registry.db`
      connections (`PRAGMA journal_mode=WAL`).
- [ ] **A3. Parameterize hardcoded sync/NAS scripts** by project slug
      (`tts_server/sync_voice_data.sh`, `~/.umbra/backup_project_to_nas.sh`,
      `~/.umbra/sync_to_nas.sh`).

### Phase B — Web project creation (UI-native new books)

- [ ] **B1. Add `POST /api/projects`** — create a fresh isolated project directory +
      DB from an uploaded manuscript (title/author/slug). Reuse `umbraaudio init`
      logic as a library call.
- [ ] **B2. Add `/projects/new` page** with upload form; on success activate the
      new project (set cookie) and redirect to dashboard.
- [ ] **B3. Add "New Project" button** to `project_list.html` + navbar.

### Phase C — Cleanup & reconciliation with `specs/15-multi-project.md`

- [ ] **C1. Remove/migrate legacy root `.umbra`** so `/projects` list is clean
      (use existing `migrate` CLI or a one-off cleanup). Keep the spec's intended
      patterns in mind; decide whether to adopt the `projects/` registry layout
      from the spec or keep the flat sibling-directory layout. **Recommendation:
      keep the current flat per-project-directory layout** (already works) and
      only add a lightweight global `projects` index if project metadata
      (title/author/slug) is needed in the list without opening each DB.
- [ ] **C2. Add tests** `tests/test_multi_project.py` (isolation, dynamic id,
      web create flow).

### Phase D — Verification scripts (idempotent, safe to run anytime)

- [ ] **D1. `verify_projects.py`** — list every project, its DB path, project id,
      chapters, speakers, voice count; assert each `project.db` has exactly one
      `project` row; flag cross-project references. (Model after the existing
      voice-consistency verifier.)
- [ ] **D2. `verify_nas.py`** — after a sync, assert each project dir exists on the
      NAS mirror and matches local (rsync `--dry-run -i`).

---

## 5. How to add a NEW book (today, before the fixes land)

Workflow that works **right now** and keeps projects isolated:

```bash
# 1. Create a new isolated project from a manuscript
cd /Volumes/1TB_APL_Storage/UmbraAudio
.venv/bin/python -m umbraaudio.cli init /path/to/manuscript.docx    # -> ProjectBase/<slug>/

# 2. Run discovery/analysis on it
.venv/bin/python -m umbraaudio.cli -p <slug> discover

# 3. Full pipeline
.venv/bin/python -m umbraaudio.cli -p <slug> pipeline -c all

# 4. Open in web UI and activate
.venv/bin/umbraaudio review   # then use /projects
```

**CAUTION:** never point a CLI `-p A` batch and the web UI at a different project
*simultaneously on the same project* (SQLite contention) until A2 (WAL) lands.

---

## 6. Deal Hunter completion checklist (gate before implementing plan above)

- [ ] Stale-voice rebuild batch finishes (PID 3600): chapters 7,9-14,16-32 regenerated clean.
- [ ] Re-run `verify_voices.py` → stale count = 0 (all voices current).
- [ ] m4b metadata entered (title/author/narrator/synopsis/cover) in m4b Meta page.
- [ ] m4b mux/publish step built and run → `.m4b` produced.
- [ ] NAS sync runs with the final Deal Hunter audio.
- [ ] THEN implement Phases A–D above.

---

## 7. Related files (for when we return)

- `umbraaudio/config.py` — `list_projects()` (line ~20), `load_config()`.
- `umbraaudio/web/routes.py` — `_get_active_config()` (~223), `/api/upload-manuscript` (259),
  `/api/projects/activate/{slug}` (~244), `/projects` (~232).
- `umbraaudio/web/templates/project_list.html`, `base.html`.
- `umbraaudio/cli.py` — `init` (~25), `projects` (~969), `migrate` (~988); hard-coded ids.
- `umbraaudio/orchestrator.py` — hard-coded ids (~188,584).
- `scripts/run_llm_attribution.py` — hard-coded id (~30).
- `umbraaudio/database.py`, `umbraaudio/registry.py`, `umbraaudio/library.py` — DB layer.
- `scripts/nas_lifecycle.py` — NAS sync/prune; `~/.umbra/*.sh`, `tts_server/sync_voice_data.sh`.
- `specs/15-multi-project.md` — target design spec.