# Session handoff — 2026-05-08 (2) This is the second handover. The first session designed and shipped track 2's project storage end-to-end: the design ADR (0015), file-backed projects, per-command persistence to YAML + CSV + history.log, rebuild-from-text on missing `.db`, explicit `rebuild` / `save` / `save as` / `new` / `load` commands, `--help` / in-app `help`, and a hardened cleanup of unmodified temp projects. Iterations 5 and 6 of track 2 are pending; the next session can pick up from this file + `CLAUDE.md` + the linked ADRs. ## State at handoff **Branch:** `main`. Working tree clean. The track-2 commits since the previous handoff: ``` 58a964d Harden temp-project cleanup with stacked safety guards b7addd6 Cleanup pass: --help, in-app help, post-rebuild message, unmodified-temp cleanup f219827 Iteration 4b: save / save as / new / load with project switching ba93d3c Iteration 4a: rebuild command with confirmation modal f0fc063 Iteration 3: existence-only load + rebuild from text on missing .db 5410075 Persistence: empty table -> no CSV (per Iteration 2 follow-up) 5c076f6 Iteration 2: per-command write-through to project.yaml, CSVs, history.log 601d3b6 Iteration 1: file-backed projects with auto-named temps, lock file, and L1 CLI 4fca862 Project storage runtime: ADR-0015 + ADR-0004/0007 amendments ``` **Tests:** 345 passing (272 lib + 73 across six integration files), 0 failing, 0 skipped. **Clippy:** clean with `nursery` lints enabled. **Release build:** ~6.3MB single binary (up from 5MB at the previous handoff; the increase is `serde_yml` + `serde` derive + `sysinfo` + `directories` + `csv` + `base64` + `gethostname`). The user's terminal is a real TTY; the TUI runs cleanly there but cannot be exercised from a non-TTY environment. `cargo test` covers everything that doesn't require a real terminal. ## What's implemented (delta vs. previous handoff) The previous handoff covered: TUI shell, in-memory SQLite, DSL grammar, type system, INSERT/UPDATE/DELETE, auto-show. All of that is still in place. **On-disk projects (Iterations 1–3):** - Auto-named temp project on startup under `/projects/`. OS-standard data root resolved via the `directories` crate (Linux / macOS / Windows); overridden by `--data-dir`. - Naming pattern: `-[temp]---` with a built-in 161-word list. The literal `[temp]` segment is what marks a directory as a temp project — brackets are rejected by `validate_user_name` so user-named projects can never collide. - Per-command write-through to `project.yaml`, `data/.csv`, and `history.log`. Atomic per-file write-tmp + fsync + rename. Commit-db-last ordering inside the SQLite transaction so a text-write failure rolls back the db and leaves disk state recoverable. - `is_temp` is dir-name-based via the `[temp]` segment — fast enough for the load picker without a YAML parse per project. - Empty tables produce no CSV (a header-only file would carry no information YAML doesn't already record). The rule is enforced by `Persistence::write_table_data`, which delegates to `delete_table_data` on an empty snapshot. - Existence-only load on startup. If `playground.db` is missing, the runtime rebuilds from `project.yaml` + `data/` before the event loop starts; the result is surfaced as a system message in the output panel rather than a silent reconstruction. - CSV reader is hand-rolled to preserve the NULL-vs-empty distinction (the `csv` crate doesn't expose whether a field was syntactically quoted). **Lifecycle commands (Iteration 4):** - `rebuild` — confirmation modal with a summary ("3 tables and 47 rows will be reconstructed; the existing playground.db will be replaced"). Y/N/Esc. The worker's `do_rebuild_from_text` wipes existing schema + metadata before reloading from text, so it works on populated as well as empty databases. - `save` — for a temp project, opens a path-entry modal (acts as save as). For a named project, prints "already auto-saved; use `save as` to copy to a different location". - `save as` — always prompts for a target. Relative names resolve under `/projects/`; absolute paths used as-is. `copy_project` is a recursive copy that excludes the per-process lock file (a fresh one is acquired on open) and preserves everything else including `playground.db`. - `new` — closes current project, creates a fresh auto-named temp, switches. - `load` — opens an in-TUI picker. List mode shows projects in the active data root sorted newest-first by `project.yaml` mtime, with `[TEMP]` prefixes for temp projects. Arrow keys navigate; Enter loads; `b` switches to a path-entry sub-mode for projects elsewhere on disk; Esc cancels. Empty data root jumps straight to path entry. - Modal infrastructure: `App.modal: Option` + per-modal key routing; renderer draws a centred overlay. **Project switching at runtime:** - The runtime owns a `Session` with `Option` + `Option` + `data_root`. `perform_switch` handles Load / SaveAs / NewTemp uniformly. The `take()` pattern drops the old project (releasing its lock) before opening the new one — required for the "load my own current project" case. **CLI / app polish:** - `--help` / `-h` prints a usage banner (options + app-level commands + DSL grammar reference) and exits. Parse errors also print the banner. - `help` in-app command notes the same listing to the output panel — a stand-in for the H3 help system that stays in sync with what's actually wired up. - `[TEMP] ` prefix in the bottom status bar when the current project is temp. **Unmodified temp cleanup (with stacked safety guards):** - On project switch and on quit, if the current project is an unmodified temp (kind=Temp, `project.yaml` parses with empty `tables` and `relationships`, AND `data/` is empty), it is deleted. - Deletion is gated by `safely_delete_temp_project` which refuses unless ALL of: path is not a symlink; path is a directory; canonical path is under `/projects/`; basename contains the literal `[temp]` segment; direct children are exclusively well-known project artefacts (project.yaml, data/, history.log, playground.db, .gitignore, lock file) plus migration `.bak` files and atomic-write `.tmp` files. Anything unexpected → refuse; never delete the wrong thing. - 7 dedicated tests cover each guard, including a `#[cfg(unix)]` symlink-rejection test. **Persistence module (`src/persistence/`):** - `mod.rs` — `Persistence` handle (project path), atomic write primitive, public types (`SchemaSnapshot` / `TableSnapshot` / `CellValue` etc.). - `yaml.rs` — hand-rolled writer, `serde_yml`-backed reader. Emits `project.created_at` (writer) and parses it (reader). Action keywords (no_action, restrict, set_null, cascade — note: `set_default` is still unsupported per ADR-0014). - `csv_io.rs` — hand-rolled writer + reader. Per-type encoding from ADR-0015 §4. Hand-rolled because the `csv` crate strips the was-quoted bit we need for NULL-vs-empty distinction. Bool stored as `INTEGER` in SQLite, written as `true`/`false` in CSV. - `history.rs` — append-only `history.log` writer. Format: `|ok|` with `\n` and `\r` escaped inside the source. ## ADR index (read these before touching the related areas) ``` 0000 Record architecture decisions (process) 0001 Language and TUI framework (Rust + Ratatui) 0002 Database engine (SQLite STRICT) 0003 Input modes and command dispatch 0004 Project file format — amended by 0015 (derived-artifact framing, rebuild-with-confirmation semantics) 0005 Column type vocabulary (ten types) 0006 Undo snapshots and replay log (deferred) 0007 Sharing and export — amended by 0015 (history.log not in export zip, but ALSO not in .gitignore — user decides) 0008 Testing approach (four tiers) 0009 DSL command syntax conventions 0010 Database access via worker thread 0011 FK column type compatibility 0012 Internal metadata for user-facing column types 0013 Relationships, naming, and rebuild-table strategy 0014 Data operations, value literals, and auto-show 0015 Project storage runtime (track 2 — implemented through Iteration 4 + cleanup; Iterations 5 and 6 pending) ``` ## What's pending — proposed next moves (in order) ### 1. Iteration 5 — `export` / `import` / `.gitignore` template Per ADR-0015 §11 and ADR-0007: - `export` — produces a zip containing `project.yaml` + `data/`, **excluding** `playground.db` and (per ADR-0007 amendment 1) `history.log`. Default filename `YYYYMMDD--export-NN.zip` with a non-clobbering two-digit sequence. The user picks the output directory (modal path entry). - `import` — accepts an exported zip, unpacks it into a named project at a chosen location, runs `rebuild` on open. The zip lacks `.db` and `history.log`, so a fresh `playground.db` is created from YAML+CSV and `history.log` starts empty. If the chosen location already exists (per the §2 collision rule from naming), `import` refuses with a friendly error. - `.gitignore` template — currently the only items inside it are `/playground.db`, `/.rdbms-playground.lock`, `/project.yaml.v*.bak`. The amendment to ADR-0007 left `history.log` out of the gitignore (user choice). That part is already implemented (`Project::initialize_skeleton`); the Iteration 5 work is just adding `export` / `import` themselves. Will need a zip dep — `zip` crate is the standard choice. Add two new modal types (`ExportPathEntry`, `ImportPathEntry`) or reuse `Modal::PathEntry` with new `PathEntryPurpose` variants. Estimated scope: moderate — ~400–600 lines + tests. ### 2. Iteration 6 — `--resume` + persistent input history + migration scaffold The remaining ADR-0015 §1 / §7 / §9 / §12 items: - `--resume` CLI flag opens the most recently used project (path tracked in `/last_project`). Errors cleanly if missing; mutually exclusive with a positional path. - Persistent input history (I2): on project open, hydrate the in-memory navigable history from `history.log`'s most recent entries. New successful commands keep appending as today. - Migration framework (F3): scaffold only — register no migrators yet, but the load path checks `version`, copies `project.yaml` to `project.yaml.v.bak`, runs the registered migrators in order, writes back at the latest version. Exercised the moment a v2 ever lands. Estimated scope: small/moderate — ~300–500 lines + tests. ### 3. Other deferred items (still untouched, in order of likely priority) - **Complex WHERE expressions (C5a)** — AND/OR/comparison/LIKE in UPDATE/DELETE/show-data filters. Bridge from DSL to real SQL. - **Indexes (C3 partial) + EXPLAIN QUERY PLAN (QA1)** — strong teaching demo. - **B2 column drops/renames/type changes** — the `rebuild_table` primitive already exists. - **Friendly error layer (H1)** — translate SQLite messages to learner-friendly ones. - **Session log + Markdown export (V4)** — bigger UX project. - **m:n convenience (C4)** — auto-generates a junction table. - **CI (TT5)** — the test infrastructure exists; the workflow file does not. ## Sharp edges and subtleties (delta vs. previous handoff) The previous handoff's sharp edges (sync `update`, worker thread, metadata transactions, rebuild-table primitive, wrap-aware scroll math) all still apply. New ones: - **`Action::ExecuteDsl` is a struct variant.** Tests can no longer pattern-match it via `vec![Action::ExecuteDsl(...)]`. The walking-skeleton tests use `assert_one_execute_dsl` which compares only the parsed `Command`, ignoring the source string. - **All public `Database` methods take `source: Option` as a final argument.** `None` skips the `history.log` append (used for tests and internal calls); `Some(text)` records the user-typed line. The Python-script bulk-update from Iteration 2 added `, None` to every existing test call site. - **`Persistence` is wired ONLY when calling `Database::open_with_persistence`.** Tests that use `Database::open(":memory:")` get no YAML/CSV/history.log writes — that's intentional and lets the SQLite layer be exercised in isolation. - **`do_rebuild_from_text` wipes existing user tables and metadata before reloading.** Works for the silent on-load case (empty db: wipe is a no-op) and the explicit `rebuild` case (replaces whatever was there). - **The runtime's `Session` holds `Option` + `Option`.** Project switches `take()` the old values (releasing the lock and stopping the worker) BEFORE opening the new ones — required for the load-my-own-current-project case where the new open would otherwise see a self-held lock. - **`safely_delete_temp_project` is the ONLY way the runtime ever removes a project directory.** Don't reach for `std::fs::remove_dir_all` directly — use the helper, which stacks containment / symlink-rejection / `[temp]`-marker / contents-allowlist guards. Refusal is non-fatal; the directory just stays put. - **Modal state lives in `App.modal: Option` and routes ALL keys when active.** New modals plug into `handle_modal_key` and the `render_modal` dispatcher in `ui.rs`. A modal's submission yields one or more `Action`s for the runtime to enact. - **`AppEvent::RebuildSucceeded` is reused for both the explicit `rebuild` command and the silent on-load rebuild.** The handler is kind enough to be a no-op on `modal = None`, so the dual use works without a special case. - **`is_unmodified_temp` requires BOTH empty schema (from YAML) AND empty `data/` directory.** The combined check is the authoritative signal; the `safely_delete` helper adds defence-in-depth on the path side. ## Repository layout (delta vs. previous handoff) ``` src/ action.rs — Action enum (Quit / ExecuteDsl / PrepareRebuild / Rebuild / OpenLoadPicker / LoadProject / SaveAs / NewProject) app.rs — App state + Modal infrastructure + per-modal key handlers cli.rs — Args + HELP_TEXT db.rs — worker, do_rebuild_from_text, finalize_persistence (+ all the previous DDL/DML) dsl/ — unchanged event.rs — AppEvent (incl. RebuildPrepared, RebuildSucceeded, RebuildFailed, PersistenceFatal, LoadPickerReady, ProjectSwitched, ProjectSwitchFailed) lib.rs — re-exports incl. persistence + project logging.rs — unchanged main.rs — handles --help before booting mode.rs — unchanged persistence/ — new module (Iteration 2+) mod.rs — Persistence handle, atomic write, public types yaml.rs — writer (hand-rolled) + reader (serde_yml) csv_io.rs — writer + reader (both hand-rolled) history.rs — history.log appender project/ — unchanged location, expanded: mod.rs — Project, ProjectKind, lifecycle, list_projects, copy_project, safely_delete_temp_project, is_unmodified_temp naming.rs — slug generator with [temp] marker, is_temp_dirname helper prettifier.rs — strips date prefix AND [temp]- lock.rs — unchanged from Iteration 1 wordlist.txt — 161 words runtime.rs — Session, perform_switch, spawn_prepare_rebuild, spawn_rebuild snapshots/ — insta snapshots (incl. new rebuild_confirm_modal_dark) theme.rs — unchanged ui.rs — modal renderers (render_rebuild_confirm, render_path_entry, render_load_picker) + status-bar [TEMP] prefix tests/ walking_skeleton.rs — Tier-3 (existing) project_lifecycle.rs — Iteration 1 iteration2_persistence.rs — Iteration 2 + the empty-CSV rule iteration3_rebuild.rs — Iteration 3 iteration4a_rebuild_command.rs — Iteration 4a iteration4b_lifecycle_commands.rs — Iteration 4b + safety guards + cleanup tests ``` ## How to take over 1. Read this file. 2. Read `CLAUDE.md` for the working-style rules and current layout. 3. Read `docs/requirements.md` for granular progress. 4. Skim `docs/adr/README.md`; read ADR-0015 in full if you'll touch the project storage runtime. 5. Run `cargo test` to confirm the 345-test green baseline. 6. `cargo run --release` to see the app — try the smoke test below. ### End-to-end smoke test Verifies temp-project lifecycle, persistence, switching, and the [TEMP] / `[TEMP]` cleanup convention: ``` # Launch — should land in a temp project named like # 20260508-[temp]---; status bar shows # "Project: [TEMP] ". $ rdbms-playground # Inside the app: help -- prints command list create table Customers with pk id:serial add column Customers: Name (text) insert into Customers ('Alice') insert into Customers ('Bob') show data Customers -- two rows save -- prompts for a name -- type "MyOrders" + Enter -- status bar updates; -- [TEMP] disappears # Confirm on disk: the OLD temp dir is deleted (was unmodified # at the point of save... no wait, save HAS modifications; # it stays). Test the unmodified-temp cleanup separately: new -- creates fresh temp -- previous (named) project -- stays put load -- picker shows MyOrders + -- the new fresh temp -- arrow-down to MyOrders, -- Enter -- the fresh temp gets -- auto-deleted (was -- unmodified) # Now rebuild from text: rebuild -- modal: Y -- "[ok] rebuild — 1 table, -- 2 rows reconstructed" delete from Customers --all-rows show data Customers -- empty quit ``` If anything in that sequence fails, something is wrong. The sequence exercises auto-temp creation, schema + DML persistence (yaml + csv + history.log), `save` (temp → named), `new` (close + create new temp), `load` picker (with unmodified-temp cleanup of the just-created fresh temp), and explicit `rebuild`. ### Manual spot-checks worth running - `--help` and `-h` both print the usage banner. - A project with a table and data: delete `playground.db`, reopen — system message "[ok] rebuild — 1 table, 2 rows reconstructed" appears in the output panel, data is intact. - Two `rdbms-playground` instances on the same project — the second refuses with a clear lock-held error. - `save` then `save as` from a named project — `save` says "already auto-saved", `save as` prompts. - `load` with `b` to switch to path-entry submode. - After `quit` from a fresh-launch (untouched) temp, the temp directory is gone from `/projects/`. - Add a `notes.md` to a temp project's directory; the cleanup refuses (warn in tracing log) and the directory stays.