rdbms-playground/docs/handoff/20260508-handoff-2.md

Session handoff — 2026-05-08 (2)

This is the second handover. The first session designed and shipped track 2's project storage end-to-end: the design ADR (0015), file-backed projects, per-command persistence to YAML

• CSV + history.log, rebuild-from-text on missing .db, explicit rebuild / save / save as / new / load commands, --help / in-app help, and a hardened cleanup of unmodified temp projects. Iterations 5 and 6 of track 2 are pending; the next session can pick up from this file + CLAUDE.md + the linked ADRs.

State at handoff

Branch: main. Working tree clean. The track-2 commits since the previous handoff:

58a964d Harden temp-project cleanup with stacked safety guards
b7addd6 Cleanup pass: --help, in-app help, post-rebuild message,
        unmodified-temp cleanup
f219827 Iteration 4b: save / save as / new / load with project
        switching
ba93d3c Iteration 4a: rebuild command with confirmation modal
f0fc063 Iteration 3: existence-only load + rebuild from text on
        missing .db
5410075 Persistence: empty table -> no CSV (per Iteration 2
        follow-up)
5c076f6 Iteration 2: per-command write-through to project.yaml,
        CSVs, history.log
601d3b6 Iteration 1: file-backed projects with auto-named temps,
        lock file, and L1 CLI
4fca862 Project storage runtime: ADR-0015 + ADR-0004/0007
        amendments

Tests: 345 passing (272 lib + 73 across six integration files), 0 failing, 0 skipped. Clippy: clean with nursery lints enabled. Release build: ~6.3MB single binary (up from 5MB at the previous handoff; the increase is serde_yml + serde derive + sysinfo + directories + csv + base64 + gethostname).

The user's terminal is a real TTY; the TUI runs cleanly there but cannot be exercised from a non-TTY environment. cargo test covers everything that doesn't require a real terminal.

What's implemented (delta vs. previous handoff)

The previous handoff covered: TUI shell, in-memory SQLite, DSL grammar, type system, INSERT/UPDATE/DELETE, auto-show. All of that is still in place.

On-disk projects (Iterations 1–3):

• Auto-named temp project on startup under <data-root>/projects/. OS-standard data root resolved via the directories crate (Linux / macOS / Windows); overridden by --data-dir.
• Naming pattern: <YYYYMMDD>-[temp]-<word>-<word>-<word> with a built-in 161-word list. The literal [temp] segment is what marks a directory as a temp project — brackets are rejected by validate_user_name so user-named projects can never collide.
• Per-command write-through to project.yaml, data/<table>.csv, and history.log. Atomic per-file write-tmp + fsync + rename. Commit-db-last ordering inside the SQLite transaction so a text-write failure rolls back the db and leaves disk state recoverable.
• is_temp is dir-name-based via the [temp] segment — fast enough for the load picker without a YAML parse per project.
• Empty tables produce no CSV (a header-only file would carry no information YAML doesn't already record). The rule is enforced by Persistence::write_table_data, which delegates to delete_table_data on an empty snapshot.
• Existence-only load on startup. If playground.db is missing, the runtime rebuilds from project.yaml + data/ before the event loop starts; the result is surfaced as a system message in the output panel rather than a silent reconstruction.
• CSV reader is hand-rolled to preserve the NULL-vs-empty distinction (the csv crate doesn't expose whether a field was syntactically quoted).

Lifecycle commands (Iteration 4):

• rebuild — confirmation modal with a summary ("3 tables and 47 rows will be reconstructed; the existing playground.db will be replaced"). Y/N/Esc. The worker's do_rebuild_from_text wipes existing schema + metadata before reloading from text, so it works on populated as well as empty databases.
• save — for a temp project, opens a path-entry modal (acts as save as). For a named project, prints "already auto-saved; use save as to copy to a different location".
• save as — always prompts for a target. Relative names resolve under <data-root>/projects/; absolute paths used as-is. copy_project is a recursive copy that excludes the per-process lock file (a fresh one is acquired on open) and preserves everything else including playground.db.
• new — closes current project, creates a fresh auto-named temp, switches.
• load — opens an in-TUI picker. List mode shows projects in the active data root sorted newest-first by project.yaml mtime, with [TEMP] prefixes for temp projects. Arrow keys navigate; Enter loads; b switches to a path-entry sub-mode for projects elsewhere on disk; Esc cancels. Empty data root jumps straight to path entry.
• Modal infrastructure: App.modal: Option<Modal> + per-modal key routing; renderer draws a centred overlay.

Project switching at runtime:

• The runtime owns a Session with Option<Project> + Option<Database> + data_root. perform_switch handles Load / SaveAs / NewTemp uniformly. The take() pattern drops the old project (releasing its lock) before opening the new one — required for the "load my own current project" case.

CLI / app polish:

• --help / -h prints a usage banner (options + app-level commands + DSL grammar reference) and exits. Parse errors also print the banner.
• help in-app command notes the same listing to the output panel — a stand-in for the H3 help system that stays in sync with what's actually wired up.
• [TEMP] prefix in the bottom status bar when the current project is temp.

Unmodified temp cleanup (with stacked safety guards):

• On project switch and on quit, if the current project is an unmodified temp (kind=Temp, project.yaml parses with empty tables and relationships, AND data/ is empty), it is deleted.
• Deletion is gated by safely_delete_temp_project which refuses unless ALL of: path is not a symlink; path is a directory; canonical path is under <active-data-root>/projects/; basename contains the literal [temp] segment; direct children are exclusively well-known project artefacts (project.yaml, data/, history.log, playground.db, .gitignore, lock file) plus migration .bak files and atomic-write .tmp files. Anything unexpected → refuse; never delete the wrong thing.
• 7 dedicated tests cover each guard, including a #[cfg(unix)] symlink-rejection test.

Persistence module (src/persistence/):

• mod.rs — Persistence handle (project path), atomic write primitive, public types (SchemaSnapshot / TableSnapshot / CellValue etc.).
• yaml.rs — hand-rolled writer, serde_yml-backed reader. Emits project.created_at (writer) and parses it (reader). Action keywords (no_action, restrict, set_null, cascade — note: set_default is still unsupported per ADR-0014).
• csv_io.rs — hand-rolled writer + reader. Per-type encoding from ADR-0015 §4. Hand-rolled because the csv crate strips the was-quoted bit we need for NULL-vs-empty distinction. Bool stored as INTEGER in SQLite, written as true/false in CSV.
• history.rs — append-only history.log writer. Format: <ISO-8601 Z>|ok|<source text> with \n and \r escaped inside the source.

ADR index (read these before touching the related areas)

0000 Record architecture decisions (process)
0001 Language and TUI framework (Rust + Ratatui)
0002 Database engine (SQLite STRICT)
0003 Input modes and command dispatch
0004 Project file format
        — amended by 0015 (derived-artifact framing,
          rebuild-with-confirmation semantics)
0005 Column type vocabulary (ten types)
0006 Undo snapshots and replay log (deferred)
0007 Sharing and export
        — amended by 0015 (history.log not in export zip,
          but ALSO not in .gitignore — user decides)
0008 Testing approach (four tiers)
0009 DSL command syntax conventions
0010 Database access via worker thread
0011 FK column type compatibility
0012 Internal metadata for user-facing column types
0013 Relationships, naming, and rebuild-table strategy
0014 Data operations, value literals, and auto-show
0015 Project storage runtime (track 2 — implemented through
     Iteration 4 + cleanup; Iterations 5 and 6 pending)

What's pending — proposed next moves (in order)

1. Iteration 5 — export / import / .gitignore template

Per ADR-0015 §11 and ADR-0007:

• export — produces a zip containing project.yaml + data/, excluding playground.db and (per ADR-0007 amendment 1) history.log. Default filename YYYYMMDD-<projectname>-export-NN.zip with a non-clobbering two-digit sequence. The user picks the output directory (modal path entry).
• import — accepts an exported zip, unpacks it into a named project at a chosen location, runs rebuild on open. The zip lacks .db and history.log, so a fresh playground.db is created from YAML+CSV and history.log starts empty. If the chosen location already exists (per the §2 collision rule from naming), import refuses with a friendly error.
• .gitignore template — currently the only items inside it are /playground.db, /.rdbms-playground.lock, /project.yaml.v*.bak. The amendment to ADR-0007 left history.log out of the gitignore (user choice). That part is already implemented (Project::initialize_skeleton); the Iteration 5 work is just adding export / import themselves.

Will need a zip dep — zip crate is the standard choice. Add two new modal types (ExportPathEntry, ImportPathEntry) or reuse Modal::PathEntry with new PathEntryPurpose variants.

Estimated scope: moderate — ~400–600 lines + tests.

2. Iteration 6 — --resume + persistent input history + migration scaffold

The remaining ADR-0015 §1 / §7 / §9 / §12 items:

• --resume CLI flag opens the most recently used project (path tracked in <data-root>/last_project). Errors cleanly if missing; mutually exclusive with a positional path.
• Persistent input history (I2): on project open, hydrate the in-memory navigable history from history.log's most recent entries. New successful commands keep appending as today.
• Migration framework (F3): scaffold only — register no migrators yet, but the load path checks version, copies project.yaml to project.yaml.v<N>.bak, runs the registered migrators in order, writes back at the latest version. Exercised the moment a v2 ever lands.

Estimated scope: small/moderate — ~300–500 lines + tests.

3. Other deferred items (still untouched, in order of likely priority)

• Complex WHERE expressions (C5a) — AND/OR/comparison/LIKE in UPDATE/DELETE/show-data filters. Bridge from DSL to real SQL.
• Indexes (C3 partial) + EXPLAIN QUERY PLAN (QA1) — strong teaching demo.
• B2 column drops/renames/type changes — the rebuild_table primitive already exists.
• Friendly error layer (H1) — translate SQLite messages to learner-friendly ones.
• Session log + Markdown export (V4) — bigger UX project.
• m:n convenience (C4) — auto-generates a junction table.
• CI (TT5) — the test infrastructure exists; the workflow file does not.

Sharp edges and subtleties (delta vs. previous handoff)

The previous handoff's sharp edges (sync update, worker thread, metadata transactions, rebuild-table primitive, wrap-aware scroll math) all still apply. New ones:

• Action::ExecuteDsl is a struct variant. Tests can no longer pattern-match it via vec![Action::ExecuteDsl(...)]. The walking-skeleton tests use assert_one_execute_dsl which compares only the parsed Command, ignoring the source string.
• All public Database methods take source: Option<String> as a final argument. None skips the history.log append (used for tests and internal calls); Some(text) records the user-typed line. The Python-script bulk-update from Iteration 2 added , None to every existing test call site.
• Persistence is wired ONLY when calling Database::open_with_persistence. Tests that use Database::open(":memory:") get no YAML/CSV/history.log writes — that's intentional and lets the SQLite layer be exercised in isolation.
• do_rebuild_from_text wipes existing user tables and metadata before reloading. Works for the silent on-load case (empty db: wipe is a no-op) and the explicit rebuild case (replaces whatever was there).
• The runtime's Session holds Option<Project> + Option<Database>. Project switches take() the old values (releasing the lock and stopping the worker) BEFORE opening the new ones — required for the load-my-own-current-project case where the new open would otherwise see a self-held lock.
• safely_delete_temp_project is the ONLY way the runtime ever removes a project directory. Don't reach for std::fs::remove_dir_all directly — use the helper, which stacks containment / symlink-rejection / [temp]-marker / contents-allowlist guards. Refusal is non-fatal; the directory just stays put.
• Modal state lives in App.modal: Option<Modal> and routes ALL keys when active. New modals plug into handle_modal_key and the render_modal dispatcher in ui.rs. A modal's submission yields one or more Actions for the runtime to enact.
• AppEvent::RebuildSucceeded is reused for both the explicit rebuild command and the silent on-load rebuild. The handler is kind enough to be a no-op on modal = None, so the dual use works without a special case.
• is_unmodified_temp requires BOTH empty schema (from YAML) AND empty data/ directory. The combined check is the authoritative signal; the safely_delete helper adds defence-in-depth on the path side.

Repository layout (delta vs. previous handoff)

src/
  action.rs              — Action enum (Quit / ExecuteDsl /
                           PrepareRebuild / Rebuild /
                           OpenLoadPicker / LoadProject /
                           SaveAs / NewProject)
  app.rs                 — App state + Modal infrastructure
                           + per-modal key handlers
  cli.rs                 — Args + HELP_TEXT
  db.rs                  — worker, do_rebuild_from_text,
                           finalize_persistence (+ all the
                           previous DDL/DML)
  dsl/                   — unchanged
  event.rs               — AppEvent (incl. RebuildPrepared,
                           RebuildSucceeded, RebuildFailed,
                           PersistenceFatal, LoadPickerReady,
                           ProjectSwitched, ProjectSwitchFailed)
  lib.rs                 — re-exports incl. persistence + project
  logging.rs             — unchanged
  main.rs                — handles --help before booting
  mode.rs                — unchanged
  persistence/           — new module (Iteration 2+)
    mod.rs               — Persistence handle, atomic write,
                           public types
    yaml.rs              — writer (hand-rolled) + reader
                           (serde_yml)
    csv_io.rs            — writer + reader (both hand-rolled)
    history.rs           — history.log appender
  project/               — unchanged location, expanded:
    mod.rs               — Project, ProjectKind, lifecycle,
                           list_projects, copy_project,
                           safely_delete_temp_project,
                           is_unmodified_temp
    naming.rs            — slug generator with [temp] marker,
                           is_temp_dirname helper
    prettifier.rs        — strips date prefix AND [temp]-
    lock.rs              — unchanged from Iteration 1
    wordlist.txt         — 161 words
  runtime.rs             — Session, perform_switch,
                           spawn_prepare_rebuild,
                           spawn_rebuild
  snapshots/             — insta snapshots (incl. new
                           rebuild_confirm_modal_dark)
  theme.rs               — unchanged
  ui.rs                  — modal renderers
                           (render_rebuild_confirm,
                           render_path_entry,
                           render_load_picker)
                           + status-bar [TEMP] prefix
tests/
  walking_skeleton.rs           — Tier-3 (existing)
  project_lifecycle.rs          — Iteration 1
  iteration2_persistence.rs     — Iteration 2 + the empty-CSV rule
  iteration3_rebuild.rs         — Iteration 3
  iteration4a_rebuild_command.rs — Iteration 4a
  iteration4b_lifecycle_commands.rs — Iteration 4b + safety
                                    guards + cleanup tests

How to take over

1. Read this file.
2. Read CLAUDE.md for the working-style rules and current layout.
3. Read docs/requirements.md for granular progress.
4. Skim docs/adr/README.md; read ADR-0015 in full if you'll touch the project storage runtime.
5. Run cargo test to confirm the 345-test green baseline.
6. cargo run --release to see the app — try the smoke test below.

End-to-end smoke test

Verifies temp-project lifecycle, persistence, switching, and the [TEMP] / [TEMP] cleanup convention:

# Launch — should land in a temp project named like
# 20260508-[temp]-<word>-<word>-<word>; status bar shows
# "Project: [TEMP] <Display>".
$ rdbms-playground

# Inside the app:
help                                         -- prints command list
create table Customers with pk id:serial
add column Customers: Name (text)
insert into Customers ('Alice')
insert into Customers ('Bob')
show data Customers                          -- two rows
save                                         -- prompts for a name
                                             -- type "MyOrders" + Enter
                                             -- status bar updates;
                                             -- [TEMP] disappears

# Confirm on disk: the OLD temp dir is deleted (was unmodified
# at the point of save... no wait, save HAS modifications;
# it stays). Test the unmodified-temp cleanup separately:
new                                          -- creates fresh temp
                                             -- previous (named) project
                                             -- stays put
load                                         -- picker shows MyOrders +
                                             -- the new fresh temp
                                             -- arrow-down to MyOrders,
                                             -- Enter
                                             -- the fresh temp gets
                                             -- auto-deleted (was
                                             -- unmodified)

# Now rebuild from text:
rebuild                                      -- modal: Y
                                             -- "[ok] rebuild — 1 table,
                                             -- 2 rows reconstructed"
delete from Customers --all-rows
show data Customers                          -- empty
quit

If anything in that sequence fails, something is wrong. The sequence exercises auto-temp creation, schema + DML persistence (yaml + csv + history.log), save (temp → named), new (close + create new temp), load picker (with unmodified-temp cleanup of the just-created fresh temp), and explicit rebuild.

Manual spot-checks worth running

• --help and -h both print the usage banner.
• A project with a table and data: delete playground.db, reopen — system message "[ok] rebuild — 1 table, 2 rows reconstructed" appears in the output panel, data is intact.
• Two rdbms-playground instances on the same project — the second refuses with a clear lock-held error.
• save then save as from a named project — save says "already auto-saved", save as prompts.
• load with b to switch to path-entry submode.
• After quit from a fresh-launch (untouched) temp, the temp directory is gone from <data-root>/projects/.
• Add a notes.md to a temp project's directory; the cleanup refuses (warn in tracing log) and the directory stays.