claude@clouddev1
ca71184678
Adds docs/handoff/20260508-handoff-2.md describing the state at the end of this session: ADR-0015 designed, Iterations 1-4 of track 2 shipped (file-backed projects with auto-named [temp] dirs, per-command write-through, rebuild from text on missing .db, save/save as/new/load/rebuild commands with modal dialogs and project switching), plus the cleanup pass (--help, in-app help, post-rebuild message, unmodified-temp cleanup) and the safety hardening of safely_delete_temp_project. Lists the next-up moves (Iteration 5: export/import, Iteration 6: --resume + persistent input history + migration scaffold) and an end-to-end smoke test. requirements.md: marks P1-P5, P-NAME-1/2/3, F1, F2, U3, L1 as [x] with iteration references; adds P-CLEAN-1 for the safe cleanup; updates A1, I2, H3, L1a progress notes. CLAUDE.md: updates the project-storage decisions and deferred-items entry to reflect what's now live vs. still pending.
457 lines
20 KiB
Markdown
457 lines
20 KiB
Markdown
# Session handoff — 2026-05-08 (2)
|
||
|
||
This is the second handover. The first session designed and
|
||
shipped track 2's project storage end-to-end: the design ADR
|
||
(0015), file-backed projects, per-command persistence to YAML
|
||
+ CSV + history.log, rebuild-from-text on missing `.db`,
|
||
explicit `rebuild` / `save` / `save as` / `new` / `load`
|
||
commands, `--help` / in-app `help`, and a hardened cleanup of
|
||
unmodified temp projects. Iterations 5 and 6 of track 2 are
|
||
pending; the next session can pick up from this file +
|
||
`CLAUDE.md` + the linked ADRs.
|
||
|
||
## State at handoff
|
||
|
||
**Branch:** `main`. Working tree clean. The track-2 commits
|
||
since the previous handoff:
|
||
|
||
```
|
||
58a964d Harden temp-project cleanup with stacked safety guards
|
||
b7addd6 Cleanup pass: --help, in-app help, post-rebuild message,
|
||
unmodified-temp cleanup
|
||
f219827 Iteration 4b: save / save as / new / load with project
|
||
switching
|
||
ba93d3c Iteration 4a: rebuild command with confirmation modal
|
||
f0fc063 Iteration 3: existence-only load + rebuild from text on
|
||
missing .db
|
||
5410075 Persistence: empty table -> no CSV (per Iteration 2
|
||
follow-up)
|
||
5c076f6 Iteration 2: per-command write-through to project.yaml,
|
||
CSVs, history.log
|
||
601d3b6 Iteration 1: file-backed projects with auto-named temps,
|
||
lock file, and L1 CLI
|
||
4fca862 Project storage runtime: ADR-0015 + ADR-0004/0007
|
||
amendments
|
||
```
|
||
|
||
**Tests:** 345 passing (272 lib + 73 across six integration
|
||
files), 0 failing, 0 skipped.
|
||
**Clippy:** clean with `nursery` lints enabled.
|
||
**Release build:** ~6.3MB single binary (up from 5MB at the
|
||
previous handoff; the increase is `serde_yml` +
|
||
`serde` derive + `sysinfo` + `directories` + `csv` +
|
||
`base64` + `gethostname`).
|
||
|
||
The user's terminal is a real TTY; the TUI runs cleanly there
|
||
but cannot be exercised from a non-TTY environment. `cargo
|
||
test` covers everything that doesn't require a real terminal.
|
||
|
||
## What's implemented (delta vs. previous handoff)
|
||
|
||
The previous handoff covered: TUI shell, in-memory SQLite,
|
||
DSL grammar, type system, INSERT/UPDATE/DELETE, auto-show.
|
||
All of that is still in place.
|
||
|
||
**On-disk projects (Iterations 1–3):**
|
||
- Auto-named temp project on startup under
|
||
`<data-root>/projects/`. OS-standard data root resolved via
|
||
the `directories` crate (Linux / macOS / Windows); overridden
|
||
by `--data-dir`.
|
||
- Naming pattern: `<YYYYMMDD>-[temp]-<word>-<word>-<word>`
|
||
with a built-in 161-word list. The literal `[temp]` segment
|
||
is what marks a directory as a temp project — brackets are
|
||
rejected by `validate_user_name` so user-named projects can
|
||
never collide.
|
||
- Per-command write-through to `project.yaml`,
|
||
`data/<table>.csv`, and `history.log`. Atomic per-file
|
||
write-tmp + fsync + rename. Commit-db-last ordering inside
|
||
the SQLite transaction so a text-write failure rolls back
|
||
the db and leaves disk state recoverable.
|
||
- `is_temp` is dir-name-based via the `[temp]` segment — fast
|
||
enough for the load picker without a YAML parse per
|
||
project.
|
||
- Empty tables produce no CSV (a header-only file would
|
||
carry no information YAML doesn't already record). The
|
||
rule is enforced by `Persistence::write_table_data`,
|
||
which delegates to `delete_table_data` on an empty
|
||
snapshot.
|
||
- Existence-only load on startup. If `playground.db` is
|
||
missing, the runtime rebuilds from `project.yaml` +
|
||
`data/` before the event loop starts; the result is
|
||
surfaced as a system message in the output panel rather
|
||
than a silent reconstruction.
|
||
- CSV reader is hand-rolled to preserve the NULL-vs-empty
|
||
distinction (the `csv` crate doesn't expose whether a
|
||
field was syntactically quoted).
|
||
|
||
**Lifecycle commands (Iteration 4):**
|
||
- `rebuild` — confirmation modal with a summary ("3 tables
|
||
and 47 rows will be reconstructed; the existing
|
||
playground.db will be replaced"). Y/N/Esc. The worker's
|
||
`do_rebuild_from_text` wipes existing schema + metadata
|
||
before reloading from text, so it works on populated as
|
||
well as empty databases.
|
||
- `save` — for a temp project, opens a path-entry modal
|
||
(acts as save as). For a named project, prints
|
||
"already auto-saved; use `save as` to copy to a different
|
||
location".
|
||
- `save as` — always prompts for a target. Relative names
|
||
resolve under `<data-root>/projects/`; absolute paths used
|
||
as-is. `copy_project` is a recursive copy that excludes
|
||
the per-process lock file (a fresh one is acquired on
|
||
open) and preserves everything else including
|
||
`playground.db`.
|
||
- `new` — closes current project, creates a fresh
|
||
auto-named temp, switches.
|
||
- `load` — opens an in-TUI picker. List mode shows projects
|
||
in the active data root sorted newest-first by
|
||
`project.yaml` mtime, with `[TEMP]` prefixes for temp
|
||
projects. Arrow keys navigate; Enter loads; `b` switches
|
||
to a path-entry sub-mode for projects elsewhere on disk;
|
||
Esc cancels. Empty data root jumps straight to path
|
||
entry.
|
||
- Modal infrastructure: `App.modal: Option<Modal>` +
|
||
per-modal key routing; renderer draws a centred overlay.
|
||
|
||
**Project switching at runtime:**
|
||
- The runtime owns a `Session` with `Option<Project>` +
|
||
`Option<Database>` + `data_root`. `perform_switch` handles
|
||
Load / SaveAs / NewTemp uniformly. The `take()` pattern
|
||
drops the old project (releasing its lock) before opening
|
||
the new one — required for the "load my own current
|
||
project" case.
|
||
|
||
**CLI / app polish:**
|
||
- `--help` / `-h` prints a usage banner (options +
|
||
app-level commands + DSL grammar reference) and exits.
|
||
Parse errors also print the banner.
|
||
- `help` in-app command notes the same listing to the
|
||
output panel — a stand-in for the H3 help system that
|
||
stays in sync with what's actually wired up.
|
||
- `[TEMP] ` prefix in the bottom status bar when the
|
||
current project is temp.
|
||
|
||
**Unmodified temp cleanup (with stacked safety guards):**
|
||
- On project switch and on quit, if the current project is
|
||
an unmodified temp (kind=Temp, `project.yaml` parses with
|
||
empty `tables` and `relationships`, AND `data/` is empty),
|
||
it is deleted.
|
||
- Deletion is gated by `safely_delete_temp_project` which
|
||
refuses unless ALL of: path is not a symlink; path is a
|
||
directory; canonical path is under
|
||
`<active-data-root>/projects/`; basename contains the
|
||
literal `[temp]` segment; direct children are exclusively
|
||
well-known project artefacts (project.yaml, data/,
|
||
history.log, playground.db, .gitignore, lock file) plus
|
||
migration `.bak` files and atomic-write `.tmp` files.
|
||
Anything unexpected → refuse; never delete the wrong
|
||
thing.
|
||
- 7 dedicated tests cover each guard, including a `#[cfg(unix)]`
|
||
symlink-rejection test.
|
||
|
||
**Persistence module (`src/persistence/`):**
|
||
- `mod.rs` — `Persistence` handle (project path), atomic
|
||
write primitive, public types
|
||
(`SchemaSnapshot` / `TableSnapshot` / `CellValue` etc.).
|
||
- `yaml.rs` — hand-rolled writer, `serde_yml`-backed
|
||
reader. Emits `project.created_at` (writer) and parses
|
||
it (reader). Action keywords (no_action, restrict,
|
||
set_null, cascade — note: `set_default` is still
|
||
unsupported per ADR-0014).
|
||
- `csv_io.rs` — hand-rolled writer + reader. Per-type
|
||
encoding from ADR-0015 §4. Hand-rolled because the `csv`
|
||
crate strips the was-quoted bit we need for NULL-vs-empty
|
||
distinction. Bool stored as `INTEGER` in SQLite, written
|
||
as `true`/`false` in CSV.
|
||
- `history.rs` — append-only `history.log` writer. Format:
|
||
`<ISO-8601 Z>|ok|<source text>` with `\n` and `\r`
|
||
escaped inside the source.
|
||
|
||
## ADR index (read these before touching the related areas)
|
||
|
||
```
|
||
0000 Record architecture decisions (process)
|
||
0001 Language and TUI framework (Rust + Ratatui)
|
||
0002 Database engine (SQLite STRICT)
|
||
0003 Input modes and command dispatch
|
||
0004 Project file format
|
||
— amended by 0015 (derived-artifact framing,
|
||
rebuild-with-confirmation semantics)
|
||
0005 Column type vocabulary (ten types)
|
||
0006 Undo snapshots and replay log (deferred)
|
||
0007 Sharing and export
|
||
— amended by 0015 (history.log not in export zip,
|
||
but ALSO not in .gitignore — user decides)
|
||
0008 Testing approach (four tiers)
|
||
0009 DSL command syntax conventions
|
||
0010 Database access via worker thread
|
||
0011 FK column type compatibility
|
||
0012 Internal metadata for user-facing column types
|
||
0013 Relationships, naming, and rebuild-table strategy
|
||
0014 Data operations, value literals, and auto-show
|
||
0015 Project storage runtime (track 2 — implemented through
|
||
Iteration 4 + cleanup; Iterations 5 and 6 pending)
|
||
```
|
||
|
||
## What's pending — proposed next moves (in order)
|
||
|
||
### 1. Iteration 5 — `export` / `import` / `.gitignore` template
|
||
|
||
Per ADR-0015 §11 and ADR-0007:
|
||
|
||
- `export` — produces a zip containing `project.yaml` +
|
||
`data/`, **excluding** `playground.db` and (per ADR-0007
|
||
amendment 1) `history.log`. Default filename
|
||
`YYYYMMDD-<projectname>-export-NN.zip` with a non-clobbering
|
||
two-digit sequence. The user picks the output directory
|
||
(modal path entry).
|
||
- `import` — accepts an exported zip, unpacks it into a named
|
||
project at a chosen location, runs `rebuild` on open. The
|
||
zip lacks `.db` and `history.log`, so a fresh `playground.db`
|
||
is created from YAML+CSV and `history.log` starts empty. If
|
||
the chosen location already exists (per the §2 collision
|
||
rule from naming), `import` refuses with a friendly error.
|
||
- `.gitignore` template — currently the only items inside it
|
||
are `/playground.db`, `/.rdbms-playground.lock`,
|
||
`/project.yaml.v*.bak`. The amendment to ADR-0007 left
|
||
`history.log` out of the gitignore (user choice). That part
|
||
is already implemented (`Project::initialize_skeleton`); the
|
||
Iteration 5 work is just adding `export` / `import`
|
||
themselves.
|
||
|
||
Will need a zip dep — `zip` crate is the standard choice. Add
|
||
two new modal types (`ExportPathEntry`, `ImportPathEntry`) or
|
||
reuse `Modal::PathEntry` with new `PathEntryPurpose` variants.
|
||
|
||
Estimated scope: moderate — ~400–600 lines + tests.
|
||
|
||
### 2. Iteration 6 — `--resume` + persistent input history + migration scaffold
|
||
|
||
The remaining ADR-0015 §1 / §7 / §9 / §12 items:
|
||
|
||
- `--resume` CLI flag opens the most recently used project
|
||
(path tracked in `<data-root>/last_project`). Errors cleanly
|
||
if missing; mutually exclusive with a positional path.
|
||
- Persistent input history (I2): on project open, hydrate the
|
||
in-memory navigable history from `history.log`'s most recent
|
||
entries. New successful commands keep appending as today.
|
||
- Migration framework (F3): scaffold only — register no
|
||
migrators yet, but the load path checks `version`, copies
|
||
`project.yaml` to `project.yaml.v<N>.bak`, runs the
|
||
registered migrators in order, writes back at the latest
|
||
version. Exercised the moment a v2 ever lands.
|
||
|
||
Estimated scope: small/moderate — ~300–500 lines + tests.
|
||
|
||
### 3. Other deferred items (still untouched, in order of likely priority)
|
||
|
||
- **Complex WHERE expressions (C5a)** — AND/OR/comparison/LIKE
|
||
in UPDATE/DELETE/show-data filters. Bridge from DSL to real
|
||
SQL.
|
||
- **Indexes (C3 partial) + EXPLAIN QUERY PLAN (QA1)** — strong
|
||
teaching demo.
|
||
- **B2 column drops/renames/type changes** — the
|
||
`rebuild_table` primitive already exists.
|
||
- **Friendly error layer (H1)** — translate SQLite messages to
|
||
learner-friendly ones.
|
||
- **Session log + Markdown export (V4)** — bigger UX project.
|
||
- **m:n convenience (C4)** — auto-generates a junction table.
|
||
- **CI (TT5)** — the test infrastructure exists; the workflow
|
||
file does not.
|
||
|
||
## Sharp edges and subtleties (delta vs. previous handoff)
|
||
|
||
The previous handoff's sharp edges (sync `update`, worker
|
||
thread, metadata transactions, rebuild-table primitive,
|
||
wrap-aware scroll math) all still apply. New ones:
|
||
|
||
- **`Action::ExecuteDsl` is a struct variant.** Tests can no
|
||
longer pattern-match it via `vec![Action::ExecuteDsl(...)]`.
|
||
The walking-skeleton tests use `assert_one_execute_dsl` which
|
||
compares only the parsed `Command`, ignoring the source
|
||
string.
|
||
- **All public `Database` methods take `source: Option<String>`
|
||
as a final argument.** `None` skips the `history.log` append
|
||
(used for tests and internal calls); `Some(text)` records the
|
||
user-typed line. The Python-script bulk-update from
|
||
Iteration 2 added `, None` to every existing test call site.
|
||
- **`Persistence` is wired ONLY when calling
|
||
`Database::open_with_persistence`.** Tests that use
|
||
`Database::open(":memory:")` get no YAML/CSV/history.log
|
||
writes — that's intentional and lets the SQLite layer be
|
||
exercised in isolation.
|
||
- **`do_rebuild_from_text` wipes existing user tables and
|
||
metadata before reloading.** Works for the silent on-load
|
||
case (empty db: wipe is a no-op) and the explicit `rebuild`
|
||
case (replaces whatever was there).
|
||
- **The runtime's `Session` holds `Option<Project>` +
|
||
`Option<Database>`.** Project switches `take()` the old
|
||
values (releasing the lock and stopping the worker) BEFORE
|
||
opening the new ones — required for the
|
||
load-my-own-current-project case where the new open would
|
||
otherwise see a self-held lock.
|
||
- **`safely_delete_temp_project` is the ONLY way the runtime
|
||
ever removes a project directory.** Don't reach for
|
||
`std::fs::remove_dir_all` directly — use the helper, which
|
||
stacks containment / symlink-rejection / `[temp]`-marker /
|
||
contents-allowlist guards. Refusal is non-fatal; the
|
||
directory just stays put.
|
||
- **Modal state lives in `App.modal: Option<Modal>` and
|
||
routes ALL keys when active.** New modals plug into
|
||
`handle_modal_key` and the `render_modal` dispatcher in
|
||
`ui.rs`. A modal's submission yields one or more `Action`s
|
||
for the runtime to enact.
|
||
- **`AppEvent::RebuildSucceeded` is reused for both the
|
||
explicit `rebuild` command and the silent on-load
|
||
rebuild.** The handler is kind enough to be a no-op on
|
||
`modal = None`, so the dual use works without a special
|
||
case.
|
||
- **`is_unmodified_temp` requires BOTH empty schema (from
|
||
YAML) AND empty `data/` directory.** The combined check is
|
||
the authoritative signal; the `safely_delete` helper adds
|
||
defence-in-depth on the path side.
|
||
|
||
## Repository layout (delta vs. previous handoff)
|
||
|
||
```
|
||
src/
|
||
action.rs — Action enum (Quit / ExecuteDsl /
|
||
PrepareRebuild / Rebuild /
|
||
OpenLoadPicker / LoadProject /
|
||
SaveAs / NewProject)
|
||
app.rs — App state + Modal infrastructure
|
||
+ per-modal key handlers
|
||
cli.rs — Args + HELP_TEXT
|
||
db.rs — worker, do_rebuild_from_text,
|
||
finalize_persistence (+ all the
|
||
previous DDL/DML)
|
||
dsl/ — unchanged
|
||
event.rs — AppEvent (incl. RebuildPrepared,
|
||
RebuildSucceeded, RebuildFailed,
|
||
PersistenceFatal, LoadPickerReady,
|
||
ProjectSwitched, ProjectSwitchFailed)
|
||
lib.rs — re-exports incl. persistence + project
|
||
logging.rs — unchanged
|
||
main.rs — handles --help before booting
|
||
mode.rs — unchanged
|
||
persistence/ — new module (Iteration 2+)
|
||
mod.rs — Persistence handle, atomic write,
|
||
public types
|
||
yaml.rs — writer (hand-rolled) + reader
|
||
(serde_yml)
|
||
csv_io.rs — writer + reader (both hand-rolled)
|
||
history.rs — history.log appender
|
||
project/ — unchanged location, expanded:
|
||
mod.rs — Project, ProjectKind, lifecycle,
|
||
list_projects, copy_project,
|
||
safely_delete_temp_project,
|
||
is_unmodified_temp
|
||
naming.rs — slug generator with [temp] marker,
|
||
is_temp_dirname helper
|
||
prettifier.rs — strips date prefix AND [temp]-
|
||
lock.rs — unchanged from Iteration 1
|
||
wordlist.txt — 161 words
|
||
runtime.rs — Session, perform_switch,
|
||
spawn_prepare_rebuild,
|
||
spawn_rebuild
|
||
snapshots/ — insta snapshots (incl. new
|
||
rebuild_confirm_modal_dark)
|
||
theme.rs — unchanged
|
||
ui.rs — modal renderers
|
||
(render_rebuild_confirm,
|
||
render_path_entry,
|
||
render_load_picker)
|
||
+ status-bar [TEMP] prefix
|
||
tests/
|
||
walking_skeleton.rs — Tier-3 (existing)
|
||
project_lifecycle.rs — Iteration 1
|
||
iteration2_persistence.rs — Iteration 2 + the empty-CSV rule
|
||
iteration3_rebuild.rs — Iteration 3
|
||
iteration4a_rebuild_command.rs — Iteration 4a
|
||
iteration4b_lifecycle_commands.rs — Iteration 4b + safety
|
||
guards + cleanup tests
|
||
```
|
||
|
||
## How to take over
|
||
|
||
1. Read this file.
|
||
2. Read `CLAUDE.md` for the working-style rules and current
|
||
layout.
|
||
3. Read `docs/requirements.md` for granular progress.
|
||
4. Skim `docs/adr/README.md`; read ADR-0015 in full if you'll
|
||
touch the project storage runtime.
|
||
5. Run `cargo test` to confirm the 345-test green baseline.
|
||
6. `cargo run --release` to see the app — try the smoke test
|
||
below.
|
||
|
||
### End-to-end smoke test
|
||
|
||
Verifies temp-project lifecycle, persistence, switching, and
|
||
the [TEMP] / `[TEMP]` cleanup convention:
|
||
|
||
```
|
||
# Launch — should land in a temp project named like
|
||
# 20260508-[temp]-<word>-<word>-<word>; status bar shows
|
||
# "Project: [TEMP] <Display>".
|
||
$ rdbms-playground
|
||
|
||
# Inside the app:
|
||
help -- prints command list
|
||
create table Customers with pk id:serial
|
||
add column Customers: Name (text)
|
||
insert into Customers ('Alice')
|
||
insert into Customers ('Bob')
|
||
show data Customers -- two rows
|
||
save -- prompts for a name
|
||
-- type "MyOrders" + Enter
|
||
-- status bar updates;
|
||
-- [TEMP] disappears
|
||
|
||
# Confirm on disk: the OLD temp dir is deleted (was unmodified
|
||
# at the point of save... no wait, save HAS modifications;
|
||
# it stays). Test the unmodified-temp cleanup separately:
|
||
new -- creates fresh temp
|
||
-- previous (named) project
|
||
-- stays put
|
||
load -- picker shows MyOrders +
|
||
-- the new fresh temp
|
||
-- arrow-down to MyOrders,
|
||
-- Enter
|
||
-- the fresh temp gets
|
||
-- auto-deleted (was
|
||
-- unmodified)
|
||
|
||
# Now rebuild from text:
|
||
rebuild -- modal: Y
|
||
-- "[ok] rebuild — 1 table,
|
||
-- 2 rows reconstructed"
|
||
delete from Customers --all-rows
|
||
show data Customers -- empty
|
||
quit
|
||
```
|
||
|
||
If anything in that sequence fails, something is wrong. The
|
||
sequence exercises auto-temp creation, schema + DML
|
||
persistence (yaml + csv + history.log), `save` (temp →
|
||
named), `new` (close + create new temp), `load` picker (with
|
||
unmodified-temp cleanup of the just-created fresh temp), and
|
||
explicit `rebuild`.
|
||
|
||
### Manual spot-checks worth running
|
||
|
||
- `--help` and `-h` both print the usage banner.
|
||
- A project with a table and data: delete `playground.db`,
|
||
reopen — system message "[ok] rebuild — 1 table, 2 rows
|
||
reconstructed" appears in the output panel, data is
|
||
intact.
|
||
- Two `rdbms-playground` instances on the same project — the
|
||
second refuses with a clear lock-held error.
|
||
- `save` then `save as` from a named project — `save` says
|
||
"already auto-saved", `save as` prompts.
|
||
- `load` with `b` to switch to path-entry submode.
|
||
- After `quit` from a fresh-launch (untouched) temp, the
|
||
temp directory is gone from `<data-root>/projects/`.
|
||
- Add a `notes.md` to a temp project's directory; the
|
||
cleanup refuses (warn in tracing log) and the directory
|
||
stays.
|