rdbms-playground/docs/handoff/20260508-handoff-3.md

Session handoff — 2026-05-08 (3)

Third handover. Continues track 2's ADR-0015 work: this session completed Iterations 5 and 6 (export/import, --resume, persistent input-history hydration, migration framework scaffold), closing out the remaining track-2 backlog from the previous handoff.

State at handoff

Branch: main. Working tree dirty (this handoff doc + the iteration changes). The track-2 commits since handoff-2 are pending the user's commit approval.

Tests: 408 passing, 0 failing, 0 skipped (up from 345 at the previous handoff). Breakdown:

unit (lib)                             295  (272 + 23 new)
project_lifecycle.rs                     0
walking_skeleton.rs                      5
iteration2_persistence.rs                6
iteration3_rebuild.rs                    9
iteration4a_rebuild_command.rs          17
iteration4b_lifecycle_commands.rs       27
iteration5_export_import.rs             14   (new)
iteration6_resume_history.rs            15   (new)
                                       ---
                                       408 total

Clippy: clean with the nursery lint group enabled.

Release build: ~6.9MB single binary (up from 6.3MB at handoff-2; the increase is the zip + flate2 + zlib-rs chain pulled in for export/import).

What's implemented (delta vs. handoff-2)

Iteration 5 — export / import (ADR-0015 §11 +

ADR-0007 amendment 1)

export [<path>] command:

• Available in both modes per ADR-0003. Wired through Action::Export and a new spawn_export task in the runtime that does the zip work on a spawn_blocking thread.
• Default output: <data-root>/YYYYMMDD-<projectname>-export-NN.zip where NN is a non-clobbering two-digit sequence found by archive::next_export_sequence — caps at 99 same-day exports per project, returns ExportSequenceExhausted past that.
• export <path> overrides the default. Relative paths resolve under the active data root (the user's stated preference: "use the data-dir as the target folder, so the export file sits 'next to' the project folder, not inside it"); absolute paths are used verbatim. Refuses if the final zip path already exists.
• Zip layout: the project's directory name is preserved as the single top-level folder inside the archive, so unzip foo.zip produces <projectname>/project.yaml etc. rather than scattering files. This is what makes the round-trip export → import pleasant: the recipient doesn't need to know the original name; it lives in the zip's structure.
• Excluded from the zip: playground.db, history.log, .rdbms-playground.lock, any *.tmp files, any project.yaml.v*.bak files. .gitignore IS included (sensible default for the recipient).

import <zip> [as <target>] command:

• Grammar: separator is the literal as (space-as-space), so a zip path containing the substring "as" without surrounding spaces is treated as a path, not a syntax marker.
• Default target: the zip's single top-level folder (verified by archive::inspect_zip). Relative <target> resolves under <data-root>/projects/; absolute paths used verbatim.
• Collision behaviour: if the resolved relative target already exists, the basename auto-suffixes -02, -03, … up to -99. This is a deliberate deviation from the §2 collision rule (which refuses), recorded as an amendment to ADR-0015 §11. Rationale: round-tripping zips (export → email → import → re-export → re-import) is a normal workflow and forcing as <target> for every collision is unnecessary friction. Absolute paths are NOT auto-suffixed — the user's explicit as <abs-path> is honored exactly or refused on collision.
• After unpack: the runtime opens the new project and runs rebuild_from_text to materialize playground.db from YAML+CSV. history.log starts empty (it was excluded from the zip).
• Switches to operating on the new project via the existing perform_switch + SwitchRequest::Import path, which means the unmodified-temp cleanup machinery from Iteration 4b also applies — the previous fresh-launch temp gets auto-deleted via safely_delete_temp_project.

Path-traversal protection in archive::extract_into:

• entry.enclosed_name() rejects .. segments and absolute paths.
• The resolved extraction path is re-validated to start with target_dir (defence-in-depth).
• Top-folder match is enforced (the inspection step recorded the single top-level folder; extraction refuses any entry under a different top folder).

Module: src/archive.rs (new, ~480 lines + 11 unit tests). Public API: default_export_filename, next_export_sequence, export_project, inspect_zip, resolve_import_target, extract_into, plus ZipInspection and ArchiveError. Dep added: zip = "5" with default-features = false, features = ["deflate"].

Iteration 6 — --resume + persistent input history +

migration framework

--resume CLI flag (L1a, ADR-0015 §7):

• Reads <data-root>/last_project (a single-line file containing the absolute project path).
• Mutually exclusive with a positional <project-path> (ArgsError::ResumeWithPath).
• Errors cleanly via stderr (printed BEFORE the alternate screen is entered, so the message lands directly above the shell prompt) if:
  • last_project is missing → "no previous project recorded under …".
  • last_project points at a path that no longer exists → "recorded project … no longer exists".
• No silent fallback to a fresh temp.
• last_project is rewritten on every successful project open: startup (resume / positional path / fresh temp), load, new, save as, import. Atomic write via temp + rename.
• Helpers: project::read_last_project, project::write_last_project. Both round-trip through disk and handle the missing-data-root case (the runtime's first launch).

Persistent input history (I2-persist, ADR-0015 §12):

• On project open (initial in run() and on every switch in handle_project_switch), the in-memory navigable input history is hydrated from the tail of the project's history.log, capped at the same 1000-entry in-memory cap.
• App::seed_history(entries: Vec<String>) is the hydration entry point; Persistence::read_recent_history is the loader (calls history::read_recent_sources).
• The hydration is delivered through AppEvent::ProjectSwitched { history_entries, .. } for switch flows (since App is owned by run_loop); for the startup flow it's called inline.
• Up/Down recall jumps to the most-recent seeded entry first, matching the in-session navigation semantics.
• Format-tolerant parser: <ts>|ok|<source> lines are parsed via splitn(3, '|') so pipes inside the source are preserved; unknown escape sequences in the source are passed through literally.

Migration framework scaffold (F3, ADR-0015 §9):

• New module src/persistence/migrations.rs.
• MigratorRegistry is an ordered list of MigrateFn function pointers, indexed by source version. production() returns an empty registry (latest_version = 1). New versions register their migrators here.
• migrate_to_latest(body, registry, project_path):
  1. Reads the version: field via a tiny serde_yml wire type (VersionOnly { version: u32 }).
  2. If file_version == latest: returns body unchanged with migrated_from = None.
  3. If file_version > latest: errors out (NewerThanSupported).
  4. Otherwise: writes <project_path>/project.yaml.v<file_version>.bak, runs each migrator in sequence, and verifies each step bumped the version: field (catches forgetful migrators).
• ensure_project_yaml_migrated(project_path, registry) is the runtime-facing wrapper that pairs migration with the read/write IO.
• Wired into runtime::run() and runtime::perform_switch() so every project open runs through the (currently no-op) migration step before the database opens.
• Tests inject a fake v1→v2 migrator to exercise the registry plumbing, the .bak write, the forgot-to-bump-version check, the newer-than-supported guard, and a propagated migrator error.

ADR / docs updates

• ADR-0015 §11 — amended to record the export zip layout (top-level folder = project name) and the import auto-suffix collision behaviour (deviates from §2's refuse-on-collision rule for save / save as).
• docs/requirements.md — A1 / I2 / F3 / E1 / L1a flipped to [x] with implementation notes; test baseline updated to 408 passing.
• CLAUDE.md — not touched this session; the rules are unchanged. The repo-layout map there is slightly out-of-date (no mention of archive.rs or persistence/migrations.rs) — a quick fix-up is fair game for the next session.

Repository layout (delta vs. handoff-2)

src/
  archive.rs                    — new (Iteration 5)
  persistence/
    mod.rs                      — read_recent_history added
    migrations.rs               — new (Iteration 6 / F3)
  project/
    mod.rs                      — read_last_project +
                                  write_last_project added
  cli.rs                        — --resume flag + ResumeWithPath
                                  error variant
  app.rs                        — export/import dispatch in
                                  submit(); seed_history;
                                  ExportSucceeded/Failed event
                                  handlers; ProjectSwitched
                                  carries history_entries
  action.rs                     — Action::Export, Action::Import
  event.rs                      — AppEvent::ExportSucceeded /
                                  ExportFailed; ProjectSwitched
                                  + history_entries
  runtime.rs                    — spawn_export, do_export,
                                  resolve_import_destination,
                                  read_history_seed,
                                  SwitchRequest::Import,
                                  --resume / last_project /
                                  migration wiring
tests/
  iteration5_export_import.rs   — new (Iteration 5)
  iteration6_resume_history.rs  — new (Iteration 6)

Sharp edges and subtleties (delta vs. handoff-2)

The previous handoff's sharp edges all still apply. New ones:

• Action::Export runs on a tokio spawn_blocking task, not the db worker. Export writes the zip directly from disk; auto-save guarantees the project's text sources are current. The history.log entry for the export command is appended synchronously from the dispatching arm BEFORE the spawn (so the user-issued command lands in history even if the export task itself fails).
• SwitchRequest::Import runs inspect_zip BEFORE dropping the current project. A failed inspection (zip not a project, multiple top folders, traversal entry, etc.) leaves the user where they were. The actual extraction also runs before the drop. Only after extraction succeeds do we drop and reopen.
• ProjectSwitched is now a 3-field event. Tests that construct it directly need the extra history_entries: Vec::new() field. Iteration-4b had one such test; updated.
• Migration runs inside perform_switch AFTER the lock on the new project is acquired but BEFORE the database opens. Order matters: a migration that mutates project.yaml while another process holds the lock would corrupt the file; doing the migration after our own lock is held prevents that.
• migrate_to_latest writes the .bak BEFORE running any migrator. If a migrator panics or returns an error mid-chain, the .bak is the only intact copy of the original. The runtime currently does not auto-restore on failure — that's part of "future work" once a real migrator lands.
• --resume errors print to stderr BEFORE the terminal is set up. If the user is debugging by reading --log-file, the resume error is in the shell, not the log.
• last_project write failures are non-fatal (logged via tracing::warn). Rationale: a failed write here surfaces on the next --resume attempt with a clear message, which is preferable to refusing to launch the app over a stat / chmod hiccup.
• The zip crate features are restricted to default-features = false, features = ["deflate"] to hold the binary-size cost down. A future cipher / compression demand can revisit.

Pending — proposed next moves (in order)

Track-2's iteration backlog is now empty; ADR-0015 ships the runtime as designed. The remaining items are the deferred features called out in handoff-2's "Other deferred items" list:

1. Complex WHERE expressions (C5a)

AND/OR/comparison operators/LIKE in UPDATE/DELETE/show-data filters. The natural progression from DSL fluency into real SQL. Needs a small ADR for the operator subset.

2. Indexes (C3 partial) + EXPLAIN QUERY PLAN (QA1)

Strong teaching demo. add index <name> on <T>(<col>) / drop index <name>, plus rendering the EXPLAIN QUERY PLAN output as an annotated tree (QA2 covers the tree rendering specifics in its own ADR).

3. Column drops/renames/type changes (B2 / C2 partial)

The rebuild_table primitive already exists (ADR-0013). The grammar additions and metadata updates are straightforward; the work is mostly tests covering the data-preservation invariants.

4. Friendly error layer (H1)

Translate raw SQLite messages to learner-friendly equivalents. Partial today (FK errors are enriched both ways); full SQL → English translation is the open work.

5. replay (U4)

The history.log format is already replay-compatible. replay <path> runs commands from a history.log or .commands file. The framework lands here; the U-series items (snapshot/undo/redo, ADR-0006) follow.

6. CI (TT5)

Test infrastructure is in place; the GitHub Actions workflow file (or equivalent) is not.

7. Bigger UX work

V4 session log + Markdown export, S2 indexes in the items list, V1/V2 pretty rendering, H1a strong syntax-help. All have their entries in docs/requirements.md and remain deferred behind their respective ADRs.

How to take over

1. Read this file.
2. Read CLAUDE.md for the working-style rules.
3. Read docs/requirements.md for granular progress.
4. Skim docs/adr/README.md; read ADR-0015 in full (especially §11 with the import-collision amendment) if you'll touch the project storage runtime, the archive module, or the migration framework.
5. Run cargo test to confirm the 408-test green baseline.
6. cargo run --release -- --help to see the updated CLI banner.

End-to-end smoke test

Verifies export, import, --resume, and persistent history. Same data-dir flag throughout so the test is contained.

# Set up: launch under a clean data dir.
$ rm -rf /tmp/rdbms-iter5-iter6-smoke
$ rdbms-playground --data-dir /tmp/rdbms-iter5-iter6-smoke

# Inside the app:
help                                    -- new commands listed
create table Customers with pk id:serial
add column Customers: Name (text)
insert into Customers ('Alice')
insert into Customers ('Bob')
save                                    -- name it "MyOrders"
export                                  -- writes
                                        -- /tmp/rdbms-iter5-iter6-smoke/
                                        --   20260508-MyOrders-export-01.zip
quit

# Verify the zip on disk:
$ unzip -l /tmp/rdbms-iter5-iter6-smoke/*.zip
# Should show:
#   MyOrders/project.yaml
#   MyOrders/data/Customers.csv
#   MyOrders/.gitignore
# and NOT:
#   MyOrders/playground.db
#   MyOrders/history.log

# Re-open via --resume and verify history is hydrated:
$ rdbms-playground --data-dir /tmp/rdbms-iter5-iter6-smoke --resume
# Up arrow should walk back through the export, save,
# inserts, add column, create table — all from the previous
# session.

# Inside the app:
import /tmp/rdbms-iter5-iter6-smoke/20260508-MyOrders-export-01.zip
                                        -- creates MyOrders-02
                                        -- (auto-suffix because
                                        --  MyOrders already exists),
                                        -- switches to it,
                                        -- rebuilds .db from text.
show data Customers                     -- 'Alice' and 'Bob' present.
quit

# Final clean-up:
$ rm -rf /tmp/rdbms-iter5-iter6-smoke

If anything in that sequence fails, something is wrong.

Manual spot-checks worth running

• --resume with a missing last_project → stderr message.
• --resume with a stale recorded path → stderr message.
• --resume <path> (combined with positional) → ResumeWithPath error from arg parsing.
• export with no current data dir created yet (rare; data root resolution still works).
• import <zip-with-multiple-top-folders> → MultipleTopFolders error in the output panel.
• import <random.zip> (no project.yaml in it) → NotAProjectArchive error.
• After the scaffold migration framework: hand-edit a project's project.yaml to version: 99, restart → migrate project.yaml context error in the run-time startup error path. (Or: write a real v1→v2 migrator and watch it execute.)