rdbms-playground/docs/handoff/20260508-handoff-3.md

# Session handoff — 2026-05-08 (3)

Third handover. Continues track 2's ADR-0015 work: this
session completed Iterations 5 and 6 (export/import,
`--resume`, persistent input-history hydration, migration
framework scaffold), closing out the remaining track-2
backlog from the previous handoff.

## State at handoff

**Branch:** `main`. Working tree dirty (this handoff doc +
the iteration changes). The track-2 commits since handoff-2
are pending the user's commit approval.

**Tests:** 408 passing, 0 failing, 0 skipped (up from 345 at
the previous handoff). Breakdown:

```
unit (lib)                             295  (272 + 23 new)
project_lifecycle.rs                     0
walking_skeleton.rs                      5
iteration2_persistence.rs                6
iteration3_rebuild.rs                    9
iteration4a_rebuild_command.rs          17
iteration4b_lifecycle_commands.rs       27
iteration5_export_import.rs             14   (new)
iteration6_resume_history.rs            15   (new)
                                       ---
                                       408 total
```

**Clippy:** clean with the nursery lint group enabled.

**Release build:** ~6.9MB single binary (up from 6.3MB at
handoff-2; the increase is the `zip` + `flate2` + `zlib-rs`
chain pulled in for export/import).

## What's implemented (delta vs. handoff-2)

### Iteration 5 — `export` / `import` (ADR-0015 §11 +
### ADR-0007 amendment 1)

**`export [<path>]` command:**

- Available in both modes per ADR-0003. Wired through
  `Action::Export` and a new `spawn_export` task in the
  runtime that does the zip work on a `spawn_blocking`
  thread.
- Default output: `<data-root>/YYYYMMDD-<projectname>-export-NN.zip`
  where `NN` is a non-clobbering two-digit sequence found by
  `archive::next_export_sequence` — caps at 99 same-day
  exports per project, returns
  `ExportSequenceExhausted` past that.
- `export <path>` overrides the default. Relative paths
  resolve under the active data root (the user's stated
  preference: "use the data-dir as the target folder, so the
  export file sits 'next to' the project folder, not inside
  it"); absolute paths are used verbatim. Refuses if the
  final zip path already exists.
- **Zip layout:** the project's directory name is preserved
  as the single top-level folder inside the archive, so
  `unzip foo.zip` produces `<projectname>/project.yaml` etc.
  rather than scattering files. This is what makes the
  round-trip `export → import` pleasant: the recipient
  doesn't need to know the original name; it lives in the
  zip's structure.
- **Excluded** from the zip: `playground.db`,
  `history.log`, `.rdbms-playground.lock`, any `*.tmp`
  files, any `project.yaml.v*.bak` files. `.gitignore` IS
  included (sensible default for the recipient).

**`import <zip> [as <target>]` command:**

- Grammar: separator is the literal ` as ` (space-as-space),
  so a zip path containing the substring "as" without
  surrounding spaces is treated as a path, not a syntax
  marker.
- Default target: the zip's single top-level folder
  (verified by `archive::inspect_zip`). Relative `<target>`
  resolves under `<data-root>/projects/`; absolute paths
  used verbatim.
- **Collision behaviour:** if the resolved relative target
  already exists, the basename auto-suffixes `-02`, `-03`, …
  up to `-99`. This is a deliberate deviation from the §2
  collision rule (which refuses), recorded as an amendment
  to ADR-0015 §11. Rationale: round-tripping zips
  (export → email → import → re-export → re-import) is a
  normal workflow and forcing `as <target>` for every
  collision is unnecessary friction. Absolute paths are NOT
  auto-suffixed — the user's explicit `as <abs-path>` is
  honored exactly or refused on collision.
- After unpack: the runtime opens the new project and runs
  `rebuild_from_text` to materialize `playground.db` from
  YAML+CSV. `history.log` starts empty (it was excluded
  from the zip).
- Switches to operating on the new project via the existing
  `perform_switch` + `SwitchRequest::Import` path, which
  means the unmodified-temp cleanup machinery from
  Iteration 4b also applies — the previous fresh-launch
  temp gets auto-deleted via
  `safely_delete_temp_project`.

**Path-traversal protection** in `archive::extract_into`:

- `entry.enclosed_name()` rejects `..` segments and
  absolute paths.
- The resolved extraction path is re-validated to start
  with `target_dir` (defence-in-depth).
- Top-folder match is enforced (the inspection step
  recorded the single top-level folder; extraction refuses
  any entry under a different top folder).

**Module:** `src/archive.rs` (new, ~480 lines + 11 unit
tests). Public API: `default_export_filename`,
`next_export_sequence`, `export_project`, `inspect_zip`,
`resolve_import_target`, `extract_into`, plus
`ZipInspection` and `ArchiveError`. Dep added: `zip = "5"`
with `default-features = false, features = ["deflate"]`.

### Iteration 6 — `--resume` + persistent input history +
### migration framework

**`--resume` CLI flag (L1a, ADR-0015 §7):**

- Reads `<data-root>/last_project` (a single-line file
  containing the absolute project path).
- Mutually exclusive with a positional `<project-path>`
  (`ArgsError::ResumeWithPath`).
- Errors cleanly via stderr (printed BEFORE the alternate
  screen is entered, so the message lands directly above
  the shell prompt) if:
  - `last_project` is missing → "no previous project
    recorded under …".
  - `last_project` points at a path that no longer exists
    → "recorded project … no longer exists".
- No silent fallback to a fresh temp.
- `last_project` is rewritten on every successful project
  open: startup (resume / positional path / fresh temp),
  `load`, `new`, `save as`, `import`. Atomic write via
  temp + rename.
- Helpers: `project::read_last_project`,
  `project::write_last_project`. Both round-trip through
  disk and handle the missing-data-root case (the runtime's
  first launch).

**Persistent input history (I2-persist, ADR-0015 §12):**

- On project open (initial in `run()` and on every switch in
  `handle_project_switch`), the in-memory navigable input
  history is hydrated from the tail of the project's
  `history.log`, capped at the same 1000-entry in-memory cap.
- `App::seed_history(entries: Vec<String>)` is the
  hydration entry point; `Persistence::read_recent_history`
  is the loader (calls `history::read_recent_sources`).
- The hydration is delivered through
  `AppEvent::ProjectSwitched { history_entries, .. }` for
  switch flows (since `App` is owned by `run_loop`); for
  the startup flow it's called inline.
- Up/Down recall jumps to the most-recent seeded entry
  first, matching the in-session navigation semantics.
- Format-tolerant parser: `<ts>|ok|<source>` lines are
  parsed via `splitn(3, '|')` so pipes inside the source
  are preserved; unknown escape sequences in the source
  are passed through literally.

**Migration framework scaffold (F3, ADR-0015 §9):**

- New module `src/persistence/migrations.rs`.
- `MigratorRegistry` is an ordered list of `MigrateFn`
  function pointers, indexed by source version.
  `production()` returns an empty registry (latest_version
  = 1). New versions register their migrators here.
- `migrate_to_latest(body, registry, project_path)`:
  1. Reads the `version:` field via a tiny `serde_yml`
     wire type (`VersionOnly { version: u32 }`).
  2. If `file_version == latest`: returns body unchanged
     with `migrated_from = None`.
  3. If `file_version > latest`: errors out
     (`NewerThanSupported`).
  4. Otherwise: writes
     `<project_path>/project.yaml.v<file_version>.bak`,
     runs each migrator in sequence, and verifies each
     step bumped the `version:` field (catches forgetful
     migrators).
- `ensure_project_yaml_migrated(project_path, registry)`
  is the runtime-facing wrapper that pairs migration with
  the read/write IO.
- Wired into `runtime::run()` and
  `runtime::perform_switch()` so every project open runs
  through the (currently no-op) migration step before the
  database opens.
- Tests inject a fake v1→v2 migrator to exercise the
  registry plumbing, the `.bak` write, the
  forgot-to-bump-version check, the
  newer-than-supported guard, and a propagated migrator
  error.

## ADR / docs updates

- **ADR-0015 §11** — amended to record the export zip
  layout (top-level folder = project name) and the
  import auto-suffix collision behaviour (deviates from
  §2's refuse-on-collision rule for `save` / `save as`).
- **`docs/requirements.md`** — A1 / I2 / F3 / E1 / L1a
  flipped to `[x]` with implementation notes; test
  baseline updated to 408 passing.
- **`CLAUDE.md`** — not touched this session; the rules
  are unchanged. The repo-layout map there is slightly
  out-of-date (no mention of `archive.rs` or
  `persistence/migrations.rs`) — a quick fix-up is fair
  game for the next session.

## Repository layout (delta vs. handoff-2)

```
src/
  archive.rs                    — new (Iteration 5)
  persistence/
    mod.rs                      — read_recent_history added
    migrations.rs               — new (Iteration 6 / F3)
  project/
    mod.rs                      — read_last_project +
                                  write_last_project added
  cli.rs                        — --resume flag + ResumeWithPath
                                  error variant
  app.rs                        — export/import dispatch in
                                  submit(); seed_history;
                                  ExportSucceeded/Failed event
                                  handlers; ProjectSwitched
                                  carries history_entries
  action.rs                     — Action::Export, Action::Import
  event.rs                      — AppEvent::ExportSucceeded /
                                  ExportFailed; ProjectSwitched
                                  + history_entries
  runtime.rs                    — spawn_export, do_export,
                                  resolve_import_destination,
                                  read_history_seed,
                                  SwitchRequest::Import,
                                  --resume / last_project /
                                  migration wiring
tests/
  iteration5_export_import.rs   — new (Iteration 5)
  iteration6_resume_history.rs  — new (Iteration 6)
```

## Sharp edges and subtleties (delta vs. handoff-2)

The previous handoff's sharp edges all still apply. New ones:

- **`Action::Export` runs on a tokio `spawn_blocking` task,
  not the db worker.** Export writes the zip directly from
  disk; auto-save guarantees the project's text sources are
  current. The `history.log` entry for the `export` command
  is appended synchronously from the dispatching arm BEFORE
  the spawn (so the user-issued command lands in history
  even if the export task itself fails).
- **`SwitchRequest::Import` runs `inspect_zip` BEFORE
  dropping the current project.** A failed inspection (zip
  not a project, multiple top folders, traversal entry,
  etc.) leaves the user where they were. The actual
  extraction also runs before the drop. Only after
  extraction succeeds do we drop and reopen.
- **`ProjectSwitched` is now a 3-field event.** Tests that
  construct it directly need the extra `history_entries:
  Vec::new()` field. Iteration-4b had one such test;
  updated.
- **Migration runs inside `perform_switch` AFTER the lock
  on the new project is acquired but BEFORE the database
  opens.** Order matters: a migration that mutates
  `project.yaml` while another process holds the lock
  would corrupt the file; doing the migration after our
  own lock is held prevents that.
- **`migrate_to_latest` writes the `.bak` BEFORE running
  any migrator.** If a migrator panics or returns an
  error mid-chain, the `.bak` is the only intact copy of
  the original. The runtime currently does not auto-restore
  on failure — that's part of "future work" once a real
  migrator lands.
- **`--resume` errors print to stderr BEFORE the terminal
  is set up.** If the user is debugging by reading
  `--log-file`, the resume error is in the shell, not the
  log.
- **`last_project` write failures are non-fatal** (logged
  via `tracing::warn`). Rationale: a failed write here
  surfaces on the *next* `--resume` attempt with a clear
  message, which is preferable to refusing to launch the
  app over a stat / chmod hiccup.
- **The `zip` crate features are restricted** to
  `default-features = false, features = ["deflate"]` to
  hold the binary-size cost down. A future cipher /
  compression demand can revisit.

## Pending — proposed next moves (in order)

Track-2's iteration backlog is now empty; ADR-0015 ships
the runtime as designed. The remaining items are the
deferred features called out in handoff-2's "Other deferred
items" list:

### 1. Complex WHERE expressions (C5a)

AND/OR/comparison operators/LIKE in UPDATE/DELETE/show-data
filters. The natural progression from DSL fluency into
real SQL. Needs a small ADR for the operator subset.

### 2. Indexes (C3 partial) + EXPLAIN QUERY PLAN (QA1)

Strong teaching demo. `add index <name> on <T>(<col>)` /
`drop index <name>`, plus rendering the `EXPLAIN QUERY PLAN`
output as an annotated tree (QA2 covers the tree rendering
specifics in its own ADR).

### 3. Column drops/renames/type changes (B2 / C2 partial)

The `rebuild_table` primitive already exists (ADR-0013).
The grammar additions and metadata updates are
straightforward; the work is mostly tests covering the
data-preservation invariants.

### 4. Friendly error layer (H1)

Translate raw SQLite messages to learner-friendly
equivalents. Partial today (FK errors are enriched both
ways); full SQL → English translation is the open work.

### 5. `replay` (U4)

The `history.log` format is already replay-compatible.
`replay <path>` runs commands from a `history.log` or
`.commands` file. The framework lands here; the U-series
items (snapshot/undo/redo, ADR-0006) follow.

### 6. CI (TT5)

Test infrastructure is in place; the GitHub Actions
workflow file (or equivalent) is not.

### 7. Bigger UX work

V4 session log + Markdown export, S2 indexes in the items
list, V1/V2 pretty rendering, H1a strong syntax-help. All
have their entries in `docs/requirements.md` and remain
deferred behind their respective ADRs.

## How to take over

1. Read this file.
2. Read `CLAUDE.md` for the working-style rules.
3. Read `docs/requirements.md` for granular progress.
4. Skim `docs/adr/README.md`; read ADR-0015 in full
   (especially §11 with the import-collision amendment)
   if you'll touch the project storage runtime, the
   archive module, or the migration framework.
5. Run `cargo test` to confirm the 408-test green
   baseline.
6. `cargo run --release -- --help` to see the updated CLI
   banner.

### End-to-end smoke test

Verifies export, import, --resume, and persistent history.
Same data-dir flag throughout so the test is contained.

```
# Set up: launch under a clean data dir.
$ rm -rf /tmp/rdbms-iter5-iter6-smoke
$ rdbms-playground --data-dir /tmp/rdbms-iter5-iter6-smoke

# Inside the app:
help                                    -- new commands listed
create table Customers with pk id:serial
add column Customers: Name (text)
insert into Customers ('Alice')
insert into Customers ('Bob')
save                                    -- name it "MyOrders"
export                                  -- writes
                                        -- /tmp/rdbms-iter5-iter6-smoke/
                                        --   20260508-MyOrders-export-01.zip
quit

# Verify the zip on disk:
$ unzip -l /tmp/rdbms-iter5-iter6-smoke/*.zip
# Should show:
#   MyOrders/project.yaml
#   MyOrders/data/Customers.csv
#   MyOrders/.gitignore
# and NOT:
#   MyOrders/playground.db
#   MyOrders/history.log

# Re-open via --resume and verify history is hydrated:
$ rdbms-playground --data-dir /tmp/rdbms-iter5-iter6-smoke --resume
# Up arrow should walk back through the export, save,
# inserts, add column, create table — all from the previous
# session.

# Inside the app:
import /tmp/rdbms-iter5-iter6-smoke/20260508-MyOrders-export-01.zip
                                        -- creates MyOrders-02
                                        -- (auto-suffix because
                                        --  MyOrders already exists),
                                        -- switches to it,
                                        -- rebuilds .db from text.
show data Customers                     -- 'Alice' and 'Bob' present.
quit

# Final clean-up:
$ rm -rf /tmp/rdbms-iter5-iter6-smoke
```

If anything in that sequence fails, something is wrong.

### Manual spot-checks worth running

- `--resume` with a missing `last_project` → stderr message.
- `--resume` with a stale recorded path → stderr message.
- `--resume <path>` (combined with positional) →
  `ResumeWithPath` error from arg parsing.
- `export` with no current data dir created yet (rare;
  data root resolution still works).
- `import <zip-with-multiple-top-folders>` →
  `MultipleTopFolders` error in the output panel.
- `import <random.zip>` (no project.yaml in it) →
  `NotAProjectArchive` error.
- After the scaffold migration framework: hand-edit a
  project's `project.yaml` to `version: 99`, restart →
  `migrate project.yaml` context error in the run-time
  startup error path. (Or: write a real v1→v2 migrator
  and watch it execute.)