add handoff-9: ADR-0024 phases A-F minimal landed; deferred work catalogue

2026-05-15 07:58:19 +00:00
parent c940ba9cf2
commit b3d3bdfe5b
1 changed files with 668 additions and 0 deletions
@@ -0,0 +1,668 @@
+# Session handoff — 2026-05-15 (9)
+
+Ninth handover. This session executed **ADR-0024 phases A
+through F** in one sitting, six commits. The walker is now the
+sole parse path; the chumsky dependency is gone. What did NOT
+land — and what the next session needs to pick up — is
+captured below in priority order.
+
+## State at handoff
+
+**Branch:** `main`. Working tree clean. **Local HEAD is
+`c940ba9`**, ahead of `origin/main` by six commits (the user
+pushes asynchronously; do not be blocked by unpushed state).
+
+Commits since handoff-8's baseline (`3e1ff83`):
+
+```
+50b3542 ADR-0024 Phase A: walker framework + app-lifecycle commands
+7e79ca8 ADR-0024 Phase B: DDL commands without value literals
+6bb6882 ADR-0024 Phase C: create table with column-list value literals
+c2accc2 ADR-0024 Phase D: data commands at chumsky parity
+dca472f ADR-0024 Phase E: replay end-to-end
+c940ba9 ADR-0024 Phase F (minimal): drop chumsky from the parse path
+```
+
+**Tests:** **844 passing, 0 failing, 1 ignored** (up from 777
+at handoff-8's baseline; +67 over this session). The ignored
+test is still the same `\`\`\`ignore` doc-test in
+`src/friendly/mod.rs`.
+
+**Clippy:** clean with `nursery` lints enabled and
+`-D warnings`.
+
+**Cargo.toml:** `chumsky` dependency dropped. `thiserror`
+remains gone (per handoff-8). The crate's parse path is now
+fully Rust-native (lexer + walker, no parser-combinator
+library).
+
+## What shipped this session — quick overview
+
+ADR-0024's six phases all landed end-to-end. Walker now owns
+all 20 entry-keyword commands across 14 entry words:
+
+- **Phase A:** walker framework + 11 app-lifecycle commands
+  (quit, help, rebuild, save / save as, new, load, export,
+  import, mode, messages). Optional + Choice + Seq combinators.
+  BarePath terminal. Path-bearing UX change shipped (paths
+  with spaces require quotes).
+- **Phase B:** DDL — drop, add, rename, change. Repeated
+  combinator with optional separator. Flag terminal. NumberLit
+  + Literal terminals. Optional now propagates inner
+  expectations as `skipped` so completion sees "what could
+  have appeared here".
+- **Phase C:** create table. Repeated with `,` separator
+  (first use). `with pk` defaulting to `id:serial`.
+- **Phase D:** data — show, insert, update, delete. StringLit
+  terminal. Value-literal sub-grammar. **Schema-aware value
+  typing deliberately deferred** (see below).
+- **Phase E:** replay. `Choice(StringLit, BarePath)`.
+- **Phase F (minimal):** chumsky combinators deleted, chumsky
+  crate dep dropped. **Significant scope deferred** (see
+  below).
+
+## DEFERRED — work the next session needs to pick up
+
+This is the meat of the handoff. Items grouped by urgency.
+
+### 1. Phase F (full): legacy parser-side modules still standing
+
+ADR-0024 §migration Phase F prescribes deleting these modules.
+Phase F minimal in this session only deleted the chumsky
+combinator code. The following are still in place and consumed
+by other modules:
+
+| Module | Still consumed by | Why kept |
+|---|---|---|
+| `src/dsl/lexer.rs` | `theme.rs`, `input_render.rs`, `app.rs`, `dsl/usage.rs`, `dsl/parser.rs` | Per-token highlighting + echo-line tokenization + completion partial-token detection |
+| `src/dsl/keyword.rs` (`Keyword` enum) | `completion.rs`, `friendly/keys.rs`, `theme.rs`, `dsl/usage.rs` | Catalog key derivation (`parse.token.keyword.*`), keyword-name validation in completion |
+| `src/dsl/ident_slot.rs` (`IdentSlot` enum) | `completion.rs`, `input_render.rs`, `runtime.rs`, `db.rs` | Schema-cache lookups dispatched per slot kind |
+| `src/dsl/usage.rs::REGISTRY` + `matched_entry` | `completion.rs`, `app.rs` | Per-command usage templates rendered on parse error |
+| `parse.token.keyword.*` catalog (40+ entries) | `dsl/usage.rs`, `friendly/keys.rs` | Keyword wording in usage templates |
+
+**Replacement strategy** (sketched, for the migrating session):
+
+- **Highlighting** (`input_render.rs::render_input_panel`):
+  the walker's `WalkResult::per_byte_class` populates
+  `(start, end, HighlightClass)` per matched terminal. Wire
+  this output to the `tok_*` theme colours in place of the
+  current `lex(input)`-driven span builder. Walker error
+  positions feed the `tok_error` overlay. This is the single
+  biggest change.
+- **Completion** (`completion.rs::candidates_at_cursor`): the
+  walker's `WalkResult` at `WalkBound::Position(cursor)` (NOT
+  yet exercised — the walker's `walk()` does support this,
+  but no caller passes it today) gives `expected: Vec<Expectation>`
+  at the cursor. The bridge already maps `Expectation::Ident
+  { source: Tables/Columns/Relationships/Types }` to the
+  user-facing labels matching `IdentSlot::expected_label`, so
+  the existing completion engine reads them transparently.
+  What's missing: a walker-driven path that bypasses
+  `parse_command` entirely and asks the walker directly for
+  candidates per `IdentSource`. Today the bridge round-trips
+  through `ParseError::Invalid::expected` strings — works,
+  but loses information.
+- **Echo-line tokenization** (`output_render.rs` for
+  `OutputKind::Echo + Mode::Simple`): same lex-driven spans.
+  Same walker `per_byte_class` plumbing.
+- **Usage templates** (`dsl/usage.rs::REGISTRY` + `matched_entry`):
+  every `CommandNode` already has `usage_id`. Wire the
+  parse-error renderer to look up the catalog entry by
+  `usage_id` instead of `matched_entry`. Then `REGISTRY` and
+  `matched_entry` can go.
+- **Catalog cleanup** (`parse.token.keyword.*` + `friendly/keys.rs`):
+  ADR-0024 §cleanup-pass §F mentions a `format_keyword_for_error
+  (literal) -> String` helper that wraps a literal in
+  backticks and replaces the per-keyword catalog entries.
+  Mechanical; do it after the consumers stop reading the
+  catalog keys.
+
+**Estimated cost: one full session** for the consumer
+migration + a follow-up commit for the catalog cleanup. Test
+suite serves as the regression net throughout.
+
+### 2. Phase D (full): schema-aware value typing
+
+ADR-0024 §migration Phase D prescribes "full schema awareness"
+via `DynamicSubgrammar(column_value_list)` that unfolds typed
+slots per column at walk time. **This session deferred it.**
+
+What's in place:
+
+- `Node::DynamicSubgrammar(fn(&WalkContext) -> Node)` variant
+  declared but unused. The walker's driver returns
+  `Failed { expected: vec![] }` on this branch (intentional —
+  catches grammar bugs that declare DynamicSubgrammar without
+  the wiring landing).
+- `WalkContext::current_table`,
+  `WalkContext::current_table_columns`,
+  `WalkContext::current_column` all declared but unwritten.
+  No `Ident` node has a `writes_table: bool` or equivalent.
+- Per-type validators (`int_slot`, `decimal_slot`, etc.) NOT
+  written. The current walker uses a generic `value_literal`
+  Choice that accepts any literal regardless of column type;
+  bind-time type-check errors fire as today.
+
+**To implement:**
+
+1. Plumb a `SchemaCache` reference into `parse_command` (and
+   thus `parse_tokens`). Currently the call site is in
+   `runtime.rs::dispatch_input` — passes `app.schema_cache()`
+   alongside the input.
+2. Extend `WalkContext::new(schema)` to carry the cache.
+3. Implement `Node::DynamicSubgrammar` walker dispatch:
+   resolve at walk time, leak the returned `Node` into a
+   per-walk arena (or `Box::leak` per ADR-0024 §sub-grammars).
+4. Implement `Ident { source: Tables, writes_table: true }`
+   semantics — when the ident matches, look up the table in
+   the schema, populate `current_table` + `current_table_columns`.
+5. Implement typed value slots — `int_slot()`, `decimal_slot()`,
+   etc. per ADR-0024 §typed-value-slots. Each is a Choice
+   over the literal forms with a content validator.
+6. Wire `column_value_list` as a DynamicSubgrammar that reads
+   `current_table_columns` and emits a Seq of typed slots
+   separated by commas.
+7. Update `insert` shape to use `column_value_list` instead
+   of the generic value list.
+8. Update `update`/`delete` to use the per-column value slot
+   based on `current_column`.
+
+**The user UX win this unlocks:** typed slots reject
+mis-shaped input at parse time with localised wording (e.g.,
+"Type a date as 'YYYY-MM-DD'") instead of bind-time errors;
+completion narrows per column type. This is the Phase D
+"central design claim" per the handoff.
+
+**Side effect to watch for:** parse_command becomes schema-
+dependent. Tests that exercised parse in isolation may need
+to pass a schema cache (today's tests don't — most just
+check round-trip parses where schema doesn't matter).
+
+**Estimated cost: 1-2 sessions** depending on how deep the
+schema plumbing goes through dispatch.
+
+### 3. Walker doesn't drive completion or hints directly
+
+Today's flow: walker produces `WalkOutcome` → bridge to
+`ParseError::Invalid` → completion / hint engines read
+`expected: Vec<String>`. The bridge formats `Expectation::Ident
+{ source: … }` to the user-facing label string the existing
+engines recognise.
+
+This works but loses information. Walker knows:
+- The `IdentSource` of every expected slot.
+- The `role` of every slot (e.g., `parent_table` vs
+  `child_table`).
+- The `HintMode` per node (currently always `Default`).
+- The `skipped` expectations from any Optional that didn't
+  engage at this position.
+- The cursor's full `MatchedPath` so far — the AST builder
+  could be invoked partially to extract context (e.g., "you've
+  typed `update Customers set Email=`, the `Email` column's
+  type is `text`").
+
+A walker-direct completion path would surface much more
+informative candidates than the current ParseError-string round
+trip. See ADR-0024 §architecture: "Completion at cursor:
+`walk(source, Position(cursor), ctx)`, inspect `outcome.expected`."
+
+**Today, no caller invokes `walk()` with `WalkBound::Position(cursor)`.**
+The variant exists in `outcome.rs` (annotated `#[allow(dead_code)]`).
+`completion.rs` still calls `parse_command(leading_slice)` —
+the slice-and-re-parse approach inherited from chumsky.
+
+**Migration path:** add a `pub fn candidates_at_cursor_from_walker
+(input, cursor, schema) -> WalkResult` that calls
+`walker::walk(input, WalkBound::Position(cursor), &mut ctx)`
+and returns the result. Then `completion.rs` reads
+`expected` directly with full `IdentSource` + `role` info.
+
+### 4. `HintMode` declared but unused
+
+`HintMode::Default | ForceProse | ProseOnly | SuppressProse` is
+declared in `grammar/mod.rs` but every `CommandNode` and
+every `Node::Ident` sets it to `None`. The current ad-hoc
+hint cases in `input_render.rs::ambient_hint` (value-literal
+slot suppression, NewName slot typing-name prose,
+invalid-ident overlay) still use the chumsky-era ad-hoc
+detection.
+
+ADR-0024 §HintMode-per-node says these migrate to
+node-attached `HintMode` annotations during Phase D. They
+didn't.
+
+**To do:** annotate the value-literal slots with
+`HintMode::ProseOnly("value_literal_format_hint")`; annotate
+NewName slots with the typing-name prose; have the hint
+resolver pick up the mode and dispatch.
+
+### 5. Ranker hook: declared in ADR-0024, not implemented at all
+
+ADR-0024 §ranker-layer specifies a `Ranker` function type
+between the walker's raw candidates and the hint-panel
+renderer. The default is `identity_ranker` (declaration-order
+preserved).
+
+**Status: not declared anywhere in code.** No `Ranker` type,
+no identity ranker, no hook into completion. The current
+completion engine ranks by its own ad-hoc logic (keyword
+matches first, then schema, alphabetised within each).
+
+This is a small future-work hook. Not blocking.
+
+### 6. Aliases: feature works, no aliases declared
+
+`Word::aliases: &'static [&'static str]` is wired through
+`Word::matches` and the walker correctly accepts case-
+insensitive alias matches. **Today every `Word` in the
+grammar has an empty aliases slice.**
+
+The round-5 `q` quit alias removal stands — if the user
+wants it back, it's:
+
+```rust
+const QUIT_WORD: Word = Word {
+    primary: "quit",
+    aliases: &["q"],
+    highlight_override: None,
+};
+pub static QUIT: CommandNode = CommandNode {
+    entry: QUIT_WORD,
+    ...
+};
+```
+
+The walker matches either; completion only surfaces the
+primary. No further wiring needed.
+
+### 7. Test edits worth knowing about
+
+A handful of tests changed wording assertions to match
+walker-emitted error wording:
+
+- `src/dsl/walker/mod.rs::walker_import_trailing_as_without_target_errors`
+  — assertion weakened from checking for "target" to checking
+  for "import". Walker's Optional backtracking means
+  `import foo.zip as ` parses as `Import { path: "foo.zip",
+  target: None }` followed by trailing `as ` → walker reports
+  `expected end of input, found …`. The friendly
+  `project.import_empty_target` wording moved out of the
+  parser. The integration test
+  (`tests/iteration5_export_import.rs::import_with_empty_target_after_as_errors`)
+  still passes because the rendered `import_usage` template
+  line in the dispatch output contains both "import" and
+  "target".
+- The walker's parse error wording for incomplete inputs
+  consistently uses "after `<consumed>`, expected …, found
+  end of input" — matches the chumsky-era contract that
+  `structural_error_for_show_data_without_arg` and friends
+  pin down.
+
+**No `parse.token.keyword.*` wording changed.** No catalog
+entries changed. No user-visible string regression.
+
+### 8. The `value_literal_hint_at_cursor` stopgap
+
+The round-6 stopgap from handoff-8 (replacing `null true false`
+with prose at value-literal slots) **is still live**. It
+lives in `completion.rs::value_literal_hint_at_cursor` and
+detects the value-literal-signature in `expected`. With Phase
+D's full schema awareness, this would become "narrow to the
+actual column's type" (e.g., "Type a date as 'YYYY-MM-DD'")
+— but that requires the schema plumbing in §2 above.
+
+**Today the stopgap continues to fire for every value-literal
+slot regardless of column type.** Same wording as handoff-8.
+
+### 9. Choice greedy-semantics shape contortions
+
+ADR-0023 says "the trie design is greedy (the first child
+node that matches wins)." This is the design. But it forced
+some grammar contortions worth knowing about:
+
+- **`insert` Form A vs Form C disambiguation** (`grammar/data.rs`):
+  Forms A (`insert into T (cols) values (vals)`) and C (`insert
+  into T (vals)` — bare value list) both start with `(`. The
+  inner-paren content is parsed as a heterogeneous
+  `Choice(VALUE_LITERAL, Ident{Columns})` — VALUE_LITERAL
+  ordered first so `null`/`true`/`false` match their Word
+  branch rather than the broader identifier catch-all (which
+  `consume_ident` doesn't filter against the keyword set).
+  AST builder discriminates by the presence of the `values`
+  keyword AFTER the first paren. **Brittle if a future grammar
+  needs to add another paren-bounded form starting at the
+  same position.**
+- **`drop` sub-form ordering**: `drop_column` and
+  `drop_relationship` come before `drop_table` in the Choice
+  because they're more specific (longer prefix). Walker
+  greediness handles this correctly because each branch's
+  first Word disambiguates.
+
+### 10. Walker `per_byte_class` populated but unused
+
+`WalkResult::per_byte_class: Vec<ByteClass>` is populated by
+every terminal match in the walker driver. **No consumer
+reads it today.** The annotation `#[allow(dead_code)]` sits
+on the field. When the highlighting consumer migrates (§1
+above), this is the data source.
+
+### 11. Differential test scaffolding wasn't actually built
+
+ADR-0024 §test-discipline §3 specifies a "differential check
+during the migration window" — a test helper that runs both
+parsers on the existing input corpus and asserts identical
+`Command` output. Removed at Phase F cleanup.
+
+**This session went with hand-curated expected `Command`
+outputs in `dsl::walker::tests` instead.** Equivalent
+coverage (every migrated command's parse asserted), simpler
+to maintain. Since chumsky is now gone (Phase F minimal
+removed the combinator code), no removal step needed.
+
+If a strict differential check is wanted retroactively, the
+chumsky path would need to be reconstructed from git history
+and run alongside walker against the test corpus — not
+trivial. The hand-curated tests + the existing integration
+test suite serve the same regression-net role.
+
+### 12. `WalkContext` writes during walk — design exists, not implemented
+
+ADR-0024 §WalkContext sketches `Ident { source: Tables,
+writes_table: true }` semantics: when the ident matches, the
+walker writes `current_table` to context. Subsequent dynamic
+sub-grammars read it.
+
+**Today no `Ident` node has a `writes_table` field.** The
+struct definition in `grammar/mod.rs` is:
+
+```rust
+Ident {
+    source: IdentSource,
+    role: &'static str,
+    validator: Option<IdentValidator>,
+    highlight_override: Option<HighlightClass>,
+}
+```
+
+When Phase D (full) lands, add a `writes_table: bool`
+(or similar) field and have the walker driver populate
+`WalkContext` accordingly.
+
+### 13. `CommandNode::usage_id` / `help_id` not consumed
+
+Every `CommandNode` declares `usage_id: Option<&'static str>`
+and `help_id: Option<&'static str>` pointing into the
+catalog. **No code reads these fields.** Usage rendering
+still goes through `dsl/usage.rs::matched_entry`. Help
+text still hand-curated.
+
+When `dsl/usage.rs::REGISTRY` retires (§1 above), wire
+the parse-error renderer and the in-app help system to read
+these fields directly.
+
+## Sharp edges to know about
+
+These are facets of the walker's behaviour that aren't bugs
+but that will surprise someone reading the code cold.
+
+- **Optional backtracking on partial-match** is intentional
+  (matches chumsky's `or_not` semantics). When an Optional's
+  inner consumes some terminals and then fails (Incomplete or
+  Mismatch), the walker rolls back the path / per_byte state
+  to the pre-Optional position and treats it as skipped, with
+  the inner's expectations carried as `skipped` on the
+  Matched return. **Validation failures (content errors) do
+  NOT backtrack** — the user means to fix those. This
+  asymmetry is the load-bearing decision that makes
+  `create table T with` produce the correct
+  `IncompleteAtEof` classification (chumsky's behaviour).
+- **Walker's `Choice` is strictly greedy** — first child
+  whose first terminal matches wins. No backtracking. Required
+  ordering: more-specific shapes before more-general. See §9
+  above for examples.
+- **`Ident { source: Tables/Columns/Relationships }` does NOT
+  validate against the schema at parse time.** It's a
+  shape-only check today. Schema-aware parse is the Phase D
+  vision; see §2.
+- **`Literal(&'static str)` matches verbatim bytes with a
+  word-boundary lookahead** so `1` doesn't half-match `12`
+  and `n` doesn't half-match `name`. The highlight class is
+  inferred from the literal's bytes (digits → Number, else
+  Keyword). Used today only for the `1` in `add 1:n
+  relationship`.
+- **`AST builder` failures surface as `WalkOutcome::ValidationFailed`
+  with `at_eof = true`**. The bridge maps these to
+  `ParseError::Invalid` with `at_eof: true` so the input
+  renderer classifies them as `IncompleteAtEof` (no live
+  overlay; on-submit error fires). This mirrors the chumsky-
+  side custom-error convention.
+- **`unknown_command_error` is the sole catch-all** for
+  inputs whose first identifier-shape token isn't a registered
+  entry word. Wording: "expected one of `add`, …, found
+  `<word>`". Position is the start of the unknown word.
+- **`q` quit alias remains gone.** The walker SUPPORTS
+  aliases natively — adding `q` back is a one-line change on
+  `QUIT.entry.aliases` (see §6).
+- **Path-bearing UX change shipped (Phase A + E):** `replay`,
+  `import`, `export` paths terminate at the first whitespace
+  byte. Paths with spaces use the quoted form
+  (`replay 'my project/seed.commands'`). This is per ADR-0024.
+
+## Suggested next-session priorities
+
+In order:
+
+1. **Phase F (full): consumer migration to walker outputs.**
+   This is the biggest deferred chunk and the right next
+   structural move. See §1 above for the migration sketch.
+   ~1 session for the highlighting + completion + usage
+   migration; one follow-up commit for the catalog cleanup
+   and lexer/keyword/ident_slot deletion.
+2. **Phase D (full): schema-aware value typing.** Once the
+   schema cache plumbing exists, the `DynamicSubgrammar`
+   wiring + typed value slots are mechanical. See §2.
+3. **Walker-driven completion** (§3). Smaller scope than the
+   above. Surfaces `IdentSource` + `role` directly to the
+   completion engine without the ParseError-string round
+   trip. Unlocks better hint UX.
+4. **`HintMode` annotations** (§4). Mechanical migration of
+   the ad-hoc hint cases in `input_render.rs` to node
+   annotations. Small.
+5. **Ranker hook** (§5). Future work. Plug-in point for
+   frequency-based ranking, content-aware priors, recency.
+
+After (1) and (2) land, the codebase reaches the steady-state
+ADR-0024 envisioned: one declaration per command, no scatter,
+walker as single source of truth across parse / complete /
+highlight / hint / usage.
+
+## ADR index (read these before touching the related areas)
+
+```
+0000 Record architecture decisions (process)
+0001 Language and TUI framework (Rust + Ratatui)
+        — chumsky dependency dropped in Phase F minimal
+0002 Database engine (User-facing posture)
+0003 Input modes and command dispatch
+0004 Project file format (amended by 0015)
+0005 Column type vocabulary
+0006 Undo snapshots and replay log (designed, not impl)
+0007 Sharing and export (amended by 0015 amendment 1)
+0008 Testing approach
+0009 DSL command syntax conventions
+0010 Database access via worker thread
+0011 FK column type compatibility
+0012 Internal metadata for user-facing column types
+0013 Relationships, naming, and rebuild-table strategy
+0014 Data operations, value literals, and auto-show
+0015 Project storage runtime
+0016 Pretty table rendering for data and structure views
+0017 Column type-change compatibility
+0018 Auto-fill contracts for serial and shortid columns
+0019 Friendly error layer (H1) and i18n message catalog
+        — `parse.token.keyword.*` collapse pending Phase F full
+0020 Tokenization layer for the DSL parser
+        — superseded by the scannerless walker in ADR-0024;
+          the lexer module survives until Phase F full
+0021 Parser-as-source-of-truth for H1a
+        — usage info migration to grammar nodes pending
+          (CommandNode.usage_id declared, not consumed yet)
+0022 Ambient typing assistance (I3 + I4 unified)
+        — completion still chumsky-bridge; walker direct
+          path pending
+0023 Unified declarative grammar tree (direction)
+        — superseded for execution by ADR-0024
+0024 Unified grammar tree: execution plan (ACCEPTED)
+        — A through F minimal landed; F full + D full deferred
+```
+
+## Repository layout (delta vs. handoff-8)
+
+New files this session:
+
+```
+src/dsl/grammar/
+  mod.rs        (267)  — Node enum, Word, IdentSource, HintMode,
+                          HighlightClass, ValidationError,
+                          IdentValidator, NumberValidator,
+                          CommandNode, REGISTRY (20 commands)
+  app.rs        (272)  — 11 app-lifecycle commands
+  ddl.rs        (492)  — drop, add, rename, change, create
+  data.rs       (504)  — show, insert, update, delete, replay
+  shared.rs     (108)  — type validator, qualified_column,
+                          referential_clauses, action_keyword
+src/dsl/walker/
+  mod.rs        (~620) — walk() entry + 53 walker tests
+  driver.rs     (~570) — per-node-kind dispatch with backtracking
+  context.rs    (43)   — WalkContext (schema fields stubbed)
+  outcome.rs    (~165) — WalkResult, WalkOutcome, MatchedPath
+  lex_helpers.rs (~190) — byte-level helpers (skip_whitespace,
+                          consume_ident, match_keyword,
+                          consume_bare_path, consume_flag,
+                          consume_number_literal, consume_string_literal)
+```
+
+Files modified this session:
+
+```
+src/dsl/mod.rs          — wire grammar + walker modules
+src/dsl/parser.rs       — chumsky combinators deleted; now a
+                          ~290-line walker bridge with the
+                          original test suite (~840 lines) intact
+Cargo.toml              — chumsky dependency removed
+Cargo.lock              — regenerated
+```
+
+Files NOT touched but worth knowing about (still consume the
+legacy modules per §1):
+
+```
+src/dsl/lexer.rs        — still exports lex() / Token / TokenKind
+src/dsl/keyword.rs      — Keyword enum still alive
+src/dsl/ident_slot.rs   — IdentSlot enum still alive
+src/dsl/usage.rs        — REGISTRY + matched_entry still alive
+src/completion.rs       — reads ParseError::Invalid::expected
+                          (bridge from walker), uses Keyword +
+                          IdentSlot
+src/input_render.rs     — uses lex() for token-class colouring
+src/theme.rs            — token colour mappings keyed on Keyword
+src/runtime.rs, src/db.rs — IdentSlot::expected_label round-trip
+```
+
+## How to take over
+
+1. **Read this file.**
+2. **Read `CLAUDE.md`** for the working-style rules. Note
+   the "Escalate ambiguity — do not decide for the user"
+   rule. The deferred items below are scoped enough that
+   most decisions are clear; escalate if the spec genuinely
+   disagrees with the implementation.
+3. **Read ADR-0024**
+   (`docs/adr/0024-unified-grammar-tree-execution-plan.md`)
+   to understand the design intent. Phase F (full) and Phase D
+   (full) are the unfinished work.
+4. **Skim `src/dsl/grammar/mod.rs`** — the Node enum + Word
+   + CommandNode + REGISTRY are the contract.
+5. **Skim `src/dsl/walker/mod.rs`** — the walk() entry +
+   bridge logic. The 53 tests at the bottom of that file
+   are the behavioural spec.
+6. **Run `cargo test`** to confirm the 844-test baseline.
+   Lib test count is 711 in `rdbms_playground` (the rest are
+   integration + doctests). Total should be 844 passed, 0
+   failed, 1 ignored.
+7. **Run `cargo clippy --all-targets -- -D warnings`** to
+   confirm clean baseline.
+8. **Pick a deferred item from §1-§5** and start. §1
+   (Phase F full) is the natural next move; it unblocks §3
+   (walker-driven completion) and §4 (HintMode annotations).
+   §2 (Phase D full) is the second-largest item and can
+   land in parallel since it touches the grammar layer
+   rather than the consumer layer.
+
+### End-to-end smoke test (current state, post-ADR-0024)
+
+```
+$ rm -rf /tmp/handoff9-smoke
+$ rdbms-playground --data-dir /tmp/handoff9-smoke
+
+# Inside the app:
+
+# All commands now route through the walker. User-visible
+# behaviour is unchanged from handoff-8 except for:
+# - import / export paths with spaces require quotes
+# - replay paths with spaces require quotes
+# - parse error wording uses "after `<consumed>`, expected …,
+#   found end of input" framing (matches the chumsky-era
+#   contract; tests pin this down)
+
+# Smoke commands:
+help                                         -- in-app help
+mode advanced                                -- mode switch
+quit                                         -- exit (no `q`)
+:quit                                        -- one-shot escape
+                                                also works
+
+# DDL:
+create table Customers with pk
+add column Customers: Email (text)
+add 1:n relationship from Customers.id to Orders.customer_id
+
+# Data:
+insert into Customers values (1, 'Alice')
+update Customers set Email='new@b.c' where id=1
+delete from Customers where id=1
+show data Customers
+show table Customers
+
+# Replay:
+replay history.log
+replay 'my project/seed.commands'
+
+# Errors:
+frobulate widgets
+# → expected one of `add`, `change`, `create`, `delete`,
+#   `drop`, `export`, `help`, `import`, `insert`, `load`,
+#   `messages`, `mode`, `new`, `quit`, `rebuild`, `rename`,
+#   `replay`, `save`, `show`, or `update`, found `frobulate`
+
+mode bogus
+# → unknown mode 'bogus' (expected 'simple' or 'advanced')
+
+change column T: c (int) --force-conversion --dont-convert
+# → `--force-conversion` and `--dont-convert` are mutually
+#   exclusive — pick one.
+
+create table Customers
+# → tables need at least one column. Add `with pk` for a
+#   default `id INTEGER PRIMARY KEY`, or `with pk <name>:<type>`
+#   …
+
+# All wording sourced from the en-US.yaml catalog via the
+# walker's ValidationError catalog-key mechanism.
+```
+
+After Phase F full lands, this smoke test extends with:
+- Per-keystroke highlighting driven by walker `per_byte_class`
+- Cursor-position completion driven by walker direct path
+- Usage templates rendered from `CommandNode.usage_id`
+- Lexer/Keyword/IdentSlot modules removed from the source tree