rdbms-playground/docs/handoff/20260519-handoff-24.md

Session handoff — 2026-05-19 (24)

Twenty-fourth handover. A design-only session: it wrote and accepted ADR-0030 — Advanced mode: the standard-SQL surface. No code changed. A fresh session implements ADR-0030 from this file + the ADR, starting at Phase 1.

§1. State at handoff

Branch: main. Working tree clean (after this docs commit). The session's commits since handoff-23 are docs-only — ADR-0030, the ADR index, the requirements.md updates, and this handoff. All local; push asynchronously, not blocking.

Tests: unchanged — 1240 passing, 0 failing, 1 ignored as of a049ff9 (ADR-0029 complete). This session touched no code. Clippy: clean as of the same commit.

§2. What was decided — ADR-0030

Read docs/adr/0030-advanced-mode-sql-surface.md — the spec, complete and accepted. Its shape, and why it landed where it did (the design went through two rounds of user input — §3 records the reasoning so it is not relitigated):

• SQL is grammar in the unified tree, not a separate parser (§1). SQL statements are authored as CommandNode / Node grammar in the ADR-0024 tree and parsed by the existing walker. sqlparser-rs is not used — ADR-0001's reservation is superseded. The reason is decisive: a batch SQL parser produces an AST and nothing else; it cannot drive completion, highlighting, or hints. The user requires SQL to have the same ambient assistance as the DSL — and that assistance comes only from the unified grammar tree (ADR-0022/0023/0024). So SQL must live in the tree.
• Mode gates the grammar (§2). One grammar tree; simple mode exposes the DSL subset, advanced mode adds the SQL forms. Shared entry words (create, insert, …) get a Choice of DSL + SQL forms under one CommandNode (how add already holds four sub-commands); select is a new SQL-only entry word.
• Execution split (§4). DDL → a typed Command → the existing executor, so metadata, the playground type vocabulary, and STRICT are preserved. DML and SELECT → executed as validated SQL (they change no schema, so a typed Command buys nothing); the worker re-persists the affected table after DML.
• Engine-neutral (§5, §7). Playground type vocabulary, never engine storage types; no STRICT surfaced; engine-neutral errors. Advanced mode is standard SQL, not an engine console.
• Assistance for free (§8). Because SQL is in the tree, the walker gives SQL completion, highlighting, hint prose, the [ERR]/[WRN] indicator, and parse-error usage — the same as the DSL, no SQL-specific assistance code.
• The DSL→SQL teaching echo (§10). A DSL command run in advanced mode also prints its equivalent SQL.
• Persistence / replay (§11). DDL via Command keeps project.yaml correct; DML re-persists its table; history.log stores the literal line; replay re-runs lines through the one walker.

§3. Decisions the user made (do not relitigate)

1. Full SQL scope — DDL + DML + the full SELECT query surface (joins, aggregates, GROUP BY/HAVING, subqueries, UNION, CTEs). No pre-emptive cuts; if a slice proves genuinely overwhelming, cutting it is an explicit escalation to the user, never a silent trim.
2. No fallback to engine types — advanced DDL keeps the playground's rich type vocabulary. The SQL → Command path for DDL is what guarantees this.
3. Advanced mode is standard SQL, engine-independent — not a SQLite console; no engine-specifics surface.
4. SQL lives in the unified grammar tree (not a separate batch parser) — chosen specifically so SQL gets full completion / highlighting / hints. The user understood and accepted that this means authoring a SQL grammar ourselves; sqlparser-rs is dropped.
5. Command only where it pays. DDL → Command (buys metadata + types); DML / SELECT → validated SQL (a Command buys nothing).
6. The DSL→SQL echo shows in advanced mode only.
7. The persisted / replayable representation must be app-enterable syntax — history.log already stores the literal line, so this holds.

§4. The phased implementation plan

ADR-0030 §Implementation is canonical. Each phase is independently shippable and test-guarded. The two large grammar slices each warrant their own focused ADR when taken up (precedent: ADR-0026 for the WHERE grammar).

1. Foundations + first SELECT. Mode-gate the grammar (advanced unlocks the SQL nodes). Author the core SQL expression grammar — the ADR-0026 superset (arithmetic, function calls, CASE, the predicate set) — its own ADR. A single-table SELECT (projection, WHERE, ORDER BY, LIMIT) as a SQL CommandNode → Command::Select → worker RunSelect → the existing data-table renderer. Replace the placeholder echo; add the simple-mode "this is SQL" hint. Proves the path end to end with full walker assistance.
2. SELECT — full. JOINs, GROUP BY/HAVING, aggregates, subqueries, UNION, CTEs — its own ADR.
3. DML. INSERT/UPDATE/DELETE grammar; the execute-as-validated-SQL path; the worker re-persist step; settle multi-row INSERT and shortid auto-fill on a SQL INSERT.
4. DDL. CREATE/DROP/ALTER TABLE, CREATE/DROP INDEX grammar → Command; the §5 type-name map; FK clauses → AddRelationship; may land table-rename (C1).
5. The DSL→SQL teaching echo (§10).
6. Polish — help sql; engine-neutral error sweep; typing-surface / matrix coverage; the DOC1 SQL-surface reference page.

§5. Seams for the implementer (from a code survey)

Anchors for Phase 1:

• The placeholder to replace: App::submit (src/app.rs ~954–1014) — the advanced-mode branch echoes the input with advanced_mode.not_implemented (~996–1012) and sends nothing to the worker. The DSL dispatch path (dispatch_dsl, ~1089) is what advanced SQL joins.
• The grammar tree: src/dsl/grammar/ — mod.rs holds Node, CommandNode, the REGISTRY; ddl.rs / data.rs hold the DSL command grammars; expr.rs holds the ADR-0026 expression grammar (the seed for the SQL expression grammar); shared.rs shared fragments. The walker is in src/dsl/walker/. SQL grammar is authored here; mode-gating is added to the REGISTRY / walker.
• Command core: Command in src/dsl/command.rs — add a Select variant. Gotcha: every exhaustive match Command breaks (verb / target_table / display_subject in command.rs; execute_command_typed in runtime.rs; build_translate_context in app.rs; command_kind_label in tests/typing_surface/mod.rs) — the ADR-0028/0029 pattern.
• Worker: Request enum + handle_request + the Database method wrappers in src/db.rs (~444 on) — add RunSelect (returns DataResult), and later a "run validated DML + re-persist" request.
• Result rendering: a Select outcome → CommandOutcome::Query → AppEvent::DslDataSucceeded → output_render::render_data_table — all reused from show data.
• Mode: src/mode.rs (Mode::Simple/Advanced); App::effective_mode and the :-strip in App::submit already work — the mode-gated grammar view plugs in there.
• sqlparser-rs is not used — do not add it (an implementer may keep it only as a test oracle, off the execution path).
• Risk: the Node taxonomy / walker may need extension to carry SQL's grammar (deeper recursion for subqueries / CTEs). Expect node-taxonomy work in Phases 1–2; ADR-0030 flags this.

§6. How to take over

1. Read this file, then docs/adr/0030-advanced-mode-sql- surface.md (the spec). Then CLAUDE.md (working-style rules), docs/requirements.md (Q4 ticked; Q1/Q2 unblocked), and skim docs/simple-mode-limitations.md — ADR-0030 §4 commits advanced mode to lifting the expression limits that file records.
2. Run cargo test — 1240 passing, 0 failing, 1 ignored.
3. Run cargo clippy --all-targets -- -D warnings — clean.
4. Start ADR-0030 Phase 1. First sub-step is the SQL expression grammar — write its focused ADR before authoring it (ADR-0026 is the model and the seed). Then the single-table SELECT end to end. Land the Command:: Select match-arm sweep with the worker request in one commit (the exhaustive-match breakage forces it).
5. Escalate, do not guess, on anything ADR-0030 left per-phase — multi-row INSERT, shortid on SQL INSERT, any Node-taxonomy extension that changes the walker's contract.

§7. What else is open

ADR-0030 is the active design. Other open clusters, unchanged from handoff-23 §5 (prioritisation is a user decision): snapshot/undo U-series (ADR-0006 written, unbuilt); m:n convenience C4; modify-relationship C3a; the show family V5; rename-table C1 (may fall out of ADR-0030 Phase 4); friendly-error sweep H1; CI TT5; session-log / Markdown export V4.