rdbms-playground/docs/handoff/20260520-handoff-26.md

Session handoff — 2026-05-20 (26)

Twenty-sixth handover. Design session. ADR-0032 (the full SQL SELECT grammar — ADR-0030 Phase 2) is Accepted and an implementation plan sits next to it under a new docs/plans/ directory. No code changed this session; the next session is implementation territory.

§1. State at handoff

Branch: main. Working tree clean. One unpushed commit (be8a4f5..a7db7dd).

Tests: 1260 passing, 0 failing, 1 ignored — unchanged from handoff-25 (no code touched).

Clippy: clean (last verified at handoff-25; no code changed since).

New commit since handoff-25:

a7db7dd docs: ADR-0032 + Phase 2 plan — full SQL SELECT grammar

A single docs commit; ADR + plan + index update ship as a coherent unit because the plan is Phase-2's mechanics for ADR-0032's decisions.

§2. What got designed — the shape now

Read in order:

1. docs/adr/0032-sql-select-grammar.md — the grammar decisions for Phase 2.
2. docs/plans/20260520-adr-0032-phase-2.md — the implementation plan walking the work in seven sub-phases.

2.1. ADR-0032 in one screen

Phase 2's grammar surface (§§1–9 of the ADR):

• Five JOIN flavours: INNER, LEFT [OUTER], RIGHT [OUTER], FULL [OUTER], CROSS. NATURAL/USING/comma-FROM are OOS.
• All four set ops: UNION, UNION ALL, INTERSECT, EXCEPT.
• CTEs: WITH and WITH RECURSIVE, with optional (col-list).
• Subquery expressions as additive sql_expr branches: scalar (SELECT …) primary, IN (SELECT …), [NOT] EXISTS (…). Redeems ADR-0031 §7 OOS-1.
• Qualified column refs t.c / alias.c as a name_or_call tail. Redeems ADR-0031 §7 OOS-2.
• LIMIT n [OFFSET m]; legacy comma form OOS.
• DISTINCT / ALL, t.* projection, bare-alias projection (lifts Phase-1 §4.2's autonomous decision).
• One new grammar file: src/dsl/grammar/sql_select.rs, parallel to sql_expr.rs, exporting SQL_SELECT_STATEMENT and SQL_SELECT_COMPOUND.

Walker-capability honesty (§10) — the part where ADR-0030 §8's "ambient assistance comes for free" got softened:

• Grammar recursion still needs nothing new — Subgrammar
  • ADR-0026's depth cap unchanged.
• Completion scope needs a new node variant Node::ScopedSubgrammar(&Node) alongside the existing Node::Subgrammar, plus a from_scope_stack: Vec<ScopeFrame> on WalkContext. Each ScopeFrame holds from_scope, cte_bindings, projection_aliases. Existing DSL Expr and sql_expr recursion stays on the non-scope-pushing variant and is unaffected.
• CTE column resolution (§10.3): SELECT * and explicit- projection CTE bodies both yield real column completion past cte_alias.| via a body-projection derivation rule that runs at the body's ScopedSubgrammar exit. The earlier "honest limitation" posture is replaced.
• Qualified-prefix completion (§10.5): at t.|, candidates are scoped to t's binding.
• Projection-before-FROM problem (§10.6): the debounced re-walk catches up via a post-walk fixup pass that re- resolves projection-list identifiers against the final from_scope, rewriting highlight class and validity diagnostic in the walker's accumulated output. Runs as a final stage of the walk itself — same convention applies to the §11.6 predicate-warnings pass.

Diagnostics (§11) — every Phase-2 validation case classified against ADR-0027's ERROR/WARNING guideline:

• Five new diagnostic.* keys (parse-time-detectable): unknown_qualifier, ambiguous_column, projection_alias_misplaced, cte_arity_mismatch, compound_arity_mismatch.
• Eight new engine.* keys for friendly-error layer translations of engine messages.
• Three diagnostic passes by end of Phase 2 (§11.7): schema-existence (extended for multi-binding scope), arity- check (new), predicate-warnings (extended with a MatchedPath-walking variant — §11.6).

Result-column type resolution (§12):

• rusqlite 0.39.0 exposes column_table_name / column_origin_name / column_database_name on RawStatement behind a column_metadata feature. Verified during this session by reading the rusqlite source. Cargo.toml change is one line — add "column_metadata" to the feature list.
• Bare column refs recover their playground type via that metadata; partially lifts Phase-1 §4.5's bool→0/1 deferral.

2.2. The implementation plan

docs/plans/20260520-adr-0032-phase-2.md (881 lines) breaks Phase 2 into seven sub-phases:

• 2a — Grammar fragment (sql_select.rs, no walker changes yet).
• 2b — Node::ScopedSubgrammar variant + the scope accumulators on WalkContext.
• 2c — Phase-1 grammar migration: move Phase-1 SELECT nodes from data.rs into sql_select.rs; the 7 Phase-1 integration tests are the safety net.
• 2d — Diagnostics passes + catalog keys.
• 2e — Completion + post-walk fixup pass.
• 2f — Type resolution via rusqlite column_metadata.
• 2g — Integration sweep + the cross-cut verification matrix (the Phase-1-gap-prevention mechanism).

Each sub-phase has explicit exit gates with required tests and a DA-prompt checklist for the gate review. The plan also encodes the user's standing authorizations:

• Walk uninterrupted between gates unless there's a real ambiguity, autonomous decision, blocker, or in-the-plan-open- question trigger.
• Commits pre-authorized at gate boundaries and natural break points within a sub-phase. Standard area: subject message style, detailed body, no AI attribution.
• Pushes remain user-only.
• End-of-plan hand-back is the one explicit pause: the verification report + DA PASS verdict get surfaced to the user before Phase 2 is declared complete.

The cross-cut verification matrix (29 rows) names every "X comes for free" claim from ADR-0030/0031/0032 with a test- location placeholder. The Phase-1 SQL-expression predicate- warning gap is row 20 by name — making sure an analogous silent gap cannot ship undetected.

§3. The Phase-1 finding worth carrying forward

While walking ADR-0027 Amendment 1's existing-cases inventory during this session, the warning-vs-error guideline check surfaced a Phase-1 carry-over gap:

ADR-0027 Amendment 1's LIKE-on-numeric warning — together with ADR-0026 §7's = NULL and type-mismatch warnings — is emitted by a pass that walks the DSL Expr AST. Phase 1's sql_expr.rs deliberately builds no AST (ADR-0031 §2). The consequence: SQL WHERE expressions today emit none of these warnings. select * from t where name like 5 parses, the engine runs it, and the learner gets the engine's verdict, not the friendly pre-flight nudge Amendment 1 promised.

Phase 1 shipped structurally green (1260 / 0 / 1) but the validity indicator was effectively dormant on SQL expressions — the test suite never asked "does this warning fire on a SQL WHERE input?" because there was no test for the cross-cut "the free-for-free promise still holds for SQL" claim.

ADR-0032 §11.6 closes the gap (Phase 2's diagnostics work implements a MatchedPath-walking predicate-warnings variant covering every sql_expr slot — WHERE, HAVING, ON, CASE, projection, ORDER BY) and the implementation plan's cross-cut matrix has it as a named row to prevent a regression. The plan is structured so this kind of gap cannot ship undetected again.

Worth carrying forward to future phase planning: every "comes for free" claim in any ADR is also a candidate for an unnoticed silent gap. The cross-cut matrix pattern in the plan generalises.

§4. Autonomous decisions during this session

Per CLAUDE.md, decisions outside ADR-0030's settled scope are flagged here. None are urgent; all were either confirmed by the user via explicit AskUserQuestion gates or are routine detail.

• Catalog key naming. diagnostic.* for parse-time diagnostics and engine.* for friendly-error translations — chosen by following the existing pattern (diagnostic.unknown_table, select.internal_table); no new naming convention introduced.
• Subgrammar push-trigger mechanism (§10.2) — Node::Scoped­ Subgrammar(&Node) chosen via explicit user AskUserQuestion gate over two alternatives (flag on existing variant; driver-side identity registry). Recommended option; user picked Recommended.
• CTE body column derivation (§10.3) — six derivation rules covering *, t.*, bare refs, qualified refs, aliased expressions, and computed-no-alias; first leg / non-recursive leg dictates for compound / recursive bodies. Per standard SQL throughout.
• Node::ScopedSubgrammar data model — ScopeFrame is a struct holding from_scope / cte_bindings / projection_aliases together rather than three parallel stacks. Cleaner data model; no semantic difference. Settled during writing.

Every scope question that affected expressiveness or surface area went through an AskUserQuestion gate. The full list of those gates appears in the conversation log; the answers shape the ADR's §§1–10 directly.

§5. What's next — Phase 2 implementation

Per ADR-0030's phasing, Phase 2 is the next slice. The user has signed off on the plan and authorized walking it without interruption between gates. The implementer for Phase 2 should:

1. Read in order: this handoff, ADR-0032, the Phase 2 plan, then CLAUDE.md (working-style rules unchanged) and docs/requirements.md (Q1/Q2 advance further; Q4 already ticked).
2. Begin sub-phase 2a — the grammar fragment. The plan's "Scope (in)" / "Build steps" / "Exit gate" / "DA gate" sections for 2a are the spec.
3. Commit at gate boundaries with detailed messages in the area: subject style (recent precedent: grammar: SQL SELECT end-to-end (ADR-0030 Phase 1), etc.).
4. Escalate when the plan says to — see §2 of the plan's "Standing authorizations" for the four trigger conditions. The plan's "Open questions to escalate before code starts" section pre-identifies three known triggers (subquery type- resolution through scalar subqueries; duplicate CTE-name detection; function name allowlist).
5. At end of 2g, produce the verification report and surface to the user; do NOT declare Phase 2 complete without the user's confirmation. This is the one explicit pause-point per §5 of the plan's standing authorizations.

After Phase 2 ships and is accepted, ADR-0030 Phase 3 (DML — INSERT / UPDATE / DELETE in SQL) is the next focus area.

§6. Seams for the implementer

These are the file-and-section pointers for Phase 2 work, copied across from the plan for fast access:

• src/dsl/grammar/sql_select.rs (new) — the §1 grammar. Parallel to sql_expr.rs (ADR-0031). Exports SQL_SELECT_STATEMENT (the full statement with optional WITH) and SQL_SELECT_COMPOUND (the embedded form subqueries recurse into).
• src/dsl/grammar/sql_expr.rs — three additive Choice branches (scalar subquery primary, IN (subquery), [NOT] EXISTS (subquery)) and one additive name_or_call tail (qualified ref t.c). All recurse through Node::ScopedSubgrammar(&SQL_SELECT_COMPOUND).
• src/dsl/grammar/mod.rs — Node enum gets the new ScopedSubgrammar(&'static Node) variant alongside the existing Subgrammar.
• src/dsl/walker/context.rs — WalkContext gains from_scope_stack: Vec<ScopeFrame>. current_table / current_table_columns become derived helpers over the top frame (DSL paths must stay green — the existing 1240+ test surface verifies this).
• src/dsl/walker/driver.rs (or wherever the Subgrammar match arm lives) — new arm for Node::ScopedSubgrammar that pushes/pops a ScopeFrame.
• src/dsl/walker/mod.rs — schema_existence_diagnostics extended to walk every from_scope binding; new arity-check pass; existing predicate_warnings gets a MatchedPath-walking variant for sql_expr slots.
• src/db.rs — do_run_select calls a new resolve_select_column_types helper before constructing the DataResult. The helper uses RawStatement::column_table_name(i) / column_origin_name(i) per result column.
• Cargo.toml — add "column_metadata" to rusqlite's feature list. One-line change.
• src/dsl/grammar/data.rs — Phase-1 SQL SELECT static nodes either move into sql_select.rs or get removed outright (the data::SELECT CommandNode is rebuilt against SQL_SELECT_STATEMENT); see plan sub-phase 2c.
• Catalog file (wherever the i18n keys live; the existing diagnostic.unknown_table neighbours) — five new diagnostic.* keys + eight new engine.* keys per ADR-0032 §11.5.

§7. How to take over

1. Read this file, then docs/adr/0032-sql-select-grammar.md and docs/plans/20260520-adr-0032-phase-2.md. Then CLAUDE.md and docs/requirements.md.
2. cargo test — expect 1260 passing, 0 failed, 1 ignored (the baseline).
3. cargo clippy --all-targets -- -D warnings — clean.
4. Start sub-phase 2a per the plan. Standing authorizations (plan §§1–5) apply.
5. Escalate per CLAUDE.md when the plan's triggers fire; never decide silently on a question the ADR/plan doesn't settle.

§8. What else is open

Unchanged from handoff-25 §8 (prioritisation is a user decision): snapshot/undo U-series (ADR-0006 written, unbuilt); m:n convenience C4; modify-relationship C3a; show family V5; rename-table C1 (may fall out of ADR-0030 Phase 4); friendly-error sweep H1; CI TT5; session-log / Markdown export V4.

ADR-0030's later phases — DML (Phase 3), DDL (Phase 4), the DSL→SQL teaching echo (Phase 5), polish (Phase 6) — remain on the roadmap behind Phase 2.

Phase-1's specific deferrals (§4.1–§4.5 in handoff-25) are addressed as follows by ADR-0032:

• §4.1 — FROM optional. Stands. ADR-0032 §1 notes SELECT 1 and SELECT upper('x') continue to parse.
• §4.2 — implicit projection aliasing (select a x). Lifted by ADR-0032 §1 — admitted in Phase 2.
• §4.3 — help_id: None on SELECT. Stands; Phase 6.
• §4.4 — walker entries defaulting to internal Mode::Simple. Stands; revisited when advanced mode grows its own ambient assistance (Phase 6).
• §4.5 — bool SELECT results render as 0/1. Partially lifted by ADR-0032 §12 — bare bool column refs now recover their type via engine column-origin metadata. Computed bool expressions stay typeless until a future enhancement.