walker: 3a — category-grouped mode-aware dispatch (ADR-0033 Amendment 1)
Replaces ADR-0033 §2's original Node::Guard + Choice(SQL,DSL) mechanism,
which was found during 3a to be unworkable: any guard-in-Choice approach
forces a walk_choice change (walk_choice falls through only on NoMatch, so
simple-mode valid-DSL would wrongly surface "this is SQL"), and walk_seq
treats a NoMatch past idx 0 as a hard Failed, breaking advanced-mode DSL
fall-through.
Mechanism (Amendment 1): each REGISTRY entry is tagged
CommandCategory::{Simple, Advanced}, generalising the whole-command
is_advanced_only gate. walk() becomes a thin dispatcher over decide()
(mode-aware candidate selection: simple commits the DSL node or emits the
"this is SQL" hint; advanced tries SQL first, DSL as a full-line fallback)
and an extracted walk_one_command(); speculative match-testing runs on a
scratch WalkContext so the caller's context is only touched by the
committed walk. No Node::Guard, no walk_choice/walk_seq change.
6 dispatch smoke tests on a shared-entry-word smoke registry; 1446 baseline
green; clippy clean.
This commit is contained in:
claude@clouddev1
@@ -1030,6 +1030,152 @@ grammar on top.
|
||||
- **No `EXPLAIN` of SQL DML** (ADR-0030 §13 OOS-2 — the DSL
|
||||
`explain` still wraps what it already wraps).
|
||||
|
||||
## Amendment 1 — Dispatch mechanism: category-grouped dispatch supersedes `Node::Guard` (2026-05-21)
|
||||
|
||||
This amendment **supersedes §2's "Mode-gating mechanism"
|
||||
subsection** and **revises sub-phase 3a's scope**. It was
|
||||
written during 3a implementation, after tracing the proposed
|
||||
mechanism against the actual walker code, and is recorded with
|
||||
explicit user approval before any code landed.
|
||||
|
||||
### The finding — option (a) does not work as §2 frames it
|
||||
|
||||
§2 selected option (a): a standalone zero-byte `Node::Guard(fn)`
|
||||
placed as the first element of the SQL branch's `Seq`, inside a
|
||||
`Choice(SQL_shape, DSL_shape)`. §2 framed this as the "smallest
|
||||
mechanism change" that "reuses the existing validator path" with
|
||||
the failure surfacing as the same `WalkOutcome::ValidationFailed`
|
||||
as the entry-word gate — implying **no `walk_choice` change**.
|
||||
|
||||
Tracing the four required behaviours against
|
||||
`src/dsl/walker/driver.rs` shows that framing is inaccurate, and
|
||||
that **any** guard-in-`Choice` mechanism is forced to modify
|
||||
`walk_choice`:
|
||||
|
||||
1. **`walk_choice` returns immediately on a branch `Failed`**
|
||||
(`driver.rs` `walk_choice`) — it only falls through to the
|
||||
next branch on `NoMatch`. So in **Simple mode**,
|
||||
`delete from t where id = 1` (which is *valid DSL*) would hit
|
||||
the SQL branch's guard, the guard would fail, and the user
|
||||
would see the "this is SQL" hint instead of the statement
|
||||
running as DSL. That contradicts §2's own "Simple mode → DSL
|
||||
only" promise. Fixing it requires a `walk_choice` change
|
||||
regardless of how the guard node is shaped.
|
||||
|
||||
2. **`walk_seq` treats a `NoMatch` at `idx > 0` as a hard
|
||||
`Failed`** (`driver.rs` `walk_seq`). A zero-byte Guard at
|
||||
`idx 0` still advances `idx` to 1, pushing the SQL branch's
|
||||
first real token to `idx 1`. So in **Advanced mode**,
|
||||
`delete from t --all-rows` (DSL-only) cannot fall through to
|
||||
DSL — the SQL branch returns `Failed`, and `walk_choice`
|
||||
won't try the DSL branch. The literal form needs an
|
||||
*additional* `walk_seq` semantics change (`idx == 0` → "no
|
||||
bytes consumed yet").
|
||||
|
||||
This is exactly the R1 risk the ADR budgeted for ("if `walk_seq`
|
||||
commits the Seq before the validator runs … the Choice can't
|
||||
fall through").
|
||||
|
||||
### The replacement — category-grouped, mode-aware dispatch
|
||||
|
||||
Rather than gate a `Choice` branch from inside the recursive
|
||||
grammar, the dispatch decision moves up to the command
|
||||
dispatcher (`walker::walk`), which is **where mode gating
|
||||
already lives**: the existing whole-command gate
|
||||
(`is_advanced_only` + the Simple-mode short-circuit in `walk`)
|
||||
is precisely this pattern, hardcoded for `select` / `with`. The
|
||||
amendment generalises that existing code instead of inventing a
|
||||
parallel in-grammar mechanism.
|
||||
|
||||
Concretely:
|
||||
|
||||
- **Each registry entry carries a category** —
|
||||
`CommandCategory::{Simple, Advanced}`. The category lives at
|
||||
the registration site (the `REGISTRY` table), not as a field
|
||||
on every `CommandNode` literal, because category is a
|
||||
dispatcher concern, not intrinsic to a command's grammar.
|
||||
`select` / `with` (and, from 3b, the SQL `insert` / `update` /
|
||||
`delete` nodes) are `Advanced`; everything else is `Simple`.
|
||||
This subsumes `ADVANCED_ONLY_ENTRIES`: `is_advanced_only(entry)`
|
||||
becomes "every registry candidate for this entry word is
|
||||
`Advanced`".
|
||||
|
||||
- **A shared entry word has a node in both groups** — e.g. a
|
||||
`Simple` DSL `insert` node and an `Advanced` SQL `insert`
|
||||
node, both with entry word `insert`. The entry-word lookup
|
||||
returns *all* candidates; the dispatcher selects by mode.
|
||||
|
||||
- **Mode-aware selection in `walk` (no combinator changes):**
|
||||
- **Simple mode** considers `Simple` candidates. If a Simple
|
||||
candidate exists, it is committed. If none exists but an
|
||||
`Advanced` candidate does (a pure-SQL entry word like
|
||||
`select`), the dispatcher emits the
|
||||
`advanced_mode.sql_in_simple` `ValidationFailed` — identical
|
||||
to today's whole-command gate.
|
||||
- **Advanced mode** tries `Advanced` candidate(s) first, then
|
||||
the `Simple` candidate as a fallback. The first candidate
|
||||
that fully matches wins (`delete from t where id = 1` →
|
||||
SQL; `delete from t --all-rows` → falls through to DSL
|
||||
because the SQL shape doesn't match `--all-rows`).
|
||||
|
||||
- **No `Node::Guard`, no `walk_choice` / `walk_seq` change.** The
|
||||
recursive combinators are untouched; each candidate is walked
|
||||
cleanly and independently. Speculative evaluation runs on a
|
||||
scratch `WalkContext`; only the committed candidate is walked
|
||||
into the caller's context, so post-walk context state
|
||||
(completion / hint accumulators) always reflects the chosen
|
||||
candidate.
|
||||
|
||||
### The two details (user-approved)
|
||||
|
||||
1. **"This is SQL" hint in Simple mode for SQL-shaped input.**
|
||||
For SQL-only input in Simple mode (e.g.
|
||||
`delete … returning *`), the Simple candidate won't match.
|
||||
To keep parity with `select` / `with`, the dispatcher
|
||||
speculatively checks whether the `Advanced` candidate *would*
|
||||
match; if so, it emits the `advanced_mode.sql_in_simple`
|
||||
hint rather than a generic DSL parse error. Lives entirely
|
||||
in `walk`.
|
||||
|
||||
2. **Advanced-mode completion is SQL-first, DSL as full-line
|
||||
fallback.** The `Choice` approach would have unioned SQL and
|
||||
DSL continuations mid-typing for free; the grouping approach
|
||||
surfaces one candidate's walk for completion / hints. The
|
||||
decision is to show SQL completions primarily in Advanced
|
||||
mode and fall to DSL only on a full DSL-shaped line — simpler
|
||||
and matching the "SQL-first, DSL as familiar fallback"
|
||||
intent. No unioned completion.
|
||||
|
||||
### Consequences of the amendment
|
||||
|
||||
- **`Node::Guard` is NOT added.** §2's mechanism caveat,
|
||||
Consequences bullet 1 ("the walker gains a new node variant
|
||||
`Node::Guard`"), and the original 3a scope are withdrawn.
|
||||
- **`REGISTRY`'s element type changes** to carry the category
|
||||
alongside each `&CommandNode`. The handful of registry
|
||||
iteration sites (entry-word listing, `command_for_entry_word`,
|
||||
the completion entry-word filters, `note_help`) destructure the
|
||||
tuple; no behavioural change for the current single-candidate
|
||||
registry.
|
||||
- **The `Command` variants (§10), worker handlers, RETURNING
|
||||
flag, cascade-summary plan, shortid auto-fill, and diagnostics
|
||||
are unaffected** — this amendment is purely about how the
|
||||
dispatcher routes a shared entry word to the right grammar.
|
||||
- **Sub-phase 3a now validates the dispatch mechanism**, not the
|
||||
guard: a smoke registry with a shared entry word (a `Simple`
|
||||
and an `Advanced` node using distinguishable tokens) exercises
|
||||
the Simple/Advanced selection, the fallback, and the
|
||||
"this is SQL" path. The 3a exit gate's four cases map onto the
|
||||
new mechanism unchanged in spirit.
|
||||
- **3b adds the first real shared entry words** to `REGISTRY`
|
||||
(the SQL `insert` / `update` / `delete` nodes). At that point
|
||||
the completion entry-word *lists* will need to de-duplicate
|
||||
the shared primary (two candidates, one entry word) — tracked
|
||||
for 3b, not 3a.
|
||||
- The mechanism remains reusable for Phase 4 DDL's analogous
|
||||
shared-entry problem (`create`, …): tag the SQL DDL nodes
|
||||
`Advanced`; the dispatcher handles the rest.
|
||||
|
||||
## See also
|
||||
|
||||
- ADR-0005 — the ten-type vocabulary INSERT works with.
|
||||
|
||||
+1
-1
@@ -38,4 +38,4 @@ This directory contains the project's ADRs, recorded per
|
||||
- [ADR-0030 — Advanced mode: the standard-SQL surface](0030-advanced-mode-sql-surface.md) — **Accepted**, SQL added as grammar *within the unified grammar tree* (ADR-0024), not a separate batch parser — so SQL gets the same completion / highlighting / hints / parse-errors as the DSL; mode gates the SQL forms; DDL routes through the typed `Command` executor (metadata + type vocabulary preserved), DML and `SELECT` execute as validated SQL; engine-neutral posture, the DSL→SQL teaching echo; supersedes ADR-0001's `sqlparser-rs` reservation; phased plan (`Q1` / `Q2` / `Q4`)
|
||||
- [ADR-0031 — The SQL expression grammar](0031-sql-expression-grammar.md) — **Accepted**, the stratified SQL expression grammar fragment commissioned by ADR-0030 §3: a single precedence ladder (`OR`/`AND`/`NOT`, the comparison/`LIKE`/`IN`/`BETWEEN`/`IS NULL` predicate set, arithmetic incl. `||`, function calls, `CASE`) — the superset of ADR-0026's DSL `WHERE` grammar, authored as a parallel fragment so simple mode is untouched; pure validation, builds **no** AST (consumers run/store SQL as text per ADR-0030 §4/§6); reuses ADR-0026's `Subgrammar` recursion + depth cap unchanged; subquery expressions and qualified column refs deferred to ADR-0030 Phase 2
|
||||
- [ADR-0032 — The full SQL `SELECT` grammar](0032-sql-select-grammar.md) — **Accepted**, the Phase-2 grammar commissioned by ADR-0030 §3: full `SELECT` with `INNER`/`LEFT`/`RIGHT`/`FULL OUTER`/`CROSS` joins, `GROUP BY`/`HAVING`, all four set ops (`UNION`/`UNION ALL`/`INTERSECT`/`EXCEPT`), `WITH` and `WITH RECURSIVE` CTEs, `LIMIT … OFFSET`, `DISTINCT`, `t.*`, and bare-alias projection (lifting Phase-1 §4.2); additive extensions to ADR-0031's `sql_expr` for scalar subqueries, `IN (SELECT …)`, `[NOT] EXISTS`, and qualified column refs (redeeming ADR-0031 §7 OOS-1/OOS-2); grammar-recursion via `Subgrammar(&SQL_SELECT_COMPOUND)` reuses ADR-0026's `MAX_SUBGRAMMAR_DEPTH = 64` cap unchanged; **softens ADR-0030 §8's "ambient assistance comes for free" claim**: completion scope needs new `WalkContext` accumulators (a `from_scope_stack` of `ScopeFrame`s holding `from_scope` / `cte_bindings` / `projection_aliases`), a **new walker node variant `Node::ScopedSubgrammar(&Node)`** as the push/pop trigger (existing `Node::Subgrammar` unchanged so DSL `Expr` and `sql_expr` recursion are unaffected), qualified-prefix completion narrowing, body-projection-derived CTE column resolution (so `SELECT *` and explicit-projection CTE bodies both yield real column completion past `cte_alias.|`), and a **post-walk fixup pass** that re-resolves projection-list identifier highlighting/validity once `FROM` is parsed (the projection-before-FROM problem); classifies every Phase-2 validation case against ADR-0027's ERROR/WARNING guideline (§11): five new `diagnostic.*` keys for parse-time-detectable cases (unknown qualifier, ambiguous column, projection-alias misplaced, CTE/compound arity mismatch) plus eight `engine.*` translation keys; a MatchedPath-walking predicate-warnings variant that closes the Phase-1 gap where SQL `WHERE` expressions emitted no `LIKE`-on-numeric / `= NULL` / type-mismatch warnings (ADR-0027 Amendment 1 finally extends to the SQL surface); adds a worker-side post-prepare type-resolution pass via engine column-origin metadata so bare column refs recover their playground type (partially lifting Phase-1 §4.5, the bool→0/1 case) — `Cargo.toml` gains `column_metadata` to rusqlite features (verified against pinned 0.39.0); `__rdbms_*` rejection extended to every new table-source slot; Amendment 1 narrows §12's resolution rule from a grammar-side structural classification to "trust the engine's column-origin metadata verbatim" after an empirical probe showed origin metadata follows through non-recursive CTEs, scalar subqueries, derived tables, set ops, and joins — the one structural exception is recursive CTE result columns, which return None and stay typeless; Amendment 2 records that §10.6's "rewrite the highlight class" prescription is realised via the two-pass schema-existence diagnostic + the renderer's diagnostic-overlay path (no separate per-byte rewrite step needed; no new HighlightClass variant), and that the projection-before-FROM completion narrowing has been improved by an `src/completion.rs` look-ahead probe when the leading walk's `from_scope` is empty but the full input parses
|
||||
- [ADR-0033 — The full SQL DML grammar (`INSERT` / `UPDATE` / `DELETE`)](0033-sql-dml-grammar.md) — **Proposed**, the Phase-3 grammar commissioned by ADR-0030 §3: single- and multi-row `INSERT` (incl. `INSERT … SELECT` recursing through ADR-0032's `SQL_SELECT_COMPOUND`), `UPDATE` with `SET` assignment list, `DELETE`, all three optionally followed by `RETURNING projection_list`, plus full `ON CONFLICT … DO NOTHING / DO UPDATE` UPSERT on INSERT; **fixes the DSL-vs-SQL dispatch architecture for shared entry words (`insert`/`update`/`delete`)**: SQL-first / DSL-fallback in Advanced mode via a `Choice(SQL_shape, DSL_shape)` per shape, gated by a new walker capability `Node::Guard(fn)` — a zero-byte-consumption gating node that fails the enclosing Seq with a `ValidationError`; carries `Command::SqlInsert` / `SqlUpdate` / `SqlDelete` variants and `do_sql_*` worker handlers each of which knows the target table (for re-persistence) and the `returning: bool` flag (for DataResult routing); `shortid` auto-fill mirrors the DSL `do_insert` mechanism via worker post-fill; SQL DELETE produces the same per-relationship cascade summary the DSL DELETE does (ADR-0014 parity), with the WHERE-clause source bytes re-injected into per-child pre-count subqueries; three new walker diagnostics (`insert_arity_mismatch` ERROR, `auto_column_overridden` WARNING, `not_null_missing` WARNING) with positive + negative tests each; OOS list explicitly carves out `DEFAULT VALUES` (the project's planned seed feature), SQLite-specific `OR REPLACE` / `OR IGNORE` / `OR ABORT` / `OR FAIL` / `OR ROLLBACK` prefixes, `UPDATE FROM` multi-table updates, and WITH-prefixed DML; the `excluded` keyword inside `ON CONFLICT DO UPDATE` is a deliberate carve-out from ADR-0030 §7's engine-neutral posture (no standard-SQL UPSERT spelling exists that SQLite and PostgreSQL share); eleven phased sub-phases each with explicit exit gates + written DA gate, opening with `Node::Guard` mechanism scaffolding before any DML grammar lands; initial DA review recorded seven critiques that were resolved before status moved to Proposed
|
||||
- [ADR-0033 — The full SQL DML grammar (`INSERT` / `UPDATE` / `DELETE`)](0033-sql-dml-grammar.md) — **Proposed**, the Phase-3 grammar commissioned by ADR-0030 §3: single- and multi-row `INSERT` (incl. `INSERT … SELECT` recursing through ADR-0032's `SQL_SELECT_COMPOUND`), `UPDATE` with `SET` assignment list, `DELETE`, all three optionally followed by `RETURNING projection_list`, plus full `ON CONFLICT … DO NOTHING / DO UPDATE` UPSERT on INSERT; **fixes the DSL-vs-SQL dispatch architecture for shared entry words (`insert`/`update`/`delete`)**: SQL-first / DSL-fallback in Advanced mode via a `Choice(SQL_shape, DSL_shape)` per shape, gated by a new walker capability `Node::Guard(fn)` — a zero-byte-consumption gating node that fails the enclosing Seq with a `ValidationError`; carries `Command::SqlInsert` / `SqlUpdate` / `SqlDelete` variants and `do_sql_*` worker handlers each of which knows the target table (for re-persistence) and the `returning: bool` flag (for DataResult routing); `shortid` auto-fill mirrors the DSL `do_insert` mechanism via worker post-fill; SQL DELETE produces the same per-relationship cascade summary the DSL DELETE does (ADR-0014 parity), with the WHERE-clause source bytes re-injected into per-child pre-count subqueries; three new walker diagnostics (`insert_arity_mismatch` ERROR, `auto_column_overridden` WARNING, `not_null_missing` WARNING) with positive + negative tests each; OOS list explicitly carves out `DEFAULT VALUES` (the project's planned seed feature), SQLite-specific `OR REPLACE` / `OR IGNORE` / `OR ABORT` / `OR FAIL` / `OR ROLLBACK` prefixes, `UPDATE FROM` multi-table updates, and WITH-prefixed DML; the `excluded` keyword inside `ON CONFLICT DO UPDATE` is a deliberate carve-out from ADR-0030 §7's engine-neutral posture (no standard-SQL UPSERT spelling exists that SQLite and PostgreSQL share); eleven phased sub-phases each with explicit exit gates + written DA gate, opening with the dispatch mechanism before any DML grammar lands; initial DA review recorded seven critiques that were resolved before status moved to Proposed; **Amendment 1 supersedes §2's dispatch mechanism**: the originally-chosen `Node::Guard(fn)` + `Choice(SQL_shape, DSL_shape)` was found during 3a to be unworkable as framed (any guard-in-`Choice` mechanism forces a `walk_choice` change — `walk_choice` only falls through on `NoMatch`, so Simple-mode valid-DSL would wrongly surface "this is SQL", and `walk_seq` treats a `NoMatch` past `idx 0` as a hard `Failed`, breaking Advanced-mode DSL fall-through); replaced by **category-grouped, mode-aware dispatch** in `walker::walk` (each `REGISTRY` entry tagged `CommandCategory::{Simple, Advanced}`, generalising the existing whole-command `is_advanced_only` gate), shared entry words carrying a node in both groups, no `Node::Guard` and no `walk_choice`/`walk_seq` change, advanced-mode completion SQL-first with DSL as a full-line fallback
|
||||
|
||||
Reference in New Issue
Block a user