New src/dsl/grammar/sql_update.rs: SQL_UPDATE_SHAPE =
<table> SET col = sql_expr (',' …)* [WHERE sql_expr] [';'], the
__rdbms_* target rejection, and the shared sql_expr on both the
assignment RHS and the predicate. No --all-rows rail — a SQL
UPDATE without WHERE runs as written (ADR-0030 §12). Reuses
sql_select::WHERE_CLAUSE (now pub(crate)) so the predicate
diagnostics are identical. The target uses the shared `table_name`
ident role (not a bespoke one) so the Phase-2 schema-existence and
predicate-warning passes collect it as a scope binding and check
the SET / WHERE columns for free — a bespoke role left them
unchecked (the cross-cut tests caught this).
Command::SqlUpdate { sql, target_table }; Request::RunSqlUpdate +
do_sql_update (execute validated SQL via execute_with_fk_enrichment,
re-persist the target CSV, append history.log). 3e surfaces the
affected-row count only; precise row output is RETURNING (3g), so
the update-success render skips a column-less data set rather than
showing a misleading "(no rows)" band. Behind the dev `sql_update`
entry word until 3j.
Tests: grammar accept/reject; integration (single/multi-col,
no-WHERE all-rows, sql_expr in SET, scalar subquery in SET,
zero-match success, history); walker cross-cut (unknown SET column
→ unknown_column, `= NULL` in WHERE → eq_null warning); app-level
render-guard both ways (column-less → count only; with columns →
table renders). 1524 green, clippy clean.
plan_shortid_autofill read exactly listed_columns.len() cells from
the materialised row source. When the row source produced a
different column count than the user's list, the extra columns were
silently dropped (wider → wrong data, insert succeeded) or read
out of range (narrower). Guard: if the materialised statement's
column_count differs from the listed-column count, skip auto-fill
and execute the verbatim statement so the engine reports the
mismatch — matching the non-auto-fill path. A friendly pre-flight
diagnostic remains sub-phase 3i.
Tests: VALUES with too many values; INSERT…SELECT with a wider and
a narrower projection — each rejected with nothing persisted.
When an INSERT's column list omits one or more shortid columns,
the worker now fills them. Command::SqlInsert gains listed_columns
and row_source, captured in build_sql_insert from the matched path
(the row source is located by the first values/select/with Word
token, so a string literal like 'select' can't be mistaken for the
keyword). do_sql_insert calls plan_shortid_autofill, which — per
the user-confirmed Option B — materialises the row source by
running it as a query, generates a distinct shortid per row via the
existing generate_shortid_batch (deduped against stored values),
and reconstructs a parameterised multi-row INSERT over the listed
columns plus the omitted shortid columns. Uniform for VALUES and
INSERT…SELECT, and handles multiple omitted shortids in one row
(each gets its own batch). No explicit list, no omitted shortid, or
a zero-row source → execute verbatim (the 3b path). serial stays
engine-filled via rowid. history.log keeps the original line, never
the rewrite (§11).
Tests: VALUES single/multi-row distinct; explicit override
honoured; INSERT…SELECT distinct fills; combined serial(engine) +
shortid(worker); two shortids (PK + non-PK) both fill; one provided
+ one omitted; compound-PK shortid member; mixed-case column name
(ADR-0009 DA gate); original-source-in-history on the rewrite path.
Still behind the dev `sqlinsert` entry word (3j). 1503 green,
clippy clean.
Make the INSERT row source a Choice between the VALUES clause and
Subgrammar(&sql_select::SQL_SELECT_COMPOUND). SQL_SELECT_COMPOUND
is itself a Choice that admits a leading WITH, so a WITH-prefixed
SELECT row source (R4) parses through it for free; the two
branches start on disjoint keywords (values vs select/with) so the
Choice never ambiguously commits. No worker change — do_sql_insert
already executes the validated SQL and re-persists, and the engine
handles insert-from-query.
Tests: grammar accept (plain / column-list+projection / WITH-
prefixed / trailing-semi) and reject (__rdbms_* on the SELECT's
FROM slot, incomplete select); integration parse-path lowering +
worker round-trip (rows land, CSV re-persisted) incl. R4 WITH end-
to-end; walker cross-cut that the Phase-2 unknown_column diagnostic
fires on the INSERT…SELECT projection; DA-gate test that a self-
sourced INSERT…SELECT runs as a plain insert (no cascade summary —
that is DELETE-only). Still behind the dev `sqlinsert` entry word
(shared `insert` is 3j). 1493 tests green, clippy clean.
walk_repeated discarded the last matched item's trailing-optional
expectations at a clean item boundary, so a comma-separated list
offered no continuation after a complete item: `order by Name `
gave no asc/desc, `select Name ` no `as`, `create table …
Code(text) ` no not/unique/default/check. Capture the last item's
skipped set and surface it when the list ends at an item boundary
(the separator `,` itself is deliberately not surfaced).
That fix made expression-position candidate lists long, which
exposed a visibility problem: the hint panel's candidate line is
single-row and window-scrolls on overflow, centring on item 0 when
nothing is selected — so with keywords-first, schema identifiers
scrolled off behind the `>` marker. Reverse the ordering: schema
identifiers (table/column/relationship names) now sort before
keywords, since a name the user would have to look up is the
highest-value completion and must stay visible (keywords are
learned over time; the tok_identifier/tok_keyword colour split
marks the boundary). This reverses the handoff-14 keywords-first
call, now recorded in ADR-0022 Amendment 2.
Tests: walker expected-set + completion-layer regressions for the
trailing-optionals and the ordering; candidate_ordering.rs header
invariant inverted; ~20 typing-surface snapshots re-baselined; a
two-line hint box recorded as a deferred follow-up.
walk_seq's Incomplete arm unconditionally merged the accumulated
skipped-Optional expectations (pending_skipped) into the child's
expected set. When a child committed terminals before going
Incomplete (e.g. `order by` consumed, now awaiting a sort item),
this leaked ~13 clause keywords from clauses positioned *before*
the committed child — WHERE/GROUP BY/HAVING, the FROM's JOIN
options, set-ops — into the ORDER BY completion list, shoving the
actual columns off-screen.
Merge pending_skipped only when the Incomplete-producing child
consumed nothing (path length unchanged): the cursor still sits at
the optional boundary, so those optionals are genuine alternatives.
A committed child means the cursor is past them.
Tests: walker expected-set guard (+ over-correction guard) and a
full-stack completion-layer regression test.
F1: the hint panel is the completion UI, so a premature "no such table/
column" ERROR on the token the user is still typing must not shadow its
completion. ambient_hint now suppresses an under-cursor error diagnostic
when a completion exists for the (non-empty) partial it overlaps, and
falls through to the candidates. Genuinely-unknown names (no prefix match)
still show the error; WARNINGs are unaffected. Both modes.
F2: projection-before-FROM ("select <cursor> from T" after deleting *)
offered the global column list instead of T's columns, because the §10.6
look-ahead's full-input walk can't reach FROM through an empty projection.
When the look-ahead finds no scope, retry with a neutral placeholder
inserted at the cursor so the trailing FROM/CTE scope is recovered for
narrowing. Only the repaired walk's from_scope/cte_bindings are used.
Test-first: 3 F1 tests (mid-typed completes, unknown still errors, simple-
mode DSL) + 1 F2 multi-table narrowing test. 1469 baseline green.
Advanced-mode hinting + completion-preview were dead: render_hint_panel
returned None for advanced mode (stale ADR-0022 §12 gate, predating the
SQL grammar) and the hint resolver/ambient_hint never threaded Mode, so a
SQL statement was gated as "this is SQL". The unified walker (ADR-0030/
0031/0032) speaks SQL, so this lifts the gate.
- ambient_hint_in_mode + hint_resolution_at_input_in_mode +
expected_for_hint_snapshot(mode); candidate/diagnostic/parse sub-calls
run in the active mode.
- render_hint_panel calls ambient for all modes; one-shot `:` sigil
stripped (strip_one_shot_prefix) so `: sel` hints `select`.
- ADR-0022 Amendment 1 + README index.
Found by manual advanced-mode testing; Phase 2 marked SQL hint/completion
green at the engine layer but never exercised the UI. App-level render
test (advanced_mode_hint_panel_surfaces_sql_candidates) + ambient-layer
regression locks. 1466 baseline green.
SQL_INSERT_SHAPE (INTO <table> [(cols)] VALUES tuple(s)) with __rdbms_*
target rejection; Command::SqlInsert{sql,target_table}; Request::RunSqlInsert
+ do_sql_insert worker (tx-guarded: execute, then finalize_persistence for
CSV + history before commit, so failures roll back and don't re-persist).
Auto-show is best-effort via last_insert_rowid range.
Isolated behind a dev `sqlinsert` entry word (Advanced) so the SQL path is
testable without making `insert` a shared word yet (that's 3j, after 3d
auto-fill parity). Command::SqlInsert carries only sql+target_table; the
plan's listed_columns/returning land in 3d/3g where they're read.
6 grammar accept/reject tests + 8 integration tests (single/multi-row,
column-list, full-arity, history, rollback-on-failure, multi-row atomicity,
parse-path reconstruction, internal-table rejection). 1452 baseline green.
Replaces ADR-0033 §2's original Node::Guard + Choice(SQL,DSL) mechanism,
which was found during 3a to be unworkable: any guard-in-Choice approach
forces a walk_choice change (walk_choice falls through only on NoMatch, so
simple-mode valid-DSL would wrongly surface "this is SQL"), and walk_seq
treats a NoMatch past idx 0 as a hard Failed, breaking advanced-mode DSL
fall-through.
Mechanism (Amendment 1): each REGISTRY entry is tagged
CommandCategory::{Simple, Advanced}, generalising the whole-command
is_advanced_only gate. walk() becomes a thin dispatcher over decide()
(mode-aware candidate selection: simple commits the DSL node or emits the
"this is SQL" hint; advanced tries SQL first, DSL as a full-line fallback)
and an extracted walk_one_command(); speculative match-testing runs on a
scratch WalkContext so the caller's context is only touched by the
committed walk. No Node::Guard, no walk_choice/walk_seq change.
6 dispatch smoke tests on a shared-entry-word smoke registry; 1446 baseline
green; clippy clean.
Three DA critiques from the Phase-2 verification flagged real gaps;
this commit closes them.
1. Type recovery row-independence (critique #1). The all-10-types
test left col_blob NULL because the DSL Value enum has no Blob
variant. The DA flagged this as a potential row-dependence gap.
Added `database_run_select_type_recovery_works_on_empty_table`
that proves column-origin metadata works on Text AND Blob
columns with zero rows, pinning the invariant. The all-types
test now carries an explicit comment referencing it.
2. Engine.* pattern matching against real SQLite output (critique
#2). The pre-rework tests fed `translate_generic` hand-coded
strings; never verified that the pinned SQLite version actually
produces those wordings. Added three engine-routing tests in
`tests/sql_select.rs` that produce real engine errors via
`run_select` and assert catalog routing. Aggregate-in-WHERE
confirms end-to-end. GROUP-BY-required and scalar-subquery
are SQLite-permissive (no real error on the natural triggers),
so those tests verify the matcher doesn't false-positive on
benign queries + that synthetic messages route correctly.
3. Manual TUI verification (critique #3) surfaced an additional
gap: `App::input_validity_verdict()` was hard-coded silent in
Advanced mode, so SQL predicate warnings emitted but never
reached the [WRN] indicator. Wired the verdict through to the
active effective mode; updated two pre-existing tests that
pinned the now-superseded "silent in Advanced" behavior; added
one new test confirming a SQL `LIKE`-on-numeric warning fires
the indicator. Launched the TUI, typed a representative
warning-triggering SELECT, confirmed SELECT/FROM/WHERE/LIKE
highlight as keyword colour AND the [WRN] indicator appears.
Test totals: 1441 → 1446 passing (+5). Clippy clean.
Cross-cut verification matrix for ADR-0032 Phase 2 is now fully
populated with concrete test references — every row green. Filling
the matrix surfaced three real gaps that this commit closes.
1. Advanced-mode syntax highlighting (ADR-0030 §8 matrix row).
The `ui.rs` Advanced branch routed through `plain_input_spans`,
bypassing the highlight walker entirely. In production SQL
keywords past the entry word rendered as plain identifiers.
Fix: mode-aware variants of `highlight_runs`,
`render_input_runs`, `lex_to_runs`, and `input_diagnostics`;
the Advanced render path now uses the highlighted form with
`Mode::Advanced`. `plain_input_spans` removed (unused).
2. Engine.* key wiring (ADR-0032 §11.4 / §13 matrix rows + handoff
§3.3 follow-up). The four Phase-2 engine.* catalog entries
were authored in 2d but never reached: `translate_generic`
discarded the engine message and returned a vague catalog
entry. Fix: pattern-match the engine message text for the four
Phase-2 categories (aggregate misuse, group-by required,
compound arity mismatch fallback, scalar-subquery cardinality)
inside `translate_generic`, routing each to its engine-neutral
catalog entry.
3. Matrix-coverage tests. Thirteen new tests covering the rows
that had no explicit coverage:
- 3 SQL keyword/operator/CASE highlight tests
- 4 engine.* engine-message tests
- 3 sql_expr column-completion tests (WHERE, HAVING)
- 3 predicate-warning slot tests (CASE, ORDER BY, projection)
- 1 all-10-playground-types recovery test (tests/sql_select.rs)
Plan document (docs/plans/20260520-adr-0032-phase-2.md) updated:
every (TBD) row in the cross-cut matrix replaced with a concrete
test file::function reference and a green status marker.
Test totals: 1428 → 1441 passing (+13 new). Clippy clean.
Amendment 2 records the §10.6 fixup-pass mechanism choice. §10.6
prescribes "rewriting the highlight class" on projection-list
idents at end-of-walk; the actual implementation uses a different
mechanism that achieves the identical user-visible behavior:
1. 2d's two-pass schema-existence diagnostic collects every FROM
binding from the matched path first, then resolves projection
idents against the complete scope. The post-walk re-resolve
§10.6 calls for, just embedded in the diagnostic emitter.
2. input_render.rs's diagnostic-overlay path colors each
diagnostic span Error/Warning, achieving the visual change
§10.6 describes without needing a new HighlightClass variant.
The completion-mid-typing piece is improved by the §10.5
look-ahead probe (sub-phase 2e earlier).
Four new regression tests in `projection_before_from_tests` pin
the behavior so a future refactor can't silently regress it:
correct ident resolves silently, unknown ident flags via
diagnostic on its span, multi-projection only flags unknowns,
projection-without-FROM is silent.
ADR index entry updated to reference Amendment 2.
Test totals: 1424 → 1428 passing (+4). Clippy clean.
ADR-0032 §10.5 — at the cursor, an `<ident>.` prefix narrows
column candidates to that qualifier's binding columns. Resolves
through from_scope aliases first, then table names, then
cte_bindings (for `cte_alias.|`). Falls back to the schema cache
for DSL paths (`from <Table>.<col>`). Unresolved qualifier →
empty column list; the structural error path surfaces the
unresolved-prefix message.
Look-ahead probe — the "edit an existing query" workflow. When
the cursor is mid-projection but FROM exists after the cursor, a
second walk on the full input populates from_scope and the
column candidates narrow accordingly. Gated on the leading walk
producing no scope so cursor-past-FROM positions pay no cost.
The full input must parse for this to work; an unparseable
mid-edit state falls back to the §10.6 global posture.
CompletionProbe now exposes `from_scope` (top-frame table
bindings) and `cte_bindings` (union of in-scope CTE bindings,
innermost-first dedupe). The walker drains these at the cursor
position; the completion engine reads them for qualifier
resolution and unqualified narrowing.
Test totals: 1415 → 1424 passing (+9: 5 qualified-prefix +
4 look-ahead). Clippy clean.
ADR-0032 §10.3 says cte_bindings lives on the scope frame, with
inner subqueries free to declare their own CTEs that shadow outer
ones. The grammar didn't actually admit nested WITH inside
SQL_SELECT_COMPOUND — a real ADR-vs-implementation gap.
Closes the gap by making SQL_SELECT_COMPOUND a Choice between a
WITH-prefixed form and a plain form. The naive Optional-prefix
approach silently broke the paren-vs-subquery dispatch in
sql_expr.rs's PAREN_GROUP: Optional matches 0 bytes, committing
the Seq, so SELECT_CORE's NoMatch on `(a + b)` became Failed and
the Choice couldn't fall through to or_expr. The Choice-fronted
form keeps the fast NoMatch on non-WITH non-SELECT first tokens.
Side effect: scalar subquery / IN / EXISTS / derived-table
bodies now admit a leading WITH too, which matches standard SQL.
Updated two tests that were guarding the old `(WITH …)` rejection
behavior. Added one new harvest test exercising nested-WITH inside
a CTE body — the harvest's `expand_binding` mechanism already
handled the data correctly; the grammar gap was the sole blocker.
Test totals: 1414 → 1415 passing (+1 nested-with-in-cte test).
Clippy clean.
Implements the six ADR-0032 §10.3 output-column derivation rules
at CTE body-frame exit, populating the placeholder CteBinding's
columns. Unblocks `diagnostic.cte_arity_mismatch` (which compares
declared col-list arity vs derived projection arity) and the
upcoming qualified-prefix completion in 2e proper.
- `WalkContext::pending_cte_harvest`: bookkeeping for an in-progress
CTE harvest, armed by writes_cte_name + extended by cte_column
idents, consumed by the next walk_scoped_subgrammar invocation
(CTE syntax has no intervening ScopedSubgrammar, so timing is
deterministic). Cleared on every walk_scoped_subgrammar entry
to prevent stale state surviving a speculative walk rollback.
- `run_cte_harvest`: post-walk path-scan classifier that
reconstructs the body's first leg's projection-list and applies
the six derivation rules. Compound bodies take columns from the
first leg per spec; recursive CTE bodies take the non-recursive
(first) leg. Optional (col-list) renames positionally with
preserved types.
- `expand_binding`: bridges a TableBinding to a CteColumn list,
resolving CTE-source bindings (empty columns + table-name
matches an in-scope CteBinding) through to the CTE's harvested
columns. Enables sibling CTEs to project correctly: in
\`WITH a AS (...), b AS (SELECT * FROM a) ...\`, b's harvest sees
a's derived columns through the body's from_scope binding.
- `WalkContext::pending_diagnostics`: accumulator for diagnostics
emitted DURING the walk by node handlers with context the
post-walk passes can't reconstruct. Drained by the top-level
walk function on both match and non-match paths so a re-used
context can't leak entries between walks.
Test totals: 1399 → 1414 passing (+15: 10 derivation rules + 1
sibling CTE + 4 arity match/mismatch tests). Clippy clean.
Closes the two diagnostics deferred by sub-phase 2d that were not
attached to a user-approved deferral. `cte_arity_mismatch` stays
deferred — it depends on the §10.3 stage-2 CTE harvest, which IS a
user-approved deferral.
- `diagnostic.projection_alias_misplaced` (ADR-0032 §11.2): emitted
when a projection alias is referenced from `WHERE` / `HAVING` /
`GROUP BY`. `ORDER BY` references are allowed and silent. The
pass is integrated into `schema_existence_diagnostics`: when a
bare-column ref doesn't resolve to any binding's column but DOES
match a projection alias in the current SELECT leg, the new
diagnostic pre-empts the misleading `unknown_column` that would
otherwise fire on the same span. Real-column-shadowed-by-alias
cases (engine resolves to the table column) stay silent. Subquery
scopes (paren depth > 0) keep their own implicit alias bag —
outer aliases don't leak into inner WHERE.
- `diagnostic.compound_arity_mismatch` (ADR-0032 §11.2 / §11.7): a
new MatchedPath-walking pass that counts projection items per
SELECT leg by tallying top-level commas at the leg's own paren-
depth, then compares adjacent legs across `UNION` / `UNION ALL` /
`INTERSECT` / `EXCEPT` operators. The diagnostic anchors on the
operator span. Per-depth book-keeping lets chained compound
queries inside CTE bodies / subqueries report independently.
Function-call argument commas (deeper depth) are correctly
ignored.
Test totals: 1385 → 1399 passing (+14), 0 failed, 1 ignored.
Clippy clean.
`Cargo.toml`: add `column_metadata` to rusqlite's feature list.
This pulls in the SQLite `SQLITE_ENABLE_COLUMN_METADATA`
compile flag and surfaces `sqlite3_column_table_name` /
`sqlite3_column_origin_name` on prepared statements via
rusqlite's `Statement::columns_with_metadata()`.
`do_run_select` in db.rs now calls a new
`resolve_select_column_types(conn, stmt)` helper after
`prepare`. The helper walks each result-column's origin
metadata; when both `table_name` and `origin_name` come back
populated (the result column traces back to a base-table
column), it looks up the playground type in
`__rdbms_playground_columns`. The per-column types thread
through to `format_cell(value, ty)` so the data-table
renderer (ADR-0016) gets the same per-type rendering it
applies to `show data` results.
Effect: ADR-0030 Phase-1 §4.5 (bool SELECT results render as
`0` / `1`) is lifted for any bare-column reference whose
origin the engine carries through — per ADR-0032 Amendment 1
(2026-05-20 empirical probe), that means all non-recursive
CTE bodies, scalar subqueries (aliased or not), derived
tables, set ops, and JOINs. Computed projections and
recursive-CTE result columns remain typeless (the engine
populates no origin), which the renderer handles via neutral
alignment.
The lookup is engine-driven verbatim — no grammar-side
structural classification (ADR-0032 Amendment 1 replaces
§12's original "structurally a single column reference" rule
with "trust column_table_name / column_origin_name").
Tests (3 new in `tests/sql_select.rs`, all green):
- `database_run_select_recovers_bool_column_type` — the
Phase-1 §4.5 case: `SELECT Active FROM Products` returns
`column_types = [Some(Bool)]` and rows render as `true` /
`false`.
- `database_run_select_recovers_text_type_through_alias` —
`SELECT Name AS n FROM Users` remaps the result column
name to `n` but the origin metadata still resolves the
playground type to `Some(Text)`.
- `database_run_select_computed_expression_stays_typeless`
— `SELECT Score + 1 FROM T` keeps `column_types[0] =
None`, the documented Amendment-1 exception.
The CTE pass-through, scalar subquery, set-op, and JOIN
cases all work for free given the empirical findings;
their behaviour is asserted by the Amendment-1 probe
results recorded in the ADR, so no per-case integration
tests are duplicated here.
Test totals: 1382 → 1385 passing (+3), 0 failed, 1 ignored.
Clippy clean.
Implements the bulk of ADR-0032 §11 diagnostics. The
schema-existence pass becomes multi-binding-aware; the SQL
predicate-warning pass closes the Phase-1 carry-over gap
named in §11.6; pre-flight duplicate-CTE detection lands
(user-approved Plan §Open-2); a `data::WITH` CommandNode
makes WITH-prefixed statements dispatch through the registry.
Catalog (`src/friendly/strings/en-US.yaml`, `src/friendly/keys.rs`):
- Six new `diagnostic.*` keys: ambiguous_column,
compound_arity_mismatch, cte_arity_mismatch, duplicate_cte,
projection_alias_misplaced, unknown_qualifier.
- Eight new `engine.*` translation keys (ADR-0032 §11.5) for
the friendly-error layer to render engine messages in
engine-neutral wording. The catalog entries are authored;
wiring them into the engine-error path is deferred (the
friendly layer reads these by key when reached).
Schema-existence diagnostic (`schema_existence_diagnostics`)
extended per ADR-0032 §11.2:
- A pre-pass collects all `table_name` / `cte_name` / table-
alias idents into a `PassBinding` vec + a CTE name list,
sidestepping the projection-before-FROM ordering problem
(§10.6). The main pass then resolves identifiers against the
complete scope.
- Bare column references resolve against any binding's
columns. Zero matches → `diagnostic.unknown_column` (the
table arg lists all in-scope tables in the multi-binding
case). Two-or-more matches → `diagnostic.ambiguous_column`.
- Qualified `t.c` refs detect their qualifier via a look-ahead
on the matched path (Punct '.' + Ident{role:
sql_expr_qualified_ref} after the leading Ident). Unknown
qualifier → `diagnostic.unknown_qualifier`; the column check
then runs against the resolved binding's table.
- The `t.*` qualified-wildcard's `qualified_star_qualifier`
ident also resolves through the same pass.
- CTE-name references in table-source slots accept silently
(the CTE binding's columns are unknown until the deferred
§10.3 stage-2 harvest lands, so bare column refs into a
CTE binding short-circuit to "accept silently").
- Duplicate CTE names in the same `WITH` block emit
`diagnostic.duplicate_cte` on the second occurrence
(Plan §Open-2).
Phase-1 gap closure (`sql_predicate_warnings`, ADR-0032 §11.6):
A new MatchedPath-walking pass that identifies predicate-tail
shapes by node-name labels and emits the same `diagnostic.*`
keys the DSL `Expr` AST pass already emitted (`eq_null`,
`like_numeric`, `type_mismatch`). Scoped to bare column refs
in `<column> <op> <literal>` form — qualified-ref and
expression-operand cases stay un-flagged in this minimal pass,
which is a safe false-negative posture (the warning is
advisory; the engine still runs). Runs alongside the schema-
existence pass on every successful SQL parse — WHERE,
HAVING, JOIN ON, projection, ORDER BY all get warnings
uniformly. Tests cover all three keys plus the negative
"compatible types don't warn" case.
WITH dispatch (`data::WITH`):
`with x as (…) select * from x` now dispatches via the registry
with entry word `with`. Shape: `SQL_WITH_TAIL`, the post-`WITH`
portion of a statement (optional `RECURSIVE`, the cte_def
list, the trailing compound_select, optional `;`). Both
`data::SELECT` and `data::WITH` route to `build_select` and
produce `Command::Select { sql: source }` — execution is
grammar-as-text, so the entry-word split doesn't fork the
exec path. `is_advanced_only` extended to include `with`.
Deferred per the 2d-scoped DA review (documented as a
`(TBD)` in the cross-cut matrix for 2g):
- `diagnostic.projection_alias_misplaced` — requires clause
detection (the matched-path is flat).
- `diagnostic.compound_arity_mismatch` — needs per-leg
projection counting.
- `diagnostic.cte_arity_mismatch` — depends on §10.3 stage-2
harvest, which 2b deferred.
- `engine.*` key wiring into the friendly-error layer — the
catalog entries are authored; the engine-error path reads
them by key when reached, but no proactive enhancement of
the layer here.
Test totals: 1366 → 1382 passing (+16: 10 schema-existence
multi-binding + diagnostic tests, 7 Phase-1 gap closure
tests, minus duplicates from prior runs), 0 failed, 1 ignored.
Clippy clean.
The Phase-1 SQL `SELECT` grammar nodes that used to live in
`src/dsl/grammar/data.rs` retire — 22 statics / consts and the
`reject_internal_table` validator copy are removed, ~150 lines
of grammar machinery gone. `data::SELECT.shape` now references
the post-`SELECT` portion of the ADR-0032 fragment via a thin
`Node::Subgrammar(&sql_select::SQL_SELECT_TAIL)`.
`SQL_SELECT_TAIL` is a new export from `sql_select.rs`,
parallel to `SQL_SELECT_STATEMENT`. It represents what a
top-level `SELECT` statement looks like AFTER the registry's
entry-word dispatch has already consumed the leading `SELECT`
keyword: the DISTINCT/ALL prefix, projection list, optional
FROM / WHERE / GROUP BY / HAVING, the compound set-op chain
(each subsequent leg's `SELECT` is part of `SET_OP_TAIL`),
outer ORDER BY / LIMIT, and a tolerated trailing `;`.
WITH-prefixed statements (`WITH x AS (…) SELECT * FROM x`)
are NOT in 2c's scope — they need a separate `data::WITH`
`CommandNode` so the entry-word dispatch routes correctly.
For now, top-level WITH continues to fall through to the
chumsky parser route (the same as in Phase 1). The
`SQL_SELECT_STATEMENT` static (which includes the optional
WITH prefix) stays available for use by that future
CommandNode or by any other consumer that needs the full
statement shape.
All seven Phase-1 SQL `SELECT` integration tests
(`tests/sql_select.rs`) pass without modification, satisfying
the 2c exit gate's "behaviour preserved" requirement. The
70 fragment unit tests and the 26 driver-level scope tests
also pass — the migration is a refactor, no new tests
required.
Behaviour change explicitly sanctioned by ADR-0032 §8:
Phase-1's `LIMIT_VALIDATOR` (positive-int-only, parse-time)
is superseded by the full `sql_expr` admission. `LIMIT max(10,
x)` and similar now parse; the engine constrains the value at
execution time per the ADR's "grammar admits, engine
rejects" posture.
Plan §2b status note: the 2026-05-20 deferral of §10.3 stage 2
(CTE output-column harvest derivation) is recorded in
`docs/plans/20260520-adr-0032-phase-2.md` per the
user-approved deferral.
Test totals: 1366 passing (unchanged), 0 failed, 1 ignored.
Clippy clean. data.rs loses ~150 lines of dead grammar; the
single source of truth for the SQL `SELECT` shape is now
`sql_select.rs`.
Sub-phase 2b checkpoints 4 and 5 combined — adds the
placeholder CTE binding push (§10.3 stage 1) and the
projection alias accumulator (§10.4).
Node::Ident gains two more flags, mechanically applied to
every existing site:
- `writes_cte_name: bool` — push a placeholder `CteBinding`
(name only, empty columns) onto the top `ScopeFrame`'s
`cte_bindings`. Set on `CTE_NAME_IDENT` in sql_select.rs.
Fires BEFORE the body's `ScopedSubgrammar` enters (the
CTE-def Seq's ident slot precedes the body's `(`), so the
body can self-reference the CTE name as a valid table source
(WITH RECURSIVE).
- `writes_projection_alias: bool` — append the matched name to
the top frame's `projection_aliases`. Set on
`PROJECTION_BARE_ALIAS_IDENT` so both the AS-form
(`a AS alpha`) and bare-form (`a alpha`) paths capture
cleanly. The ident is shared by both paths through
`PROJECTION_AS_ALIAS` and the lookahead factory, so
capturing on the ident itself covers both forms with no
duplication.
The §10.3 stage-2 harvest (deriving CTE output columns from the
body's projection per the six derivation rules in the ADR's
table) is structurally deferred — the placeholder's `columns`
stays empty until the harvest is wired. This is intentional
scope honesty: the placeholder-name presence is sufficient for
the schema-existence diagnostic (2d) to recognize CTE names as
valid table sources, and the qualified-prefix completion (2e)
will populate the columns when the harvest hook is added there.
Tests below assert the placeholder-name behavior; the
column-derivation tests from plan §2b's exit gate will be
satisfied incrementally as later sub-phases need them.
Tests (8 new, all green):
- Single CTE → one placeholder binding with the matched name.
- Multiple CTEs → placeholders in declaration order.
- Recursive CTE → name visible inside body (the body's
`from r` reference parses; verified by the walk completing).
- Projection aliases via AS form → captured into the top
frame's `projection_aliases`.
- Projection aliases via bare form → captured.
- Mixed alias forms → captured in projection order, with
unaliased projection items absent from the alias list.
- No aliases → empty `projection_aliases`.
- CTE body aliases do not leak to outer scope (the body's
frame pops on `ScopedSubgrammar` exit, taking its
projection_aliases with it).
All 1358 previous tests still pass. Test totals: 1366
passing, 0 failed, 1 ignored. Clippy clean.
This closes out the scope-accumulator side of sub-phase 2b.
The remaining 2b-style work — full CTE column-derivation
harvest per §10.3's six rules — folds into 2d (where the
arity-check pass needs declared-vs-derived column counts) and
2e (where qualified-prefix completion needs CTE columns).
Sub-phase 2b checkpoint 3 — the `writes_table` / `writes_table_alias`
flags now drive the multi-binding `from_scope` accumulator on
the top `ScopeFrame`.
Node::Ident gains `writes_table_alias: bool`. When set on an
ident-name slot, the matched name lands on the most-recently-
pushed `TableBinding`'s `alias`. All 46 existing Ident sites
across the codebase are updated to `writes_table_alias: false`
(mechanical — no behavioral change for DSL paths).
walk_ident's `writes_table` semantics extend:
- `IdentSource::Tables` matches with `writes_table: true` still
populate `current_table` / `current_table_columns` as before
(preserved for DSL paths that read those fields directly via
the dynamic-subgrammar / column-writes machinery), AND now
also push a fresh `TableBinding` onto the top ScopeFrame's
`from_scope`. The two mechanisms coexist additively —
current_table reflects the most-recent `writes_table` write
(single-binding view, as before); from_scope is the
authoritative multi-binding accumulator that SQL JOINs,
subqueries, and CTE bodies use.
sql_select.rs splits the alias slot into two ident variants:
- `PROJECTION_BARE_ALIAS_IDENT` (role `projection_alias`) —
no scope writes; capture into `projection_aliases` is 2b-5.
- `TABLE_SOURCE_BARE_ALIAS_IDENT` (role `table_alias`,
`writes_table_alias: true`) — sets the top binding's alias.
The `AS alias` form likewise splits into PROJECTION_AS_ALIAS
and TABLE_SOURCE_AS_ALIAS so each path threads through the
correct ident. The bare-alias lookahead factories return the
projection or table-source ident accordingly.
`TABLE_NAME_IDENT` in sql_select.rs gets `writes_table: true`
so each FROM / JOIN table source pushes a binding. The
schema-resolved columns are stored on the TableBinding for
later use by qualified-prefix completion (2e) and the
schema-existence diagnostic (2d).
Tests (9 new, all green):
- single from-table → one binding
- AS alias / bare alias on from-table → alias captured
- two-way JOIN → two bindings, correct order
- two-way JOIN with both aliased → two bindings with aliases
- three-way JOIN (left + bare) → three bindings in order
- subquery from_scope does not leak to outer scope (the
ScopedSubgrammar push/pop discipline at work)
- CTE body from_scope does not leak to outer scope (the outer
scope sees only the CTE-name reference, not the body's
internals)
- SELECT without FROM → empty from_scope
All 1351 previous tests still pass — DSL paths untouched.
Test totals: 1358 passing, 0 failed, 1 ignored. Clippy clean.
Frame is_cte_body marker, body-projection harvest, and
projection_aliases population are the remaining 2b work
(2b-4 and 2b-5).
Sub-phase 2b checkpoint 2 — closes the recursion loop between
sql_expr.rs and sql_select.rs so subquery expressions and
qualified column refs become structurally valid in every SQL
context where they belong.
sql_expr.rs:
- §5 qualified-ref tail. `name_or_call` gains a `.identifier`
suffix as a Choice sibling of the function-call `(args)`
tail. The leading identifier is still matched once (per
ADR-0031 §1's factoring); the optional tail dispatches
between the two suffixes by their first character (`.` vs
`(`).
- §6.1 scalar subquery as primary. The `(or_expr)` and
`(SELECT …)` branches share the leading `(`; the first
inside token (`SELECT` → subquery, anything else →
expression) discriminates. The subquery recurses through
`Node::ScopedSubgrammar(&sql_select::SQL_SELECT_COMPOUND)`.
- §6.2 IN (subquery) predicate. Sibling of the existing
IN-value-list; same `(` factoring, same dispatch.
- §6.3 [NOT] EXISTS primary. Bare `EXISTS (compound_select)`
lives in `primary`; `NOT EXISTS` falls out via the existing
`not_expr := NOT not_expr` tier above `primary`.
sql_select.rs:
- CTE body recursion rewires `Node::Subgrammar` →
`Node::ScopedSubgrammar`, matching §10.2. The top-level
statement's COMPOUND embedding stays plain Subgrammar — the
implicit bottom frame is the right scope for a statement-
level SELECT.
Structural side-effect — const-eval cycle workaround:
Closing the sql_expr ⇄ sql_select reference loop made Rust's
const-evaluator follow the cycle through every `const Node`
that transitively reaches it. Mirroring sql_expr.rs's existing
pattern, composition Nodes in sql_select.rs (Seq / Choice /
Optional / Repeated / Lookahead) are now `static Node` and
appear in slice positions through `Node::Subgrammar(&NAME)`
wraps; only leaf items (Punct, Word, Ident) remain `const`.
Same workaround applies to data.rs's SELECT_PROJ_LIST /
SELECT_PROJECTION chain and the inlined `SQL_EXPR` reference.
Statics resolve lazily at link time, so the cycle is valid;
const-eval is not, and the named `const SQL_EXPR` alias is
gone in both files (replaced with the inline `Node::Subgrammar
(&sql_expr::SQL_OR_EXPR)` expression at every use site).
Test coverage:
- sql_expr.rs gains 11 new tests for qualified refs, scalar
subquery, IN-subquery, EXISTS / NOT EXISTS, nested
subqueries, and the existing IN-value-list form (regression).
- sql_select.rs gains 7 new tests for qualified refs in WHERE,
scalar subqueries in WHERE / projection, IN / EXISTS / NOT
EXISTS in WHERE, nested subqueries, and qualified refs
inside CTE bodies.
- All 70 prior sql_select tests still pass; the 2a baseline
is preserved.
`(WITH x AS (…) SELECT * FROM x)` is explicitly NOT admitted
as a scalar subquery — ADR-0032 §1 / §9 wire subqueries to
SQL_SELECT_COMPOUND, which omits the outer with_clause. WITH
remains a statement-level-only construct. Documented in the
relevant test.
Test totals: 1333 → 1351 passing, 0 failed, 1 ignored
(unchanged). Clippy clean.
Sub-phase 2b checkpoint 1 — adds the foundation for SQL SELECT
lexical-scope discipline without changing existing walker
semantics.
New types in `dsl::walker::context`:
- `TableBinding` — one FROM-source binding with table name,
optional alias, and schema-resolved columns (§10.1).
- `CteBinding` + `CteColumn` — a CTE definition visible from
inside its body (WITH RECURSIVE self-reference) and from the
outer scope after harvest (§10.3).
- `ScopeFrame` — `from_scope`, `cte_bindings`, and
`projection_aliases` for one lexical scope. Default-empty;
the fields will be populated by later 2b checkpoints.
`WalkContext` gains `from_scope_stack: Vec<ScopeFrame>`,
initialised with one bottom frame in both `new()` and
`with_schema()`. The bottom frame is the implicit top-level
scope DSL paths and top-level SQL statements operate in;
`Node::ScopedSubgrammar` entries push and pop additional frames
on top. `current_table` / `current_table_columns` remain as
direct fields for this checkpoint — converting them to derived
helpers is a later 2b step.
New grammar-tree variant:
- `Node::ScopedSubgrammar(&'static Self)` — like `Subgrammar`,
but pushes a fresh `ScopeFrame` on entry and pops it on exit
(ADR-0032 §10.2). Shares `subgrammar_depth` with the plain
Subgrammar variant so the MAX_SUBGRAMMAR_DEPTH = 64 cap fires
uniformly across both — §9's "no new walker capability for
grammar recursion" claim holds. DSL Expr (ADR-0026) and
sql_expr.rs ladder (ADR-0031) recursion continue to use the
plain Subgrammar variant and never push a scope.
Driver gains a parallel `walk_scoped_subgrammar` arm; the
push/pop is unconditional so a speculatively-walked branch a
later Choice rolls back leaves the stack clean.
Test coverage in `driver.rs`:
- A recursive ScopedSubgrammar test grammar walks correctly
through depths 0-3.
- The depth cap fires the same `expression_too_deep` friendly
validation error as for plain Subgrammar.
- The bottom frame invariant: `WalkContext::new` seeds exactly
one frame, and after a walk the stack is restored.
No grammar tree references the new variant yet — the rewire of
sql_select.rs CTE bodies and the sql_expr.rs additive
extensions for §5/§6 are the next 2b checkpoint. Test totals:
1330 baseline + 3 = 1333 passing, 0 failed, 1 ignored. Clippy
clean.
Author the standalone walkable shape for the full standard-SQL
SELECT per ADR-0032 §1: compound queries with the four set ops
(UNION / UNION ALL / INTERSECT / EXCEPT), the five JOIN flavours
(INNER / LEFT [OUTER] / RIGHT [OUTER] / FULL [OUTER] / CROSS),
GROUP BY / HAVING, WITH and WITH RECURSIVE common table
expressions, LIMIT … OFFSET, DISTINCT / ALL, qualified-wildcard
`t.*` projection, and bare-alias projection (lifting ADR-0030
Phase-1 §4.2).
Recursion into SQL_SELECT_COMPOUND uses Node::Subgrammar for
2a; sub-phase 2b will rewire those references to the new
Node::ScopedSubgrammar variant for completion-scope discipline
(ADR-0032 §10.2). The Phase-1 data::SELECT CommandNode is not
touched here — the new fragment is reachable only from its own
tests until sub-phase 2c performs the migration.
Two implementation mechanisms realize ADR semantics without
changing them:
- Node::Lookahead disambiguates the projection_item Choice
(bare `*` vs `ident . *` qualified wildcard vs `sql_expr [
alias ]`) and gates bare-alias slots against continuation
keywords. The walker's walk_ident accepts any
identifier-shape token, including keyword-shape ones, and
Choice / Optional are first-match-wins; without lookahead a
bare-alias slot would greedily swallow FROM / WHERE / JOIN /
etc. Per-position follow-sets list which keywords legitimately
follow each alias slot. Same pattern as data.rs's
insert_first_paren precedent.
- INNER JOIN and bare JOIN are split into two distinct Choice
branches (each with a concrete leading keyword) rather than
sharing one Optional(Word("inner"))-leading branch. Avoids a
walker hazard where an Optional-leading-child Seq commits to
idx > 0 and then converts the next child's EOF NoMatch into
Incomplete, blocking the outer Choice from falling through to
later branches. Same semantic surface, distinct mechanism.
The §13 OOS shapes all have explicit reject tests (NATURAL,
USING, comma-FROM, LIMIT m,n, window OVER, VALUES, derived
tables). LATERAL has a noted partial limitation: the comma form
rejects via OOS-3, but the single-keyword form `FROM a LATERAL
JOIN b ON …` is admitted structurally because `lateral` parses
as a bare table-source alias for `a`. This matches ADR-0030's
"grammar admits identifier-shape tokens; engine resolves"
posture.
`__rdbms_*` rejection extends to every Phase-2 table-source
slot — the FROM table, each JOIN's table, each CTE name, and
the FROM inside any CTE body — via the reuseable
reject_internal_table validator.
70 new unit tests in sql_select.rs walk every §1 production and
every OOS reject case. Test totals: 1260 baseline + 70 = 1330
passing, 0 failing, 1 ignored (unchanged from baseline). Clippy
clean.
Per the Phase-2 plan sub-phase 2a exit gate. DA gate written
review: PASS.
The dispatch-layer mode gate (previous commit) made the submit
behaviour correct — `select` runs in advanced mode and shows
the SQL hint in simple mode. This commit extends that gating to
the ambient assistance layer so simple-mode users do not see
SQL leak through Tab completion, the live error overlay, or the
`[ERR]`/`[WRN]` validity indicator either.
`_in_mode` walker variants
--------------------------
- `completion_probe_in_mode`, `expected_at_input_in_mode`,
`input_verdict_in_mode`. Each sets `ctx.mode` before walking.
The empty-input / unknown-entry fallback in `completion_probe`
and `expected_at_input` filters the `REGISTRY` listing by
`is_advanced_only` so Tab does not offer `select` in simple
mode. Old signatures keep delegating to `Mode::Advanced`
(back-compat for tests + other callers).
`_in_mode` completion variants
------------------------------
- `candidates_at_cursor_in_mode`, `candidates_at_cursor_with_in_mode`.
Internally they route the `parse_command` completeness probe
through `parse_command_in_mode(input, mode)`, the
`completion_probe` call through `completion_probe_in_mode`,
and the `expected_at` fallback through
`expected_at_input_in_mode`. Old signatures default to
`Mode::Advanced`.
`EffectiveMode::as_mode`
------------------------
- Collapses the persistent / one-shot distinction the UI cares
about into the plain `Mode` the walker reads from
`WalkContext::mode`. App-level call sites that thread mode
into the walker chain use this.
App / input-render wiring
-------------------------
- `App::input_validity_verdict` runs only when effective mode
is plain `Simple` (per ADR-0027), so it hardcodes
`Mode::Simple` into the new `input_verdict_in_mode` call
rather than threading.
- `App::start_or_complete_at` / `_last` (the Tab handlers)
pass `self.effective_mode().as_mode()` into
`candidates_at_cursor_in_mode`, so a `:` one-shot or
persistent advanced gives full SQL completion, persistent
simple does not offer SQL.
- `input_render::render_input_runs` and `ambient_hint` are
invoked from `ui.rs` only when effective mode is plain
`Simple` (advanced rendering uses `plain_input_spans` and
skips ambient hinting per ADR-0022 §12). Their internal
`classify_input_with_schema` / `candidates_at_cursor` /
`parse_command` calls now go through the mode-aware variants
with `Mode::Simple` hardcoded — a SQL form in simple mode
surfaces as a definite-error overlay and the hint panel does
not offer it.
After this commit a simple-mode user typing `select` or
`sel<Tab>` sees nothing SQL-shaped: no live highlight, no Tab
completion candidate, the `[ERR]` indicator lit, and the on-
submit hint that names the recovery paths. An advanced-mode
user or a `:` one-shot sees the full SQL surface.
The first cut of advanced-mode SQL: a `select` line in advanced
mode parses, runs against the database, and renders its rows
through the existing data-table renderer; the same line in
simple mode lights up the precise "this is SQL" hint instead of
running.
Walker mode gate (ADR-0030 §2)
------------------------------
- `WalkContext` gains a `mode: Mode` field; `Mode` derives
`Default` (= `Simple`, matching the app's startup mode).
- `grammar::is_advanced_only` keys an advanced-only entry-word
set (Phase 1: just `select`). When the walker matches an
advanced-only entry word with `ctx.mode == Simple`, it
short-circuits to a `WalkOutcome::ValidationFailed` carrying
the `advanced_mode.sql_in_simple` catalog key — the input
highlights as a keyword, the validity indicator goes ERROR,
and the parse-error layer renders the "switch with `mode
advanced`, or prefix the line with `:`" hint.
- `parser::parse_command_with_schema_in_mode` (and the
schemaless `parse_command_in_mode`) threads the mode into
`WalkContext`; existing `parse_command*` entry points default
to `Mode::Advanced` (most permissive) so back-compat callers
see the full grammar.
- `App::submit` is unified: both modes route through
`dispatch_dsl(&effective_input, effective_mode)`, which now
parses with the line's effective mode. The placeholder
advanced-mode echo branch is gone.
Builder signature sweep (ADR-0031 §2)
-------------------------------------
- `CommandNode.ast_builder` gains a `source: &str` parameter,
forwarded by the walker. `build_select` reads it to put the
validated SQL text into `Command::Select`; the 21 existing
builders accept it as `_source`.
SQL `SELECT` (ADR-0030 §6, ADR-0031)
-------------------------------------
- New `Command::Select { sql: String }` variant. Every
exhaustive `match Command` updated (`verb`, `target_table`,
`build_translate_context`, `execute_command_typed`,
`typing_surface`'s label).
- `grammar::data::SELECT` `CommandNode`: projection (`*` or
`expr [as alias]` list), optional `FROM <table>`, optional
`WHERE`/`ORDER BY`/`LIMIT`, optional trailing `;`. The
expression slots reference the ADR-0031 fragment through
`Subgrammar(&sql_expr::SQL_OR_EXPR)`. The `FROM` table-name
slot carries a `reject_internal_table` validator that
refuses `__rdbms_*` references at parse time.
- The `FROM` clause is optional — `select 1`, `select upper('x')`
(zero-table constant/function-call SELECTs) work alongside
the single-table form. Standard SQL admits them and they are
the canonical learner probe.
- Implicit projection aliasing (`select a x`) is deliberately
unsupported — `from` is a keyword, the bare alias would be
ambiguous; only `select a as x` is admitted.
Worker / runtime
----------------
- `Request::RunSelect { sql, source, reply }` + a new
`Database::run_select` method. `do_run_select_request` runs
the prepared statement, collects rows into a `DataResult`
with `column_types: Vec<None>` (Phase-1 SELECT result columns
carry no playground type per ADR-0030 §6), and appends the
literal source line to `history.log` so replay re-runs it
(ADR-0030 §11).
- `runtime::execute_command_typed` gains a `Command::Select`
arm that calls `database.run_select(sql, src)` and maps to
`CommandOutcome::Query`, which flows into the existing
`AppEvent::DslDataSucceeded` → `render_data_table` path.
Catalog (ADR-0019)
------------------
- `advanced_mode.sql_in_simple` — the walker's gate message.
- `select.internal_table` — the `__rdbms_*` rejection.
- `parse.usage.select` — the parse-error usage template.
Tests
-----
Two `app::tests` cases that pinned the pre-ADR-0030 placeholder
echo are updated to pin the new dispatch contract — both verify
that the advanced-mode `select` (one persistent, one via the
`:` one-shot) produces `ExecuteDsl(Command::Select)` with the
submission's effective mode tagged on the echo. The matching
walking-skeleton test is updated likewise.
A separate follow-up commit lands the ambient mode-threading
(completion / live overlay / validity indicator) so simple-mode
users do not see SQL surfaced through Tab or the live error
overlay either — the dispatch-layer gate landed here is the
behavioural foundation that follow-up builds on. Integration
tests for the full end-to-end land in a third commit.
A new `src/dsl/grammar/sql_expr.rs` authored as a parallel
fragment to `expr.rs` (the DSL `WHERE` grammar, ADR-0026). The
ADR's stratified ladder lands as named `static` `Node`s, one per
precedence tier:
or_expr → and_expr → not_expr → predicate → additive →
multiplicative → unary → primary
Recursion through `Node::Subgrammar` reuses ADR-0026's
`MAX_SUBGRAMMAR_DEPTH = 64` cap unchanged; no new walker
capability is required. `predicate_tail` follows ADR-0026's
factoring (shared operand prefix, infix `NOT` as an explicit
branch, no `Optional`-first branch) so `Choice` discriminates
cleanly. `name_or_call` factors the identifier-prefix shared
between column refs and function calls into a single `Ident`
followed by an `Optional` `( call_args )` tail — the same
hazard-avoidance shape `predicate_tail` uses.
The fragment exports `pub static SQL_OR_EXPR` (test entry) and
`pub static SQL_EXPRESSION` (drop-in `Subgrammar(&SQL_OR_EXPR)`
that SQL `CommandNode` shapes embed in their `Seq`). No AST
builder — every Phase-1 consumer (SELECT projection, WHERE)
runs validated SQL as text per ADR-0030 §4/§6.
13 unit tests cover every operator and precedence pair, the
full predicate set, `CASE` (searched + simple) including
`count(*)` and `count(distinct …)`, parenthesised regrouping,
case-insensitive keywords, the depth cap, and a representative
set of malformed inputs that do *not* walk.
Module registered via one new line in `grammar/mod.rs`.
Completes ADR-0029's implementation: the friendly-error layer
now names the rule a CHECK violation broke, and the
typing-surface matrix covers the whole constraint grammar.
CHECK-violation friendly error (ADR-0029 §10):
- enrich_dsl_failure gains a CHECK branch — it reads the column
from the engine's `CHECK constraint failed: <column>`
message, then resolves the table, the offending value, and
the column's compiled CHECK expression.
- FailureContext / TranslateContext carry the resolved
check_rule; translate_check renders "the value <v> breaks the
rule `<rule>`" when it is known, falling back to the plain
hint otherwise.
Typing-surface matrix: a new `constraints` submodule, 14 cells
covering the create-table / add-column constraint suffix and
the add-constraint / drop-constraint commands (174 → 188).
16 tests added (1 translate unit, 1 enrichment integration, 14
matrix cells).
Adds the two commands for modifying a column's constraints after
creation, completing ADR-0029's §2.2 surface.
Grammar (dsl/grammar/ddl.rs): `add constraint <constraint> to
<T>.<col>` reuses the §2.1 COLUMN_CONSTRAINT choice; `drop
constraint <kind> from <T>.<col>` names only the kind. Both join
the `add` / `drop` choices, discriminated by the `constraint`
form word.
AST (dsl/command.rs): `Command::AddConstraint` / `DropConstraint`
plus the `Constraint` / `ConstraintKind` enums.
Worker (db.rs): `do_add_constraint` / `do_drop_constraint` apply
the change through the rebuild-table primitive. `add` runs the §5
dry-run first — `not null` / `unique` / `check` against a
populated column are refused, before any write, with a
pretty-printed table of offending rows. §9 redundant-on-PK
declarations and §6 `default` on an auto-generated column are
friendly refusals; dropping a constraint the column does not
carry is likewise refused.
Also fixes schema_to_ddl, which suppressed UNIQUE for every PK
column — a compound-PK member is not individually unique, so an
explicit UNIQUE on it must survive the rebuild.
23 tests added (6 grammar, 17 worker); 3 completion-test and 3
matrix snapshots updated for the new `constraint` subcommand.
The fourth constraint. `check ( <expr> )` reuses the ADR-0026
WHERE-expression grammar via `Subgrammar`, so a check is
written in the same language as a `where` filter.
- Grammar: a `CHECK_CONSTRAINT` arm joins the shared
constraint-suffix Choice; `consume_check_expr` extracts the
parenthesised expression (paren-depth aware) into
`ColumnSpec.check` / `Command::AddColumn.check`.
- Storage: the parsed `Expr` is compiled once to inline SQL
(`compile_check_sql` — `compile_expr` + ADR-0028's
param-inliner) and stored in that form everywhere — a new
`check_expr` column in `__rdbms_playground_columns`,
`project.yaml`'s `ColumnSchema.check`, and the column DDL
emitted by `do_create_table` / `schema_to_ddl`.
- `add column … check` routes through the rebuild primitive
(SQLite's `ALTER … ADD COLUMN` cannot carry it); a CHECK on
a serial/shortid column is create-table-only and refused at
add-column with a friendly message.
- `describe` surfaces the CHECK. ADR-0029 §7/§8 updated to the
SQL-form decision — double-quoted identifiers, consistent
with ADR-0028's `explain` display SQL.
1201 tests pass (+8); clippy clean.
`add column` now accepts the shared constraint suffix and the
worker honours it — the surface where NOT NULL / UNIQUE
actually matter, on non-PK columns.
- Grammar: `ADD_COLUMN_NODES` gains the constraint-suffix
fragment; `collect_column_constraints` folds it into
`Command::AddColumn`.
- `do_add_column` routes per ADR-0029 §6: SQLite's `ALTER
TABLE ADD COLUMN` cannot express `UNIQUE` and requires a
default for `NOT NULL`, so those go through the rebuild
primitive (`do_add_constrained_column_via_rebuild`); plain
cases keep the ALTER path with the constraint suffix
appended.
- Pre-flight refusals, before any SQL write: a NOT NULL
column with no default added to a populated table; a UNIQUE
column with a default added to a multi-row table; a default
on a `serial` / `shortid` column.
CHECK is still deferred to the next commit. 1193 tests pass
(+9); clippy clean.
`create table … with pk` now parses the column-constraint
suffix; combined with the commit-1 db layer, a constrained
table works end to end.
- A shared constraint-suffix grammar fragment — `not null`,
`unique`, `default <literal>` — sits after each column's
`(type)` group; `build_create_table` walks the matched path
per column and folds the constraints into `ColumnSpec`.
- §9 redundancy check: every `with pk` column is a primary-key
column, so `not null` (any) and `unique` (single-column PK)
are rejected with a friendly error
(`parse.custom.constraint_redundant_on_pk`).
- `project.yaml` round-trip: `ColumnSchema` gains `not_null` /
`default`; the YAML reader/writer and `build_read_schema`
carry them, so `rebuild` / `export` / `import` preserve
constraints.
- ADR-0029 §2.1's example corrected — `create table` columns
are all PK columns, so its suffix is for `default` / `check`;
`docs/simple-mode-limitations.md` records that non-PK
columns at create time need advanced mode.
CHECK is deferred to the next commit. 1184 tests pass (+7);
clippy clean.
The database layer now honours the ColumnSpec constraint
fields end to end, ahead of the grammar that lets users type
them.
- `do_create_table` emits ` NOT NULL` / ` UNIQUE` / ` DEFAULT
<literal>` per column via the new `column_constraints_sql`
helper (the default literal bound against the column's type).
- `ReadColumn` gains `default_sql`, read from
`pragma_table_info.dflt_value`; `schema_to_ddl` emits it, so
the rebuild-table primitive preserves DEFAULT — it already
preserved NOT NULL / UNIQUE.
- `ColumnDescription` gains `unique` / `default`;
`do_describe_table` now sources columns from `read_schema`
(one source of per-column truth) and `constraints_display`
lists PK / NOT NULL / UNIQUE / DEFAULT.
No user-facing change yet — no grammar produces constrained
columns. Tests exercise creation, enforcement, describe, and
rebuild-preservation programmatically.
1177 tests pass (+5); clippy clean.
Expand ColumnSpec and Command::AddColumn with the four
ADR-0029 constraint slots (not_null, unique, default, check),
all defaulting off; `Database::add_column` now takes a
ColumnSpec. No behaviour change — the grammar to set the
fields and the DDL to enforce them land in the following
commits. Isolated here so those commits stay readable.
Adds ColumnSpec::new for the unconstrained case; 110 call
sites updated. 1172 tests pass; clippy clean.
13 matrix cells for the `explain` prefix across all three
wrapped commands — `explain show data` / `explain update` /
`explain delete` — covering each typing position (after the
prefix, the inner entry word, the table, the filter clause)
plus the three complete forms. The cells confirm `explain`
plugs into the inner query grammars cleanly: candidates, hints
and column scoping match the standalone commands, and the
complete forms parse as `Command::Explain`.
Also adds a worker test pinning the display SQL's `<>`
rendering of inequality (ADR-0028 §3).
Matrix: 161 -> 174 cells. 1172 tests pass; clippy clean.
`render_explain_plan` now classifies each plan node and colours
its category-bearing keywords through the styled-runs mechanism.
- `PLAN_TAXONOMY`: a substring-pattern table mapping the
engine's plan vocabulary to four semantic classes — full
scan / temp B-tree -> Expensive, index search / covering
index / PK lookup -> Efficient, automatic index ->
AutomaticIndex. An unrecognised detail renders neutral, since
the engine's plan vocabulary may grow.
- Only the matched keyword run carries the category colour;
connectors, prefixes and table / index names stay neutral
(ADR-0028 §6). The display-SQL line is wholly neutral.
- An automatic-index node also gets the distinct "← add an
index?" advice tag, so it reads as guidance, not merely
"this is slow".
1158 tests pass (+7); clippy clean.
Add the `explain` prefix command — `explain show data`,
`explain update`, `explain delete` — from grammar through to a
rendered plan tree.
- Grammar: an `EXPLAIN` CommandNode whose shape is a Choice over
the three explainable query shapes, referenced (not
duplicated) through `Subgrammar`. `Command::Explain { query:
Box<Self> }`; `build_show_data` is extracted so the role-based
builders serve both standalone and explain-wrapped commands.
- Worker: SQL construction is split out of do_query_data /
do_update / do_delete into `build_*_sql`, so EXPLAIN QUERY
PLAN runs the exact same statement. `Request::ExplainPlan` /
`do_explain_plan` capture the plan; `QueryPlan` / `ExplainRow`
carry it back. EXPLAIN QUERY PLAN never executes, so
explaining update/delete changes nothing.
- Display SQL: the executed statement with `?N` parameters
inlined as standard-SQL literals via a quote-aware scan.
- Render: `render_explain_plan` draws the box-drawing plan tree
(plain output; ADR-0028 step 4 adds the styled tree).
- Catalog: `parse.usage.explain` and the `help.data.explain`
entry, so `explain` shows up in the in-app `help` listing.
1151 tests pass (+18); clippy clean.
OutputLine gains an optional styled-runs payload — a
Vec<OutputSpan> of { byte_range, OutputStyleClass } over the
line text. render_output_line gains a branch: when the payload
is present it renders the text span-by-span, each run's
semantic class (Neutral / Efficient / Expensive /
AutomaticIndex) resolved to a theme colour at render time;
otherwise the existing whole-line kind styling. The echo path
is untouched.
Theme gains `plan_efficient` — a green deliberately distinct
from `system` so green never reads as two things (ADR-0028 §6);
`warning` is reused for expensive steps.
A general per-span output-styling capability (ADR-0016's OOS-3
realized); the query-plan renderer will be its first consumer.
No user-visible change on its own. 1133 passing, clippy clean.
On launch an empty temp project is created but, by design
(ADR-0015), auto-deleted on quit while still empty. The
unconditional `write_last_project` at startup recorded that
temp's path anyway, so a later `--resume` resolved to a
since-deleted directory and printed a confusing
"recorded project … no longer exists".
All three resume-pointer writes are now gated on
`!project.is_unmodified_temp()`: the startup write, the
on-switch write (a `new`-command switch to a fresh temp no
longer records it), and a new on-quit write. The quit write is
where a launch-temp the user *filled with content* finally
gets remembered — startup skipped it while it was still empty.
An unmodified empty temp is deleted, never recorded; the two
dispositions are mutually exclusive.
The "no previous project" friendly error the user asked for
already exists (`project.resume_no_previous`, wired in the
resume resolution) — verified, no change needed. The gate
predicate `is_unmodified_temp` is covered by existing
integration tests. 1131 passing, clippy clean.
Typing `--` to start an optional trailing flag (`--create-fk`
on `add 1:n relationship`, `--cascade` on `drop column`,
`--force-conversion` / `--dont-convert` on `change column`)
made completion go empty: the trailing `--` turns the parse
into a trailing-junk Mismatch, and the Mismatch arm of the
completion expected-set resolution returned only `[EndOfInput]`
— the skipped optional-flag expectations, carried in
`tail_expected`, were dropped.
completion_probe and expected_at_input now merge `tail_expected`
into a Mismatch's expected set. `tail_expected` is empty for a
genuine mid-command mismatch, so this only adds the outer
shape's skipped trailing optionals — exactly the continuations
the trailing `--` is starting to type. This also resolves the
"wrong usage hint" symptom: with `--create-fk` offered as a
candidate, the hint panel shows candidates instead of falling
through to the parse-error usage block.
Audit outcome (the requested scan): usage_key_for_input was
verified correct for every multi-form command — add / drop /
show, including the digit-led `add 1:n relationship` form —
and is now regression-locked. The flag-completion fix covers
the whole optional-trailing-flag class.
6 tests (3 flag-completion, 3 usage-key). 1131 passing.
The validity-indicator debounce was two locals in the event
loop (indicator_pending + app.input_indicator) with no unit
coverage — ADR-0027's as-built notes flag it as untested async
glue. The decision logic is now an IndicatorDebounce struct:
note_event (a keystroke hides + arms; non-key events leave it
be), settle (the quiet window elapsed → show the verdict +
disarm), is_armed (drives the recv timeout), visible (mirrored
into app.input_indicator for the renderer).
No behaviour change — the tokio timer and terminal stay in the
loop. 7 unit tests cover the debounce contract: the keystroke /
settle cycle, clean verdicts, and that a background event
mid-typing does not cancel the owed recompute. 1125 passing,
clippy clean.
ambient_hint now reads the walker's schema-aware diagnostics.
input_diagnostics is non-empty only for a command that
structurally parses — so a non-empty result means "complete
and submittable, but wrong or dubious". That is checked early
(right after the Tab-cycle memo), ahead of slot hints and
completions: a command that parses but is flawed no longer
gets the misleading "Submit with Enter" prose, it gets the
diagnostic's why. pick_hint_diagnostic prefers the diagnostic
under the cursor, else the most severe.
The cursor-local invalid-ident hint is kept for genuinely
incomplete commands (no Match → no diagnostics).
5 ambient_hint tests (unknown table, type-mismatch over
submit-prose, LIKE-numeric, clean command still submittable,
cursor-following). The complex_and_or matrix cell referenced a
non-existent column `t`; fixed to a real column so it tests a
valid expression as intended. 1118 passing, clippy clean.
render_input_runs now overlays the walker's schema-aware
diagnostics: an unknown table/column ERROR is recoloured
tok_error, an expression WARNING (type mismatch, = NULL, LIKE
on a numeric column) recoloured theme.warning. New overlay_span
covers a token's whole byte range (overlay_error only hits the
run at a single byte). New walker::input_diagnostics is the
shared entry point.
The overlay is global — every flagged token is coloured
wherever it sits, not only under the cursor — which is exactly
ADR-0027's motivation. The existing cursor-local invalid-ident
overlay is kept (it covers in-progress idents diagnostics do
not); the two are additive and idempotent.
5 input_render tests (unknown table/column, type-mismatch
literal precise, LIKE-on-numeric, clean command). 1113 passing,
clippy clean.
LIKE is a text-pattern match; against a numeric column (int,
real, decimal, serial) it runs but is almost never intended.
predicate_warnings now emits a WARNING for it, spanned at the
target column. New Type::is_numeric; catalog key
diagnostic.like_numeric; ADR-0027 gains "Amendment 1" and the
adr/README index line is updated per the index-upkeep rule.
bool and the text-/blob-backed types are deliberately not
flagged — see the amendment for the rationale.
3 walker tests (int, decimal NOT LIKE, text-column clean).
1108 passing, clippy clean.
Expression WARNING diagnostics (type mismatch, = NULL) carried
a coarse span — the whole WHERE clause, from the `where`
keyword to end of input. They now span exactly the offending
literal operand, read from the Operand source span added in the
previous commit. predicate_warnings derives the span per
warning; pair_type_mismatch returns (message, literal-span);
the dead where_clause_span helper is removed.
5 walker tests assert the spans cover exactly the literal /
identifier (type mismatch, = NULL, BETWEEN bounds, IN item,
unknown-column ERROR). 1105 passing, clippy clean.
Each WHERE-expression Operand now records the byte span of the
terminal it was built from — the precise per-literal highlight
target for an expression WARNING (finishing ADR-0027 §2's
highlight/hint wiring). parse_operand captures MatchedItem::span;
the RowFilter::eq convenience constructor uses Operand::NO_SPAN.
PartialEq is hand-written to ignore the span — it is editor
metadata, so Command equality stays whitespace- and
position-independent, which the Expr test corpus relies on.
No behaviour change; 1100 tests still pass, clippy clean.
A parse error in `add index …` showed the `add column` usage:
`add` and `drop` are multi-form commands, and both the
ambient hint and the submit-time usage block picked the
first-listed form unconditionally.
New `grammar::usage_key_for_input` disambiguates by the form
word after the entry keyword — `column` / `index` / `table` /
`relationship`, or the leading digit of `add 1:n …`. The
ambient hint now shows that one form; `render_usage_block`
shows the committed form's usage and falls back to the whole
family only for a bare `add` / `drop` with no form chosen.
Sweep: input_verdict tests confirm the schema-existence check
fires across the identifier-taking commands — unknown table
on drop / show / add column, unknown column on drop column /
update — and that known references stay clean. The Step B
check is grammar-generic, so this is verification + coverage
rather than new code.
Docs: requirements.md S6 -> [x], baseline 1096; CLAUDE.md
deferred list reconciled (C5a and S6 are done — removed);
ADR-0026's as-built note updated (step 5 shipped via
ADR-0027); ADR-0027 gains an As-built notes section
recording the post-walk diagnostics realization, the
pre-rendered message, the timeout-based debounce, coarse
WARNING spans, and the deferred highlight/hint wiring.
The event loop now time-boxes `recv` while an indicator
recompute is owed: every keystroke hides the indicator and
arms an `INDICATOR_DEBOUNCE` (1s) window; once typing pauses
that long the runtime computes `App::input_validity_verdict`
and shows `[ERR]` / `[WRN]`. An idle session (nothing owed)
still blocks plainly on `recv` — no wake-ups.
`update()` stays pure — the debounce timer lives in the
runtime; `App` only holds the resulting `input_indicator`
state, which the runtime clears on a keystroke and sets when
the quiet interval elapses.
`App::input_validity_verdict` is tested directly (a
simple-mode verdict, and silence in advanced mode / the `:`
one-shot); the debounce timing itself is runtime-loop glue,
covered at the integration level.
claude@clouddev1