Implements the six ADR-0032 §10.3 output-column derivation rules
at CTE body-frame exit, populating the placeholder CteBinding's
columns. Unblocks `diagnostic.cte_arity_mismatch` (which compares
declared col-list arity vs derived projection arity) and the
upcoming qualified-prefix completion in 2e proper.
- `WalkContext::pending_cte_harvest`: bookkeeping for an in-progress
CTE harvest, armed by writes_cte_name + extended by cte_column
idents, consumed by the next walk_scoped_subgrammar invocation
(CTE syntax has no intervening ScopedSubgrammar, so timing is
deterministic). Cleared on every walk_scoped_subgrammar entry
to prevent stale state surviving a speculative walk rollback.
- `run_cte_harvest`: post-walk path-scan classifier that
reconstructs the body's first leg's projection-list and applies
the six derivation rules. Compound bodies take columns from the
first leg per spec; recursive CTE bodies take the non-recursive
(first) leg. Optional (col-list) renames positionally with
preserved types.
- `expand_binding`: bridges a TableBinding to a CteColumn list,
resolving CTE-source bindings (empty columns + table-name
matches an in-scope CteBinding) through to the CTE's harvested
columns. Enables sibling CTEs to project correctly: in
\`WITH a AS (...), b AS (SELECT * FROM a) ...\`, b's harvest sees
a's derived columns through the body's from_scope binding.
- `WalkContext::pending_diagnostics`: accumulator for diagnostics
emitted DURING the walk by node handlers with context the
post-walk passes can't reconstruct. Drained by the top-level
walk function on both match and non-match paths so a re-used
context can't leak entries between walks.
Test totals: 1399 → 1414 passing (+15: 10 derivation rules + 1
sibling CTE + 4 arity match/mismatch tests). Clippy clean.
Closes the two diagnostics deferred by sub-phase 2d that were not
attached to a user-approved deferral. `cte_arity_mismatch` stays
deferred — it depends on the §10.3 stage-2 CTE harvest, which IS a
user-approved deferral.
- `diagnostic.projection_alias_misplaced` (ADR-0032 §11.2): emitted
when a projection alias is referenced from `WHERE` / `HAVING` /
`GROUP BY`. `ORDER BY` references are allowed and silent. The
pass is integrated into `schema_existence_diagnostics`: when a
bare-column ref doesn't resolve to any binding's column but DOES
match a projection alias in the current SELECT leg, the new
diagnostic pre-empts the misleading `unknown_column` that would
otherwise fire on the same span. Real-column-shadowed-by-alias
cases (engine resolves to the table column) stay silent. Subquery
scopes (paren depth > 0) keep their own implicit alias bag —
outer aliases don't leak into inner WHERE.
- `diagnostic.compound_arity_mismatch` (ADR-0032 §11.2 / §11.7): a
new MatchedPath-walking pass that counts projection items per
SELECT leg by tallying top-level commas at the leg's own paren-
depth, then compares adjacent legs across `UNION` / `UNION ALL` /
`INTERSECT` / `EXCEPT` operators. The diagnostic anchors on the
operator span. Per-depth book-keeping lets chained compound
queries inside CTE bodies / subqueries report independently.
Function-call argument commas (deeper depth) are correctly
ignored.
Test totals: 1385 → 1399 passing (+14), 0 failed, 1 ignored.
Clippy clean.
Session summary covering nine implementation commits since
handoff-26 (e032f01..0c3847a). Records the user-approved
deferral of §10.3 stage 2 (CTE column-derivation harvest),
flags three further 2d diagnostic-key deferrals that need
user confirmation, and points the next implementer at the
seams for sub-phases 2e (qualified-prefix completion +
post-walk fixup) and 2g (verification sweep + final
report).
Test totals at handoff: 1385 passing, 0 failed, 1 ignored
(+125 from the 1260 baseline at handoff-26).
`Cargo.toml`: add `column_metadata` to rusqlite's feature list.
This pulls in the SQLite `SQLITE_ENABLE_COLUMN_METADATA`
compile flag and surfaces `sqlite3_column_table_name` /
`sqlite3_column_origin_name` on prepared statements via
rusqlite's `Statement::columns_with_metadata()`.
`do_run_select` in db.rs now calls a new
`resolve_select_column_types(conn, stmt)` helper after
`prepare`. The helper walks each result-column's origin
metadata; when both `table_name` and `origin_name` come back
populated (the result column traces back to a base-table
column), it looks up the playground type in
`__rdbms_playground_columns`. The per-column types thread
through to `format_cell(value, ty)` so the data-table
renderer (ADR-0016) gets the same per-type rendering it
applies to `show data` results.
Effect: ADR-0030 Phase-1 §4.5 (bool SELECT results render as
`0` / `1`) is lifted for any bare-column reference whose
origin the engine carries through — per ADR-0032 Amendment 1
(2026-05-20 empirical probe), that means all non-recursive
CTE bodies, scalar subqueries (aliased or not), derived
tables, set ops, and JOINs. Computed projections and
recursive-CTE result columns remain typeless (the engine
populates no origin), which the renderer handles via neutral
alignment.
The lookup is engine-driven verbatim — no grammar-side
structural classification (ADR-0032 Amendment 1 replaces
§12's original "structurally a single column reference" rule
with "trust column_table_name / column_origin_name").
Tests (3 new in `tests/sql_select.rs`, all green):
- `database_run_select_recovers_bool_column_type` — the
Phase-1 §4.5 case: `SELECT Active FROM Products` returns
`column_types = [Some(Bool)]` and rows render as `true` /
`false`.
- `database_run_select_recovers_text_type_through_alias` —
`SELECT Name AS n FROM Users` remaps the result column
name to `n` but the origin metadata still resolves the
playground type to `Some(Text)`.
- `database_run_select_computed_expression_stays_typeless`
— `SELECT Score + 1 FROM T` keeps `column_types[0] =
None`, the documented Amendment-1 exception.
The CTE pass-through, scalar subquery, set-op, and JOIN
cases all work for free given the empirical findings;
their behaviour is asserted by the Amendment-1 probe
results recorded in the ADR, so no per-case integration
tests are duplicated here.
Test totals: 1382 → 1385 passing (+3), 0 failed, 1 ignored.
Clippy clean.
Implements the bulk of ADR-0032 §11 diagnostics. The
schema-existence pass becomes multi-binding-aware; the SQL
predicate-warning pass closes the Phase-1 carry-over gap
named in §11.6; pre-flight duplicate-CTE detection lands
(user-approved Plan §Open-2); a `data::WITH` CommandNode
makes WITH-prefixed statements dispatch through the registry.
Catalog (`src/friendly/strings/en-US.yaml`, `src/friendly/keys.rs`):
- Six new `diagnostic.*` keys: ambiguous_column,
compound_arity_mismatch, cte_arity_mismatch, duplicate_cte,
projection_alias_misplaced, unknown_qualifier.
- Eight new `engine.*` translation keys (ADR-0032 §11.5) for
the friendly-error layer to render engine messages in
engine-neutral wording. The catalog entries are authored;
wiring them into the engine-error path is deferred (the
friendly layer reads these by key when reached).
Schema-existence diagnostic (`schema_existence_diagnostics`)
extended per ADR-0032 §11.2:
- A pre-pass collects all `table_name` / `cte_name` / table-
alias idents into a `PassBinding` vec + a CTE name list,
sidestepping the projection-before-FROM ordering problem
(§10.6). The main pass then resolves identifiers against the
complete scope.
- Bare column references resolve against any binding's
columns. Zero matches → `diagnostic.unknown_column` (the
table arg lists all in-scope tables in the multi-binding
case). Two-or-more matches → `diagnostic.ambiguous_column`.
- Qualified `t.c` refs detect their qualifier via a look-ahead
on the matched path (Punct '.' + Ident{role:
sql_expr_qualified_ref} after the leading Ident). Unknown
qualifier → `diagnostic.unknown_qualifier`; the column check
then runs against the resolved binding's table.
- The `t.*` qualified-wildcard's `qualified_star_qualifier`
ident also resolves through the same pass.
- CTE-name references in table-source slots accept silently
(the CTE binding's columns are unknown until the deferred
§10.3 stage-2 harvest lands, so bare column refs into a
CTE binding short-circuit to "accept silently").
- Duplicate CTE names in the same `WITH` block emit
`diagnostic.duplicate_cte` on the second occurrence
(Plan §Open-2).
Phase-1 gap closure (`sql_predicate_warnings`, ADR-0032 §11.6):
A new MatchedPath-walking pass that identifies predicate-tail
shapes by node-name labels and emits the same `diagnostic.*`
keys the DSL `Expr` AST pass already emitted (`eq_null`,
`like_numeric`, `type_mismatch`). Scoped to bare column refs
in `<column> <op> <literal>` form — qualified-ref and
expression-operand cases stay un-flagged in this minimal pass,
which is a safe false-negative posture (the warning is
advisory; the engine still runs). Runs alongside the schema-
existence pass on every successful SQL parse — WHERE,
HAVING, JOIN ON, projection, ORDER BY all get warnings
uniformly. Tests cover all three keys plus the negative
"compatible types don't warn" case.
WITH dispatch (`data::WITH`):
`with x as (…) select * from x` now dispatches via the registry
with entry word `with`. Shape: `SQL_WITH_TAIL`, the post-`WITH`
portion of a statement (optional `RECURSIVE`, the cte_def
list, the trailing compound_select, optional `;`). Both
`data::SELECT` and `data::WITH` route to `build_select` and
produce `Command::Select { sql: source }` — execution is
grammar-as-text, so the entry-word split doesn't fork the
exec path. `is_advanced_only` extended to include `with`.
Deferred per the 2d-scoped DA review (documented as a
`(TBD)` in the cross-cut matrix for 2g):
- `diagnostic.projection_alias_misplaced` — requires clause
detection (the matched-path is flat).
- `diagnostic.compound_arity_mismatch` — needs per-leg
projection counting.
- `diagnostic.cte_arity_mismatch` — depends on §10.3 stage-2
harvest, which 2b deferred.
- `engine.*` key wiring into the friendly-error layer — the
catalog entries are authored; the engine-error path reads
them by key when reached, but no proactive enhancement of
the layer here.
Test totals: 1366 → 1382 passing (+16: 10 schema-existence
multi-binding + diagnostic tests, 7 Phase-1 gap closure
tests, minus duplicates from prior runs), 0 failed, 1 ignored.
Clippy clean.
The Phase-1 SQL `SELECT` grammar nodes that used to live in
`src/dsl/grammar/data.rs` retire — 22 statics / consts and the
`reject_internal_table` validator copy are removed, ~150 lines
of grammar machinery gone. `data::SELECT.shape` now references
the post-`SELECT` portion of the ADR-0032 fragment via a thin
`Node::Subgrammar(&sql_select::SQL_SELECT_TAIL)`.
`SQL_SELECT_TAIL` is a new export from `sql_select.rs`,
parallel to `SQL_SELECT_STATEMENT`. It represents what a
top-level `SELECT` statement looks like AFTER the registry's
entry-word dispatch has already consumed the leading `SELECT`
keyword: the DISTINCT/ALL prefix, projection list, optional
FROM / WHERE / GROUP BY / HAVING, the compound set-op chain
(each subsequent leg's `SELECT` is part of `SET_OP_TAIL`),
outer ORDER BY / LIMIT, and a tolerated trailing `;`.
WITH-prefixed statements (`WITH x AS (…) SELECT * FROM x`)
are NOT in 2c's scope — they need a separate `data::WITH`
`CommandNode` so the entry-word dispatch routes correctly.
For now, top-level WITH continues to fall through to the
chumsky parser route (the same as in Phase 1). The
`SQL_SELECT_STATEMENT` static (which includes the optional
WITH prefix) stays available for use by that future
CommandNode or by any other consumer that needs the full
statement shape.
All seven Phase-1 SQL `SELECT` integration tests
(`tests/sql_select.rs`) pass without modification, satisfying
the 2c exit gate's "behaviour preserved" requirement. The
70 fragment unit tests and the 26 driver-level scope tests
also pass — the migration is a refactor, no new tests
required.
Behaviour change explicitly sanctioned by ADR-0032 §8:
Phase-1's `LIMIT_VALIDATOR` (positive-int-only, parse-time)
is superseded by the full `sql_expr` admission. `LIMIT max(10,
x)` and similar now parse; the engine constrains the value at
execution time per the ADR's "grammar admits, engine
rejects" posture.
Plan §2b status note: the 2026-05-20 deferral of §10.3 stage 2
(CTE output-column harvest derivation) is recorded in
`docs/plans/20260520-adr-0032-phase-2.md` per the
user-approved deferral.
Test totals: 1366 passing (unchanged), 0 failed, 1 ignored.
Clippy clean. data.rs loses ~150 lines of dead grammar; the
single source of truth for the SQL `SELECT` shape is now
`sql_select.rs`.
Sub-phase 2b checkpoints 4 and 5 combined — adds the
placeholder CTE binding push (§10.3 stage 1) and the
projection alias accumulator (§10.4).
Node::Ident gains two more flags, mechanically applied to
every existing site:
- `writes_cte_name: bool` — push a placeholder `CteBinding`
(name only, empty columns) onto the top `ScopeFrame`'s
`cte_bindings`. Set on `CTE_NAME_IDENT` in sql_select.rs.
Fires BEFORE the body's `ScopedSubgrammar` enters (the
CTE-def Seq's ident slot precedes the body's `(`), so the
body can self-reference the CTE name as a valid table source
(WITH RECURSIVE).
- `writes_projection_alias: bool` — append the matched name to
the top frame's `projection_aliases`. Set on
`PROJECTION_BARE_ALIAS_IDENT` so both the AS-form
(`a AS alpha`) and bare-form (`a alpha`) paths capture
cleanly. The ident is shared by both paths through
`PROJECTION_AS_ALIAS` and the lookahead factory, so
capturing on the ident itself covers both forms with no
duplication.
The §10.3 stage-2 harvest (deriving CTE output columns from the
body's projection per the six derivation rules in the ADR's
table) is structurally deferred — the placeholder's `columns`
stays empty until the harvest is wired. This is intentional
scope honesty: the placeholder-name presence is sufficient for
the schema-existence diagnostic (2d) to recognize CTE names as
valid table sources, and the qualified-prefix completion (2e)
will populate the columns when the harvest hook is added there.
Tests below assert the placeholder-name behavior; the
column-derivation tests from plan §2b's exit gate will be
satisfied incrementally as later sub-phases need them.
Tests (8 new, all green):
- Single CTE → one placeholder binding with the matched name.
- Multiple CTEs → placeholders in declaration order.
- Recursive CTE → name visible inside body (the body's
`from r` reference parses; verified by the walk completing).
- Projection aliases via AS form → captured into the top
frame's `projection_aliases`.
- Projection aliases via bare form → captured.
- Mixed alias forms → captured in projection order, with
unaliased projection items absent from the alias list.
- No aliases → empty `projection_aliases`.
- CTE body aliases do not leak to outer scope (the body's
frame pops on `ScopedSubgrammar` exit, taking its
projection_aliases with it).
All 1358 previous tests still pass. Test totals: 1366
passing, 0 failed, 1 ignored. Clippy clean.
This closes out the scope-accumulator side of sub-phase 2b.
The remaining 2b-style work — full CTE column-derivation
harvest per §10.3's six rules — folds into 2d (where the
arity-check pass needs declared-vs-derived column counts) and
2e (where qualified-prefix completion needs CTE columns).
Sub-phase 2b checkpoint 3 — the `writes_table` / `writes_table_alias`
flags now drive the multi-binding `from_scope` accumulator on
the top `ScopeFrame`.
Node::Ident gains `writes_table_alias: bool`. When set on an
ident-name slot, the matched name lands on the most-recently-
pushed `TableBinding`'s `alias`. All 46 existing Ident sites
across the codebase are updated to `writes_table_alias: false`
(mechanical — no behavioral change for DSL paths).
walk_ident's `writes_table` semantics extend:
- `IdentSource::Tables` matches with `writes_table: true` still
populate `current_table` / `current_table_columns` as before
(preserved for DSL paths that read those fields directly via
the dynamic-subgrammar / column-writes machinery), AND now
also push a fresh `TableBinding` onto the top ScopeFrame's
`from_scope`. The two mechanisms coexist additively —
current_table reflects the most-recent `writes_table` write
(single-binding view, as before); from_scope is the
authoritative multi-binding accumulator that SQL JOINs,
subqueries, and CTE bodies use.
sql_select.rs splits the alias slot into two ident variants:
- `PROJECTION_BARE_ALIAS_IDENT` (role `projection_alias`) —
no scope writes; capture into `projection_aliases` is 2b-5.
- `TABLE_SOURCE_BARE_ALIAS_IDENT` (role `table_alias`,
`writes_table_alias: true`) — sets the top binding's alias.
The `AS alias` form likewise splits into PROJECTION_AS_ALIAS
and TABLE_SOURCE_AS_ALIAS so each path threads through the
correct ident. The bare-alias lookahead factories return the
projection or table-source ident accordingly.
`TABLE_NAME_IDENT` in sql_select.rs gets `writes_table: true`
so each FROM / JOIN table source pushes a binding. The
schema-resolved columns are stored on the TableBinding for
later use by qualified-prefix completion (2e) and the
schema-existence diagnostic (2d).
Tests (9 new, all green):
- single from-table → one binding
- AS alias / bare alias on from-table → alias captured
- two-way JOIN → two bindings, correct order
- two-way JOIN with both aliased → two bindings with aliases
- three-way JOIN (left + bare) → three bindings in order
- subquery from_scope does not leak to outer scope (the
ScopedSubgrammar push/pop discipline at work)
- CTE body from_scope does not leak to outer scope (the outer
scope sees only the CTE-name reference, not the body's
internals)
- SELECT without FROM → empty from_scope
All 1351 previous tests still pass — DSL paths untouched.
Test totals: 1358 passing, 0 failed, 1 ignored. Clippy clean.
Frame is_cte_body marker, body-projection harvest, and
projection_aliases population are the remaining 2b work
(2b-4 and 2b-5).
Sub-phase 2b checkpoint 2 — closes the recursion loop between
sql_expr.rs and sql_select.rs so subquery expressions and
qualified column refs become structurally valid in every SQL
context where they belong.
sql_expr.rs:
- §5 qualified-ref tail. `name_or_call` gains a `.identifier`
suffix as a Choice sibling of the function-call `(args)`
tail. The leading identifier is still matched once (per
ADR-0031 §1's factoring); the optional tail dispatches
between the two suffixes by their first character (`.` vs
`(`).
- §6.1 scalar subquery as primary. The `(or_expr)` and
`(SELECT …)` branches share the leading `(`; the first
inside token (`SELECT` → subquery, anything else →
expression) discriminates. The subquery recurses through
`Node::ScopedSubgrammar(&sql_select::SQL_SELECT_COMPOUND)`.
- §6.2 IN (subquery) predicate. Sibling of the existing
IN-value-list; same `(` factoring, same dispatch.
- §6.3 [NOT] EXISTS primary. Bare `EXISTS (compound_select)`
lives in `primary`; `NOT EXISTS` falls out via the existing
`not_expr := NOT not_expr` tier above `primary`.
sql_select.rs:
- CTE body recursion rewires `Node::Subgrammar` →
`Node::ScopedSubgrammar`, matching §10.2. The top-level
statement's COMPOUND embedding stays plain Subgrammar — the
implicit bottom frame is the right scope for a statement-
level SELECT.
Structural side-effect — const-eval cycle workaround:
Closing the sql_expr ⇄ sql_select reference loop made Rust's
const-evaluator follow the cycle through every `const Node`
that transitively reaches it. Mirroring sql_expr.rs's existing
pattern, composition Nodes in sql_select.rs (Seq / Choice /
Optional / Repeated / Lookahead) are now `static Node` and
appear in slice positions through `Node::Subgrammar(&NAME)`
wraps; only leaf items (Punct, Word, Ident) remain `const`.
Same workaround applies to data.rs's SELECT_PROJ_LIST /
SELECT_PROJECTION chain and the inlined `SQL_EXPR` reference.
Statics resolve lazily at link time, so the cycle is valid;
const-eval is not, and the named `const SQL_EXPR` alias is
gone in both files (replaced with the inline `Node::Subgrammar
(&sql_expr::SQL_OR_EXPR)` expression at every use site).
Test coverage:
- sql_expr.rs gains 11 new tests for qualified refs, scalar
subquery, IN-subquery, EXISTS / NOT EXISTS, nested
subqueries, and the existing IN-value-list form (regression).
- sql_select.rs gains 7 new tests for qualified refs in WHERE,
scalar subqueries in WHERE / projection, IN / EXISTS / NOT
EXISTS in WHERE, nested subqueries, and qualified refs
inside CTE bodies.
- All 70 prior sql_select tests still pass; the 2a baseline
is preserved.
`(WITH x AS (…) SELECT * FROM x)` is explicitly NOT admitted
as a scalar subquery — ADR-0032 §1 / §9 wire subqueries to
SQL_SELECT_COMPOUND, which omits the outer with_clause. WITH
remains a statement-level-only construct. Documented in the
relevant test.
Test totals: 1333 → 1351 passing, 0 failed, 1 ignored
(unchanged). Clippy clean.
Sub-phase 2b checkpoint 1 — adds the foundation for SQL SELECT
lexical-scope discipline without changing existing walker
semantics.
New types in `dsl::walker::context`:
- `TableBinding` — one FROM-source binding with table name,
optional alias, and schema-resolved columns (§10.1).
- `CteBinding` + `CteColumn` — a CTE definition visible from
inside its body (WITH RECURSIVE self-reference) and from the
outer scope after harvest (§10.3).
- `ScopeFrame` — `from_scope`, `cte_bindings`, and
`projection_aliases` for one lexical scope. Default-empty;
the fields will be populated by later 2b checkpoints.
`WalkContext` gains `from_scope_stack: Vec<ScopeFrame>`,
initialised with one bottom frame in both `new()` and
`with_schema()`. The bottom frame is the implicit top-level
scope DSL paths and top-level SQL statements operate in;
`Node::ScopedSubgrammar` entries push and pop additional frames
on top. `current_table` / `current_table_columns` remain as
direct fields for this checkpoint — converting them to derived
helpers is a later 2b step.
New grammar-tree variant:
- `Node::ScopedSubgrammar(&'static Self)` — like `Subgrammar`,
but pushes a fresh `ScopeFrame` on entry and pops it on exit
(ADR-0032 §10.2). Shares `subgrammar_depth` with the plain
Subgrammar variant so the MAX_SUBGRAMMAR_DEPTH = 64 cap fires
uniformly across both — §9's "no new walker capability for
grammar recursion" claim holds. DSL Expr (ADR-0026) and
sql_expr.rs ladder (ADR-0031) recursion continue to use the
plain Subgrammar variant and never push a scope.
Driver gains a parallel `walk_scoped_subgrammar` arm; the
push/pop is unconditional so a speculatively-walked branch a
later Choice rolls back leaves the stack clean.
Test coverage in `driver.rs`:
- A recursive ScopedSubgrammar test grammar walks correctly
through depths 0-3.
- The depth cap fires the same `expression_too_deep` friendly
validation error as for plain Subgrammar.
- The bottom frame invariant: `WalkContext::new` seeds exactly
one frame, and after a walk the stack is restored.
No grammar tree references the new variant yet — the rewire of
sql_select.rs CTE bodies and the sql_expr.rs additive
extensions for §5/§6 are the next 2b checkpoint. Test totals:
1330 baseline + 3 = 1333 passing, 0 failed, 1 ignored. Clippy
clean.
Author the standalone walkable shape for the full standard-SQL
SELECT per ADR-0032 §1: compound queries with the four set ops
(UNION / UNION ALL / INTERSECT / EXCEPT), the five JOIN flavours
(INNER / LEFT [OUTER] / RIGHT [OUTER] / FULL [OUTER] / CROSS),
GROUP BY / HAVING, WITH and WITH RECURSIVE common table
expressions, LIMIT … OFFSET, DISTINCT / ALL, qualified-wildcard
`t.*` projection, and bare-alias projection (lifting ADR-0030
Phase-1 §4.2).
Recursion into SQL_SELECT_COMPOUND uses Node::Subgrammar for
2a; sub-phase 2b will rewire those references to the new
Node::ScopedSubgrammar variant for completion-scope discipline
(ADR-0032 §10.2). The Phase-1 data::SELECT CommandNode is not
touched here — the new fragment is reachable only from its own
tests until sub-phase 2c performs the migration.
Two implementation mechanisms realize ADR semantics without
changing them:
- Node::Lookahead disambiguates the projection_item Choice
(bare `*` vs `ident . *` qualified wildcard vs `sql_expr [
alias ]`) and gates bare-alias slots against continuation
keywords. The walker's walk_ident accepts any
identifier-shape token, including keyword-shape ones, and
Choice / Optional are first-match-wins; without lookahead a
bare-alias slot would greedily swallow FROM / WHERE / JOIN /
etc. Per-position follow-sets list which keywords legitimately
follow each alias slot. Same pattern as data.rs's
insert_first_paren precedent.
- INNER JOIN and bare JOIN are split into two distinct Choice
branches (each with a concrete leading keyword) rather than
sharing one Optional(Word("inner"))-leading branch. Avoids a
walker hazard where an Optional-leading-child Seq commits to
idx > 0 and then converts the next child's EOF NoMatch into
Incomplete, blocking the outer Choice from falling through to
later branches. Same semantic surface, distinct mechanism.
The §13 OOS shapes all have explicit reject tests (NATURAL,
USING, comma-FROM, LIMIT m,n, window OVER, VALUES, derived
tables). LATERAL has a noted partial limitation: the comma form
rejects via OOS-3, but the single-keyword form `FROM a LATERAL
JOIN b ON …` is admitted structurally because `lateral` parses
as a bare table-source alias for `a`. This matches ADR-0030's
"grammar admits identifier-shape tokens; engine resolves"
posture.
`__rdbms_*` rejection extends to every Phase-2 table-source
slot — the FROM table, each JOIN's table, each CTE name, and
the FROM inside any CTE body — via the reuseable
reject_internal_table validator.
70 new unit tests in sql_select.rs walk every §1 production and
every OOS reject case. Test totals: 1260 baseline + 70 = 1330
passing, 0 failing, 1 ignored (unchanged from baseline). Clippy
clean.
Per the Phase-2 plan sub-phase 2a exit gate. DA gate written
review: PASS.
§12 was written conservatively, classifying projection items
structurally and listing "subquery expressions" alongside
arithmetic / CASE as cases that stay None. The Phase-2 plan's
Open Question 1 captured the matching uncertainty about CTEs
and scalar subqueries.
A throwaway probe against the pinned bundled SQLite +
rusqlite 0.39.0 (with the `column_metadata` feature) settles
the question across 20 representative query shapes. The
engine's column_table_name / column_origin_name metadata
follows through non-recursive CTEs (SELECT *, bare-ref,
qualified-ref, and (col-list)-renamed bodies; CTE chains),
scalar subqueries (aliased and unaliased), derived tables
(out of scope per §13 OOS-1 but useful to note), all four
set ops, multi-table JOIN projections, and IN-subquery
WHERE clauses (the inner subquery does not affect the
outer projection's origin).
The structural-None classes reduce to computed projections
(function calls, arithmetic, CASE, literals, wildcards —
expected and pedagogically obvious) and recursive CTE result
columns (the one structural surprise — the recursive
temporary table has no base-column origin to point at).
Amendment 1 supersedes §12's "Resolution rule" with a simpler
engine-driven rule: trust column_table_name(i) /
column_origin_name(i) verbatim, with no grammar-side
structural classification. The speculative MatchedPath-walk
fallback is moot. The Phase-2 plan's sub-phase 2f exit gate
gains explicit positive assertions for CTE pass-through and
scalar-subquery type recovery, and a new explicit negative
assertion for the recursive-CTE limitation.
README.md index entry extended in the same style as ADR-0027's
Amendment-1 line. Closes Plan §Open-1.
Twenty-sixth handover. Design session, no code touched. Tests
unchanged at 1260 / 0 / 1.
Captures: the Phase 2 grammar decisions (ADR-0032 accepted),
the implementation plan at docs/plans/20260520-adr-0032-phase-2.md
with seven sub-phases and a cross-cut verification matrix that
explicitly names every "X comes for free" claim from ADR-0030/
0031/0032, and the Phase-1 carry-over finding the warning/error
guideline check surfaced — SQL WHERE expressions currently emit
no LIKE-on-numeric / = NULL / type-mismatch warnings because
sql_expr builds no AST. ADR-0032 §11.6 closes the gap; the
plan's cross-cut matrix has a named row to prevent regression.
The next session is Phase 2 sub-phase 2a (grammar fragment) per
the plan; standing authorizations apply.
Status: ready to hand over.
ADR-0030 §3 commissioned a focused ADR for the full SELECT
grammar (the "SELECT — full" phase). ADR-0032 records the
decisions; docs/plans/20260520-adr-0032-phase-2.md is the
implementation plan walking the work.
Phase 2's grammar surface:
- Five JOIN flavours (INNER, LEFT, RIGHT, FULL OUTER, CROSS).
NATURAL/USING/comma-FROM explicitly OOS.
- All four set ops (UNION, UNION ALL, INTERSECT, EXCEPT).
- WITH and WITH RECURSIVE CTEs, with optional (col-list) renaming.
- Scalar subqueries, IN (SELECT …), [NOT] EXISTS as additive
primary branches in sql_expr (redeems ADR-0031 §7 OOS-1).
- Qualified column refs t.c / alias.c as a name_or_call tail
(redeems ADR-0031 §7 OOS-2).
- LIMIT n [OFFSET m]; legacy `LIMIT m, n` OOS.
- DISTINCT/ALL, t.* projection, bare-alias projection (lifts
Phase-1 §4.2's autonomous decision).
Walker-capability honesty (§10): ADR-0030 §8's "ambient
assistance comes for free" holds for grammar recursion (reuses
ADR-0026's Subgrammar + depth cap unchanged) but not for
completion scope. Phase 2 adds a new Node::ScopedSubgrammar
variant alongside the existing Node::Subgrammar (DSL Expr and
sql_expr recursion untouched), a from_scope_stack of
ScopeFrames holding from_scope / cte_bindings /
projection_aliases, qualified-prefix completion narrowing, and
a post-walk fixup pass that re-resolves projection-list
identifier highlighting/validity once FROM is parsed (the
projection-before-FROM problem).
CTE column resolution (§10.3): SELECT * and explicit-projection
CTE bodies both yield real column completion past cte_alias.|
via a body-projection derivation rule that runs at the body's
ScopedSubgrammar exit and writes derived columns back into the
binding.
Diagnostics (§11): every Phase-2 validation case classified
against ADR-0027's ERROR/WARNING guideline. Five new diagnostic.*
catalog keys for parse-time-detectable cases (unknown_qualifier,
ambiguous_column, projection_alias_misplaced, cte_arity_mismatch,
compound_arity_mismatch) plus eight engine.* translation keys.
A MatchedPath-walking predicate-warnings variant closes the
Phase-1 carry-over gap where SQL WHERE expressions emitted no
LIKE-on-numeric / = NULL / type-mismatch warnings — ADR-0027
Amendment 1 finally extends to the SQL surface.
Result-column type resolution (§12): rusqlite 0.39.0 exposes
column_table_name / column_origin_name / column_database_name
behind a `column_metadata` feature; verified. Bare column refs
recover their playground type — partially lifts Phase-1 §4.5's
bool→0/1 deferral.
The implementation plan breaks Phase 2 into seven sub-phases
(2a–2g) with explicit exit gates per sub-phase and a cross-cut
verification matrix that names every "X comes for free" claim
from ADR-0030/0031/0032. The Phase-1 SQL-expression
predicate-warning gap is a named row, preventing an analogous
silent gap from shipping. The plan encodes the user's standing
authorization for the implementer to walk uninterrupted between
gates and commit with standard messages — escalation
discipline preserved for design ambiguities and real blockers.
Pushes remain user-only.
New docs/plans/ directory sets a pattern for future phase plans.
Status: Accepted.
Implementation handoff: a SQL `select` typed in advanced mode
parses, runs, and renders end to end; the same line in simple
mode lights up the precise "this is SQL" hint instead. ADR-0031
(the SQL expression grammar) and ADR-0030 Phase 1 ("Foundations
+ first SELECT") landed across five commits. Tests 1240 → 1260,
clippy clean.
The handoff records: the walker mode gate + `is_advanced_only`
set, the `ast_builder` source-param sweep, `Command::Select`
carrying the validated SQL text, the `data::SELECT` shape, the
worker `Request::RunSelect` round-trip, the ambient mode
threading through completion / overlay / validity indicator,
the autonomous calls made during execution (FROM optional,
implicit alias unsupported, etc.), and the seams the next
session uses to take up ADR-0030 Phase 2 (full SELECT) — which
gets its own focused ADR before code, per ADR-0030 §3.
`tests/sql_select.rs` covers the full advanced-mode SELECT path
end to end (ADR-0030 Phase 1, ADR-0031):
App-level dispatch
- `advanced_mode_select_dispatches_as_command_select`: an
advanced-mode `select 1` produces exactly one
`Action::ExecuteDsl { command: Command::Select { sql }, .. }`
carrying the validated SQL text.
- `simple_mode_select_yields_sql_hint_and_does_not_dispatch`:
a simple-mode `select` produces no dispatch action and the
error output contains the SQL hint naming both recovery
paths (`mode advanced` / the `:` one-shot).
- `colon_one_shot_from_simple_mode_dispatches_select`:
`:select 1` keeps the persistent mode as `Simple` while
dispatching `Command::Select` with the `:` stripped.
- `advanced_mode_select_from_internal_table_is_rejected`:
a SELECT against `__rdbms_playground_columns` is refused by
the grammar's `reject_internal_table` validator.
Worker round-trip
- `database_run_select_constant_returns_a_single_row`:
`select 1` runs through `Database::run_select` and returns
a `DataResult` with one row whose only cell is `1`; all
`column_types` are `None` (ADR-0030 §6).
- `database_run_select_from_user_table_returns_inserted_rows`:
create-table → insert → `select Name from T` round-trips
the inserted row through the worker.
- `database_run_select_appends_to_history_when_source_present`:
the literal source line lands in `history.log` so replay
re-runs it (ADR-0030 §11).
The dispatch-layer mode gate (previous commit) made the submit
behaviour correct — `select` runs in advanced mode and shows
the SQL hint in simple mode. This commit extends that gating to
the ambient assistance layer so simple-mode users do not see
SQL leak through Tab completion, the live error overlay, or the
`[ERR]`/`[WRN]` validity indicator either.
`_in_mode` walker variants
--------------------------
- `completion_probe_in_mode`, `expected_at_input_in_mode`,
`input_verdict_in_mode`. Each sets `ctx.mode` before walking.
The empty-input / unknown-entry fallback in `completion_probe`
and `expected_at_input` filters the `REGISTRY` listing by
`is_advanced_only` so Tab does not offer `select` in simple
mode. Old signatures keep delegating to `Mode::Advanced`
(back-compat for tests + other callers).
`_in_mode` completion variants
------------------------------
- `candidates_at_cursor_in_mode`, `candidates_at_cursor_with_in_mode`.
Internally they route the `parse_command` completeness probe
through `parse_command_in_mode(input, mode)`, the
`completion_probe` call through `completion_probe_in_mode`,
and the `expected_at` fallback through
`expected_at_input_in_mode`. Old signatures default to
`Mode::Advanced`.
`EffectiveMode::as_mode`
------------------------
- Collapses the persistent / one-shot distinction the UI cares
about into the plain `Mode` the walker reads from
`WalkContext::mode`. App-level call sites that thread mode
into the walker chain use this.
App / input-render wiring
-------------------------
- `App::input_validity_verdict` runs only when effective mode
is plain `Simple` (per ADR-0027), so it hardcodes
`Mode::Simple` into the new `input_verdict_in_mode` call
rather than threading.
- `App::start_or_complete_at` / `_last` (the Tab handlers)
pass `self.effective_mode().as_mode()` into
`candidates_at_cursor_in_mode`, so a `:` one-shot or
persistent advanced gives full SQL completion, persistent
simple does not offer SQL.
- `input_render::render_input_runs` and `ambient_hint` are
invoked from `ui.rs` only when effective mode is plain
`Simple` (advanced rendering uses `plain_input_spans` and
skips ambient hinting per ADR-0022 §12). Their internal
`classify_input_with_schema` / `candidates_at_cursor` /
`parse_command` calls now go through the mode-aware variants
with `Mode::Simple` hardcoded — a SQL form in simple mode
surfaces as a definite-error overlay and the hint panel does
not offer it.
After this commit a simple-mode user typing `select` or
`sel<Tab>` sees nothing SQL-shaped: no live highlight, no Tab
completion candidate, the `[ERR]` indicator lit, and the on-
submit hint that names the recovery paths. An advanced-mode
user or a `:` one-shot sees the full SQL surface.
The first cut of advanced-mode SQL: a `select` line in advanced
mode parses, runs against the database, and renders its rows
through the existing data-table renderer; the same line in
simple mode lights up the precise "this is SQL" hint instead of
running.
Walker mode gate (ADR-0030 §2)
------------------------------
- `WalkContext` gains a `mode: Mode` field; `Mode` derives
`Default` (= `Simple`, matching the app's startup mode).
- `grammar::is_advanced_only` keys an advanced-only entry-word
set (Phase 1: just `select`). When the walker matches an
advanced-only entry word with `ctx.mode == Simple`, it
short-circuits to a `WalkOutcome::ValidationFailed` carrying
the `advanced_mode.sql_in_simple` catalog key — the input
highlights as a keyword, the validity indicator goes ERROR,
and the parse-error layer renders the "switch with `mode
advanced`, or prefix the line with `:`" hint.
- `parser::parse_command_with_schema_in_mode` (and the
schemaless `parse_command_in_mode`) threads the mode into
`WalkContext`; existing `parse_command*` entry points default
to `Mode::Advanced` (most permissive) so back-compat callers
see the full grammar.
- `App::submit` is unified: both modes route through
`dispatch_dsl(&effective_input, effective_mode)`, which now
parses with the line's effective mode. The placeholder
advanced-mode echo branch is gone.
Builder signature sweep (ADR-0031 §2)
-------------------------------------
- `CommandNode.ast_builder` gains a `source: &str` parameter,
forwarded by the walker. `build_select` reads it to put the
validated SQL text into `Command::Select`; the 21 existing
builders accept it as `_source`.
SQL `SELECT` (ADR-0030 §6, ADR-0031)
-------------------------------------
- New `Command::Select { sql: String }` variant. Every
exhaustive `match Command` updated (`verb`, `target_table`,
`build_translate_context`, `execute_command_typed`,
`typing_surface`'s label).
- `grammar::data::SELECT` `CommandNode`: projection (`*` or
`expr [as alias]` list), optional `FROM <table>`, optional
`WHERE`/`ORDER BY`/`LIMIT`, optional trailing `;`. The
expression slots reference the ADR-0031 fragment through
`Subgrammar(&sql_expr::SQL_OR_EXPR)`. The `FROM` table-name
slot carries a `reject_internal_table` validator that
refuses `__rdbms_*` references at parse time.
- The `FROM` clause is optional — `select 1`, `select upper('x')`
(zero-table constant/function-call SELECTs) work alongside
the single-table form. Standard SQL admits them and they are
the canonical learner probe.
- Implicit projection aliasing (`select a x`) is deliberately
unsupported — `from` is a keyword, the bare alias would be
ambiguous; only `select a as x` is admitted.
Worker / runtime
----------------
- `Request::RunSelect { sql, source, reply }` + a new
`Database::run_select` method. `do_run_select_request` runs
the prepared statement, collects rows into a `DataResult`
with `column_types: Vec<None>` (Phase-1 SELECT result columns
carry no playground type per ADR-0030 §6), and appends the
literal source line to `history.log` so replay re-runs it
(ADR-0030 §11).
- `runtime::execute_command_typed` gains a `Command::Select`
arm that calls `database.run_select(sql, src)` and maps to
`CommandOutcome::Query`, which flows into the existing
`AppEvent::DslDataSucceeded` → `render_data_table` path.
Catalog (ADR-0019)
------------------
- `advanced_mode.sql_in_simple` — the walker's gate message.
- `select.internal_table` — the `__rdbms_*` rejection.
- `parse.usage.select` — the parse-error usage template.
Tests
-----
Two `app::tests` cases that pinned the pre-ADR-0030 placeholder
echo are updated to pin the new dispatch contract — both verify
that the advanced-mode `select` (one persistent, one via the
`:` one-shot) produces `ExecuteDsl(Command::Select)` with the
submission's effective mode tagged on the echo. The matching
walking-skeleton test is updated likewise.
A separate follow-up commit lands the ambient mode-threading
(completion / live overlay / validity indicator) so simple-mode
users do not see SQL surfaced through Tab or the live error
overlay either — the dispatch-layer gate landed here is the
behavioural foundation that follow-up builds on. Integration
tests for the full end-to-end land in a third commit.
A new `src/dsl/grammar/sql_expr.rs` authored as a parallel
fragment to `expr.rs` (the DSL `WHERE` grammar, ADR-0026). The
ADR's stratified ladder lands as named `static` `Node`s, one per
precedence tier:
or_expr → and_expr → not_expr → predicate → additive →
multiplicative → unary → primary
Recursion through `Node::Subgrammar` reuses ADR-0026's
`MAX_SUBGRAMMAR_DEPTH = 64` cap unchanged; no new walker
capability is required. `predicate_tail` follows ADR-0026's
factoring (shared operand prefix, infix `NOT` as an explicit
branch, no `Optional`-first branch) so `Choice` discriminates
cleanly. `name_or_call` factors the identifier-prefix shared
between column refs and function calls into a single `Ident`
followed by an `Optional` `( call_args )` tail — the same
hazard-avoidance shape `predicate_tail` uses.
The fragment exports `pub static SQL_OR_EXPR` (test entry) and
`pub static SQL_EXPRESSION` (drop-in `Subgrammar(&SQL_OR_EXPR)`
that SQL `CommandNode` shapes embed in their `Seq`). No AST
builder — every Phase-1 consumer (SELECT projection, WHERE)
runs validated SQL as text per ADR-0030 §4/§6.
13 unit tests cover every operator and precedence pair, the
full predicate set, `CASE` (searched + simple) including
`count(*)` and `count(distinct …)`, parenthesised regrouping,
case-insensitive keywords, the depth cap, and a representative
set of malformed inputs that do *not* walk.
Module registered via one new line in `grammar/mod.rs`.
ADR-0030 §3 commissioned a focused ADR for the stratified SQL
expression grammar fragment. ADR-0031 records the decisions:
- One unified precedence ladder (OR/AND/NOT, comparison/LIKE/IN/
BETWEEN/IS NULL predicates, arithmetic incl. `||`, function
calls, CASE) — SQL treats booleans as values, so unlike
ADR-0026's bool/scalar split this is a single ladder.
- No AST — every Phase-1 consumer (SELECT projection, WHERE)
runs validated SQL as text per ADR-0030 §4/§6; CHECK/DEFAULT
in Phase 4 store text too. The fragment's job is accept /
reject + the matched-terminal path + a source span.
- Recursion via Subgrammar with ADR-0026's depth cap reused.
- A parallel `grammar/sql_expr.rs` — separate from `expr.rs` so
simple mode's 1240-test surface is untouched by construction.
- Subquery expressions and qualified `t.c` column refs deferred
to ADR-0030 Phase 2 (they need the recursive SELECT grammar).
`%` modulo is included alongside `+ - * /` and `||` — it isn't
ISO SQL but is near-universal across mainstream engines and
matches learner expectations (pedagogy wins ties, ADR-0030).
Status: Accepted. The implementation lands in subsequent
commits.
Decides the architecture for SQL in advanced mode (Q1/Q2/Q4):
SQL is authored as grammar within the unified grammar tree
(ADR-0024) and parsed by the existing walker — not a separate
batch parser — so SQL gets the same completion, highlighting,
hints, and parse-error reporting as the DSL. Mode gates the
SQL forms. DDL routes through the typed Command executor
(metadata and the playground type vocabulary preserved); DML
and SELECT execute as validated SQL. Engine-neutral posture;
DSL→SQL teaching echo; phased plan.
Supersedes ADR-0001's sqlparser-rs reservation. Ticks Q4;
updates the ADR index and the Q1/Q2 notes. handoff-24 orients
the implementation session at Phase 1.
ADR-0029 (column constraints — NOT NULL / UNIQUE / CHECK /
DEFAULT) is fully implemented across the handoff-22 and
handoff-23 sessions. Ticks requirement C3, and corrects
ADR §10's CHECK-error wording to the compiled-SQL form per
the §7 storage deviation.
Completes ADR-0029's implementation: the friendly-error layer
now names the rule a CHECK violation broke, and the
typing-surface matrix covers the whole constraint grammar.
CHECK-violation friendly error (ADR-0029 §10):
- enrich_dsl_failure gains a CHECK branch — it reads the column
from the engine's `CHECK constraint failed: <column>`
message, then resolves the table, the offending value, and
the column's compiled CHECK expression.
- FailureContext / TranslateContext carry the resolved
check_rule; translate_check renders "the value <v> breaks the
rule `<rule>`" when it is known, falling back to the plain
hint otherwise.
Typing-surface matrix: a new `constraints` submodule, 14 cells
covering the create-table / add-column constraint suffix and
the add-constraint / drop-constraint commands (174 → 188).
16 tests added (1 translate unit, 1 enrichment integration, 14
matrix cells).
Adds the two commands for modifying a column's constraints after
creation, completing ADR-0029's §2.2 surface.
Grammar (dsl/grammar/ddl.rs): `add constraint <constraint> to
<T>.<col>` reuses the §2.1 COLUMN_CONSTRAINT choice; `drop
constraint <kind> from <T>.<col>` names only the kind. Both join
the `add` / `drop` choices, discriminated by the `constraint`
form word.
AST (dsl/command.rs): `Command::AddConstraint` / `DropConstraint`
plus the `Constraint` / `ConstraintKind` enums.
Worker (db.rs): `do_add_constraint` / `do_drop_constraint` apply
the change through the rebuild-table primitive. `add` runs the §5
dry-run first — `not null` / `unique` / `check` against a
populated column are refused, before any write, with a
pretty-printed table of offending rows. §9 redundant-on-PK
declarations and §6 `default` on an auto-generated column are
friendly refusals; dropping a constraint the column does not
carry is likewise refused.
Also fixes schema_to_ddl, which suppressed UNIQUE for every PK
column — a compound-PK member is not individually unique, so an
explicit UNIQUE on it must survive the rebuild.
23 tests added (6 grammar, 17 worker); 3 completion-test and 3
matrix snapshots updated for the new `constraint` subcommand.
ADR-0028 complete (per handoff-21); ADR-0029 (column
constraints) written, accepted, and implemented through
commit 4 of 6 — NOT NULL / UNIQUE / DEFAULT / CHECK at
`create table` and `add column`. Commits 5 (`add constraint`
/ `drop constraint` + the §5 dry-run) and 6 (friendly errors
+ typing-surface matrix) are planned in full in §4.
The fourth constraint. `check ( <expr> )` reuses the ADR-0026
WHERE-expression grammar via `Subgrammar`, so a check is
written in the same language as a `where` filter.
- Grammar: a `CHECK_CONSTRAINT` arm joins the shared
constraint-suffix Choice; `consume_check_expr` extracts the
parenthesised expression (paren-depth aware) into
`ColumnSpec.check` / `Command::AddColumn.check`.
- Storage: the parsed `Expr` is compiled once to inline SQL
(`compile_check_sql` — `compile_expr` + ADR-0028's
param-inliner) and stored in that form everywhere — a new
`check_expr` column in `__rdbms_playground_columns`,
`project.yaml`'s `ColumnSchema.check`, and the column DDL
emitted by `do_create_table` / `schema_to_ddl`.
- `add column … check` routes through the rebuild primitive
(SQLite's `ALTER … ADD COLUMN` cannot carry it); a CHECK on
a serial/shortid column is create-table-only and refused at
add-column with a friendly message.
- `describe` surfaces the CHECK. ADR-0029 §7/§8 updated to the
SQL-form decision — double-quoted identifiers, consistent
with ADR-0028's `explain` display SQL.
1201 tests pass (+8); clippy clean.
`add column` now accepts the shared constraint suffix and the
worker honours it — the surface where NOT NULL / UNIQUE
actually matter, on non-PK columns.
- Grammar: `ADD_COLUMN_NODES` gains the constraint-suffix
fragment; `collect_column_constraints` folds it into
`Command::AddColumn`.
- `do_add_column` routes per ADR-0029 §6: SQLite's `ALTER
TABLE ADD COLUMN` cannot express `UNIQUE` and requires a
default for `NOT NULL`, so those go through the rebuild
primitive (`do_add_constrained_column_via_rebuild`); plain
cases keep the ALTER path with the constraint suffix
appended.
- Pre-flight refusals, before any SQL write: a NOT NULL
column with no default added to a populated table; a UNIQUE
column with a default added to a multi-row table; a default
on a `serial` / `shortid` column.
CHECK is still deferred to the next commit. 1193 tests pass
(+9); clippy clean.
`create table … with pk` now parses the column-constraint
suffix; combined with the commit-1 db layer, a constrained
table works end to end.
- A shared constraint-suffix grammar fragment — `not null`,
`unique`, `default <literal>` — sits after each column's
`(type)` group; `build_create_table` walks the matched path
per column and folds the constraints into `ColumnSpec`.
- §9 redundancy check: every `with pk` column is a primary-key
column, so `not null` (any) and `unique` (single-column PK)
are rejected with a friendly error
(`parse.custom.constraint_redundant_on_pk`).
- `project.yaml` round-trip: `ColumnSchema` gains `not_null` /
`default`; the YAML reader/writer and `build_read_schema`
carry them, so `rebuild` / `export` / `import` preserve
constraints.
- ADR-0029 §2.1's example corrected — `create table` columns
are all PK columns, so its suffix is for `default` / `check`;
`docs/simple-mode-limitations.md` records that non-PK
columns at create time need advanced mode.
CHECK is deferred to the next commit. 1184 tests pass (+7);
clippy clean.
The database layer now honours the ColumnSpec constraint
fields end to end, ahead of the grammar that lets users type
them.
- `do_create_table` emits ` NOT NULL` / ` UNIQUE` / ` DEFAULT
<literal>` per column via the new `column_constraints_sql`
helper (the default literal bound against the column's type).
- `ReadColumn` gains `default_sql`, read from
`pragma_table_info.dflt_value`; `schema_to_ddl` emits it, so
the rebuild-table primitive preserves DEFAULT — it already
preserved NOT NULL / UNIQUE.
- `ColumnDescription` gains `unique` / `default`;
`do_describe_table` now sources columns from `read_schema`
(one source of per-column truth) and `constraints_display`
lists PK / NOT NULL / UNIQUE / DEFAULT.
No user-facing change yet — no grammar produces constrained
columns. Tests exercise creation, enforcement, describe, and
rebuild-preservation programmatically.
1177 tests pass (+5); clippy clean.
Expand ColumnSpec and Command::AddColumn with the four
ADR-0029 constraint slots (not_null, unique, default, check),
all defaulting off; `Database::add_column` now takes a
ColumnSpec. No behaviour change — the grammar to set the
fields and the DDL to enforce them land in the following
commits. Isolated here so those commits stay readable.
Adds ColumnSpec::new for the unconstrained case; 110 call
sites updated. 1172 tests pass; clippy clean.
Designs the remaining C3 surface: the four column-level
constraints declared in the column-spec suffix at `create
table` / `add column`, and modified on existing columns via
`add constraint … to` / `drop constraint … from`.
- A pre-flight dry-run (the ADR-0017 ethos) scans a populated
column before applying NOT NULL / UNIQUE / CHECK and refuses
with a pretty-table of offending rows; no `--force`.
- CHECK reuses the ADR-0026 expression grammar via Subgrammar.
- `__rdbms_playground_columns` carries a new `check_expr`
column; the other three are recoverable from SQLite pragmas.
- README index updated.
`add index` / `drop index` (ADR-0025) and `explain` (ADR-0028)
are both done, so they no longer belong under "Things
deliberately deferred". Folded into a new "Indexes & query
plans" entry in the decisions-at-a-glance list.
ADR-0028 (query plans / `explain`) is fully implemented; the
handoff-16 design trio (ADR-0026 / 0027 / 0028) is now closed.
- handoff-21: session summary, the two deliberate deviations
from handoff-20's plan, test coverage, open clusters.
- requirements.md: QA1 / QA2 ticked.
- CLAUDE.md: the `EXPLAIN QUERY PLAN` deferred-items line
updated to "implemented per ADR-0028".
13 matrix cells for the `explain` prefix across all three
wrapped commands — `explain show data` / `explain update` /
`explain delete` — covering each typing position (after the
prefix, the inner entry word, the table, the filter clause)
plus the three complete forms. The cells confirm `explain`
plugs into the inner query grammars cleanly: candidates, hints
and column scoping match the standalone commands, and the
complete forms parse as `Command::Explain`.
Also adds a worker test pinning the display SQL's `<>`
rendering of inequality (ADR-0028 §3).
Matrix: 161 -> 174 cells. 1172 tests pass; clippy clean.
`render_explain_plan` now classifies each plan node and colours
its category-bearing keywords through the styled-runs mechanism.
- `PLAN_TAXONOMY`: a substring-pattern table mapping the
engine's plan vocabulary to four semantic classes — full
scan / temp B-tree -> Expensive, index search / covering
index / PK lookup -> Efficient, automatic index ->
AutomaticIndex. An unrecognised detail renders neutral, since
the engine's plan vocabulary may grow.
- Only the matched keyword run carries the category colour;
connectors, prefixes and table / index names stay neutral
(ADR-0028 §6). The display-SQL line is wholly neutral.
- An automatic-index node also gets the distinct "← add an
index?" advice tag, so it reads as guidance, not merely
"this is slow".
1158 tests pass (+7); clippy clean.
Add the `explain` prefix command — `explain show data`,
`explain update`, `explain delete` — from grammar through to a
rendered plan tree.
- Grammar: an `EXPLAIN` CommandNode whose shape is a Choice over
the three explainable query shapes, referenced (not
duplicated) through `Subgrammar`. `Command::Explain { query:
Box<Self> }`; `build_show_data` is extracted so the role-based
builders serve both standalone and explain-wrapped commands.
- Worker: SQL construction is split out of do_query_data /
do_update / do_delete into `build_*_sql`, so EXPLAIN QUERY
PLAN runs the exact same statement. `Request::ExplainPlan` /
`do_explain_plan` capture the plan; `QueryPlan` / `ExplainRow`
carry it back. EXPLAIN QUERY PLAN never executes, so
explaining update/delete changes nothing.
- Display SQL: the executed statement with `?N` parameters
inlined as standard-SQL literals via a quote-aware scan.
- Render: `render_explain_plan` draws the box-drawing plan tree
(plain output; ADR-0028 step 4 adds the styled tree).
- Catalog: `parse.usage.explain` and the `help.data.explain`
entry, so `explain` shows up in the in-app `help` listing.
1151 tests pass (+18); clippy clean.
Interim handoff. ADR-0028 (query plans / explain) is started:
step 1 (the styled-output-line mechanism, 03d8a09) is done and
committed. handoff-20 carries the full validated build plan
for steps 2-5 with file:line anchors and three implementation
gotchas (the const/static Subgrammar wrinkle, build_show's
positional dispatch, and why steps 2+3 must land as one
commit) so a fresh session implements them without
re-exploring.
OutputLine gains an optional styled-runs payload — a
Vec<OutputSpan> of { byte_range, OutputStyleClass } over the
line text. render_output_line gains a branch: when the payload
is present it renders the text span-by-span, each run's
semantic class (Neutral / Efficient / Expensive /
AutomaticIndex) resolved to a theme colour at render time;
otherwise the existing whole-line kind styling. The echo path
is untouched.
Theme gains `plan_efficient` — a green deliberately distinct
from `system` so green never reads as two things (ADR-0028 §6);
`warning` is reused for expensive steps.
A general per-span output-styling capability (ADR-0016's OOS-3
realized); the query-plan renderer will be its first consumer.
No user-visible change on its own. 1133 passing, clippy clean.
handoff-19 §2 now records the two bugs as fixed (was "queued
next"): the optional trailing-flag completion fix (f239ca5)
and the --resume temp-project pointer fix (3a40ae2). §5 drops
them from "what's next" — ADR-0028 is now the natural next
pick. State/§6 updated to 10 commits and 1131 tests;
requirements.md test baseline → 1131.
On launch an empty temp project is created but, by design
(ADR-0015), auto-deleted on quit while still empty. The
unconditional `write_last_project` at startup recorded that
temp's path anyway, so a later `--resume` resolved to a
since-deleted directory and printed a confusing
"recorded project … no longer exists".
All three resume-pointer writes are now gated on
`!project.is_unmodified_temp()`: the startup write, the
on-switch write (a `new`-command switch to a fresh temp no
longer records it), and a new on-quit write. The quit write is
where a launch-temp the user *filled with content* finally
gets remembered — startup skipped it while it was still empty.
An unmodified empty temp is deleted, never recorded; the two
dispositions are mutually exclusive.
The "no previous project" friendly error the user asked for
already exists (`project.resume_no_previous`, wired in the
resume resolution) — verified, no change needed. The gate
predicate `is_unmodified_temp` is covered by existing
integration tests. 1131 passing, clippy clean.
Typing `--` to start an optional trailing flag (`--create-fk`
on `add 1:n relationship`, `--cascade` on `drop column`,
`--force-conversion` / `--dont-convert` on `change column`)
made completion go empty: the trailing `--` turns the parse
into a trailing-junk Mismatch, and the Mismatch arm of the
completion expected-set resolution returned only `[EndOfInput]`
— the skipped optional-flag expectations, carried in
`tail_expected`, were dropped.
completion_probe and expected_at_input now merge `tail_expected`
into a Mismatch's expected set. `tail_expected` is empty for a
genuine mid-command mismatch, so this only adds the outer
shape's skipped trailing optionals — exactly the continuations
the trailing `--` is starting to type. This also resolves the
"wrong usage hint" symptom: with `--create-fk` offered as a
candidate, the hint panel shows candidates instead of falling
through to the parse-error usage block.
Audit outcome (the requested scan): usage_key_for_input was
verified correct for every multi-form command — add / drop /
show, including the digit-led `add 1:n relationship` form —
and is now regression-locked. The flag-completion fix covers
the whole optional-trailing-flag class.
6 tests (3 flag-completion, 3 usage-key). 1131 passing.
ADR-0027 gains a "Follow-up" section recording the completed
§2 highlight + hint wiring and precise per-literal WARNING
spans; the three stale As-built bullets point at it.
requirements.md test baseline → 1125 and the S6 entry notes
the completion + Amendment 1. handoff-19 records the run and
queues the two deferred manual-testing bugs (add 1:n
relationship completion/usage hint; --resume / last_project)
as the next session's first work.
The validity-indicator debounce was two locals in the event
loop (indicator_pending + app.input_indicator) with no unit
coverage — ADR-0027's as-built notes flag it as untested async
glue. The decision logic is now an IndicatorDebounce struct:
note_event (a keystroke hides + arms; non-key events leave it
be), settle (the quiet window elapsed → show the verdict +
disarm), is_armed (drives the recv timeout), visible (mirrored
into app.input_indicator for the renderer).
No behaviour change — the tokio timer and terminal stay in the
loop. 7 unit tests cover the debounce contract: the keystroke /
settle cycle, clean verdicts, and that a background event
mid-typing does not cancel the owed recompute. 1125 passing,
clippy clean.
ambient_hint now reads the walker's schema-aware diagnostics.
input_diagnostics is non-empty only for a command that
structurally parses — so a non-empty result means "complete
and submittable, but wrong or dubious". That is checked early
(right after the Tab-cycle memo), ahead of slot hints and
completions: a command that parses but is flawed no longer
gets the misleading "Submit with Enter" prose, it gets the
diagnostic's why. pick_hint_diagnostic prefers the diagnostic
under the cursor, else the most severe.
The cursor-local invalid-ident hint is kept for genuinely
incomplete commands (no Match → no diagnostics).
5 ambient_hint tests (unknown table, type-mismatch over
submit-prose, LIKE-numeric, clean command still submittable,
cursor-following). The complex_and_or matrix cell referenced a
non-existent column `t`; fixed to a real column so it tests a
valid expression as intended. 1118 passing, clippy clean.
render_input_runs now overlays the walker's schema-aware
diagnostics: an unknown table/column ERROR is recoloured
tok_error, an expression WARNING (type mismatch, = NULL, LIKE
on a numeric column) recoloured theme.warning. New overlay_span
covers a token's whole byte range (overlay_error only hits the
run at a single byte). New walker::input_diagnostics is the
shared entry point.
The overlay is global — every flagged token is coloured
wherever it sits, not only under the cursor — which is exactly
ADR-0027's motivation. The existing cursor-local invalid-ident
overlay is kept (it covers in-progress idents diagnostics do
not); the two are additive and idempotent.
5 input_render tests (unknown table/column, type-mismatch
literal precise, LIKE-on-numeric, clean command). 1113 passing,
clippy clean.
LIKE is a text-pattern match; against a numeric column (int,
real, decimal, serial) it runs but is almost never intended.
predicate_warnings now emits a WARNING for it, spanned at the
target column. New Type::is_numeric; catalog key
diagnostic.like_numeric; ADR-0027 gains "Amendment 1" and the
adr/README index line is updated per the index-upkeep rule.
bool and the text-/blob-backed types are deliberately not
flagged — see the amendment for the rationale.
3 walker tests (int, decimal NOT LIKE, text-column clean).
1108 passing, clippy clean.
Expression WARNING diagnostics (type mismatch, = NULL) carried
a coarse span — the whole WHERE clause, from the `where`
keyword to end of input. They now span exactly the offending
literal operand, read from the Operand source span added in the
previous commit. predicate_warnings derives the span per
warning; pair_type_mismatch returns (message, literal-span);
the dead where_clause_span helper is removed.
5 walker tests assert the spans cover exactly the literal /
identifier (type mismatch, = NULL, BETWEEN bounds, IN item,
unknown-column ERROR). 1105 passing, clippy clean.
Each WHERE-expression Operand now records the byte span of the
terminal it was built from — the precise per-literal highlight
target for an expression WARNING (finishing ADR-0027 §2's
highlight/hint wiring). parse_operand captures MatchedItem::span;
the RowFilter::eq convenience constructor uses Operand::NO_SPAN.
PartialEq is hand-written to ignore the span — it is editor
metadata, so Command equality stays whitespace- and
position-independent, which the Expr test corpus relies on.
No behaviour change; 1100 tests still pass, clippy clean.
The multi-form usage-template fix (151ed08) and the reviewed
`add index` syntax decision (kept as-is), so the next agent
does not re-flag a settled question.
A parse error in `add index …` showed the `add column` usage:
`add` and `drop` are multi-form commands, and both the
ambient hint and the submit-time usage block picked the
first-listed form unconditionally.
New `grammar::usage_key_for_input` disambiguates by the form
word after the entry keyword — `column` / `index` / `table` /
`relationship`, or the leading digit of `add 1:n …`. The
ambient hint now shows that one form; `render_usage_block`
shows the committed form's usage and falls back to the whole
family only for a bare `add` / `drop` with no form chosen.
Sweep: input_verdict tests confirm the schema-existence check
fires across the identifier-taking commands — unknown table
on drop / show / add column, unknown column on drop column /
update — and that known references stay clean. The Step B
check is grammar-generic, so this is verification + coverage
rather than new code.
Docs: requirements.md S6 -> [x], baseline 1096; CLAUDE.md
deferred list reconciled (C5a and S6 are done — removed);
ADR-0026's as-built note updated (step 5 shipped via
ADR-0027); ADR-0027 gains an As-built notes section
recording the post-walk diagnostics realization, the
pre-rendered message, the timeout-based debounce, coarse
WARNING spans, and the deferred highlight/hint wiring.
The event loop now time-boxes `recv` while an indicator
recompute is owed: every keystroke hides the indicator and
arms an `INDICATOR_DEBOUNCE` (1s) window; once typing pauses
that long the runtime computes `App::input_validity_verdict`
and shows `[ERR]` / `[WRN]`. An idle session (nothing owed)
still blocks plainly on `recv` — no wake-ups.
`update()` stays pure — the debounce timer lives in the
runtime; `App` only holds the resulting `input_indicator`
state, which the runtime clears on a keystroke and sets when
the quiet interval elapses.
`App::input_validity_verdict` is tested directly (a
simple-mode verdict, and silence in advanced mode / the `:`
one-shot); the debounce timing itself is runtime-loop glue,
covered at the integration level.
Adds the `[ERR]` / `[WRN]` validity indicator to the input
row. `App` gains `input_indicator: Option<Severity>` (the
runtime owns its timing — step E) and a pure
`input_validity_verdict()` query that runs `input_verdict`
in simple mode only (advanced mode is raw SQL, ADR-0027 §7).
`render_input_panel` reserves the rightmost six columns of
the input row unconditionally (ADR-0027 §4) — a five-column
label plus a one-column gap — so the typed command never
shifts sideways when the indicator appears or hides. The
label renders only when `input_indicator` is set: `[ERR]` in
`theme.error`, `[WRN]` in the new amber `theme.warning`
(defined for both light and dark themes).
The indicator is not yet wired live — `input_indicator`
stays `None` until the debounce lands (step E). Covered by a
render test and the theme contrast test; the input-panel
snapshot is updated for the six-column reservation.
Type-mismatched comparisons and `= NULL` / `!= NULL` in a
WHERE expression now yield WARNING diagnostics — the command
still parses and runs (the ADR-0026 §7 permissive posture is
unchanged), but the validity indicator can flag it before
submission.
Computed post-walk from the built command's `Expr` against
the table's column types: a Compare / Between / In with a
column operand and a non-null literal whose type the column
cannot hold, or a Compare with `=` / `!=` against NULL. New
catalog keys `diagnostic.type_mismatch` / `diagnostic.eq_null`.
This is ADR-0026's deferred step 5, folded into ADR-0027's
diagnostics-severity model as the user requested.
`MatchedKind::Ident` now carries its `IdentSource`. A
post-walk pass over a structurally-valid parse flags a
matched `Tables` ident that is absent from the schema, or a
`Columns` ident absent from the table in scope, as an ERROR
diagnostic — the command parses but would fail at execution
(ADR-0027 §2). New behaviour: an unknown table / column used
to parse cleanly and fail only when run.
Column scope is resolved by one left-to-right pass over the
matched path (every command places its table ident before
the columns that belong to it); an unknown table clears the
scope, so its columns are not cascaded into a second
diagnostic. New catalog keys `diagnostic.unknown_table` /
`diagnostic.unknown_column`.
Adds `Severity` (Error / Warning, ordered so Error > Warning)
and `Diagnostic { severity, span, message }` in
`walker::outcome`, plus a `diagnostics` field on `WalkResult`
— the schema-aware findings layered on a structurally-valid
parse (ADR-0027 §2).
`input_verdict(source, schema)` is the validity-indicator
entry point: `None` when the input would run clean (and for
empty input), `Some(Error)` for a parse failure or unknown
command, `Some(Warning)` for the ADR-0026 expression flags.
The verdict is the highest severity across the parse outcome
and the diagnostics set.
`diagnostics` is empty at this step — the schema-existence
(ERROR) and expression (WARNING) passes that fill it land
next. Covered by `input_verdict` unit tests.
Adds tests/typing_surface/where_expression.rs — 9 matrix
cells for the complex WHERE / show-data limit typing surface:
operator candidates after an operand, AND / OR after a
predicate, NOT, BETWEEN / IN bounds, and `show data`
where / limit.
Writing the cells surfaced a grammar bug. `predicate_tail`'s
`[NOT] negatable` branch started with `Optional(not)`, and an
Optional-first `Seq` always "commits" — so on an incomplete
input the walker's `Choice` returned that branch's
`Incomplete` early and discarded every sibling branch's
expected set, dropping `is` and the comparison operators from
completion after a column. Fixed by splitting it into
explicit `NOT negatable` and bare `negatable` branches — no
`predicate_tail` branch starts with an `Optional` now. The
matched terminal sequence is unchanged, so `build_expr` is
untouched.
Docs: ADR-0026 gains an "As-built notes" section recording
the option-1 builder realization, its two deviations from the
§3 sketch, and the deferral of §7 diagnostic flagging to
ADR-0027. requirements.md C5a -> [x] (steps 1-4) with the
test baseline refreshed to 1079; CLAUDE.md's deferred list
reconciled (C5a implemented; the QA1/QA2 note now points at
ADR-0028).
Wires the stratified WHERE-expression fragment into the three
filter commands and compiles the resulting Expr to SQL.
Grammar (data.rs): the `update` / `delete` `where` clause is
now the expression fragment (`Subgrammar(&expr::OR_EXPR)`) in
place of the single `col = val` slot; `show data` gains an
optional `where <expr>` and an optional `limit <n>` (a
non-negative integer, validated at parse time). The
expression's right-hand operands are a schema-aware
`DynamicSubgrammar` so the hint panel still narrows to the
left column's type (ADR-0026 §8) — but the inner grammar is
permissive: a type-mismatched literal still parses (§7).
AST: `RowFilter::Where{column,value}` -> `RowFilter::Where(Expr)`;
`ShowData` gains `filter: Option<Expr>` and `limit: Option<u64>`.
A `RowFilter::eq` convenience constructor keeps simple-equality
call sites and tests readable.
SQL (db.rs): `compile_expr` lowers an `Expr` to a
parameterised WHERE — every literal a `?` placeholder,
identifiers `quote_ident`-quoted, `<>` for inequality. A
literal compared against a column binds through that column's
type where compatible and falls back to its syntactic shape on
a mismatch (§7 — permissive). `show data ... limit n` emits
`LIMIT ?` with an implicit primary-key `ORDER BY`, so it is a
stable "first n by primary key".
completion.rs: `invalid_ident_at_cursor` no longer mis-flags a
digit-led literal (`1`) as an unknown column now that the
WHERE operand slot also accepts a column reference; a
`ProseOnly` slot suppresses keyword candidates even when the
expected set also carries a column ident.
11 db integration tests cover AND / OR / NOT, BETWEEN, IN,
LIKE, filtered `show data`, and limit ordering; walker and
expr unit tests cover the parse surface. Type-mismatch /
`= NULL` diagnostic flagging (§7 highlight + hint) is the
remaining ADR-0026 piece.
The stratified WHERE-expression grammar — or / and / not /
bool_primary / predicate tiers as named `static` Node
fragments, recursing through `Subgrammar`. Covers the six
comparison operators (`<>` and `!=` both NotEq), AND / OR /
NOT, parentheses, LIKE / IN / BETWEEN with optional infix NOT,
and IS [NOT] NULL. `predicate_tail` factors the shared operand
prefix and the infix NOT so the Choice branches discriminate
on a cleanly-failing first token.
New recursive Expr / Predicate / Operand / CompareOp AST in
dsl::command. `build_expr` folds the flat matched-terminal
slice into an Expr — a deterministic recursive descent
mirroring the grammar tiers, with single-child tiers
collapsing. Per ADR-0026 §3 option 1: the walker stays a pure
structural matcher; Expr is assembled only in this
submit-time fold.
Fragment + builder are unit-tested standalone (walk against
&OR_EXPR, then build_expr); not yet wired into any command.
New `Node::Subgrammar(&'static Node)` variant lets a named
static grammar fragment recurse through a reference — `Seq` /
`Choice` embed children by value and cannot close a cycle, but
a `&'static Node` can point back at an enclosing fragment. This
is the mechanism the stratified WHERE-expression grammar
(ADR-0026 §2) recurses through.
The walker counts active Subgrammar frames in
`WalkContext::subgrammar_depth` and refuses past
`MAX_SUBGRAMMAR_DEPTH` (64), surfacing a friendly
`parse.custom.expression_too_deep` error instead of a stack
overflow. Depth is saved/restored per frame so a
speculatively-walked-then-rolled-back Choice branch leaves no
residue.
No grammar references the node yet; covered by walker unit
tests with a small recursive `( x )` test grammar.
`create table … with pk` parsed column types as `name:type`,
while `add column` uses `name(type)`. Unify on the parens
form so column-type syntax is consistent across the DSL:
create table T with pk id(serial), name(text)
Only `COL_SPEC` changes (`:` → `( … )`); `build_create_table`
reads columns by role, so it is unaffected. The `:` that
separates table from column in `add column` / `drop column`
is unchanged. Sweeps the test suite, the typing-surface
matrix (two `after_colon` cells renamed to `after_paren`,
4 snapshots regenerated), the friendly catalog's usage
templates, ADR-0009's example, and requirements.md.
1039 passing / 0 failing / 1 ignored; clippy clean.
The QA1/QA2 design: an `explain` prefix command over
`show data` / `update` / `delete` that runs
EXPLAIN QUERY PLAN (without executing the statement) and
renders the result as an annotated tree. Plan steps keep
the engine's own wording; an annotation taxonomy marks
full scans, index use, and the automatic-index "you
should add an index here" case. Introduces a general
styled-output-line mechanism — an OutputLine may carry
per-span styling — realising the per-span theming
ADR-0016 deferred; the plan renderer is its first
consumer. The explained SQL is shown above the tree as
standard, copy-pasteable SQL.
- docs/adr/0028-query-plans.md — the ADR.
- docs/adr/README.md — index entry.
- docs/requirements.md — QA2 [~] -> [ ]; QA1 note
reconciled (designed in ADR-0028).
A debounced `[ERR]` / `[WRN]` marker at the right edge of
the input row, summarising — before submit — whether the
current command would run. Backed by a small
diagnostics-severity model: the walker emits severity-
tagged diagnostics (parse outcome, schema-existence of
table / column names) that the indicator summarises and
the existing highlighting / hint layers detail. Advisory
only — submission is never blocked.
- docs/adr/0027-input-validity-indicator.md — the ADR.
- docs/adr/README.md — index entry.
- docs/requirements.md — new S6 (TUI shell).
The C5a design: a stratified, recursive WHERE-expression
grammar (AND/OR/NOT, comparisons, LIKE, IS NULL, IN,
BETWEEN) for update / delete / show-data filters; show
data gains optional `where` and `limit`. Adds the
`Subgrammar` reference-following grammar node and a
recursive `Expr` AST, built selectively for the
expression fragment.
- docs/adr/0026-complex-where-expressions.md — the ADR.
- docs/adr/README.md — index entry.
- docs/simple-mode-limitations.md — new running list of
simple-mode query boundaries vs. advanced SQL, seeded
from ADR-0026.
- docs/requirements.md — C5a [~] -> [ ] (designed, not
yet implemented); new Documentation section with DOC1.
`push_history` skipped its `history_cursor` / `history_draft`
reset on the consecutive-duplicate early-return path. Recalling
a command with Up and re-submitting it unchanged left the cursor
stranded at that entry, so the next Up stepped backwards from
there instead of restarting at the newest entry.
Move the reset ahead of the early-return guards. Adds a Tier-1
regression test driving the recall/resubmit keystroke sequence.
Implement ADR-0025 — indexes as a DSL DDL feature.
- Grammar: `add index [as <name>] on <T> (<cols>)`, `drop index
<name>` / `drop index on <T> (<cols>)`, plus a `--cascade`
flag on `drop column`.
- db.rs: index operations over the engine's native index
catalog (no metadata table). The rebuild-table primitive now
captures and recreates indexes, so `change column` and the
relationship operations no longer silently drop them.
- `drop column` refuses an indexed column unless `--cascade`,
which drops the covering indexes and reports each.
- Persistence: additive `indexes:` list in `project.yaml`
(version unchanged); round-trips through rebuild/export/import.
- Display: an `Indexes:` section in the structure view and a
nested tables/indexes items panel (S2).
Reconciles requirements.md (C3 index portion, S2 satisfied)
and CLAUDE.md. 1038 tests passing (+31), clippy clean.
ADR-0024 audited as fully implemented. Amend the ADR with a "Phase F
minimal" implementation note (parser.rs retained as the router +
ParseError home) and update the README index line to match.
Reconcile docs/requirements.md against handoffs 10-14: refresh the
test baseline (449 -> 1006), mark U4 (replay) satisfied, correct the
A1 / H1a / H3 progress notes.
Amend handoff-14: §3 flagged items both resolved (ranker kept,
CommandNode.hint_mode removed); §4 rewritten as a concrete next-work
pointer at the reconciled requirements.md.
HintMode became per-node (Node::Hinted) in the node-attached refactor;
the per-command hint_mode field was never the mechanism and is now
read by nothing. Removed the field and its 20 `None` initialisers.
Every CommandNode declared a help_id that nothing read; the in-app
`help` body was a single hand-kept catalog block that drifted from
the command set (handoff-12 §2.1).
note_help now iterates the command REGISTRY and translates each
CommandNode's help_id (`help.<id>`), framed by help.intro /
help.dsl_section / help.types_reference. A newly-registered command
appears in `help` automatically — no edit to note_help or a hand-kept
list. Added 20 per-command help entries plus the 3 framing entries;
removed help.in_app_body.
Per-command entries use block scalars: a libyml 0.0.5 scanner bug
panics on long internal space runs in double-quoted scalars, and the
entries are space-aligned.
run_replay parsed each line with the schemaless parse_command, so
Phase D typed-slot rejections (wrong-count value lists, wrong-type
column values) fired only at bind time during replay — inconsistent
with the interactive path (handoff-12 §2.1).
run_replay now re-snapshots the schema per line (the schema mutates
as replayed create-table / add-column commands run) and parses with
parse_command_with_schema. Extracted build_schema_cache, shared with
the interactive refresh_schema_cache.
Added a replay integration test asserting a typed-slot violation is
caught at parse time (through the replay.error_parse wrapper).
Form C (`insert into T (vals)`) shared the `(` opener with Form A,
so its paren was an untyped Repeated(Choice(literal, ident)) — values
weren't type- or count-checked at parse time (handoff-12 §2.2).
New Node::Lookahead variant: a factory that peeks the source. The
insert first-paren factory inspects the first token — a value literal
routes the contents through the typed column_value_list (Form B
dispatch contract: per-non-auto-column typed slots); an identifier or
empty paren routes to a Form A column-name list. So Form C now gets
the same per-column typed slots, hints, and parse-time type/count
checking Form B has.
The explicit-Choice-branch split is impossible here (committed-choice
semantics commit after `(` matches); lookahead is the only route, and
DynamicSubgrammar factories couldn't see the source. Node::Lookahead
is not memoized — its output depends on source — but it returns only
a small node (a Repeated, or a thin DynamicSubgrammar wrapper that
delegates to the memoized column_value_list).
`insert into T (` now cleanly shows Form A column candidates instead
of mixed Form-A/C suggestions. Form C matrix tests updated for the
type-aware behaviour.
Node::DynamicSubgrammar factories build a Node from the WalkContext and
must Box::leak it (the Node enum's combinator children are &'static).
Leaking per walk grew unbounded under per-keystroke completion
(handoff-12 §2.1).
resolve_dynamic now memoizes on the schema state a factory reads
(table columns, current column, user-listed columns) keyed by factory
fn-pointer. Each distinct value-list shape leaks exactly once — total
leak bounded by distinct (schema × form) combinations, not keystroke
count. TableColumn gains Hash for the cache key.
The handoff's original arena sketch needed a lifetime-generic Node
(major refactor); memoization gets the same bound without it.
Replaces the hint resolver's signature-matching (does the expected set
contain all five literal forms? an Ident{NewName}?) with a grammar-
declared annotation. New Node::Hinted { mode, inner } wrapper; the
walker records the mode in WalkContext::pending_hint_mode on entry and
clears it on any successful match (cursor moved past the slot — this
also undoes the leak where a failed Hinted branch of a Choice would
otherwise strand a stale mode). The resolver reads pending_hint_mode
directly.
Value-literal fallback slots carry ProseOnly; NewName ident slots carry
ForceProse. hint_mode_at_input_inner now delegates to
hint_resolution_at_input — one resolution path, no duplicated logic.
No behaviour change; the typing-surface matrix guards it.
Handoff-12 §2.2: Form B `insert into T values (…)` silently skips
auto-generated columns from the value list, so a user who wants to
set a serial/shortid column explicitly could only discover Form A by
reading help. Now the hint at the first Form B value slot appends a
note naming the skipped column(s) and pointing at the explicit-column
form.
hint_resolution_at_input derives the skipped columns from the
post-walk WalkContext (Form B = no user_listed_columns + table has
serial/shortid columns) and reports them on HintResolution; the note
fires only at the first slot so it doesn't repeat at every comma.
ambient_hint composes it onto the per-column prose.
8 tests covering completion-candidate order: connective keywords in
reading order (`to`/`from`/`in` before `table`), and command-part
keywords before schema identifiers. The ordering already held via
declaration-order preservation + keywords-first sectioning in
candidates_at_cursor; nothing pinned it until now, so a future
grammar or sort change could silently break the hint panel's
left-to-right reading.
The 5 structural-class entries (identifier/number/string_literal/flag/
end_of_input) and 3 lex-error entries (bad_flag/unknown_char/
unterminated_string) were unreachable: ADR-0024 Phase F made the
walker render keyword wording verbatim and the lex errors never
surface through today's walker. Handoff-12 §2.1 kept them as a
conservative call; removed now per user request. Re-adding is cheap
if a future need arises.
55 tests covering create table, drop column, drop relationship
(endpoints + by-name), add relationship, rename/change column, and
all app-lifecycle commands. The drop-column and relationship tests
drove the §2.2 writes_table fix in the previous commit.
Documents one UX wrinkle as a flagged finding: partial entry words
(`qu`) classify as DefiniteErrorAt — same as unknown commands —
because the walker only engages on a complete entry word.
859 baseline -> 989 passing; 1 ignored (pre-existing doc-test).
Handoff-12 §2.2: the DDL TABLE_NAME_EXISTING slot and the
relationship-endpoint table idents didn't set writes_table, so
column-name slots downstream (drop/rename/change column; relationship
qualified columns) couldn't narrow to the active table — candidates
leaked from every table. Set writes_table: true on TABLE_NAME_EXISTING
and on DR_PARENT/DR_CHILD/AR_PARENT/AR_CHILD table idents. The
deliberately-documenting completion test now asserts per-table
narrowing.
34 new tests covering:
- Form C bare-value-list (happy path + Form-A-recovery + type-unaware grammar limitation per handoff §2.2)
- update with WHERE (column-narrowing invariant per handoff §1 bug E1; typed-slot prose for assignments and where filters)
- update --all-rows (filter-clause requirement per ADR-0014)
- delete with WHERE (column-narrowing; typed-slot prose for where filters)
- delete --all-rows
859 baseline -> 931 passing. No bugs surfaced — the data-mutation
command family was already well-shaped post-Phase-D.
12 tests across schema_serial_pk / text_pk / multi_table / every_type.
Pins (a) Form B skips auto-gen columns from the slot list (regression
for handoff-12 §B fix); (b) wrong-count value lists are now flagged
at typing time, not only at submit (the previous commit's fix); and
(c) per-type slot prose advances correctly through every Type variant.
classify_input was schemaless; wrong-count Form B value lists
(`insert into Customers values ('Alice')` against a 3-column table)
showed as Valid until submit. Add classify_input_with_schema that
threads the SchemaCache through parse_command_with_schema and wire
render_input_runs to use it. Schemaless classify_input is kept public
for handoff-11/12 regression tests that exercise schema-independent
positions.
Per docs/handoff/20260515-handoff-12.md §1. Systematic per-position
coverage of (state, hint, completion, parse_result) across canonical
schema shapes; submodule per command family. Insert Form A covers 23
cursor positions across serial-PK, text-PK, multi-table, and
every-Type schemas. Both bugs fixed in the previous commit were
surfaced by these tests.
Shared helpers under tests/typing_surface/mod.rs: 5 canonical schema
shapes, assess() helper, property-assertion shortcuts, and a snap!
macro that wraps insta with a stable per-cell suffix.
859 -> 885 tests passing; 1 ignored (pre-existing doc-test).
The typing-surface matrix exposed two bugs the existing 859-test suite
missed:
walk_repeated: when the separator consumed but the inner item failed
at EOF, the old path rolled the separator back and reported a definite
error at the rollback position (`insert into T (a, ` flashed red on
the `,` after each comma). Now propagates Incomplete with the inner's
expected set so the input renderer treats it as mid-typing.
build_insert Form C path: `insert into T (col)` walked to a complete
match but produced `values: []` because Form C's value collector drops
ident-shaped items. The user almost certainly meant Form A and just
hasn't typed `values (...)` yet. Reject with a ValidationError naming
the Form-A continuation; classify_input now reports IncompleteAtEof.
completion_probe / expected_at_input: ValidationFailed used to return
an empty expected set, leaving Tab with nothing to offer at the new
Form-A flag point. Now surface result.tail_expected (skipped-Optional
expectations captured before validation fired) so `values` is still
offered as a candidate.
Two related fixes:
1. \`update MyTable set \` was offering columns from every
table in the project — completion fetched
\`cache.for_source(IdentSource::Columns)\` which returns the
flat \`cache.columns\` (union of every table's columns).
The walker's WalkContext had \`current_table_columns\`
populated (because the update-table-name slot is
\`writes_table: true\`) but the completion engine never
consulted it.
2. \`insert into MyTable (\` was offering nothing — the
value-literal suppression fired because the expected set at
this position contains both Form A column-list candidates
(\`Ident{Columns}\`) and Form C bare-value-list literals
(null/true/false/NumberLit/StringLit). \`is_value_literal_signature\`
matched and the engine returned \`None\` before the column
candidates were considered.
The fix threads the walker's \`current_table_columns\` through
to the completion engine and narrows the suppression rule:
**Walker:**
- New \`walker::CompletionProbe { expected, current_table_columns }\`
struct.
- New \`walker::completion_probe(source, schema) -> CompletionProbe\`
runs one schema-aware walk and reports both the expected
set (or tail_expected on Match) and the resolved table-column
snapshot.
**Completion engine:**
- \`candidates_at_cursor_with\` calls \`completion_probe\` and
reads \`current_table_columns\` for the \`Columns\` ident
source. Schemaless or unknown-table falls back to the flat
\`cache.columns\` (preserves pre-fix behavior).
- Value-literal suppression now gated on
\`!has_schema_ident\` — if the expected set also offers a
schema-listable Ident, the user has actionable candidates
beyond the misleading null/true/false trio and we shouldn't
hide them.
Tests:
- \`update_set_offers_only_current_table_columns\` confirms
Customers' columns appear while Orders' columns don't.
- \`update_where_offers_only_current_table_columns\` covers
the where path.
- \`insert_into_open_paren_offers_current_table_columns\` and
\`insert_into_open_paren_does_not_offer_unrelated_columns\`
cover the Form A column-list position.
- \`drop_column_from_offers_only_current_table_columns\`
documents the DDL fallback (drop-column's table-name slot
doesn't currently \`writes_table\` — falls back to the flat
list).
For the user: \`update MyTable set \` now offers only
MyTable's columns. \`insert into MyTable (\` offers all of
MyTable's columns so Form A is fully discoverable.
Tests: 859 passing, 0 failing, 1 ignored. Clippy clean.
Two related fixes from a user-reported snag:
1. After typing \`insert into Orders \`, the hint suggested only
\`values\` even though the user could also choose \`(\` to
open Form A (the explicit-column-list variant). The walker
reports both \`Expectation::Word("values")\` and
\`Expectation::Punct('(')\` at that position, but
\`candidates_at_cursor\` had a blanket "no punctuation as Tab
candidate" policy.
Loosened the policy to surface branching punct
(specifically \`(\` opening a sub-shape). Closing punct
(\`)\`), separators (\`,\`), and content-trailing punct (\`:\`,
\`=\`, \`.\`) stay out — the user types those naturally and
advertising them in the Tab menu is noise. New
\`CandidateKind::Punct\` so the renderer colors it as punct
rather than mis-classifying as a keyword.
2. While typing \`insert into Orders (id, CustId, Total) values
(42, 89, 17.59\` (no closing paren yet), the word \`values\`
was rendered in \`tok_error\` red. The walker's
\`Optional(Seq[values, '(', list, ')'])\` was rolling back on
the partial inner match — treating \`(id, CustId, Total)\` as
Form C (bare value list) followed by trailing junk starting
at \`values\`. The classify_input call thus returned
\`DefiniteErrorAt(<values byte>)\` and the renderer overlaid.
Tightened \`walk_optional\`: roll back only when the inner
reports NoMatch (or Incomplete / Mismatch without consuming
anything). Once the inner has committed to at least one
terminal (e.g. matched the \`values\` keyword), propagate
Incomplete / Mismatch up — the user is mid-typing the
optional's content and rolling back would lose their
intent.
The pre-existing chumsky-or_not-style aggressive rollback
covered cases like \`save Customers\` (Optional(\`as\`)
inner is a single Word that returns NoMatch without
consuming, so rollback still fires). Those keep working.
3. Side effect: with \`Optional\` no longer hiding the
in-progress Form A from the leading slice, the walker on
\`create table T with \` correctly reports the next-expected
keyword as \`pk\` — so cursor at the end of the complete
command \`create table T with pk\` would now re-offer \`pk\`
as a Tab candidate against the partial \"pk\". Added a final
filter: when the full input is a valid parse AND the
partial prefix is non-empty, drop candidates that equal the
partial exactly. Preserves schema narrowing
(\`show data Cu\` → \`Customers\` is not an exact match).
Tests:
- New \`in_progress_form_a_values_list_classifies_as_incomplete\`
asserts the input-state for the user's exact scenario.
- New \`open_paren_branching_punct_surfaces_after_insert_into_table\`
and \`open_paren_candidate_is_classified_as_punct_kind\` cover
the punct-as-candidate surface.
- Renamed and rewrote \`punctuation_expected_does_not_produce_candidates\`
to \`non_branching_punctuation_is_not_surfaced_as_candidate\`
to document the new finer-grained policy.
- Existing tests for \`save Tab → as\` and the schema-
narrowing case continue to pass.
Tests: 854 passing, 0 failing, 1 ignored. Clippy clean.
Bug: hint at \`insert into Customers values (\` for a Customers
table with id:serial PK suggested typing an integer for \`id\`,
but the dispatch path (\`db::do_insert\`) deliberately doesn't
accept user-supplied values for auto-generated columns in
Form B. The grammar prompted for a value the dispatch would
refuse.
The fix aligns Phase D's \`column_value_list\` dynamic sub-grammar
with do_insert's three forms (ADR-0014 + ADR-0018 §3):
- **Form A** \`insert into <T> (col1, col2, …) values (…)\` —
user explicitly lists columns. Slot list mirrors that
selection; serial / shortid columns CAN appear if the user
lists them.
- **Form B** \`insert into <T> values (…)\` — bare values. Slot
list = non-auto-generated columns of the table in
declaration order. Serial / shortid get auto-filled by the
dispatch; the grammar doesn't prompt for them.
- **Form C** \`insert into <T> (v1, v2, …)\` — bare value list.
Not affected by this change (column_value_list isn't on this
path; Form C's literals route through the schemaless
INSERT_PAREN_LIST).
Implementation:
\`WalkContext.user_listed_columns: Option<Vec<String>>\` — when
\`Some\`, signals Form A; \`None\` is Form B. Populated by walking
the first paren's column-list idents.
\`Node::Ident.writes_user_listed_column: bool\` — new field;
\`true\` on the INSERT_PAREN_ITEM's Ident child. When the
walker matches that ident in Form A, it appends the
schema-canonical column name (case-corrected against the
schema) to user_listed_columns.
\`column_value_list\` factory:
- If user_listed_columns is Some → resolve each name from the
schema; one typed slot per listed column.
- Else → filter current_table_columns to non-auto-generated;
one typed slot per remaining column.
- Empty result → fall back to the schemaless value-literal
list (a serial-only table in Form B has nothing for the
user to type).
Tests:
- New \`phase_d_insert_form_b_skips_serial_column\` confirms the
bug: \`insert into Customers values (1, 'Alice')\` against a
Customers with serial id rejects at parse time (Form B
expects 1 value for Name, not 2).
- New \`phase_d_insert_form_a_accepts_serial_when_listed\`
confirms \`insert into Customers (id, Name) values (1, 'Alice')\`
works.
- New \`phase_d_insert_form_a_filters_to_user_listed_columns\`
confirms partial Form A (\`(Name) values ('Alice')\`).
- Updated \`phase_d_insert_with_schema_accepts_typed_values_per_column\`
to match the new Form B contract (2 user-typed values, not 3).
- Updated typed-hint test matrix split into form-B (8 types)
and form-A (serial / shortid).
- New \`typed_hint_form_b_skips_serial_column_to_generic_or_text_neighbor\`
pins the fallback behavior for a serial-only table.
For the user: \`insert into Customers values (\` for a Customers
with \`(id:serial, Name:text, Email:text)\` now hints
\`for \`Name\`: Type a quoted string …\` (skipping id entirely)
and accepts exactly 2 values. To set the serial explicitly,
use Form A: \`insert into Customers (id, Name, Email) values
(1, 'Alice', 'a@b.c')\`.
Tests: 851 passing, 0 failing, 1 ignored. Clippy clean.
User-facing improvement: typing into a value slot now surfaces
the column name in the hint. The hint at `insert into Customers
values (` (first column id:int) reads "for `id`: Type an
integer (e.g. 42, -7) or null" instead of the generic
"Type an integer …" prose. After `1, ` the panel updates to
the second column ("for `Name`: Type a quoted string …"). The
same applies to `update T set Email=` and `delete from T where
ts=` — the catalog wrapper threads the column name through.
Implementation:
**`Node::TypedValueSlot.column_name: Option<&'static str>`**
(new field, `src/dsl/grammar/mod.rs`). When `Some`, walker
writes `WalkContext::pending_value_column` on entry; clears
along with `pending_value_type` on inner success.
**Walker driver writes both names** (`src/dsl/walker/driver.rs`):
- `Node::TypedValueSlot` dispatch reads `column_name` and
populates `pending_value_column`.
- `Ident { writes_column: true }` dispatch also writes
`pending_value_column` (using the schema-canonical name when
available, falling back to the user's spelling) so update
set / where positions surface the column name.
**Shared sub-grammars** (`src/dsl/grammar/shared.rs`):
- New `slot_for_column(ty, name)` builds a `TypedValueSlot`
with the embedded leaked column name. Used by
`column_value_list`.
- New `slot_inner_for_type(ty)` returns just the Choice
(without TypedValueSlot wrapper) for slot_for_column to
rebuild.
- `column_value_list` factory now constructs per-column slots
via `slot_for_column(col.user_type, &col.name)`. Each slot
leaks its column name string with the same per-walk Box::leak
pattern the rest of dynamic dispatch uses.
**`WalkContext::pending_value_column: Option<String>`** (new
field, `src/dsl/walker/context.rs`). Pairs with
`pending_value_type` to give the hint resolver both pieces.
**Single-walk hint resolver** (`src/dsl/walker/mod.rs`):
- New `HintResolution { mode: HintMode, column: Option<String> }`
struct.
- New `hint_resolution_at_input(source, schema) -> Option<
HintResolution>` runs one walk and reports both pieces. The
ambient_hint dispatch composes per-column prose from the
result.
- Existing `hint_mode_at_input` / `hint_mode_at_input_with_schema`
preserved as thinner wrappers for tests / future callers
that don't need the column name.
**Catalog wrapper** (`src/friendly/strings/en-US.yaml`,
`src/friendly/keys.rs`):
- New `hint.value_slot_for_column: "for `{column}`: {detail}"`
prefixes the per-type prose with the actual column name when
the walker has it bound. Schemaless fallback continues to use
the generic value-literal prose with no column prefix.
**ambient_hint composes** (`src/input_render.rs`): consults
`hint_resolution_at_input`; when `column` is `Some`, wraps the
type prose through `hint.value_slot_for_column`; otherwise
emits the bare type prose.
Tests (846 total, 0 failing):
- 4 new input_render tests assert column names appear in the
prose at insert/update/where positions plus the
second-insert-value position (proves column tracking advances
with comma).
- All existing tests pass unchanged — the column-name addition
is layered on top of the type-only prose path.
Clippy clean.
The Phase D commit landed parse-time validation but not the
user-facing payoff — per-column-type hints. Typing
`insert into Customers values (` rightfully expected a hint
like "Type an integer (e.g. 42, -7) or null" at an int column.
This commit closes that gap.
End-to-end:
**`Node::TypedValueSlot { ty, inner }`** (new variant in
`src/dsl/grammar/mod.rs`):
- Walker walks `inner` to consume the literal but tags
`WalkContext::pending_value_type = Some(ty)` on entry, then
clears it on a successful inner match. Positions BETWEEN
slots (`insert into T values (1` mid-input) thus don't carry
a stale hint type.
**Typed slot factories wrapped in `TypedValueSlot`**
(`src/dsl/grammar/shared.rs`):
- `INT_SLOT`, `REAL_SLOT`, `DECIMAL_SLOT`, `BOOL_SLOT`,
`TEXT_SLOT`, `DATE_SLOT`, `DATETIME_SLOT`, `BLOB_SLOT`,
`SERIAL_SLOT`, `SHORTID_SLOT` — each pairs an inner literal
Choice with its `Type` so the walker can tag context.
- `slot_for_type(ty)` dispatches to the appropriate constant.
- Bug fix: `ShortId` previously dispatched to `INT_SLOT` (a
pre-Phase-D holdover from the chumsky-side generic
fallback). `shortid` columns store base58 text (ADR-0011
fk_target_type shortid → text); the corrected slot accepts
`StringLit` or `null`.
**Schema-aware hint resolver** (`src/dsl/walker/mod.rs`):
- `hint_mode_at_input_with_schema(source, &SchemaCache) ->
Option<HintMode>` is the new public entry point. Reads
`pending_value_type` from the walker's WalkContext and
emits `HintMode::ProseOnly("hint.value_slot_<type>")` —
one per Type.
- The schemaless `hint_mode_at_input(source)` falls back to
the generic `hint.value_literal_slot` at value-literal slots
(no per-type narrowing without a schema).
- `catalog_key_for_value_type(ty)` is the type → key
dispatcher.
**Catalog entries** (`src/friendly/strings/en-US.yaml`,
`src/friendly/keys.rs`):
- 10 new `hint.value_slot_<type>` keys with per-type prose:
- int/serial → "Type an integer (e.g. 42, -7) or null"
- real/decimal → "Type a number (e.g. 3.14, -0.5) or null"
- bool → "Type true, false, or null"
- text → "Type a quoted string (e.g. 'Alice') or null"
- date → "Type a quoted date as 'YYYY-MM-DD' or null"
- datetime → "Type a quoted datetime as 'YYYY-MM-DD
HH:MM:SS' or null"
- blob → "Type a quoted blob literal or null"
- shortid → "Type a quoted shortid (or omit to auto-generate)
or null"
**Ambient-hint dispatch** (`src/input_render.rs::ambient_hint`):
- Passes the SchemaCache through to
`hint_mode_at_input_with_schema`, so the live hint panel
surfaces per-column-type prose as the user types into a
value slot.
Tests:
- 8 walker-side tests cover insert / update / where typed-slot
hint dispatch, mid-value no-stale-hint behaviour, and a
full-coverage routing matrix for every `Type` variant.
- 4 input_render integration tests cover the end-to-end
ambient_hint path: insert first/second value, update set
value, and the schemaless fallback to generic prose.
Tests: 842 passing, 0 failing, 1 ignored. Clippy clean.
For the user: typing `insert into Customers values (` against
a Customers table whose first column is `id:int` now shows
"Type an integer (e.g. 42, -7) or null" in the hint panel,
replacing the previous generic value-literal prose. After
typing `1, `, the panel updates to whatever the second column
requires — "Type a quoted string (e.g. 'Alice') or null"
for text, "Type a quoted date as 'YYYY-MM-DD'" for date, etc.
Pre-Phase-D, `save ` parsed as a complete `save` command, so
the completion engine had nothing to mine: `as` never surfaced
as a Tab candidate. This is the round-5 gap the handoffs have
been tracking.
`WalkResult` gains a `tail_expected: Vec<Expectation>` field.
The walker's top-level `Matched` branch copies the outer
shape's skipped-Optional expectations into it. `expected_at_input`
returns `tail_expected` on `Match` so the completion engine
sees the optional-suffix continuations and offers them as Tab
candidates.
`hint_mode_at_input` deliberately does NOT consume
`tail_expected` — surfacing prose like "Type a name" at the
end of a valid command would be misleading. A new private
`expected_for_hint` returns empty on `Match` to preserve this.
The split distinguishes "valid + could continue" (completion
helps) from "invalid + must continue" (hint resolver helps).
Tests:
- `save ` Tab → `as` (new test, the original round-5 gap).
- `messages ` Tab → `short` and `verbose` (same shape).
- Existing `hint_mode_none_for_complete_command` stays green
because hint resolver ignores tail_expected.
- 830 total passing, 0 failing, 1 ignored. Clippy clean.
Schema-aware typed value slots — the central design claim of
ADR-0024 §Phase D. Insert / update / delete value slots now
dispatch on the user-facing column type at parse time, rejecting
mis-shaped input with localised wording instead of waiting for
the bind-time error.
What changed:
**SchemaCache extension** (`src/completion.rs`):
- New `TableColumn { name, user_type }` for per-table column
metadata.
- `SchemaCache.table_columns: HashMap<String, Vec<TableColumn>>`.
- `SchemaCache::columns_for_table(name)` — case-insensitive
lookup, mirrors the walker's case-insensitive entry-word
resolution.
**WalkContext schema plumbing** (`src/dsl/walker/context.rs`):
- `WalkContext<'a>` gains a lifetime and a `schema: Option<&'a
SchemaCache>`. `WalkContext::new()` keeps the schemaless
default; `with_schema(s)` is the new schema-aware constructor.
**Parser entry point** (`src/dsl/parser.rs`):
- `parse_command_with_schema(input, schema)` is the new public
schema-aware variant. `parse_command(input)` becomes a thin
wrapper that delegates with `None` for back-compat.
- Internal `try_walker_route` accepts an `Option<&SchemaCache>`
and threads it into the WalkContext.
**Node::Ident writes_table/writes_column** (`src/dsl/grammar/mod.rs`):
- Two new fields on `Node::Ident`. When `writes_table: true` and
`source: Tables`, the walker writes the matched ident's name
into `current_table` and resolves `current_table_columns`
against the schema cache. When `writes_column: true` and
`source: Columns`, the walker writes the resolved
`TableColumn` into `current_column`.
**Walker driver DynamicSubgrammar dispatch** (`src/dsl/walker/driver.rs`):
- The `Node::DynamicSubgrammar(factory)` branch now resolves the
factory at walk time and `Box::leak`s the result so its inner
static-slice fields (Choice/Seq) have the lifetime the walker
expects (per ADR-0024 §sub-grammars). The leak is bounded by
command-shape complexity per walk; per-walk arena is a future
optimisation.
- `walk_ident` extends to perform the schema writes when the
flags are set.
**Typed value slot factories + dynamic sub-grammars** (`src/dsl/grammar/shared.rs`):
- `int_slot` / `real_slot` / `decimal_slot` / `bool_slot` /
`text_slot` / `date_slot` / `datetime_slot` / `blob_slot` —
one per `Type`. Each accepts the appropriate literal kind plus
`null`; integer-only validator rejects `3.14` at int columns;
decimal validator pins numeric shape.
- `slot_for_type(ty) -> Node` is the dispatcher.
- `current_column_value(ctx) -> Node` is the dynamic sub-grammar
for `set col = …` and `where col = …` values; reads
`current_column` and dispatches via `slot_for_type`.
- `column_value_list(ctx) -> Node` is the dynamic sub-grammar
for `insert into T values (…)`; reads `current_table_columns`
and unfolds a Seq of typed slots separated by commas.
- Both fall back to the schemaless `VALUE_LITERAL` choice when
the context lacks the schema-resolved entries — keeps
schemaless `parse_command` callers (tests, replay path)
working.
**Data-command grammar wires the new types** (`src/dsl/grammar/data.rs`):
- `TABLE_NAME_INSERT` / `TABLE_NAME_WRITES` (new): table-name
slots that set `writes_table: true`. Used by insert / update /
delete to populate `current_table_columns`.
- `SET_COLUMN` / `FILTER_COLUMN` (new): column-name slots in
`set col=…` / `where col=…` set `writes_column: true`.
- `INSERT_VALUES_LIST` becomes `DynamicSubgrammar(column_value_list)`.
- `UPDATE_ASSIGNMENT` and `WHERE_CLAUSE` use
`PER_COLUMN_VALUE = DynamicSubgrammar(current_column_value)`.
**Runtime plumbs schema-with-types** (`src/runtime.rs`):
- `refresh_schema_cache` calls `describe_table` for each table
and populates `SchemaCache::table_columns` with
`TableColumn { name, user_type }` entries. Best-effort: a
`describe_table` miss leaves that table unpopulated and the
walker falls back to schemaless dispatch.
**App dispatches with schema** (`src/app.rs`):
- `dispatch_dsl` routes through `parse_command_with_schema(&self
.schema_cache, …)` so live typing/dispatch sees the typed
slots. The replay path stays schemaless (deferred — replay
bind-time errors still catch type mismatches).
**Catalog** (`src/friendly/strings/en-US.yaml`, `src/friendly/keys.rs`):
- New `parse.custom.bind_type_mismatch` entry with `{found}` and
`{expected}` placeholders. Surfaced by the int_slot /
decimal_slot validators.
Tests:
- 11 new walker-side Phase D tests cover insert / update /
delete with schemas — typed acceptance per column, decimal
rejection at int columns, null acceptance at any slot,
multi-assignment per-column dispatch, schemaless fallback.
- The pre-existing `parse_command(input)` test suite (no
schema) still passes — the fallback path is behaviour-
preserving.
- 828 passing total, 0 failing, 1 ignored. Clippy clean.
Adds the `HintMode` dispatch layer the ADR specified: the
ambient-hint resolver now consults a single
`walker::hint_mode_at_input(source) -> Option<HintMode>` to
decide between the prose / candidates ladder, rather than
discovering each slot kind through three separate post-hoc
helpers (`value_literal_hint_at_cursor`,
`typing_name_at_cursor`, and so on).
Behaviour at slot positions today:
- **Value-literal slot** (`null`/`true`/`false`/number/string
all in the expected set) → `HintMode::ProseOnly
("hint.value_literal_slot")`. The ambient-hint ladder
emits the catalog prose at empty prefix; once the user types
a partial (`n`, `tr`, `fa`) the partial check declines and
normal candidate completion takes over.
- **NewName ident slot** → `HintMode::ForceProse
("hint.ambient_typing_name")`. The ladder still consults
`typing_name_at_cursor` to learn what comes after the name
(the post-name probe is unchanged); `ForceProse` is the
declarative tag telling the resolver *that* we're in this
mode.
`HintMode` itself gains `PartialEq + Eq` for tests, and
its docstring is rewritten to describe the live semantics.
This is the structural shape ADR-0024 §HintMode-per-node
describes: one slot → one hint mode → one dispatch arm. The
detection inside `hint_mode_at_input` is transitional — it
pattern-matches the walker's expected-set today, which is
exactly what the previous ad-hoc detectors did. Phase D will
replace the signature match with node-attached `HintMode`
annotations on the typed value slots (so `date_slot`,
`int_slot`, etc. each carry a type-specific catalog key).
Two helpers move into `input_render.rs`:
- `hint_leading_slice(input, cursor)` mirrors the look-back
used by `candidates_at_cursor` so the hint resolver sees the
same token-boundary view of the world.
- `cursor_partial_is_empty(input, cursor)` distinguishes
empty-prefix from in-progress identifier shapes.
8 new walker tests pin the hint-mode resolver across
value-literal-after-paren, value-literal-after-set-assign,
value-literal-in-where, two NewName-slot cases, the
entry-keyword position, the complete-command position, and
the schema-ident position.
Tests: 817 passing, 0 failing, 1 ignored. Clippy clean.
Adds the `Ranker` plug-in point ADR-0024 §ranker-layer
specified. The grammar tree declares *what's valid*; the
ranker decides *what's likely useful first*. Default
`identity_ranker` preserves declaration order from the
grammar.
API:
- `pub type Ranker = fn(Vec<Candidate>) -> Vec<Candidate>;`
- `pub const fn identity_ranker(c) -> Vec<Candidate>` returns
its input unchanged.
- `candidates_at_cursor_with(input, cursor, cache, ranker)`
applies a custom ranker; the default `candidates_at_cursor`
delegates with `identity_ranker`.
Three new tests cover identity preservation, custom reordering,
and the empty-list-collapses-to-None edge.
This is a future-work hook — no production caller passes a
non-identity ranker yet. Hooks for frequency-based ranking,
content-aware priors, or recency plug in here without touching
the grammar declarations.
Tests: 809 passing, 0 failing, 1 ignored. Clippy clean.
claude@clouddev1