Commit Graph

84 Commits

Author SHA1 Message Date
claude@clouddev1 ed881eea59 2g: advanced-mode highlight + engine.* wiring + matrix tests
Cross-cut verification matrix for ADR-0032 Phase 2 is now fully
populated with concrete test references — every row green. Filling
the matrix surfaced three real gaps that this commit closes.

1. Advanced-mode syntax highlighting (ADR-0030 §8 matrix row).
   The `ui.rs` Advanced branch routed through `plain_input_spans`,
   bypassing the highlight walker entirely. In production SQL
   keywords past the entry word rendered as plain identifiers.
   Fix: mode-aware variants of `highlight_runs`,
   `render_input_runs`, `lex_to_runs`, and `input_diagnostics`;
   the Advanced render path now uses the highlighted form with
   `Mode::Advanced`. `plain_input_spans` removed (unused).

2. Engine.* key wiring (ADR-0032 §11.4 / §13 matrix rows + handoff
   §3.3 follow-up). The four Phase-2 engine.* catalog entries
   were authored in 2d but never reached: `translate_generic`
   discarded the engine message and returned a vague catalog
   entry. Fix: pattern-match the engine message text for the four
   Phase-2 categories (aggregate misuse, group-by required,
   compound arity mismatch fallback, scalar-subquery cardinality)
   inside `translate_generic`, routing each to its engine-neutral
   catalog entry.

3. Matrix-coverage tests. Thirteen new tests covering the rows
   that had no explicit coverage:
   - 3 SQL keyword/operator/CASE highlight tests
   - 4 engine.* engine-message tests
   - 3 sql_expr column-completion tests (WHERE, HAVING)
   - 3 predicate-warning slot tests (CASE, ORDER BY, projection)
   - 1 all-10-playground-types recovery test (tests/sql_select.rs)

Plan document (docs/plans/20260520-adr-0032-phase-2.md) updated:
every (TBD) row in the cross-cut matrix replaced with a concrete
test file::function reference and a green status marker.

Test totals: 1428 → 1441 passing (+13 new). Clippy clean.
2026-05-20 21:38:08 +00:00
claude@clouddev1 ee0dafd86b docs: ADR-0032 Amendment 2 + §10.6 regression tests
Amendment 2 records the §10.6 fixup-pass mechanism choice. §10.6
prescribes "rewriting the highlight class" on projection-list
idents at end-of-walk; the actual implementation uses a different
mechanism that achieves the identical user-visible behavior:

1. 2d's two-pass schema-existence diagnostic collects every FROM
   binding from the matched path first, then resolves projection
   idents against the complete scope. The post-walk re-resolve
   §10.6 calls for, just embedded in the diagnostic emitter.

2. input_render.rs's diagnostic-overlay path colors each
   diagnostic span Error/Warning, achieving the visual change
   §10.6 describes without needing a new HighlightClass variant.

The completion-mid-typing piece is improved by the §10.5
look-ahead probe (sub-phase 2e earlier).

Four new regression tests in `projection_before_from_tests` pin
the behavior so a future refactor can't silently regress it:
correct ident resolves silently, unknown ident flags via
diagnostic on its span, multi-projection only flags unknowns,
projection-without-FROM is silent.

ADR index entry updated to reference Amendment 2.

Test totals: 1424 → 1428 passing (+4). Clippy clean.
2026-05-20 21:19:57 +00:00
claude@clouddev1 0fc7b082b2 completion: §10.5 qualified-prefix + edit-scenario look-ahead
ADR-0032 §10.5 — at the cursor, an `<ident>.` prefix narrows
column candidates to that qualifier's binding columns. Resolves
through from_scope aliases first, then table names, then
cte_bindings (for `cte_alias.|`). Falls back to the schema cache
for DSL paths (`from <Table>.<col>`). Unresolved qualifier →
empty column list; the structural error path surfaces the
unresolved-prefix message.

Look-ahead probe — the "edit an existing query" workflow. When
the cursor is mid-projection but FROM exists after the cursor, a
second walk on the full input populates from_scope and the
column candidates narrow accordingly. Gated on the leading walk
producing no scope so cursor-past-FROM positions pay no cost.
The full input must parse for this to work; an unparseable
mid-edit state falls back to the §10.6 global posture.

CompletionProbe now exposes `from_scope` (top-frame table
bindings) and `cte_bindings` (union of in-scope CTE bindings,
innermost-first dedupe). The walker drains these at the cursor
position; the completion engine reads them for qualifier
resolution and unqualified narrowing.

Test totals: 1415 → 1424 passing (+9: 5 qualified-prefix +
4 look-ahead). Clippy clean.
2026-05-20 21:05:27 +00:00
claude@clouddev1 fd259048da grammar: admit WITH inside subqueries / CTE bodies (ADR-0032 §10.3)
ADR-0032 §10.3 says cte_bindings lives on the scope frame, with
inner subqueries free to declare their own CTEs that shadow outer
ones. The grammar didn't actually admit nested WITH inside
SQL_SELECT_COMPOUND — a real ADR-vs-implementation gap.

Closes the gap by making SQL_SELECT_COMPOUND a Choice between a
WITH-prefixed form and a plain form. The naive Optional-prefix
approach silently broke the paren-vs-subquery dispatch in
sql_expr.rs's PAREN_GROUP: Optional matches 0 bytes, committing
the Seq, so SELECT_CORE's NoMatch on `(a + b)` became Failed and
the Choice couldn't fall through to or_expr. The Choice-fronted
form keeps the fast NoMatch on non-WITH non-SELECT first tokens.

Side effect: scalar subquery / IN / EXISTS / derived-table
bodies now admit a leading WITH too, which matches standard SQL.

Updated two tests that were guarding the old `(WITH …)` rejection
behavior. Added one new harvest test exercising nested-WITH inside
a CTE body — the harvest's `expand_binding` mechanism already
handled the data correctly; the grammar gap was the sole blocker.

Test totals: 1414 → 1415 passing (+1 nested-with-in-cte test).
Clippy clean.
2026-05-20 20:34:42 +00:00
claude@clouddev1 dd37a1cbfc walker: 2e prereq — §10.3 stage-2 CTE harvest + cte_arity_mismatch
Implements the six ADR-0032 §10.3 output-column derivation rules
at CTE body-frame exit, populating the placeholder CteBinding's
columns. Unblocks `diagnostic.cte_arity_mismatch` (which compares
declared col-list arity vs derived projection arity) and the
upcoming qualified-prefix completion in 2e proper.

- `WalkContext::pending_cte_harvest`: bookkeeping for an in-progress
  CTE harvest, armed by writes_cte_name + extended by cte_column
  idents, consumed by the next walk_scoped_subgrammar invocation
  (CTE syntax has no intervening ScopedSubgrammar, so timing is
  deterministic). Cleared on every walk_scoped_subgrammar entry
  to prevent stale state surviving a speculative walk rollback.

- `run_cte_harvest`: post-walk path-scan classifier that
  reconstructs the body's first leg's projection-list and applies
  the six derivation rules. Compound bodies take columns from the
  first leg per spec; recursive CTE bodies take the non-recursive
  (first) leg. Optional (col-list) renames positionally with
  preserved types.

- `expand_binding`: bridges a TableBinding to a CteColumn list,
  resolving CTE-source bindings (empty columns + table-name
  matches an in-scope CteBinding) through to the CTE's harvested
  columns. Enables sibling CTEs to project correctly: in
  \`WITH a AS (...), b AS (SELECT * FROM a) ...\`, b's harvest sees
  a's derived columns through the body's from_scope binding.

- `WalkContext::pending_diagnostics`: accumulator for diagnostics
  emitted DURING the walk by node handlers with context the
  post-walk passes can't reconstruct. Drained by the top-level
  walk function on both match and non-match paths so a re-used
  context can't leak entries between walks.

Test totals: 1399 → 1414 passing (+15: 10 derivation rules + 1
sibling CTE + 4 arity match/mismatch tests). Clippy clean.
2026-05-20 17:42:17 +00:00
claude@clouddev1 c20c6e05ca walker: 2d.1 — projection-alias misplaced + compound-arity ERROR passes
Closes the two diagnostics deferred by sub-phase 2d that were not
attached to a user-approved deferral. `cte_arity_mismatch` stays
deferred — it depends on the §10.3 stage-2 CTE harvest, which IS a
user-approved deferral.

- `diagnostic.projection_alias_misplaced` (ADR-0032 §11.2): emitted
  when a projection alias is referenced from `WHERE` / `HAVING` /
  `GROUP BY`. `ORDER BY` references are allowed and silent. The
  pass is integrated into `schema_existence_diagnostics`: when a
  bare-column ref doesn't resolve to any binding's column but DOES
  match a projection alias in the current SELECT leg, the new
  diagnostic pre-empts the misleading `unknown_column` that would
  otherwise fire on the same span. Real-column-shadowed-by-alias
  cases (engine resolves to the table column) stay silent. Subquery
  scopes (paren depth > 0) keep their own implicit alias bag —
  outer aliases don't leak into inner WHERE.

- `diagnostic.compound_arity_mismatch` (ADR-0032 §11.2 / §11.7): a
  new MatchedPath-walking pass that counts projection items per
  SELECT leg by tallying top-level commas at the leg's own paren-
  depth, then compares adjacent legs across `UNION` / `UNION ALL` /
  `INTERSECT` / `EXCEPT` operators. The diagnostic anchors on the
  operator span. Per-depth book-keeping lets chained compound
  queries inside CTE bodies / subqueries report independently.
  Function-call argument commas (deeper depth) are correctly
  ignored.

Test totals: 1385 → 1399 passing (+14), 0 failed, 1 ignored.
Clippy clean.
2026-05-20 17:11:44 +00:00
claude@clouddev1 c5cf03b152 walker: SQL diagnostics — multi-binding scope, qualified refs, Phase-1 gap closure (sub-phase 2d)
Implements the bulk of ADR-0032 §11 diagnostics. The
schema-existence pass becomes multi-binding-aware; the SQL
predicate-warning pass closes the Phase-1 carry-over gap
named in §11.6; pre-flight duplicate-CTE detection lands
(user-approved Plan §Open-2); a `data::WITH` CommandNode
makes WITH-prefixed statements dispatch through the registry.

Catalog (`src/friendly/strings/en-US.yaml`, `src/friendly/keys.rs`):

- Six new `diagnostic.*` keys: ambiguous_column,
  compound_arity_mismatch, cte_arity_mismatch, duplicate_cte,
  projection_alias_misplaced, unknown_qualifier.
- Eight new `engine.*` translation keys (ADR-0032 §11.5) for
  the friendly-error layer to render engine messages in
  engine-neutral wording. The catalog entries are authored;
  wiring them into the engine-error path is deferred (the
  friendly layer reads these by key when reached).

Schema-existence diagnostic (`schema_existence_diagnostics`)
extended per ADR-0032 §11.2:

- A pre-pass collects all `table_name` / `cte_name` / table-
  alias idents into a `PassBinding` vec + a CTE name list,
  sidestepping the projection-before-FROM ordering problem
  (§10.6). The main pass then resolves identifiers against the
  complete scope.
- Bare column references resolve against any binding's
  columns. Zero matches → `diagnostic.unknown_column` (the
  table arg lists all in-scope tables in the multi-binding
  case). Two-or-more matches → `diagnostic.ambiguous_column`.
- Qualified `t.c` refs detect their qualifier via a look-ahead
  on the matched path (Punct '.' + Ident{role:
  sql_expr_qualified_ref} after the leading Ident). Unknown
  qualifier → `diagnostic.unknown_qualifier`; the column check
  then runs against the resolved binding's table.
- The `t.*` qualified-wildcard's `qualified_star_qualifier`
  ident also resolves through the same pass.
- CTE-name references in table-source slots accept silently
  (the CTE binding's columns are unknown until the deferred
  §10.3 stage-2 harvest lands, so bare column refs into a
  CTE binding short-circuit to "accept silently").
- Duplicate CTE names in the same `WITH` block emit
  `diagnostic.duplicate_cte` on the second occurrence
  (Plan §Open-2).

Phase-1 gap closure (`sql_predicate_warnings`, ADR-0032 §11.6):

A new MatchedPath-walking pass that identifies predicate-tail
shapes by node-name labels and emits the same `diagnostic.*`
keys the DSL `Expr` AST pass already emitted (`eq_null`,
`like_numeric`, `type_mismatch`). Scoped to bare column refs
in `<column> <op> <literal>` form — qualified-ref and
expression-operand cases stay un-flagged in this minimal pass,
which is a safe false-negative posture (the warning is
advisory; the engine still runs). Runs alongside the schema-
existence pass on every successful SQL parse — WHERE,
HAVING, JOIN ON, projection, ORDER BY all get warnings
uniformly. Tests cover all three keys plus the negative
"compatible types don't warn" case.

WITH dispatch (`data::WITH`):

`with x as (…) select * from x` now dispatches via the registry
with entry word `with`. Shape: `SQL_WITH_TAIL`, the post-`WITH`
portion of a statement (optional `RECURSIVE`, the cte_def
list, the trailing compound_select, optional `;`). Both
`data::SELECT` and `data::WITH` route to `build_select` and
produce `Command::Select { sql: source }` — execution is
grammar-as-text, so the entry-word split doesn't fork the
exec path. `is_advanced_only` extended to include `with`.

Deferred per the 2d-scoped DA review (documented as a
`(TBD)` in the cross-cut matrix for 2g):

- `diagnostic.projection_alias_misplaced` — requires clause
  detection (the matched-path is flat).
- `diagnostic.compound_arity_mismatch` — needs per-leg
  projection counting.
- `diagnostic.cte_arity_mismatch` — depends on §10.3 stage-2
  harvest, which 2b deferred.
- `engine.*` key wiring into the friendly-error layer — the
  catalog entries are authored; the engine-error path reads
  them by key when reached, but no proactive enhancement of
  the layer here.

Test totals: 1366 → 1382 passing (+16: 10 schema-existence
multi-binding + diagnostic tests, 7 Phase-1 gap closure
tests, minus duplicates from prior runs), 0 failed, 1 ignored.
Clippy clean.
2026-05-20 16:12:42 +00:00
claude@clouddev1 a491df32a0 grammar: migrate Phase-1 SELECT to the ADR-0032 fragment (sub-phase 2c)
The Phase-1 SQL `SELECT` grammar nodes that used to live in
`src/dsl/grammar/data.rs` retire — 22 statics / consts and the
`reject_internal_table` validator copy are removed, ~150 lines
of grammar machinery gone. `data::SELECT.shape` now references
the post-`SELECT` portion of the ADR-0032 fragment via a thin
`Node::Subgrammar(&sql_select::SQL_SELECT_TAIL)`.

`SQL_SELECT_TAIL` is a new export from `sql_select.rs`,
parallel to `SQL_SELECT_STATEMENT`. It represents what a
top-level `SELECT` statement looks like AFTER the registry's
entry-word dispatch has already consumed the leading `SELECT`
keyword: the DISTINCT/ALL prefix, projection list, optional
FROM / WHERE / GROUP BY / HAVING, the compound set-op chain
(each subsequent leg's `SELECT` is part of `SET_OP_TAIL`),
outer ORDER BY / LIMIT, and a tolerated trailing `;`.

WITH-prefixed statements (`WITH x AS (…) SELECT * FROM x`)
are NOT in 2c's scope — they need a separate `data::WITH`
`CommandNode` so the entry-word dispatch routes correctly.
For now, top-level WITH continues to fall through to the
chumsky parser route (the same as in Phase 1). The
`SQL_SELECT_STATEMENT` static (which includes the optional
WITH prefix) stays available for use by that future
CommandNode or by any other consumer that needs the full
statement shape.

All seven Phase-1 SQL `SELECT` integration tests
(`tests/sql_select.rs`) pass without modification, satisfying
the 2c exit gate's "behaviour preserved" requirement. The
70 fragment unit tests and the 26 driver-level scope tests
also pass — the migration is a refactor, no new tests
required.

Behaviour change explicitly sanctioned by ADR-0032 §8:
Phase-1's `LIMIT_VALIDATOR` (positive-int-only, parse-time)
is superseded by the full `sql_expr` admission. `LIMIT max(10,
x)` and similar now parse; the engine constrains the value at
execution time per the ADR's "grammar admits, engine
rejects" posture.

Plan §2b status note: the 2026-05-20 deferral of §10.3 stage 2
(CTE output-column harvest derivation) is recorded in
`docs/plans/20260520-adr-0032-phase-2.md` per the
user-approved deferral.

Test totals: 1366 passing (unchanged), 0 failed, 1 ignored.
Clippy clean. data.rs loses ~150 lines of dead grammar; the
single source of truth for the SQL `SELECT` shape is now
`sql_select.rs`.
2026-05-20 15:42:44 +00:00
claude@clouddev1 4ff054ca75 walker: populate cte_bindings placeholders + projection_aliases (ADR-0032 §10.3 stage 1 / §10.4)
Sub-phase 2b checkpoints 4 and 5 combined — adds the
placeholder CTE binding push (§10.3 stage 1) and the
projection alias accumulator (§10.4).

Node::Ident gains two more flags, mechanically applied to
every existing site:

- `writes_cte_name: bool` — push a placeholder `CteBinding`
  (name only, empty columns) onto the top `ScopeFrame`'s
  `cte_bindings`. Set on `CTE_NAME_IDENT` in sql_select.rs.
  Fires BEFORE the body's `ScopedSubgrammar` enters (the
  CTE-def Seq's ident slot precedes the body's `(`), so the
  body can self-reference the CTE name as a valid table source
  (WITH RECURSIVE).
- `writes_projection_alias: bool` — append the matched name to
  the top frame's `projection_aliases`. Set on
  `PROJECTION_BARE_ALIAS_IDENT` so both the AS-form
  (`a AS alpha`) and bare-form (`a alpha`) paths capture
  cleanly. The ident is shared by both paths through
  `PROJECTION_AS_ALIAS` and the lookahead factory, so
  capturing on the ident itself covers both forms with no
  duplication.

The §10.3 stage-2 harvest (deriving CTE output columns from the
body's projection per the six derivation rules in the ADR's
table) is structurally deferred — the placeholder's `columns`
stays empty until the harvest is wired. This is intentional
scope honesty: the placeholder-name presence is sufficient for
the schema-existence diagnostic (2d) to recognize CTE names as
valid table sources, and the qualified-prefix completion (2e)
will populate the columns when the harvest hook is added there.
Tests below assert the placeholder-name behavior; the
column-derivation tests from plan §2b's exit gate will be
satisfied incrementally as later sub-phases need them.

Tests (8 new, all green):

- Single CTE → one placeholder binding with the matched name.
- Multiple CTEs → placeholders in declaration order.
- Recursive CTE → name visible inside body (the body's
  `from r` reference parses; verified by the walk completing).
- Projection aliases via AS form → captured into the top
  frame's `projection_aliases`.
- Projection aliases via bare form → captured.
- Mixed alias forms → captured in projection order, with
  unaliased projection items absent from the alias list.
- No aliases → empty `projection_aliases`.
- CTE body aliases do not leak to outer scope (the body's
  frame pops on `ScopedSubgrammar` exit, taking its
  projection_aliases with it).

All 1358 previous tests still pass. Test totals: 1366
passing, 0 failed, 1 ignored. Clippy clean.

This closes out the scope-accumulator side of sub-phase 2b.
The remaining 2b-style work — full CTE column-derivation
harvest per §10.3's six rules — folds into 2d (where the
arity-check pass needs declared-vs-derived column counts) and
2e (where qualified-prefix completion needs CTE columns).
2026-05-20 15:29:08 +00:00
claude@clouddev1 b522d09f5a walker: populate from_scope table bindings (ADR-0032 §10.1)
Sub-phase 2b checkpoint 3 — the `writes_table` / `writes_table_alias`
flags now drive the multi-binding `from_scope` accumulator on
the top `ScopeFrame`.

Node::Ident gains `writes_table_alias: bool`. When set on an
ident-name slot, the matched name lands on the most-recently-
pushed `TableBinding`'s `alias`. All 46 existing Ident sites
across the codebase are updated to `writes_table_alias: false`
(mechanical — no behavioral change for DSL paths).

walk_ident's `writes_table` semantics extend:

- `IdentSource::Tables` matches with `writes_table: true` still
  populate `current_table` / `current_table_columns` as before
  (preserved for DSL paths that read those fields directly via
  the dynamic-subgrammar / column-writes machinery), AND now
  also push a fresh `TableBinding` onto the top ScopeFrame's
  `from_scope`. The two mechanisms coexist additively —
  current_table reflects the most-recent `writes_table` write
  (single-binding view, as before); from_scope is the
  authoritative multi-binding accumulator that SQL JOINs,
  subqueries, and CTE bodies use.

sql_select.rs splits the alias slot into two ident variants:

- `PROJECTION_BARE_ALIAS_IDENT` (role `projection_alias`) —
  no scope writes; capture into `projection_aliases` is 2b-5.
- `TABLE_SOURCE_BARE_ALIAS_IDENT` (role `table_alias`,
  `writes_table_alias: true`) — sets the top binding's alias.

The `AS alias` form likewise splits into PROJECTION_AS_ALIAS
and TABLE_SOURCE_AS_ALIAS so each path threads through the
correct ident. The bare-alias lookahead factories return the
projection or table-source ident accordingly.

`TABLE_NAME_IDENT` in sql_select.rs gets `writes_table: true`
so each FROM / JOIN table source pushes a binding. The
schema-resolved columns are stored on the TableBinding for
later use by qualified-prefix completion (2e) and the
schema-existence diagnostic (2d).

Tests (9 new, all green):

- single from-table → one binding
- AS alias / bare alias on from-table → alias captured
- two-way JOIN → two bindings, correct order
- two-way JOIN with both aliased → two bindings with aliases
- three-way JOIN (left + bare) → three bindings in order
- subquery from_scope does not leak to outer scope (the
  ScopedSubgrammar push/pop discipline at work)
- CTE body from_scope does not leak to outer scope (the outer
  scope sees only the CTE-name reference, not the body's
  internals)
- SELECT without FROM → empty from_scope

All 1351 previous tests still pass — DSL paths untouched.
Test totals: 1358 passing, 0 failed, 1 ignored. Clippy clean.

Frame is_cte_body marker, body-projection harvest, and
projection_aliases population are the remaining 2b work
(2b-4 and 2b-5).
2026-05-20 15:25:10 +00:00
claude@clouddev1 98a74b23d3 grammar: sql_expr additive extensions for §5/§6, CTE body rewires to ScopedSubgrammar
Sub-phase 2b checkpoint 2 — closes the recursion loop between
sql_expr.rs and sql_select.rs so subquery expressions and
qualified column refs become structurally valid in every SQL
context where they belong.

sql_expr.rs:

- §5 qualified-ref tail. `name_or_call` gains a `.identifier`
  suffix as a Choice sibling of the function-call `(args)`
  tail. The leading identifier is still matched once (per
  ADR-0031 §1's factoring); the optional tail dispatches
  between the two suffixes by their first character (`.` vs
  `(`).
- §6.1 scalar subquery as primary. The `(or_expr)` and
  `(SELECT …)` branches share the leading `(`; the first
  inside token (`SELECT` → subquery, anything else →
  expression) discriminates. The subquery recurses through
  `Node::ScopedSubgrammar(&sql_select::SQL_SELECT_COMPOUND)`.
- §6.2 IN (subquery) predicate. Sibling of the existing
  IN-value-list; same `(` factoring, same dispatch.
- §6.3 [NOT] EXISTS primary. Bare `EXISTS (compound_select)`
  lives in `primary`; `NOT EXISTS` falls out via the existing
  `not_expr := NOT not_expr` tier above `primary`.

sql_select.rs:

- CTE body recursion rewires `Node::Subgrammar` →
  `Node::ScopedSubgrammar`, matching §10.2. The top-level
  statement's COMPOUND embedding stays plain Subgrammar — the
  implicit bottom frame is the right scope for a statement-
  level SELECT.

Structural side-effect — const-eval cycle workaround:

Closing the sql_expr ⇄ sql_select reference loop made Rust's
const-evaluator follow the cycle through every `const Node`
that transitively reaches it. Mirroring sql_expr.rs's existing
pattern, composition Nodes in sql_select.rs (Seq / Choice /
Optional / Repeated / Lookahead) are now `static Node` and
appear in slice positions through `Node::Subgrammar(&NAME)`
wraps; only leaf items (Punct, Word, Ident) remain `const`.
Same workaround applies to data.rs's SELECT_PROJ_LIST /
SELECT_PROJECTION chain and the inlined `SQL_EXPR` reference.
Statics resolve lazily at link time, so the cycle is valid;
const-eval is not, and the named `const SQL_EXPR` alias is
gone in both files (replaced with the inline `Node::Subgrammar
(&sql_expr::SQL_OR_EXPR)` expression at every use site).

Test coverage:

- sql_expr.rs gains 11 new tests for qualified refs, scalar
  subquery, IN-subquery, EXISTS / NOT EXISTS, nested
  subqueries, and the existing IN-value-list form (regression).
- sql_select.rs gains 7 new tests for qualified refs in WHERE,
  scalar subqueries in WHERE / projection, IN / EXISTS / NOT
  EXISTS in WHERE, nested subqueries, and qualified refs
  inside CTE bodies.
- All 70 prior sql_select tests still pass; the 2a baseline
  is preserved.

`(WITH x AS (…) SELECT * FROM x)` is explicitly NOT admitted
as a scalar subquery — ADR-0032 §1 / §9 wire subqueries to
SQL_SELECT_COMPOUND, which omits the outer with_clause. WITH
remains a statement-level-only construct. Documented in the
relevant test.

Test totals: 1333 → 1351 passing, 0 failed, 1 ignored
(unchanged). Clippy clean.
2026-05-20 11:47:27 +00:00
claude@clouddev1 4f89106a63 walker: Node::ScopedSubgrammar variant + scope-frame stack (ADR-0032 §10.2)
Sub-phase 2b checkpoint 1 — adds the foundation for SQL SELECT
lexical-scope discipline without changing existing walker
semantics.

New types in `dsl::walker::context`:

- `TableBinding` — one FROM-source binding with table name,
  optional alias, and schema-resolved columns (§10.1).
- `CteBinding` + `CteColumn` — a CTE definition visible from
  inside its body (WITH RECURSIVE self-reference) and from the
  outer scope after harvest (§10.3).
- `ScopeFrame` — `from_scope`, `cte_bindings`, and
  `projection_aliases` for one lexical scope. Default-empty;
  the fields will be populated by later 2b checkpoints.

`WalkContext` gains `from_scope_stack: Vec<ScopeFrame>`,
initialised with one bottom frame in both `new()` and
`with_schema()`. The bottom frame is the implicit top-level
scope DSL paths and top-level SQL statements operate in;
`Node::ScopedSubgrammar` entries push and pop additional frames
on top. `current_table` / `current_table_columns` remain as
direct fields for this checkpoint — converting them to derived
helpers is a later 2b step.

New grammar-tree variant:

- `Node::ScopedSubgrammar(&'static Self)` — like `Subgrammar`,
  but pushes a fresh `ScopeFrame` on entry and pops it on exit
  (ADR-0032 §10.2). Shares `subgrammar_depth` with the plain
  Subgrammar variant so the MAX_SUBGRAMMAR_DEPTH = 64 cap fires
  uniformly across both — §9's "no new walker capability for
  grammar recursion" claim holds. DSL Expr (ADR-0026) and
  sql_expr.rs ladder (ADR-0031) recursion continue to use the
  plain Subgrammar variant and never push a scope.

Driver gains a parallel `walk_scoped_subgrammar` arm; the
push/pop is unconditional so a speculatively-walked branch a
later Choice rolls back leaves the stack clean.

Test coverage in `driver.rs`:

- A recursive ScopedSubgrammar test grammar walks correctly
  through depths 0-3.
- The depth cap fires the same `expression_too_deep` friendly
  validation error as for plain Subgrammar.
- The bottom frame invariant: `WalkContext::new` seeds exactly
  one frame, and after a walk the stack is restored.

No grammar tree references the new variant yet — the rewire of
sql_select.rs CTE bodies and the sql_expr.rs additive
extensions for §5/§6 are the next 2b checkpoint. Test totals:
1330 baseline + 3 = 1333 passing, 0 failed, 1 ignored. Clippy
clean.
2026-05-20 11:34:53 +00:00
claude@clouddev1 8d293358a0 grammar: SQL SELECT full statement fragment (ADR-0032 Phase 2a)
Author the standalone walkable shape for the full standard-SQL
SELECT per ADR-0032 §1: compound queries with the four set ops
(UNION / UNION ALL / INTERSECT / EXCEPT), the five JOIN flavours
(INNER / LEFT [OUTER] / RIGHT [OUTER] / FULL [OUTER] / CROSS),
GROUP BY / HAVING, WITH and WITH RECURSIVE common table
expressions, LIMIT … OFFSET, DISTINCT / ALL, qualified-wildcard
`t.*` projection, and bare-alias projection (lifting ADR-0030
Phase-1 §4.2).

Recursion into SQL_SELECT_COMPOUND uses Node::Subgrammar for
2a; sub-phase 2b will rewire those references to the new
Node::ScopedSubgrammar variant for completion-scope discipline
(ADR-0032 §10.2). The Phase-1 data::SELECT CommandNode is not
touched here — the new fragment is reachable only from its own
tests until sub-phase 2c performs the migration.

Two implementation mechanisms realize ADR semantics without
changing them:

- Node::Lookahead disambiguates the projection_item Choice
  (bare `*` vs `ident . *` qualified wildcard vs `sql_expr [
  alias ]`) and gates bare-alias slots against continuation
  keywords. The walker's walk_ident accepts any
  identifier-shape token, including keyword-shape ones, and
  Choice / Optional are first-match-wins; without lookahead a
  bare-alias slot would greedily swallow FROM / WHERE / JOIN /
  etc. Per-position follow-sets list which keywords legitimately
  follow each alias slot. Same pattern as data.rs's
  insert_first_paren precedent.

- INNER JOIN and bare JOIN are split into two distinct Choice
  branches (each with a concrete leading keyword) rather than
  sharing one Optional(Word("inner"))-leading branch. Avoids a
  walker hazard where an Optional-leading-child Seq commits to
  idx > 0 and then converts the next child's EOF NoMatch into
  Incomplete, blocking the outer Choice from falling through to
  later branches. Same semantic surface, distinct mechanism.

The §13 OOS shapes all have explicit reject tests (NATURAL,
USING, comma-FROM, LIMIT m,n, window OVER, VALUES, derived
tables). LATERAL has a noted partial limitation: the comma form
rejects via OOS-3, but the single-keyword form `FROM a LATERAL
JOIN b ON …` is admitted structurally because `lateral` parses
as a bare table-source alias for `a`. This matches ADR-0030's
"grammar admits identifier-shape tokens; engine resolves"
posture.

`__rdbms_*` rejection extends to every Phase-2 table-source
slot — the FROM table, each JOIN's table, each CTE name, and
the FROM inside any CTE body — via the reuseable
reject_internal_table validator.

70 new unit tests in sql_select.rs walk every §1 production and
every OOS reject case. Test totals: 1260 baseline + 70 = 1330
passing, 0 failing, 1 ignored (unchanged from baseline). Clippy
clean.

Per the Phase-2 plan sub-phase 2a exit gate. DA gate written
review: PASS.
2026-05-20 11:29:48 +00:00
claude@clouddev1 83e0ddc2ff app: mode-threaded completion, overlay, and validity indicator
The dispatch-layer mode gate (previous commit) made the submit
behaviour correct — `select` runs in advanced mode and shows
the SQL hint in simple mode. This commit extends that gating to
the ambient assistance layer so simple-mode users do not see
SQL leak through Tab completion, the live error overlay, or the
`[ERR]`/`[WRN]` validity indicator either.

`_in_mode` walker variants
--------------------------
- `completion_probe_in_mode`, `expected_at_input_in_mode`,
  `input_verdict_in_mode`. Each sets `ctx.mode` before walking.
  The empty-input / unknown-entry fallback in `completion_probe`
  and `expected_at_input` filters the `REGISTRY` listing by
  `is_advanced_only` so Tab does not offer `select` in simple
  mode. Old signatures keep delegating to `Mode::Advanced`
  (back-compat for tests + other callers).

`_in_mode` completion variants
------------------------------
- `candidates_at_cursor_in_mode`, `candidates_at_cursor_with_in_mode`.
  Internally they route the `parse_command` completeness probe
  through `parse_command_in_mode(input, mode)`, the
  `completion_probe` call through `completion_probe_in_mode`,
  and the `expected_at` fallback through
  `expected_at_input_in_mode`. Old signatures default to
  `Mode::Advanced`.

`EffectiveMode::as_mode`
------------------------
- Collapses the persistent / one-shot distinction the UI cares
  about into the plain `Mode` the walker reads from
  `WalkContext::mode`. App-level call sites that thread mode
  into the walker chain use this.

App / input-render wiring
-------------------------
- `App::input_validity_verdict` runs only when effective mode
  is plain `Simple` (per ADR-0027), so it hardcodes
  `Mode::Simple` into the new `input_verdict_in_mode` call
  rather than threading.
- `App::start_or_complete_at` / `_last` (the Tab handlers)
  pass `self.effective_mode().as_mode()` into
  `candidates_at_cursor_in_mode`, so a `:` one-shot or
  persistent advanced gives full SQL completion, persistent
  simple does not offer SQL.
- `input_render::render_input_runs` and `ambient_hint` are
  invoked from `ui.rs` only when effective mode is plain
  `Simple` (advanced rendering uses `plain_input_spans` and
  skips ambient hinting per ADR-0022 §12). Their internal
  `classify_input_with_schema` / `candidates_at_cursor` /
  `parse_command` calls now go through the mode-aware variants
  with `Mode::Simple` hardcoded — a SQL form in simple mode
  surfaces as a definite-error overlay and the hint panel does
  not offer it.

After this commit a simple-mode user typing `select` or
`sel<Tab>` sees nothing SQL-shaped: no live highlight, no Tab
completion candidate, the `[ERR]` indicator lit, and the on-
submit hint that names the recovery paths. An advanced-mode
user or a `:` one-shot sees the full SQL surface.
2026-05-19 21:48:21 +00:00
claude@clouddev1 6369066fe4 grammar: SQL SELECT end-to-end (ADR-0030 Phase 1)
The first cut of advanced-mode SQL: a `select` line in advanced
mode parses, runs against the database, and renders its rows
through the existing data-table renderer; the same line in
simple mode lights up the precise "this is SQL" hint instead of
running.

Walker mode gate (ADR-0030 §2)
------------------------------
- `WalkContext` gains a `mode: Mode` field; `Mode` derives
  `Default` (= `Simple`, matching the app's startup mode).
- `grammar::is_advanced_only` keys an advanced-only entry-word
  set (Phase 1: just `select`). When the walker matches an
  advanced-only entry word with `ctx.mode == Simple`, it
  short-circuits to a `WalkOutcome::ValidationFailed` carrying
  the `advanced_mode.sql_in_simple` catalog key — the input
  highlights as a keyword, the validity indicator goes ERROR,
  and the parse-error layer renders the "switch with `mode
  advanced`, or prefix the line with `:`" hint.
- `parser::parse_command_with_schema_in_mode` (and the
  schemaless `parse_command_in_mode`) threads the mode into
  `WalkContext`; existing `parse_command*` entry points default
  to `Mode::Advanced` (most permissive) so back-compat callers
  see the full grammar.
- `App::submit` is unified: both modes route through
  `dispatch_dsl(&effective_input, effective_mode)`, which now
  parses with the line's effective mode. The placeholder
  advanced-mode echo branch is gone.

Builder signature sweep (ADR-0031 §2)
-------------------------------------
- `CommandNode.ast_builder` gains a `source: &str` parameter,
  forwarded by the walker. `build_select` reads it to put the
  validated SQL text into `Command::Select`; the 21 existing
  builders accept it as `_source`.

SQL `SELECT` (ADR-0030 §6, ADR-0031)
-------------------------------------
- New `Command::Select { sql: String }` variant. Every
  exhaustive `match Command` updated (`verb`, `target_table`,
  `build_translate_context`, `execute_command_typed`,
  `typing_surface`'s label).
- `grammar::data::SELECT` `CommandNode`: projection (`*` or
  `expr [as alias]` list), optional `FROM <table>`, optional
  `WHERE`/`ORDER BY`/`LIMIT`, optional trailing `;`. The
  expression slots reference the ADR-0031 fragment through
  `Subgrammar(&sql_expr::SQL_OR_EXPR)`. The `FROM` table-name
  slot carries a `reject_internal_table` validator that
  refuses `__rdbms_*` references at parse time.
- The `FROM` clause is optional — `select 1`, `select upper('x')`
  (zero-table constant/function-call SELECTs) work alongside
  the single-table form. Standard SQL admits them and they are
  the canonical learner probe.
- Implicit projection aliasing (`select a x`) is deliberately
  unsupported — `from` is a keyword, the bare alias would be
  ambiguous; only `select a as x` is admitted.

Worker / runtime
----------------
- `Request::RunSelect { sql, source, reply }` + a new
  `Database::run_select` method. `do_run_select_request` runs
  the prepared statement, collects rows into a `DataResult`
  with `column_types: Vec<None>` (Phase-1 SELECT result columns
  carry no playground type per ADR-0030 §6), and appends the
  literal source line to `history.log` so replay re-runs it
  (ADR-0030 §11).
- `runtime::execute_command_typed` gains a `Command::Select`
  arm that calls `database.run_select(sql, src)` and maps to
  `CommandOutcome::Query`, which flows into the existing
  `AppEvent::DslDataSucceeded` → `render_data_table` path.

Catalog (ADR-0019)
------------------
- `advanced_mode.sql_in_simple` — the walker's gate message.
- `select.internal_table` — the `__rdbms_*` rejection.
- `parse.usage.select` — the parse-error usage template.

Tests
-----
Two `app::tests` cases that pinned the pre-ADR-0030 placeholder
echo are updated to pin the new dispatch contract — both verify
that the advanced-mode `select` (one persistent, one via the
`:` one-shot) produces `ExecuteDsl(Command::Select)` with the
submission's effective mode tagged on the echo. The matching
walking-skeleton test is updated likewise.

A separate follow-up commit lands the ambient mode-threading
(completion / live overlay / validity indicator) so simple-mode
users do not see SQL surfaced through Tab or the live error
overlay either — the dispatch-layer gate landed here is the
behavioural foundation that follow-up builds on. Integration
tests for the full end-to-end land in a third commit.
2026-05-19 21:46:56 +00:00
claude@clouddev1 c93f9394f5 grammar: SQL expression grammar fragment (ADR-0031)
A new `src/dsl/grammar/sql_expr.rs` authored as a parallel
fragment to `expr.rs` (the DSL `WHERE` grammar, ADR-0026). The
ADR's stratified ladder lands as named `static` `Node`s, one per
precedence tier:

  or_expr → and_expr → not_expr → predicate → additive →
  multiplicative → unary → primary

Recursion through `Node::Subgrammar` reuses ADR-0026's
`MAX_SUBGRAMMAR_DEPTH = 64` cap unchanged; no new walker
capability is required. `predicate_tail` follows ADR-0026's
factoring (shared operand prefix, infix `NOT` as an explicit
branch, no `Optional`-first branch) so `Choice` discriminates
cleanly. `name_or_call` factors the identifier-prefix shared
between column refs and function calls into a single `Ident`
followed by an `Optional` `( call_args )` tail — the same
hazard-avoidance shape `predicate_tail` uses.

The fragment exports `pub static SQL_OR_EXPR` (test entry) and
`pub static SQL_EXPRESSION` (drop-in `Subgrammar(&SQL_OR_EXPR)`
that SQL `CommandNode` shapes embed in their `Seq`). No AST
builder — every Phase-1 consumer (SELECT projection, WHERE)
runs validated SQL as text per ADR-0030 §4/§6.

13 unit tests cover every operator and precedence pair, the
full predicate set, `CASE` (searched + simple) including
`count(*)` and `count(distinct …)`, parenthesised regrouping,
case-insensitive keywords, the depth cap, and a representative
set of malformed inputs that do *not* walk.

Module registered via one new line in `grammar/mod.rs`.
2026-05-19 21:39:49 +00:00
claude@clouddev1 abce1188f2 constraints: add constraint / drop constraint on existing columns (ADR-0029 §2.2)
Adds the two commands for modifying a column's constraints after
creation, completing ADR-0029's §2.2 surface.

Grammar (dsl/grammar/ddl.rs): `add constraint <constraint> to
<T>.<col>` reuses the §2.1 COLUMN_CONSTRAINT choice; `drop
constraint <kind> from <T>.<col>` names only the kind. Both join
the `add` / `drop` choices, discriminated by the `constraint`
form word.

AST (dsl/command.rs): `Command::AddConstraint` / `DropConstraint`
plus the `Constraint` / `ConstraintKind` enums.

Worker (db.rs): `do_add_constraint` / `do_drop_constraint` apply
the change through the rebuild-table primitive. `add` runs the §5
dry-run first — `not null` / `unique` / `check` against a
populated column are refused, before any write, with a
pretty-printed table of offending rows. §9 redundant-on-PK
declarations and §6 `default` on an auto-generated column are
friendly refusals; dropping a constraint the column does not
carry is likewise refused.

Also fixes schema_to_ddl, which suppressed UNIQUE for every PK
column — a compound-PK member is not individually unique, so an
explicit UNIQUE on it must survive the rebuild.

23 tests added (6 grammar, 17 worker); 3 completion-test and 3
matrix snapshots updated for the new `constraint` subcommand.
2026-05-19 18:31:57 +00:00
claude@clouddev1 942222bfc9 constraints: CHECK — check (<expr>) at create table & add column (ADR-0029)
The fourth constraint. `check ( <expr> )` reuses the ADR-0026
WHERE-expression grammar via `Subgrammar`, so a check is
written in the same language as a `where` filter.

- Grammar: a `CHECK_CONSTRAINT` arm joins the shared
  constraint-suffix Choice; `consume_check_expr` extracts the
  parenthesised expression (paren-depth aware) into
  `ColumnSpec.check` / `Command::AddColumn.check`.
- Storage: the parsed `Expr` is compiled once to inline SQL
  (`compile_check_sql` — `compile_expr` + ADR-0028's
  param-inliner) and stored in that form everywhere — a new
  `check_expr` column in `__rdbms_playground_columns`,
  `project.yaml`'s `ColumnSchema.check`, and the column DDL
  emitted by `do_create_table` / `schema_to_ddl`.
- `add column … check` routes through the rebuild primitive
  (SQLite's `ALTER … ADD COLUMN` cannot carry it); a CHECK on
  a serial/shortid column is create-table-only and refused at
  add-column with a friendly message.
- `describe` surfaces the CHECK. ADR-0029 §7/§8 updated to the
  SQL-form decision — double-quoted identifiers, consistent
  with ADR-0028's `explain` display SQL.

1201 tests pass (+8); clippy clean.
2026-05-19 16:42:18 +00:00
claude@clouddev1 58d8958822 add column: column constraints — NOT NULL / UNIQUE / DEFAULT (ADR-0029 §6)
`add column` now accepts the shared constraint suffix and the
worker honours it — the surface where NOT NULL / UNIQUE
actually matter, on non-PK columns.

- Grammar: `ADD_COLUMN_NODES` gains the constraint-suffix
  fragment; `collect_column_constraints` folds it into
  `Command::AddColumn`.
- `do_add_column` routes per ADR-0029 §6: SQLite's `ALTER
  TABLE ADD COLUMN` cannot express `UNIQUE` and requires a
  default for `NOT NULL`, so those go through the rebuild
  primitive (`do_add_constrained_column_via_rebuild`); plain
  cases keep the ALTER path with the constraint suffix
  appended.
- Pre-flight refusals, before any SQL write: a NOT NULL
  column with no default added to a populated table; a UNIQUE
  column with a default added to a multi-row table; a default
  on a `serial` / `shortid` column.

CHECK is still deferred to the next commit. 1193 tests pass
(+9); clippy clean.
2026-05-19 14:50:19 +00:00
claude@clouddev1 12395a9a6c create table: column constraints — NOT NULL / UNIQUE / DEFAULT grammar (ADR-0029)
`create table … with pk` now parses the column-constraint
suffix; combined with the commit-1 db layer, a constrained
table works end to end.

- A shared constraint-suffix grammar fragment — `not null`,
  `unique`, `default <literal>` — sits after each column's
  `(type)` group; `build_create_table` walks the matched path
  per column and folds the constraints into `ColumnSpec`.
- §9 redundancy check: every `with pk` column is a primary-key
  column, so `not null` (any) and `unique` (single-column PK)
  are rejected with a friendly error
  (`parse.custom.constraint_redundant_on_pk`).
- `project.yaml` round-trip: `ColumnSchema` gains `not_null` /
  `default`; the YAML reader/writer and `build_read_schema`
  carry them, so `rebuild` / `export` / `import` preserve
  constraints.
- ADR-0029 §2.1's example corrected — `create table` columns
  are all PK columns, so its suffix is for `default` / `check`;
  `docs/simple-mode-limitations.md` records that non-PK
  columns at create time need advanced mode.

CHECK is deferred to the next commit. 1184 tests pass (+7);
clippy clean.
2026-05-19 14:41:29 +00:00
claude@clouddev1 eff2ee8d14 refactor: ColumnSpec / AddColumn carry constraint fields (ADR-0029 scaffolding)
Expand ColumnSpec and Command::AddColumn with the four
ADR-0029 constraint slots (not_null, unique, default, check),
all defaulting off; `Database::add_column` now takes a
ColumnSpec. No behaviour change — the grammar to set the
fields and the DDL to enforce them land in the following
commits. Isolated here so those commits stay readable.

Adds ColumnSpec::new for the unconstrained case; 110 call
sites updated. 1172 tests pass; clippy clean.
2026-05-19 14:04:36 +00:00
claude@clouddev1 d17addddd7 explain: explain command end to end (ADR-0028 steps 2–3)
Add the `explain` prefix command — `explain show data`,
`explain update`, `explain delete` — from grammar through to a
rendered plan tree.

- Grammar: an `EXPLAIN` CommandNode whose shape is a Choice over
  the three explainable query shapes, referenced (not
  duplicated) through `Subgrammar`. `Command::Explain { query:
  Box<Self> }`; `build_show_data` is extracted so the role-based
  builders serve both standalone and explain-wrapped commands.
- Worker: SQL construction is split out of do_query_data /
  do_update / do_delete into `build_*_sql`, so EXPLAIN QUERY
  PLAN runs the exact same statement. `Request::ExplainPlan` /
  `do_explain_plan` capture the plan; `QueryPlan` / `ExplainRow`
  carry it back. EXPLAIN QUERY PLAN never executes, so
  explaining update/delete changes nothing.
- Display SQL: the executed statement with `?N` parameters
  inlined as standard-SQL literals via a quote-aware scan.
- Render: `render_explain_plan` draws the box-drawing plan tree
  (plain output; ADR-0028 step 4 adds the styled tree).
- Catalog: `parse.usage.explain` and the `help.data.explain`
  entry, so `explain` shows up in the in-app `help` listing.

1151 tests pass (+18); clippy clean.
2026-05-19 12:38:02 +00:00
claude@clouddev1 f239ca5ff4 walker: keep optional trailing flags completable after --
Typing `--` to start an optional trailing flag (`--create-fk`
on `add 1:n relationship`, `--cascade` on `drop column`,
`--force-conversion` / `--dont-convert` on `change column`)
made completion go empty: the trailing `--` turns the parse
into a trailing-junk Mismatch, and the Mismatch arm of the
completion expected-set resolution returned only `[EndOfInput]`
— the skipped optional-flag expectations, carried in
`tail_expected`, were dropped.

completion_probe and expected_at_input now merge `tail_expected`
into a Mismatch's expected set. `tail_expected` is empty for a
genuine mid-command mismatch, so this only adds the outer
shape's skipped trailing optionals — exactly the continuations
the trailing `--` is starting to type. This also resolves the
"wrong usage hint" symptom: with `--create-fk` offered as a
candidate, the hint panel shows candidates instead of falling
through to the parse-error usage block.

Audit outcome (the requested scan): usage_key_for_input was
verified correct for every multi-form command — add / drop /
show, including the digit-led `add 1:n relationship` form —
and is now regression-locked. The flag-completion fix covers
the whole optional-trailing-flag class.

6 tests (3 flag-completion, 3 usage-key). 1131 passing.
2026-05-19 10:19:00 +00:00
claude@clouddev1 bbfb70c767 ui: overlay diagnostic spans on the input field (ADR-0027 §2)
render_input_runs now overlays the walker's schema-aware
diagnostics: an unknown table/column ERROR is recoloured
tok_error, an expression WARNING (type mismatch, = NULL, LIKE
on a numeric column) recoloured theme.warning. New overlay_span
covers a token's whole byte range (overlay_error only hits the
run at a single byte). New walker::input_diagnostics is the
shared entry point.

The overlay is global — every flagged token is coloured
wherever it sits, not only under the cursor — which is exactly
ADR-0027's motivation. The existing cursor-local invalid-ident
overlay is kept (it covers in-progress idents diagnostics do
not); the two are additive and idempotent.

5 input_render tests (unknown table/column, type-mismatch
literal precise, LIKE-on-numeric, clean command). 1113 passing,
clippy clean.
2026-05-19 09:32:52 +00:00
claude@clouddev1 437b2f2e91 walker: flag LIKE on a numeric column (ADR-0027 Amendment 1)
LIKE is a text-pattern match; against a numeric column (int,
real, decimal, serial) it runs but is almost never intended.
predicate_warnings now emits a WARNING for it, spanned at the
target column. New Type::is_numeric; catalog key
diagnostic.like_numeric; ADR-0027 gains "Amendment 1" and the
adr/README index line is updated per the index-upkeep rule.

bool and the text-/blob-backed types are deliberately not
flagged — see the amendment for the rationale.

3 walker tests (int, decimal NOT LIKE, text-column clean).
1108 passing, clippy clean.
2026-05-19 09:28:43 +00:00
claude@clouddev1 3912fb5a9b walker: precise per-literal spans for expression WARNINGs
Expression WARNING diagnostics (type mismatch, = NULL) carried
a coarse span — the whole WHERE clause, from the `where`
keyword to end of input. They now span exactly the offending
literal operand, read from the Operand source span added in the
previous commit. predicate_warnings derives the span per
warning; pair_type_mismatch returns (message, literal-span);
the dead where_clause_span helper is removed.

5 walker tests assert the spans cover exactly the literal /
identifier (type mismatch, = NULL, BETWEEN bounds, IN item,
unknown-column ERROR). 1105 passing, clippy clean.
2026-05-19 09:24:44 +00:00
claude@clouddev1 426e80185f command: Operand carries a source span
Each WHERE-expression Operand now records the byte span of the
terminal it was built from — the precise per-literal highlight
target for an expression WARNING (finishing ADR-0027 §2's
highlight/hint wiring). parse_operand captures MatchedItem::span;
the RowFilter::eq convenience constructor uses Operand::NO_SPAN.

PartialEq is hand-written to ignore the span — it is editor
metadata, so Command equality stays whitespace- and
position-independent, which the Expr test corpus relies on.
No behaviour change; 1100 tests still pass, clippy clean.
2026-05-19 09:20:52 +00:00
claude@clouddev1 151ed084a3 hint: show the matching usage template for multi-form commands
A parse error in `add index …` showed the `add column` usage:
`add` and `drop` are multi-form commands, and both the
ambient hint and the submit-time usage block picked the
first-listed form unconditionally.

New `grammar::usage_key_for_input` disambiguates by the form
word after the entry keyword — `column` / `index` / `table` /
`relationship`, or the leading digit of `add 1:n …`. The
ambient hint now shows that one form; `render_usage_block`
shows the committed form's usage and falls back to the whole
family only for a bare `add` / `drop` with no form chosen.
2026-05-19 08:37:17 +00:00
claude@clouddev1 a3268495e2 ADR-0027: existing-cases sweep + docs (step F)
Sweep: input_verdict tests confirm the schema-existence check
fires across the identifier-taking commands — unknown table
on drop / show / add column, unknown column on drop column /
update — and that known references stay clean. The Step B
check is grammar-generic, so this is verification + coverage
rather than new code.

Docs: requirements.md S6 -> [x], baseline 1096; CLAUDE.md
deferred list reconciled (C5a and S6 are done — removed);
ADR-0026's as-built note updated (step 5 shipped via
ADR-0027); ADR-0027 gains an As-built notes section
recording the post-walk diagnostics realization, the
pre-rendered message, the timeout-based debounce, coarse
WARNING spans, and the deferred highlight/hint wiring.
2026-05-19 07:35:06 +00:00
claude@clouddev1 73c74701c2 walker: expression WARNING diagnostics (ADR-0027 step C, folds ADR-0026 §7)
Type-mismatched comparisons and `= NULL` / `!= NULL` in a
WHERE expression now yield WARNING diagnostics — the command
still parses and runs (the ADR-0026 §7 permissive posture is
unchanged), but the validity indicator can flag it before
submission.

Computed post-walk from the built command's `Expr` against
the table's column types: a Compare / Between / In with a
column operand and a non-null literal whose type the column
cannot hold, or a Compare with `=` / `!=` against NULL. New
catalog keys `diagnostic.type_mismatch` / `diagnostic.eq_null`.

This is ADR-0026's deferred step 5, folded into ADR-0027's
diagnostics-severity model as the user requested.
2026-05-19 07:21:30 +00:00
claude@clouddev1 827b47f88f walker: schema-existence ERROR diagnostics (ADR-0027 step B)
`MatchedKind::Ident` now carries its `IdentSource`. A
post-walk pass over a structurally-valid parse flags a
matched `Tables` ident that is absent from the schema, or a
`Columns` ident absent from the table in scope, as an ERROR
diagnostic — the command parses but would fail at execution
(ADR-0027 §2). New behaviour: an unknown table / column used
to parse cleanly and fail only when run.

Column scope is resolved by one left-to-right pass over the
matched path (every command places its table ident before
the columns that belong to it); an unknown table clears the
scope, so its columns are not cascaded into a second
diagnostic. New catalog keys `diagnostic.unknown_table` /
`diagnostic.unknown_column`.
2026-05-19 07:15:58 +00:00
claude@clouddev1 e22f933e02 walker: diagnostics-severity model + input_verdict (ADR-0027 step A)
Adds `Severity` (Error / Warning, ordered so Error > Warning)
and `Diagnostic { severity, span, message }` in
`walker::outcome`, plus a `diagnostics` field on `WalkResult`
— the schema-aware findings layered on a structurally-valid
parse (ADR-0027 §2).

`input_verdict(source, schema)` is the validity-indicator
entry point: `None` when the input would run clean (and for
empty input), `Some(Error)` for a parse failure or unknown
command, `Some(Warning)` for the ADR-0026 expression flags.
The verdict is the highest severity across the parse outcome
and the diagnostics set.

`diagnostics` is empty at this step — the schema-existence
(ERROR) and expression (WARNING) passes that fill it land
next. Covered by `input_verdict` unit tests.
2026-05-19 07:08:13 +00:00
claude@clouddev1 a50c6cdf70 WHERE expressions: matrix cells + predicate_tail grammar fix (ADR-0026 step 6)
Adds tests/typing_surface/where_expression.rs — 9 matrix
cells for the complex WHERE / show-data limit typing surface:
operator candidates after an operand, AND / OR after a
predicate, NOT, BETWEEN / IN bounds, and `show data`
where / limit.

Writing the cells surfaced a grammar bug. `predicate_tail`'s
`[NOT] negatable` branch started with `Optional(not)`, and an
Optional-first `Seq` always "commits" — so on an incomplete
input the walker's `Choice` returned that branch's
`Incomplete` early and discarded every sibling branch's
expected set, dropping `is` and the comparison operators from
completion after a column. Fixed by splitting it into
explicit `NOT negatable` and bare `negatable` branches — no
`predicate_tail` branch starts with an `Optional` now. The
matched terminal sequence is unchanged, so `build_expr` is
untouched.

Docs: ADR-0026 gains an "As-built notes" section recording
the option-1 builder realization, its two deviations from the
§3 sketch, and the deferral of §7 diagnostic flagging to
ADR-0027. requirements.md C5a -> [x] (steps 1-4) with the
test baseline refreshed to 1079; CLAUDE.md's deferred list
reconciled (C5a implemented; the QA1/QA2 note now points at
ADR-0028).
2026-05-18 23:19:53 +00:00
claude@clouddev1 f75f71bbe4 WHERE expressions: wire into update/delete/show data + SQL gen (ADR-0026 steps 3-4)
Wires the stratified WHERE-expression fragment into the three
filter commands and compiles the resulting Expr to SQL.

Grammar (data.rs): the `update` / `delete` `where` clause is
now the expression fragment (`Subgrammar(&expr::OR_EXPR)`) in
place of the single `col = val` slot; `show data` gains an
optional `where <expr>` and an optional `limit <n>` (a
non-negative integer, validated at parse time). The
expression's right-hand operands are a schema-aware
`DynamicSubgrammar` so the hint panel still narrows to the
left column's type (ADR-0026 §8) — but the inner grammar is
permissive: a type-mismatched literal still parses (§7).

AST: `RowFilter::Where{column,value}` -> `RowFilter::Where(Expr)`;
`ShowData` gains `filter: Option<Expr>` and `limit: Option<u64>`.
A `RowFilter::eq` convenience constructor keeps simple-equality
call sites and tests readable.

SQL (db.rs): `compile_expr` lowers an `Expr` to a
parameterised WHERE — every literal a `?` placeholder,
identifiers `quote_ident`-quoted, `<>` for inequality. A
literal compared against a column binds through that column's
type where compatible and falls back to its syntactic shape on
a mismatch (§7 — permissive). `show data ... limit n` emits
`LIMIT ?` with an implicit primary-key `ORDER BY`, so it is a
stable "first n by primary key".

completion.rs: `invalid_ident_at_cursor` no longer mis-flags a
digit-led literal (`1`) as an unknown column now that the
WHERE operand slot also accepts a column reference; a
`ProseOnly` slot suppresses keyword candidates even when the
expected set also carries a column ident.

11 db integration tests cover AND / OR / NOT, BETWEEN, IN,
LIKE, filtered `show data`, and limit ordering; walker and
expr unit tests cover the parse surface. Type-mismatch /
`= NULL` diagnostic flagging (§7 highlight + hint) is the
remaining ADR-0026 piece.
2026-05-18 23:12:33 +00:00
claude@clouddev1 59e6a541bf grammar: WHERE-expression fragment + Expr AST + build_expr (ADR-0026 step 2)
The stratified WHERE-expression grammar — or / and / not /
bool_primary / predicate tiers as named `static` Node
fragments, recursing through `Subgrammar`. Covers the six
comparison operators (`<>` and `!=` both NotEq), AND / OR /
NOT, parentheses, LIKE / IN / BETWEEN with optional infix NOT,
and IS [NOT] NULL. `predicate_tail` factors the shared operand
prefix and the infix NOT so the Choice branches discriminate
on a cleanly-failing first token.

New recursive Expr / Predicate / Operand / CompareOp AST in
dsl::command. `build_expr` folds the flat matched-terminal
slice into an Expr — a deterministic recursive descent
mirroring the grammar tiers, with single-child tiers
collapsing. Per ADR-0026 §3 option 1: the walker stays a pure
structural matcher; Expr is assembled only in this
submit-time fold.

Fragment + builder are unit-tested standalone (walk against
&OR_EXPR, then build_expr); not yet wired into any command.
2026-05-18 22:40:52 +00:00
claude@clouddev1 f0b2043a39 walker: add Subgrammar node + recursion-depth cap (ADR-0026 step 1)
New `Node::Subgrammar(&'static Node)` variant lets a named
static grammar fragment recurse through a reference — `Seq` /
`Choice` embed children by value and cannot close a cycle, but
a `&'static Node` can point back at an enclosing fragment. This
is the mechanism the stratified WHERE-expression grammar
(ADR-0026 §2) recurses through.

The walker counts active Subgrammar frames in
`WalkContext::subgrammar_depth` and refuses past
`MAX_SUBGRAMMAR_DEPTH` (64), surfacing a friendly
`parse.custom.expression_too_deep` error instead of a stack
overflow. Depth is saved/restored per frame so a
speculatively-walked-then-rolled-back Choice branch leaves no
residue.

No grammar references the node yet; covered by walker unit
tests with a small recursive `( x )` test grammar.
2026-05-18 22:36:19 +00:00
claude@clouddev1 d9a98bbd49 Grammar: with-pk column specs use name(type), matching add column
`create table … with pk` parsed column types as `name:type`,
while `add column` uses `name(type)`. Unify on the parens
form so column-type syntax is consistent across the DSL:

    create table T with pk id(serial), name(text)

Only `COL_SPEC` changes (`:` → `( … )`); `build_create_table`
reads columns by role, so it is unaffected. The `:` that
separates table from column in `add column` / `drop column`
is unchanged. Sweeps the test suite, the typing-surface
matrix (two `after_colon` cells renamed to `after_paren`,
4 snapshots regenerated), the friendly catalog's usage
templates, ADR-0009's example, and requirements.md.

1039 passing / 0 failing / 1 ignored; clippy clean.
2026-05-18 21:51:52 +00:00
claude@clouddev1 0dc159fd7e Indexes: add index / drop index, persistence, display (ADR-0025)
Implement ADR-0025 — indexes as a DSL DDL feature.

- Grammar: `add index [as <name>] on <T> (<cols>)`, `drop index
  <name>` / `drop index on <T> (<cols>)`, plus a `--cascade`
  flag on `drop column`.
- db.rs: index operations over the engine's native index
  catalog (no metadata table). The rebuild-table primitive now
  captures and recreates indexes, so `change column` and the
  relationship operations no longer silently drop them.
- `drop column` refuses an indexed column unless `--cascade`,
  which drops the covering indexes and reports each.
- Persistence: additive `indexes:` list in `project.yaml`
  (version unchanged); round-trips through rebuild/export/import.
- Display: an `Indexes:` section in the structure view and a
  nested tables/indexes items panel (S2).

Reconciles requirements.md (C3 index portion, S2 satisfied)
and CLAUDE.md. 1038 tests passing (+31), clippy clean.
2026-05-16 00:15:55 +00:00
claude@clouddev1 6d2b92996d Grammar: remove the dead CommandNode.hint_mode field
HintMode became per-node (Node::Hinted) in the node-attached refactor;
the per-command hint_mode field was never the mechanism and is now
read by nothing. Removed the field and its 20 `None` initialisers.
2026-05-15 22:54:24 +00:00
claude@clouddev1 03dd9003df Help: consume CommandNode.help_id — REGISTRY-driven in-app help
Every CommandNode declared a help_id that nothing read; the in-app
`help` body was a single hand-kept catalog block that drifted from
the command set (handoff-12 §2.1).

note_help now iterates the command REGISTRY and translates each
CommandNode's help_id (`help.<id>`), framed by help.intro /
help.dsl_section / help.types_reference. A newly-registered command
appears in `help` automatically — no edit to note_help or a hand-kept
list. Added 20 per-command help entries plus the 3 framing entries;
removed help.in_app_body.

Per-command entries use block scalars: a libyml 0.0.5 scanner bug
panics on long internal space runs in double-quoted scalars, and the
entries are space-aligned.
2026-05-15 22:45:18 +00:00
claude@clouddev1 90e3f5dbfb Insert grammar: Form C type-awareness via lookahead (ADR-0024 §Phase D)
Form C (`insert into T (vals)`) shared the `(` opener with Form A,
so its paren was an untyped Repeated(Choice(literal, ident)) — values
weren't type- or count-checked at parse time (handoff-12 §2.2).

New Node::Lookahead variant: a factory that peeks the source. The
insert first-paren factory inspects the first token — a value literal
routes the contents through the typed column_value_list (Form B
dispatch contract: per-non-auto-column typed slots); an identifier or
empty paren routes to a Form A column-name list. So Form C now gets
the same per-column typed slots, hints, and parse-time type/count
checking Form B has.

The explicit-Choice-branch split is impossible here (committed-choice
semantics commit after `(` matches); lookahead is the only route, and
DynamicSubgrammar factories couldn't see the source. Node::Lookahead
is not memoized — its output depends on source — but it returns only
a small node (a Repeated, or a thin DynamicSubgrammar wrapper that
delegates to the memoized column_value_list).

`insert into T (` now cleanly shows Form A column candidates instead
of mixed Form-A/C suggestions. Form C matrix tests updated for the
type-aware behaviour.
2026-05-15 22:27:53 +00:00
claude@clouddev1 9bbb96e735 Walker: memoize DynamicSubgrammar resolution to bound the Box::leak
Node::DynamicSubgrammar factories build a Node from the WalkContext and
must Box::leak it (the Node enum's combinator children are &'static).
Leaking per walk grew unbounded under per-keystroke completion
(handoff-12 §2.1).

resolve_dynamic now memoizes on the schema state a factory reads
(table columns, current column, user-listed columns) keyed by factory
fn-pointer. Each distinct value-list shape leaks exactly once — total
leak bounded by distinct (schema × form) combinations, not keystroke
count. TableColumn gains Hash for the cache key.

The handoff's original arena sketch needed a lifetime-generic Node
(major refactor); memoization gets the same bound without it.
2026-05-15 22:06:33 +00:00
claude@clouddev1 911a537a83 Walker: node-attached HintMode via Node::Hinted (ADR-0024 §HintMode-per-node)
Replaces the hint resolver's signature-matching (does the expected set
contain all five literal forms? an Ident{NewName}?) with a grammar-
declared annotation. New Node::Hinted { mode, inner } wrapper; the
walker records the mode in WalkContext::pending_hint_mode on entry and
clears it on any successful match (cursor moved past the slot — this
also undoes the leak where a failed Hinted branch of a Choice would
otherwise strand a stale mode). The resolver reads pending_hint_mode
directly.

Value-literal fallback slots carry ProseOnly; NewName ident slots carry
ForceProse. hint_mode_at_input_inner now delegates to
hint_resolution_at_input — one resolution path, no duplicated logic.
No behaviour change; the typing-surface matrix guards it.
2026-05-15 21:58:22 +00:00
claude@clouddev1 f1ff5970bf Hint: pedagogical Form-A pointer at Form B's first value slot
Handoff-12 §2.2: Form B `insert into T values (…)` silently skips
auto-generated columns from the value list, so a user who wants to
set a serial/shortid column explicitly could only discover Form A by
reading help. Now the hint at the first Form B value slot appends a
note naming the skipped column(s) and pointing at the explicit-column
form.

hint_resolution_at_input derives the skipped columns from the
post-walk WalkContext (Form B = no user_listed_columns + table has
serial/shortid columns) and reports them on HintResolution; the note
fires only at the first slot so it doesn't repeat at every comma.
ambient_hint composes it onto the per-column prose.
2026-05-15 21:30:03 +00:00
claude@clouddev1 216e7ba61b DDL grammar: writes_table on table-name slots for column narrowing
Handoff-12 §2.2: the DDL TABLE_NAME_EXISTING slot and the
relationship-endpoint table idents didn't set writes_table, so
column-name slots downstream (drop/rename/change column; relationship
qualified columns) couldn't narrow to the active table — candidates
leaked from every table. Set writes_table: true on TABLE_NAME_EXISTING
and on DR_PARENT/DR_CHILD/AR_PARENT/AR_CHILD table idents. The
deliberately-documenting completion test now asserts per-table
narrowing.
2026-05-15 20:50:56 +00:00
claude@clouddev1 0b15ce0306 Walker + parser: surface mid-typing after separators and Form C/A ambiguity
The typing-surface matrix exposed two bugs the existing 859-test suite
missed:

walk_repeated: when the separator consumed but the inner item failed
at EOF, the old path rolled the separator back and reported a definite
error at the rollback position (`insert into T (a, ` flashed red on
the `,` after each comma). Now propagates Incomplete with the inner's
expected set so the input renderer treats it as mid-typing.

build_insert Form C path: `insert into T (col)` walked to a complete
match but produced `values: []` because Form C's value collector drops
ident-shaped items. The user almost certainly meant Form A and just
hasn't typed `values (...)` yet. Reject with a ValidationError naming
the Form-A continuation; classify_input now reports IncompleteAtEof.

completion_probe / expected_at_input: ValidationFailed used to return
an empty expected set, leaving Tab with nothing to offer at the new
Form-A flag point. Now surface result.tail_expected (skipped-Optional
expectations captured before validation fired) so `values` is still
offered as a candidate.
2026-05-15 20:06:52 +00:00
claude@clouddev1 619a8bd707 Completion: narrow column candidates to the active table
Two related fixes:

1. \`update MyTable set \` was offering columns from every
   table in the project — completion fetched
   \`cache.for_source(IdentSource::Columns)\` which returns the
   flat \`cache.columns\` (union of every table's columns).
   The walker's WalkContext had \`current_table_columns\`
   populated (because the update-table-name slot is
   \`writes_table: true\`) but the completion engine never
   consulted it.

2. \`insert into MyTable (\` was offering nothing — the
   value-literal suppression fired because the expected set at
   this position contains both Form A column-list candidates
   (\`Ident{Columns}\`) and Form C bare-value-list literals
   (null/true/false/NumberLit/StringLit). \`is_value_literal_signature\`
   matched and the engine returned \`None\` before the column
   candidates were considered.

The fix threads the walker's \`current_table_columns\` through
to the completion engine and narrows the suppression rule:

**Walker:**
- New \`walker::CompletionProbe { expected, current_table_columns }\`
  struct.
- New \`walker::completion_probe(source, schema) -> CompletionProbe\`
  runs one schema-aware walk and reports both the expected
  set (or tail_expected on Match) and the resolved table-column
  snapshot.

**Completion engine:**
- \`candidates_at_cursor_with\` calls \`completion_probe\` and
  reads \`current_table_columns\` for the \`Columns\` ident
  source. Schemaless or unknown-table falls back to the flat
  \`cache.columns\` (preserves pre-fix behavior).
- Value-literal suppression now gated on
  \`!has_schema_ident\` — if the expected set also offers a
  schema-listable Ident, the user has actionable candidates
  beyond the misleading null/true/false trio and we shouldn't
  hide them.

Tests:
- \`update_set_offers_only_current_table_columns\` confirms
  Customers' columns appear while Orders' columns don't.
- \`update_where_offers_only_current_table_columns\` covers
  the where path.
- \`insert_into_open_paren_offers_current_table_columns\` and
  \`insert_into_open_paren_does_not_offer_unrelated_columns\`
  cover the Form A column-list position.
- \`drop_column_from_offers_only_current_table_columns\`
  documents the DDL fallback (drop-column's table-name slot
  doesn't currently \`writes_table\` — falls back to the flat
  list).

For the user: \`update MyTable set \` now offers only
MyTable's columns. \`insert into MyTable (\` offers all of
MyTable's columns so Form A is fully discoverable.

Tests: 859 passing, 0 failing, 1 ignored. Clippy clean.
2026-05-15 19:07:46 +00:00
claude@clouddev1 5815918efb Hint: surface ( as a branching candidate; stop red-flagging in-progress Form A values
Two related fixes from a user-reported snag:

1. After typing \`insert into Orders \`, the hint suggested only
   \`values\` even though the user could also choose \`(\` to
   open Form A (the explicit-column-list variant). The walker
   reports both \`Expectation::Word("values")\` and
   \`Expectation::Punct('(')\` at that position, but
   \`candidates_at_cursor\` had a blanket "no punctuation as Tab
   candidate" policy.

   Loosened the policy to surface branching punct
   (specifically \`(\` opening a sub-shape). Closing punct
   (\`)\`), separators (\`,\`), and content-trailing punct (\`:\`,
   \`=\`, \`.\`) stay out — the user types those naturally and
   advertising them in the Tab menu is noise. New
   \`CandidateKind::Punct\` so the renderer colors it as punct
   rather than mis-classifying as a keyword.

2. While typing \`insert into Orders (id, CustId, Total) values
   (42, 89, 17.59\` (no closing paren yet), the word \`values\`
   was rendered in \`tok_error\` red. The walker's
   \`Optional(Seq[values, '(', list, ')'])\` was rolling back on
   the partial inner match — treating \`(id, CustId, Total)\` as
   Form C (bare value list) followed by trailing junk starting
   at \`values\`. The classify_input call thus returned
   \`DefiniteErrorAt(<values byte>)\` and the renderer overlaid.

   Tightened \`walk_optional\`: roll back only when the inner
   reports NoMatch (or Incomplete / Mismatch without consuming
   anything). Once the inner has committed to at least one
   terminal (e.g. matched the \`values\` keyword), propagate
   Incomplete / Mismatch up — the user is mid-typing the
   optional's content and rolling back would lose their
   intent.

   The pre-existing chumsky-or_not-style aggressive rollback
   covered cases like \`save Customers\` (Optional(\`as\`)
   inner is a single Word that returns NoMatch without
   consuming, so rollback still fires). Those keep working.

3. Side effect: with \`Optional\` no longer hiding the
   in-progress Form A from the leading slice, the walker on
   \`create table T with \` correctly reports the next-expected
   keyword as \`pk\` — so cursor at the end of the complete
   command \`create table T with pk\` would now re-offer \`pk\`
   as a Tab candidate against the partial \"pk\". Added a final
   filter: when the full input is a valid parse AND the
   partial prefix is non-empty, drop candidates that equal the
   partial exactly. Preserves schema narrowing
   (\`show data Cu\` → \`Customers\` is not an exact match).

Tests:
- New \`in_progress_form_a_values_list_classifies_as_incomplete\`
  asserts the input-state for the user's exact scenario.
- New \`open_paren_branching_punct_surfaces_after_insert_into_table\`
  and \`open_paren_candidate_is_classified_as_punct_kind\` cover
  the punct-as-candidate surface.
- Renamed and rewrote \`punctuation_expected_does_not_produce_candidates\`
  to \`non_branching_punctuation_is_not_surfaced_as_candidate\`
  to document the new finer-grained policy.
- Existing tests for \`save Tab → as\` and the schema-
  narrowing case continue to pass.

Tests: 854 passing, 0 failing, 1 ignored. Clippy clean.
2026-05-15 18:58:28 +00:00
claude@clouddev1 b3f1a20652 Phase D: insert value list mirrors do_insert's user_cols contract
Bug: hint at \`insert into Customers values (\` for a Customers
table with id:serial PK suggested typing an integer for \`id\`,
but the dispatch path (\`db::do_insert\`) deliberately doesn't
accept user-supplied values for auto-generated columns in
Form B. The grammar prompted for a value the dispatch would
refuse.

The fix aligns Phase D's \`column_value_list\` dynamic sub-grammar
with do_insert's three forms (ADR-0014 + ADR-0018 §3):

- **Form A** \`insert into <T> (col1, col2, …) values (…)\` —
  user explicitly lists columns. Slot list mirrors that
  selection; serial / shortid columns CAN appear if the user
  lists them.
- **Form B** \`insert into <T> values (…)\` — bare values. Slot
  list = non-auto-generated columns of the table in
  declaration order. Serial / shortid get auto-filled by the
  dispatch; the grammar doesn't prompt for them.
- **Form C** \`insert into <T> (v1, v2, …)\` — bare value list.
  Not affected by this change (column_value_list isn't on this
  path; Form C's literals route through the schemaless
  INSERT_PAREN_LIST).

Implementation:

\`WalkContext.user_listed_columns: Option<Vec<String>>\` — when
\`Some\`, signals Form A; \`None\` is Form B. Populated by walking
the first paren's column-list idents.

\`Node::Ident.writes_user_listed_column: bool\` — new field;
\`true\` on the INSERT_PAREN_ITEM's Ident child. When the
walker matches that ident in Form A, it appends the
schema-canonical column name (case-corrected against the
schema) to user_listed_columns.

\`column_value_list\` factory:
- If user_listed_columns is Some → resolve each name from the
  schema; one typed slot per listed column.
- Else → filter current_table_columns to non-auto-generated;
  one typed slot per remaining column.
- Empty result → fall back to the schemaless value-literal
  list (a serial-only table in Form B has nothing for the
  user to type).

Tests:
- New \`phase_d_insert_form_b_skips_serial_column\` confirms the
  bug: \`insert into Customers values (1, 'Alice')\` against a
  Customers with serial id rejects at parse time (Form B
  expects 1 value for Name, not 2).
- New \`phase_d_insert_form_a_accepts_serial_when_listed\`
  confirms \`insert into Customers (id, Name) values (1, 'Alice')\`
  works.
- New \`phase_d_insert_form_a_filters_to_user_listed_columns\`
  confirms partial Form A (\`(Name) values ('Alice')\`).
- Updated \`phase_d_insert_with_schema_accepts_typed_values_per_column\`
  to match the new Form B contract (2 user-typed values, not 3).
- Updated typed-hint test matrix split into form-B (8 types)
  and form-A (serial / shortid).
- New \`typed_hint_form_b_skips_serial_column_to_generic_or_text_neighbor\`
  pins the fallback behavior for a serial-only table.

For the user: \`insert into Customers values (\` for a Customers
with \`(id:serial, Name:text, Email:text)\` now hints
\`for \`Name\`: Type a quoted string …\` (skipping id entirely)
and accepts exactly 2 values. To set the serial explicitly,
use Form A: \`insert into Customers (id, Name, Email) values
(1, 'Alice', 'a@b.c')\`.

Tests: 851 passing, 0 failing, 1 ignored. Clippy clean.
2026-05-15 18:45:47 +00:00
claude@clouddev1 c485189da8 ADR-0024 Phase D: include column name in value-slot hint prose
User-facing improvement: typing into a value slot now surfaces
the column name in the hint. The hint at `insert into Customers
values (` (first column id:int) reads "for `id`: Type an
integer (e.g. 42, -7) or null" instead of the generic
"Type an integer …" prose. After `1, ` the panel updates to
the second column ("for `Name`: Type a quoted string …"). The
same applies to `update T set Email=` and `delete from T where
ts=` — the catalog wrapper threads the column name through.

Implementation:

**`Node::TypedValueSlot.column_name: Option<&'static str>`**
(new field, `src/dsl/grammar/mod.rs`). When `Some`, walker
writes `WalkContext::pending_value_column` on entry; clears
along with `pending_value_type` on inner success.

**Walker driver writes both names** (`src/dsl/walker/driver.rs`):
- `Node::TypedValueSlot` dispatch reads `column_name` and
  populates `pending_value_column`.
- `Ident { writes_column: true }` dispatch also writes
  `pending_value_column` (using the schema-canonical name when
  available, falling back to the user's spelling) so update
  set / where positions surface the column name.

**Shared sub-grammars** (`src/dsl/grammar/shared.rs`):
- New `slot_for_column(ty, name)` builds a `TypedValueSlot`
  with the embedded leaked column name. Used by
  `column_value_list`.
- New `slot_inner_for_type(ty)` returns just the Choice
  (without TypedValueSlot wrapper) for slot_for_column to
  rebuild.
- `column_value_list` factory now constructs per-column slots
  via `slot_for_column(col.user_type, &col.name)`. Each slot
  leaks its column name string with the same per-walk Box::leak
  pattern the rest of dynamic dispatch uses.

**`WalkContext::pending_value_column: Option<String>`** (new
field, `src/dsl/walker/context.rs`). Pairs with
`pending_value_type` to give the hint resolver both pieces.

**Single-walk hint resolver** (`src/dsl/walker/mod.rs`):
- New `HintResolution { mode: HintMode, column: Option<String> }`
  struct.
- New `hint_resolution_at_input(source, schema) -> Option<
  HintResolution>` runs one walk and reports both pieces. The
  ambient_hint dispatch composes per-column prose from the
  result.
- Existing `hint_mode_at_input` / `hint_mode_at_input_with_schema`
  preserved as thinner wrappers for tests / future callers
  that don't need the column name.

**Catalog wrapper** (`src/friendly/strings/en-US.yaml`,
`src/friendly/keys.rs`):
- New `hint.value_slot_for_column: "for `{column}`: {detail}"`
  prefixes the per-type prose with the actual column name when
  the walker has it bound. Schemaless fallback continues to use
  the generic value-literal prose with no column prefix.

**ambient_hint composes** (`src/input_render.rs`): consults
`hint_resolution_at_input`; when `column` is `Some`, wraps the
type prose through `hint.value_slot_for_column`; otherwise
emits the bare type prose.

Tests (846 total, 0 failing):
- 4 new input_render tests assert column names appear in the
  prose at insert/update/where positions plus the
  second-insert-value position (proves column tracking advances
  with comma).
- All existing tests pass unchanged — the column-name addition
  is layered on top of the type-only prose path.

Clippy clean.
2026-05-15 18:33:52 +00:00