Add table-level CHECK (e.g. `CREATE TABLE t (a int, b int, CHECK (a < b))`)
to advanced-mode SQL CREATE TABLE. Since SQLite exposes no PRAGMA for CHECK
constraints, a table-level CHECK cannot be read back from the engine and
becomes the source of truth in a new internal metadata table
`__rdbms_playground_table_checks (table_name, seq, check_expr)`.
- Grammar: new TABLE_CHECK element in ELEMENT_CHOICES.
- Builder: distinguishes a table-level CHECK from a column-level one by
element position (no column-def open in the element), using depth-aware
boundary tracking so a length-arg comma (`numeric(10,2)`) or a
table-PRIMARY KEY's inner comma is not mistaken for an element separator.
- Worker: do_create_table emits the CHECK clauses and writes the metadata
rows in its transaction; schema_to_ddl emits them identically on rebuild;
read_schema / read_schema_snapshot read them from the metadata table;
do_drop_table clears them.
- Persistence: TableSchema.check_constraints round-trips through project.yaml
(#[serde(default)], optional on read), mirroring unique_constraints.
- Composite UNIQUE deliberately stays PRAGMA-detected (engine-reportable,
unlike CHECK) — user-confirmed.
DA/runda round added cross-cutting tests and a forward-looking doc fix:
- table CHECK survives a rebuild triggered by `add column`, and a later
rebuild_from_text (the ADR-0013 rebuild primitive uses a raw DROP, so the
metadata rows keyed on the final name are preserved);
- dropping a column a table CHECK references fails cleanly (rollback, table
intact); detection is 4e, friendly wording is H1;
- dropping a table clears its CHECK metadata (no orphan rows on re-create);
- amended ADR §6 so 4h's RENAME also updates the new metadata table.
20 Tier-3 + 9 grammar/builder + 2 YAML tests. Docs: ADR-0035 Status/§13/§6,
README index, requirements.md Q1. Help/usage skeleton + describe display of
table-level constraints deferred to 4i (symmetric with 4a.2).
Tests: 1769 passing, 0 failing, 1 ignored. Clippy clean.
Advanced-mode SQL CREATE TABLE implemented through sub-phase 4a.2
(columns/types/aliases, NOT NULL/UNIQUE/PRIMARY KEY, IF NOT EXISTS,
per-column CHECK/DEFAULT, composite UNIQUE), ADR-0035 flipped to
Accepted, /runda pass on 4a fixed two defects. Handoff details the next
step (4a.3 — table-level CHECK + a new __rdbms_* metadata table), the
remaining Phase-4 sub-phases (4b–4i), the cross-cutting patterns (two
DDL generators must stay in sync; round-trip via PRAGMA-or-metadata;
the litmus test; raw-text capture), and process pins. Baseline
1752/0/0/1, clippy clean.
Advanced-mode SQL CREATE TABLE gains the constraints that need no new
internal table (the 4a.2 slice):
- Grammar (sql_create_table.rs): column-level DEFAULT/CHECK and
table-level UNIQUE(cols). DEFAULT is a literal or a *parenthesised*
expression (standard SQL) — a bare sql_expr greedily eats a following
NOT (NOT IN/LIKE/BETWEEN), breaking `DEFAULT 0 NOT NULL`; the parens
bound it. CHECK is paren-bounded already.
- Builder (ddl.rs): captures CHECK/DEFAULT raw SQL text by byte span
(sql_expr builds no AST) via capture_parenthesised_span /
capture_expr_span; routes single-column table UNIQUE into the
column's flag and composite UNIQUE into unique_constraints.
- Command/worker: ColumnSpec gains check_sql/default_sql (raw, preferred
over the typed Expr/Value); Command::SqlCreateTable + Request +
do_create_table gain unique_constraints; do_create_table emits raw
CHECK/DEFAULT and composite UNIQUE clauses.
- Round-trip (part D): ReadSchema/TableSchema gain unique_constraints;
read_schema detects composite UNIQUE via PRAGMA index_list origin 'u'
(single-column still folds to the column flag); schema_to_ddl emits
them; YAML RawTable/write_table round-trips (optional-on-read).
CHECK round-trips via __rdbms_playground_columns.check_expr, DEFAULT
via PRAGMA table_info — no new metadata table.
Table-level/multi-column CHECK remains 4a.3 (rejected "not yet
supported"); FK is 4b.
Tests: +7 builder (raw-text capture incl. the DEFAULT 0 NOT NULL
boundary the fix was found by; single/composite UNIQUE routing) and +4
Tier-3 (CHECK enforced, DEFAULT applied, composite UNIQUE enforced, and
all three survive a rebuild — the part-D round-trip). 1752 pass / 0 fail
/ 1 ignored; clippy clean. Plan + requirements.md updated.
Survey of the constraint persistence machinery revealed that
table-level/multi-column CHECK needs a NEW __rdbms_* metadata table
(SQLite exposes no PRAGMA for CHECK), unlike per-column CHECK/DEFAULT
(reuse __rdbms_playground_columns.check_expr + PRAGMA dflt_value) and
composite UNIQUE (PRAGMA index_list origin 'u' + a TableSchema field).
User-confirmed split: 4a.2 = per-column CHECK/DEFAULT (raw sql_expr
text) + composite UNIQUE(a,b), no new internal table; 4a.3 = table-level
CHECK + the new metadata table. ADR §13 and README updated in lockstep;
4a.2 plan doc added.
Command + builder + worker for advanced-mode SQL CREATE TABLE
(sub-phase 4a), executed structurally through do_create_table:
- Command::SqlCreateTable + build_sql_create_table (ddl.rs): aliases via
from_sql_name (incl. double precision), column- and table-level
PRIMARY KEY, redundant-flag de-dup off a sole PK, IF NOT EXISTS.
Advanced REGISTRY entry on the shared `create` word (SQL-first, DSL
fallback); no-PK tables allowed (user-confirmed).
- Worker (db.rs): Request::SqlCreateTable + CreateOutcome + snapshot_then
(one undo step); IF NOT EXISTS no-op (no snapshot, but journalled, like
read-only commands). do_create_table inline-PK rule aligned with the
rebuild generator schema_to_ddl — no round-trip DDL drift; serial
autoincrement is independent of inline-PK (verified by round-trip
tests).
- Runtime/App: dispatch + CommandOutcome::SchemaSkipped +
AppEvent::DslCreateSkipped (structure + "already exists — skipped"
note). Friendly catalog keys added (engine-neutral).
DEFAULT/CHECK/table-level UNIQUE are absent from the 4a grammar (parse
error with usage skeleton; friendly message + support land in the 4a.2
constraint slice) — user-confirmed.
Tests: type resolver, grammar shape, builder (incl. the PK
detection bug they caught), and tests/sql_create_table.rs (worker
round-trip, serial autoincrement first/non-first across rebuild, IF NOT
EXISTS no-op + journalling, no-PK table, one undo step) + a replay-as-
write test. 1739 pass / 0 fail / 1 ignored; clippy clean.
Exit gate: ADR-0035 Proposed -> Accepted (validated end-to-end by 4a);
README + requirements.md Q1 updated.
Three design questions settled during 4a implementation (plan + ADR §13
+ README in lockstep):
- CHECK/DEFAULT defer to the 4a.2 constraint slice: sql_expr is
validate-only (no Expr AST), so they need raw-SQL-text storage on a
separate path, not do_create_table's Expr->compile reuse. 4a.2 now
also covers composite UNIQUE / multi-column table CHECK.
- double precision (the lone two-word alias) handled via a keyword-pair
branch; single-word aliases + discarded (len) cover the rest.
- serial sole-PK in a multi-column table must inline PRIMARY KEY to keep
autoincrement (worker-step do_create_table extension).
4a core narrows to columns + types + NOT NULL/UNIQUE/PRIMARY KEY +
IF NOT EXISTS; everything else errors "not yet supported".
Add the sub-phase 4a implementation plan (docs/plans/), test-first,
mirroring the ADR-0033 DML sub-phase model: SqlCreateTable as its own
command executed structurally through the existing do_create_table
helper; shared-entry-word dispatch (SQL-first, simple fallback); the
type-alias resolver; IF NOT EXISTS no-op-with-note (CreateOutcome
enum); INTEGER PRIMARY KEY -> plain int; one-undo-step wiring.
Records the user-confirmed 4a/4a.2 split: composite UNIQUE(a,b) and
multi-column table CHECK move to a dedicated slice because they are the
first structures TableSchema cannot already represent, so they need a
persistence-model + round-trip extension rather than parse+execute
reuse. ADR-0035 §13 gains 4a.2; README sub-phase line updated in
lockstep.
Pre-implementation /runda round settled two open micro-calls before 4a,
both user-confirmed:
- IF [NOT] EXISTS admitted (no-op-that-succeeds-with-a-note), not
refused — a near-universal cross-vendor idiom (PostgreSQL, MySQL,
SQLite, Oracle 23ai), reclassified into scope rather than treated as
an engine-specific spelling. Touches §3/§4/§12/§13 (4a, 4c).
- INTEGER PRIMARY KEY maps to a plain int PK, not auto-increment;
serial stays the sole auto-increment type (§3).
README index updated in the same edit per the lockstep rule.
Phase 4 of the ADR-0030 roadmap; clarifies §4. Advanced-mode
CREATE/DROP/ALTER TABLE + CREATE/DROP INDEX get their own
per-statement Sql* commands, executed structurally (not verbatim)
so the playground's types, named relationships, and STRICT stay
intact. Full surface (no pre-emptive cuts): constraints, compound
PK, FK -> named relationships (one statement = one undo step),
ALTER incl. advanced-only table rename (C1), [UNIQUE] indexes.
Unified column-type-conversion: lossy refuses in simple mode but
proceeds-with-a-note in advanced, with undo as the safety net.
Integration (parser/hint/completion/diagnostics/history/replay/undo)
is structural via the unified grammar; replay treats DDL as a write.
Nine sub-phases (4a-4i). Updates the ADR README index.
Status: Proposed (design agreed; implementation pending).
/runda found silent data loss: with the non-fatal snapshot-failure
policy, a committed mutation whose snapshot couldn't be staged left
the redo stack stale (redo-clear was only a side effect of finalize),
so a later redo silently discarded the new work. Same gap in batches.
- SnapshotStore::clear_redo() drops the redo stack + payloads
- snapshot_then / end_batch call it when committed user work has no
staged snapshot; for disk-full it succeeds where a full backup
couldn't (tiny index write + payload deletes)
- unit test + integration regression (forced staging failure)
- ADR-0006 implementation note records the fix + residual edge
1698 passed / 0 failed / 1 ignored; clippy clean.
Settles the undo/snapshot half (U1/U2) before implementation:
- every-mutation single-step undo (supersedes destructive-only model)
- hybrid whole-project snapshot (db backup API + yaml/csv copy),
reconciling ADR-0006 with ADR-0015's derived-db model
- persisted N=50 ring; redo discarded on new work
- batch ops (replay + future) record one undo step; import excluded
- --no-undo disable switch
Adds the implementation plan and updates README index, requirements
U1/U2, and CLAUDE.md in lockstep.
Records this session's close-out: ADR-0033 Phase 3 marked Accepted; ADR-0034 (history journal + replay filter, incl. Amendment 1 replay app-command skip) implemented and verified. Tees up ADR-0006's undo/snapshot half (U1/U2) as the next job with scope considerations and open design calls.
Replay (§3): run_replay parses <ts>|<status>|<source> journal records — runs ok, skips non-ok — while still accepting bare .commands scripts (prefix-detected so a | inside a bare command isn't misread). Fixes replay history.log, which died on line 1.
Journal failures (§1/§2): failed commands are recorded err via a new Action::JournalFailure, emitted by the pure-sync App for both parse failures and worker-execution failures (runtime appends best-effort, never fatal). Hydration reads all records so typo'd/rejected commands are recallable across sessions.
Amendment 1 — replay filters app-lifecycle commands: a working replay history.log exposed that the journal also records save as/load/new/export/import/rebuild/mode (which would panic the worker dispatch or abort replay). Replay now re-applies only schema/data writes and skips every app-lifecycle command + nested replay, classified by entry word so modal/incomplete forms (save as, bare mode) and quit skip uniformly rather than aborting. All skips continue (reversing the nested-replay refusal); import and nested replay warn. replay.error_nested removed; replay.skipped_import/_replay added; ReplayCompleted carries warnings. requirements.md U3/U4 updated; app-command runtime-failure journalling tracked as a follow-up.
1659 passing / 0 failing / 0 skipped / 1 ignored. Clippy clean.
Two-sub-task test-first plan mapping 1:1 to ADR-0034's named sub-tasks: (1) journal failures + per-consumer filtering (status-tagged append, best-effort err writes, hydration reads all), (2) replay parses the journal format (ok-only filter, dual-shape input). Opens with a headline failing test that reproduces the live replay history.log bug.
Phase 3 of the ADR-0030 SQL surface (DML) is implemented and verified through sub-phase 3k; mark ADR-0033 Accepted in the ADR and the README index (index-upkeep rule). Add handoff 34 tracking the close-out and teeing up ADR-0034 (history journal + replay filter) as the next job.
1645 passing / 0 failing / 0 skipped / 1 ignored. Clippy clean.
Wire `insert`/`update`/`delete` as shared DSL/SQL entry words through the
category-grouped dispatcher (ADR-0033 Amendment 1): the Advanced SQL nodes
move off the dev words (`sqlinsert`/`sql_update`/`sql_delete`) to the real
keywords, registered alongside the Simple DSL nodes. Remove the dev-word
scaffold; collapse build_sql_{insert,update,delete} to source.trim();
de-duplicate the two REGISTRY entry-word listing sites.
Dispatch model (ADR-0033 Amendment 3, written this round):
- A command is the mode-rooted grammar-path outcome; identity is intrinsic.
Advanced mode tries SQL first, falling back to the Simple DSL command when
no SQL branch matches a token (`delete … --all-rows` falls back;
`update … --all-rows` does not — the SET expression absorbs it, harmless
since the engine treats `--all-rows` as a comment).
- Simple mode commits the DSL candidate for a shared word, surfacing the real
DSL error; bare "this is SQL" is reserved for SQL-only entry words
(`select`/`with`). A content rejection on the SQL candidate (internal
table) is committed, never masked by the DSL fallback.
Combined DSL-error + advanced-SQL pointer (ADR-0033 Amendment 3): a Simple-mode
definite DSL error that would run as SQL in advanced mode gains the
`advanced_mode.also_valid_sql` suffix — in the live hint (ambient_hint_in_mode)
and on submit (dispatch_dsl), via the shared advanced_alternative_note — so the
actionable DSL fix and the mode pointer coexist (submit covers constructs that
surface only on submit, e.g. `delete … returning`).
Internal-table rejection symmetrised (/runda finding B, ADR-0030 §6): the DSL
data-command target slots (insert/update/delete/show data/show table) gained
reject_internal_table, so `__rdbms_*` tables are refused in Simple mode too —
previously only the advanced SQL grammar rejected them.
Mode-awareness: classify_input_with_schema_in_mode and
invalid_ident_at_cursor_in_mode stop leaking the advanced SQL view into
simple-mode hints for shared words.
Tests: dev-word inputs migrated to the real words (advanced); DSL grammar /
completion / phase-D / db tests parse in Simple mode (the DSL surface); replay
keeps its advanced-mode model (one stale assertion fixed); dispatcher routing,
combined-pointer, and internal-table tests added. Suite 1626 pass / 0 fail /
1 ignored; clippy --all-targets -D warnings clean.
Defer M4 (execution-time mode side-channel; tracked in requirements.md) to its
own ADR.
Found while implementing 3f: history.log is success-only, but the
in-memory Up/Down recall ring records every submission — and the ring
is re-seeded from the log on open, so failed commands are recallable
in-session yet lost across sessions. Replay and recall also want
different inputs (state-builders vs everything-typed), which one
success-only file can't serve. And replay never parsed the pipe
format (run_replay parses whole lines), so `replay history.log` fails
on line 1 with no test covering it.
Decision: history.log becomes a complete journal tagged ok/err;
hydration reads all, replay reads ok-only and learns the format.
Amends ADR-0006 + ADR-0015 §5/§12. Code deferred to two tracked
sub-tasks. No migration for existing all-ok logs.
New src/dsl/grammar/sql_delete.rs (FROM <table> [WHERE] [;]),
Command::SqlDelete, Request::RunSqlDelete, do_sql_delete worker.
do_sql_delete mirrors the DSL do_delete: detect FK cascade by
before/after child row-count diffing, re-persist target + every
cascade-affected child, history-on-success inside the tx. Reuses
CommandOutcome::Delete -> handle_dsl_delete_success, so the
per-relationship cascade summary formatter is shared, not duplicated.
ADR-0033 Amendment 2: supersedes §7's WHERE-injected pre-count. Its
premise (DSL handler builds pre-counts from the typed Expr) was wrong
— do_delete uses count-diff. The pre-count would also have broken the
§2 parity promise by reporting SET NULL the DSL path doesn't. Count-
diff gives exact parity, no WHERE-byte extraction, and withdraws R2.
SET NULL reporting deferred for both paths (user-confirmed).
Tests: +6 grammar unit, +12 integration (cascade parity with DSL,
both R2 subquery cases, before-execute order, no-WHERE, FK-rejection
rollback, childless-parent, two-child cascade). 1542 pass / 0 fail /
1 ignored. Clippy clean. Dev sql_delete entry word removed in 3j.
walk_repeated discarded the last matched item's trailing-optional
expectations at a clean item boundary, so a comma-separated list
offered no continuation after a complete item: `order by Name `
gave no asc/desc, `select Name ` no `as`, `create table …
Code(text) ` no not/unique/default/check. Capture the last item's
skipped set and surface it when the list ends at an item boundary
(the separator `,` itself is deliberately not surfaced).
That fix made expression-position candidate lists long, which
exposed a visibility problem: the hint panel's candidate line is
single-row and window-scrolls on overflow, centring on item 0 when
nothing is selected — so with keywords-first, schema identifiers
scrolled off behind the `>` marker. Reverse the ordering: schema
identifiers (table/column/relationship names) now sort before
keywords, since a name the user would have to look up is the
highest-value completion and must stay visible (keywords are
learned over time; the tok_identifier/tok_keyword colour split
marks the boundary). This reverses the handoff-14 keywords-first
call, now recorded in ADR-0022 Amendment 2.
Tests: walker expected-set + completion-layer regressions for the
trailing-optionals and the ordering; candidate_ordering.rs header
invariant inverted; ~20 typing-surface snapshots re-baselined; a
two-line hint box recorded as a deferred follow-up.
Advanced-mode hinting + completion-preview were dead: render_hint_panel
returned None for advanced mode (stale ADR-0022 §12 gate, predating the
SQL grammar) and the hint resolver/ambient_hint never threaded Mode, so a
SQL statement was gated as "this is SQL". The unified walker (ADR-0030/
0031/0032) speaks SQL, so this lifts the gate.
- ambient_hint_in_mode + hint_resolution_at_input_in_mode +
expected_for_hint_snapshot(mode); candidate/diagnostic/parse sub-calls
run in the active mode.
- render_hint_panel calls ambient for all modes; one-shot `:` sigil
stripped (strip_one_shot_prefix) so `: sel` hints `select`.
- ADR-0022 Amendment 1 + README index.
Found by manual advanced-mode testing; Phase 2 marked SQL hint/completion
green at the engine layer but never exercised the UI. App-level render
test (advanced_mode_hint_panel_surfaces_sql_candidates) + ambient-layer
regression locks. 1466 baseline green.
Replaces ADR-0033 §2's original Node::Guard + Choice(SQL,DSL) mechanism,
which was found during 3a to be unworkable: any guard-in-Choice approach
forces a walk_choice change (walk_choice falls through only on NoMatch, so
simple-mode valid-DSL would wrongly surface "this is SQL"), and walk_seq
treats a NoMatch past idx 0 as a hard Failed, breaking advanced-mode DSL
fall-through.
Mechanism (Amendment 1): each REGISTRY entry is tagged
CommandCategory::{Simple, Advanced}, generalising the whole-command
is_advanced_only gate. walk() becomes a thin dispatcher over decide()
(mode-aware candidate selection: simple commits the DSL node or emits the
"this is SQL" hint; advanced tries SQL first, DSL as a full-line fallback)
and an extracted walk_one_command(); speculative match-testing runs on a
scratch WalkContext so the caller's context is only touched by the
committed walk. No Node::Guard, no walk_choice/walk_seq change.
6 dispatch smoke tests on a shared-entry-word smoke registry; 1446 baseline
green; clippy clean.
Build-order plan (sub-phases 3a–3k) with per-sub-phase scope, exit
gates, and written DA gates, modelled on
docs/plans/20260520-adr-0032-phase-2.md. Centrepiece is the cross-cut
verification matrix scaffold (~75 rows) grouped by ADR-0033 section
(statement shapes, dispatch, RETURNING, shortid, cascade, diagnostics,
UPSERT, inherited Phase-2 diagnostics, ambient assistance,
engine-neutrality, persistence, OOS rejections), to be filled in
during 3k.
Carries the handoff-29 §4 process pins into the relevant sections: DA
critiques listed before verdict, no silent out-of-scope classification,
and matrix attribution requiring SQL-input tests for SQL claims. Records
four open questions to escalate before code starts (shortid SELECT
row-source path, R1 mechanism fallback, cascade pre-count construction,
UPSERT catalog wording).
Twenty-ninth handover. This session drafted ADR-0033 as the
planning artifact for ADR-0030 Phase 3 (SQL DML in Advanced mode).
Documents the ten settled design decisions (Q1-Q10), the dispatch
architecture (SQL-first / DSL-fallback via the new Node::Guard
mechanism), the eleven phased sub-phases (3a-3k) each with their
exit gates, and the open implementation risks (R1-R4).
Points the next session at sub-phase 3a (Node::Guard scaffolding)
as the concrete entry point, with the plan-doc cross-cut matrix
as the immediate prerequisite task. Pins four process lessons
from the Phase-2 session (DA rubber-stamp risk, defer-trap
reflexes, tests-first on gap closure, matrix attribution
verification).
State: 1446 / 0 / 1 passing (unchanged — planning-only session).
Clippy clean.
Phase 3 of ADR-0030's SQL-surface roadmap. Status: Proposed.
Statement shapes (§1): single- and multi-row INSERT, INSERT…SELECT
(recursing through ADR-0032's SQL_SELECT_COMPOUND), UPDATE with
SET assignment list, DELETE, all three optionally followed by
RETURNING projection_list. Full UPSERT (ON CONFLICT … DO NOTHING /
DO UPDATE with the SQLite/PostgreSQL `excluded` pseudo-table)
on INSERT.
Dispatch (§2): SQL-first / DSL-fallback in Advanced mode via
Choice(SQL_shape, DSL_shape) per shared entry word. Requires a
new walker capability — Node::Guard(fn), a zero-byte-consumption
gating node — landed as the first sub-phase's work (R1 mitigation
budgeted).
Execution (§10): three typed Command variants (SqlInsert /
SqlUpdate / SqlDelete) carrying target_table, listed_columns,
and a returning: bool flag. Worker handlers know per-kind
specialisations: shortid auto-fill (§6, parity with DSL),
cascade summary (§7, WHERE byte-range injection into pre-count
subqueries), DataResult routing on RETURNING (§5).
Diagnostics (§8): three new keys (insert_arity_mismatch ERROR,
auto_column_overridden WARNING, not_null_missing WARNING) with
positive + negative test requirements.
OOS list (§13): DEFAULT VALUES (seed feature), SQLite OR-prefixes,
UPDATE FROM, WITH-prefixed DML, indexed-by hints,
multi-statement batches.
Implementation notes: eleven phased sub-phases (3a–3k) each with
explicit exit gates + written DA gates. Ordering puts Node::Guard
scaffolding (3a) FIRST so the dispatch mechanism is proven before
DML grammar lands on top.
Initial DA review (Initial DA review section) recorded seven
critiques that were resolved before status moved to Proposed; a
second-pass DA surfaced an eighth (Node::Guard wasn't an
existing walker capability) and added it to §2 + sub-phase 3a's
scope.
Twenty-eighth handover. Captures this session's Phase-2 completion
(sub-phases 2d.1, 2e, 2g + DA-driven rework) and points the next
session at ADR-0030 Phase 3 (DML in Advanced mode) as the natural
continuation.
Pinned items for the next session:
- Four non-blocking DA observations from this session's
verification report (group-by pattern overbreadth, look-ahead
probe cost, tests-after-code on matrix coverage, matrix
attribution wasn't row-by-row verified).
- A process lesson on DA discipline: rubber-stamp PASS verdicts
must die. Next DA review lists critiques first, concludes after.
- Phase 3 design considerations: UPDATE/DELETE table-source
scope, INSERT…SELECT, RETURNING, UPSERT, DML-specific
diagnostics.
State: 1446 / 0 / 1 passing. Clippy clean. Phase 2 fully pushed.
The §5 deliverable from the implementation plan, this time with
a non-rubber-stamp DA review.
Documents:
- Final test state (1446 / 0 / 1 — clippy clean).
- Cross-cut matrix outcome (29 rows, all green per the plan doc).
- Requirements-to-test mapping for ADR-0032 §§1–13 + both
Amendments.
- Autonomous-decision audit (7 implementation decisions, each
with explicit user-confirmation pointer).
- DA's written final review with three blocking critiques
(now closed in commit 05884bd) and four non-blocking
observations recorded as known trade-offs.
- Process critique on the first DA pass being a rubber stamp.
Verdict: PASS, with non-blocking observations pinned in the
report rather than carried into the next phase as folklore.
Cross-cut verification matrix for ADR-0032 Phase 2 is now fully
populated with concrete test references — every row green. Filling
the matrix surfaced three real gaps that this commit closes.
1. Advanced-mode syntax highlighting (ADR-0030 §8 matrix row).
The `ui.rs` Advanced branch routed through `plain_input_spans`,
bypassing the highlight walker entirely. In production SQL
keywords past the entry word rendered as plain identifiers.
Fix: mode-aware variants of `highlight_runs`,
`render_input_runs`, `lex_to_runs`, and `input_diagnostics`;
the Advanced render path now uses the highlighted form with
`Mode::Advanced`. `plain_input_spans` removed (unused).
2. Engine.* key wiring (ADR-0032 §11.4 / §13 matrix rows + handoff
§3.3 follow-up). The four Phase-2 engine.* catalog entries
were authored in 2d but never reached: `translate_generic`
discarded the engine message and returned a vague catalog
entry. Fix: pattern-match the engine message text for the four
Phase-2 categories (aggregate misuse, group-by required,
compound arity mismatch fallback, scalar-subquery cardinality)
inside `translate_generic`, routing each to its engine-neutral
catalog entry.
3. Matrix-coverage tests. Thirteen new tests covering the rows
that had no explicit coverage:
- 3 SQL keyword/operator/CASE highlight tests
- 4 engine.* engine-message tests
- 3 sql_expr column-completion tests (WHERE, HAVING)
- 3 predicate-warning slot tests (CASE, ORDER BY, projection)
- 1 all-10-playground-types recovery test (tests/sql_select.rs)
Plan document (docs/plans/20260520-adr-0032-phase-2.md) updated:
every (TBD) row in the cross-cut matrix replaced with a concrete
test file::function reference and a green status marker.
Test totals: 1428 → 1441 passing (+13 new). Clippy clean.
Amendment 2 records the §10.6 fixup-pass mechanism choice. §10.6
prescribes "rewriting the highlight class" on projection-list
idents at end-of-walk; the actual implementation uses a different
mechanism that achieves the identical user-visible behavior:
1. 2d's two-pass schema-existence diagnostic collects every FROM
binding from the matched path first, then resolves projection
idents against the complete scope. The post-walk re-resolve
§10.6 calls for, just embedded in the diagnostic emitter.
2. input_render.rs's diagnostic-overlay path colors each
diagnostic span Error/Warning, achieving the visual change
§10.6 describes without needing a new HighlightClass variant.
The completion-mid-typing piece is improved by the §10.5
look-ahead probe (sub-phase 2e earlier).
Four new regression tests in `projection_before_from_tests` pin
the behavior so a future refactor can't silently regress it:
correct ident resolves silently, unknown ident flags via
diagnostic on its span, multi-projection only flags unknowns,
projection-without-FROM is silent.
ADR index entry updated to reference Amendment 2.
Test totals: 1424 → 1428 passing (+4). Clippy clean.
Session summary covering nine implementation commits since
handoff-26 (e032f01..0c3847a). Records the user-approved
deferral of §10.3 stage 2 (CTE column-derivation harvest),
flags three further 2d diagnostic-key deferrals that need
user confirmation, and points the next implementer at the
seams for sub-phases 2e (qualified-prefix completion +
post-walk fixup) and 2g (verification sweep + final
report).
Test totals at handoff: 1385 passing, 0 failed, 1 ignored
(+125 from the 1260 baseline at handoff-26).
The Phase-1 SQL `SELECT` grammar nodes that used to live in
`src/dsl/grammar/data.rs` retire — 22 statics / consts and the
`reject_internal_table` validator copy are removed, ~150 lines
of grammar machinery gone. `data::SELECT.shape` now references
the post-`SELECT` portion of the ADR-0032 fragment via a thin
`Node::Subgrammar(&sql_select::SQL_SELECT_TAIL)`.
`SQL_SELECT_TAIL` is a new export from `sql_select.rs`,
parallel to `SQL_SELECT_STATEMENT`. It represents what a
top-level `SELECT` statement looks like AFTER the registry's
entry-word dispatch has already consumed the leading `SELECT`
keyword: the DISTINCT/ALL prefix, projection list, optional
FROM / WHERE / GROUP BY / HAVING, the compound set-op chain
(each subsequent leg's `SELECT` is part of `SET_OP_TAIL`),
outer ORDER BY / LIMIT, and a tolerated trailing `;`.
WITH-prefixed statements (`WITH x AS (…) SELECT * FROM x`)
are NOT in 2c's scope — they need a separate `data::WITH`
`CommandNode` so the entry-word dispatch routes correctly.
For now, top-level WITH continues to fall through to the
chumsky parser route (the same as in Phase 1). The
`SQL_SELECT_STATEMENT` static (which includes the optional
WITH prefix) stays available for use by that future
CommandNode or by any other consumer that needs the full
statement shape.
All seven Phase-1 SQL `SELECT` integration tests
(`tests/sql_select.rs`) pass without modification, satisfying
the 2c exit gate's "behaviour preserved" requirement. The
70 fragment unit tests and the 26 driver-level scope tests
also pass — the migration is a refactor, no new tests
required.
Behaviour change explicitly sanctioned by ADR-0032 §8:
Phase-1's `LIMIT_VALIDATOR` (positive-int-only, parse-time)
is superseded by the full `sql_expr` admission. `LIMIT max(10,
x)` and similar now parse; the engine constrains the value at
execution time per the ADR's "grammar admits, engine
rejects" posture.
Plan §2b status note: the 2026-05-20 deferral of §10.3 stage 2
(CTE output-column harvest derivation) is recorded in
`docs/plans/20260520-adr-0032-phase-2.md` per the
user-approved deferral.
Test totals: 1366 passing (unchanged), 0 failed, 1 ignored.
Clippy clean. data.rs loses ~150 lines of dead grammar; the
single source of truth for the SQL `SELECT` shape is now
`sql_select.rs`.
§12 was written conservatively, classifying projection items
structurally and listing "subquery expressions" alongside
arithmetic / CASE as cases that stay None. The Phase-2 plan's
Open Question 1 captured the matching uncertainty about CTEs
and scalar subqueries.
A throwaway probe against the pinned bundled SQLite +
rusqlite 0.39.0 (with the `column_metadata` feature) settles
the question across 20 representative query shapes. The
engine's column_table_name / column_origin_name metadata
follows through non-recursive CTEs (SELECT *, bare-ref,
qualified-ref, and (col-list)-renamed bodies; CTE chains),
scalar subqueries (aliased and unaliased), derived tables
(out of scope per §13 OOS-1 but useful to note), all four
set ops, multi-table JOIN projections, and IN-subquery
WHERE clauses (the inner subquery does not affect the
outer projection's origin).
The structural-None classes reduce to computed projections
(function calls, arithmetic, CASE, literals, wildcards —
expected and pedagogically obvious) and recursive CTE result
columns (the one structural surprise — the recursive
temporary table has no base-column origin to point at).
Amendment 1 supersedes §12's "Resolution rule" with a simpler
engine-driven rule: trust column_table_name(i) /
column_origin_name(i) verbatim, with no grammar-side
structural classification. The speculative MatchedPath-walk
fallback is moot. The Phase-2 plan's sub-phase 2f exit gate
gains explicit positive assertions for CTE pass-through and
scalar-subquery type recovery, and a new explicit negative
assertion for the recursive-CTE limitation.
README.md index entry extended in the same style as ADR-0027's
Amendment-1 line. Closes Plan §Open-1.
Twenty-sixth handover. Design session, no code touched. Tests
unchanged at 1260 / 0 / 1.
Captures: the Phase 2 grammar decisions (ADR-0032 accepted),
the implementation plan at docs/plans/20260520-adr-0032-phase-2.md
with seven sub-phases and a cross-cut verification matrix that
explicitly names every "X comes for free" claim from ADR-0030/
0031/0032, and the Phase-1 carry-over finding the warning/error
guideline check surfaced — SQL WHERE expressions currently emit
no LIKE-on-numeric / = NULL / type-mismatch warnings because
sql_expr builds no AST. ADR-0032 §11.6 closes the gap; the
plan's cross-cut matrix has a named row to prevent regression.
The next session is Phase 2 sub-phase 2a (grammar fragment) per
the plan; standing authorizations apply.
Status: ready to hand over.
ADR-0030 §3 commissioned a focused ADR for the full SELECT
grammar (the "SELECT — full" phase). ADR-0032 records the
decisions; docs/plans/20260520-adr-0032-phase-2.md is the
implementation plan walking the work.
Phase 2's grammar surface:
- Five JOIN flavours (INNER, LEFT, RIGHT, FULL OUTER, CROSS).
NATURAL/USING/comma-FROM explicitly OOS.
- All four set ops (UNION, UNION ALL, INTERSECT, EXCEPT).
- WITH and WITH RECURSIVE CTEs, with optional (col-list) renaming.
- Scalar subqueries, IN (SELECT …), [NOT] EXISTS as additive
primary branches in sql_expr (redeems ADR-0031 §7 OOS-1).
- Qualified column refs t.c / alias.c as a name_or_call tail
(redeems ADR-0031 §7 OOS-2).
- LIMIT n [OFFSET m]; legacy `LIMIT m, n` OOS.
- DISTINCT/ALL, t.* projection, bare-alias projection (lifts
Phase-1 §4.2's autonomous decision).
Walker-capability honesty (§10): ADR-0030 §8's "ambient
assistance comes for free" holds for grammar recursion (reuses
ADR-0026's Subgrammar + depth cap unchanged) but not for
completion scope. Phase 2 adds a new Node::ScopedSubgrammar
variant alongside the existing Node::Subgrammar (DSL Expr and
sql_expr recursion untouched), a from_scope_stack of
ScopeFrames holding from_scope / cte_bindings /
projection_aliases, qualified-prefix completion narrowing, and
a post-walk fixup pass that re-resolves projection-list
identifier highlighting/validity once FROM is parsed (the
projection-before-FROM problem).
CTE column resolution (§10.3): SELECT * and explicit-projection
CTE bodies both yield real column completion past cte_alias.|
via a body-projection derivation rule that runs at the body's
ScopedSubgrammar exit and writes derived columns back into the
binding.
Diagnostics (§11): every Phase-2 validation case classified
against ADR-0027's ERROR/WARNING guideline. Five new diagnostic.*
catalog keys for parse-time-detectable cases (unknown_qualifier,
ambiguous_column, projection_alias_misplaced, cte_arity_mismatch,
compound_arity_mismatch) plus eight engine.* translation keys.
A MatchedPath-walking predicate-warnings variant closes the
Phase-1 carry-over gap where SQL WHERE expressions emitted no
LIKE-on-numeric / = NULL / type-mismatch warnings — ADR-0027
Amendment 1 finally extends to the SQL surface.
Result-column type resolution (§12): rusqlite 0.39.0 exposes
column_table_name / column_origin_name / column_database_name
behind a `column_metadata` feature; verified. Bare column refs
recover their playground type — partially lifts Phase-1 §4.5's
bool→0/1 deferral.
The implementation plan breaks Phase 2 into seven sub-phases
(2a–2g) with explicit exit gates per sub-phase and a cross-cut
verification matrix that names every "X comes for free" claim
from ADR-0030/0031/0032. The Phase-1 SQL-expression
predicate-warning gap is a named row, preventing an analogous
silent gap from shipping. The plan encodes the user's standing
authorization for the implementer to walk uninterrupted between
gates and commit with standard messages — escalation
discipline preserved for design ambiguities and real blockers.
Pushes remain user-only.
New docs/plans/ directory sets a pattern for future phase plans.
Status: Accepted.
Implementation handoff: a SQL `select` typed in advanced mode
parses, runs, and renders end to end; the same line in simple
mode lights up the precise "this is SQL" hint instead. ADR-0031
(the SQL expression grammar) and ADR-0030 Phase 1 ("Foundations
+ first SELECT") landed across five commits. Tests 1240 → 1260,
clippy clean.
The handoff records: the walker mode gate + `is_advanced_only`
set, the `ast_builder` source-param sweep, `Command::Select`
carrying the validated SQL text, the `data::SELECT` shape, the
worker `Request::RunSelect` round-trip, the ambient mode
threading through completion / overlay / validity indicator,
the autonomous calls made during execution (FROM optional,
implicit alias unsupported, etc.), and the seams the next
session uses to take up ADR-0030 Phase 2 (full SELECT) — which
gets its own focused ADR before code, per ADR-0030 §3.
ADR-0030 §3 commissioned a focused ADR for the stratified SQL
expression grammar fragment. ADR-0031 records the decisions:
- One unified precedence ladder (OR/AND/NOT, comparison/LIKE/IN/
BETWEEN/IS NULL predicates, arithmetic incl. `||`, function
calls, CASE) — SQL treats booleans as values, so unlike
ADR-0026's bool/scalar split this is a single ladder.
- No AST — every Phase-1 consumer (SELECT projection, WHERE)
runs validated SQL as text per ADR-0030 §4/§6; CHECK/DEFAULT
in Phase 4 store text too. The fragment's job is accept /
reject + the matched-terminal path + a source span.
- Recursion via Subgrammar with ADR-0026's depth cap reused.
- A parallel `grammar/sql_expr.rs` — separate from `expr.rs` so
simple mode's 1240-test surface is untouched by construction.
- Subquery expressions and qualified `t.c` column refs deferred
to ADR-0030 Phase 2 (they need the recursive SELECT grammar).
`%` modulo is included alongside `+ - * /` and `||` — it isn't
ISO SQL but is near-universal across mainstream engines and
matches learner expectations (pedagogy wins ties, ADR-0030).
Status: Accepted. The implementation lands in subsequent
commits.
Decides the architecture for SQL in advanced mode (Q1/Q2/Q4):
SQL is authored as grammar within the unified grammar tree
(ADR-0024) and parsed by the existing walker — not a separate
batch parser — so SQL gets the same completion, highlighting,
hints, and parse-error reporting as the DSL. Mode gates the
SQL forms. DDL routes through the typed Command executor
(metadata and the playground type vocabulary preserved); DML
and SELECT execute as validated SQL. Engine-neutral posture;
DSL→SQL teaching echo; phased plan.
Supersedes ADR-0001's sqlparser-rs reservation. Ticks Q4;
updates the ADR index and the Q1/Q2 notes. handoff-24 orients
the implementation session at Phase 1.
ADR-0029 (column constraints — NOT NULL / UNIQUE / CHECK /
DEFAULT) is fully implemented across the handoff-22 and
handoff-23 sessions. Ticks requirement C3, and corrects
ADR §10's CHECK-error wording to the compiled-SQL form per
the §7 storage deviation.
ADR-0028 complete (per handoff-21); ADR-0029 (column
constraints) written, accepted, and implemented through
commit 4 of 6 — NOT NULL / UNIQUE / DEFAULT / CHECK at
`create table` and `add column`. Commits 5 (`add constraint`
/ `drop constraint` + the §5 dry-run) and 6 (friendly errors
+ typing-surface matrix) are planned in full in §4.
The fourth constraint. `check ( <expr> )` reuses the ADR-0026
WHERE-expression grammar via `Subgrammar`, so a check is
written in the same language as a `where` filter.
- Grammar: a `CHECK_CONSTRAINT` arm joins the shared
constraint-suffix Choice; `consume_check_expr` extracts the
parenthesised expression (paren-depth aware) into
`ColumnSpec.check` / `Command::AddColumn.check`.
- Storage: the parsed `Expr` is compiled once to inline SQL
(`compile_check_sql` — `compile_expr` + ADR-0028's
param-inliner) and stored in that form everywhere — a new
`check_expr` column in `__rdbms_playground_columns`,
`project.yaml`'s `ColumnSchema.check`, and the column DDL
emitted by `do_create_table` / `schema_to_ddl`.
- `add column … check` routes through the rebuild primitive
(SQLite's `ALTER … ADD COLUMN` cannot carry it); a CHECK on
a serial/shortid column is create-table-only and refused at
add-column with a friendly message.
- `describe` surfaces the CHECK. ADR-0029 §7/§8 updated to the
SQL-form decision — double-quoted identifiers, consistent
with ADR-0028's `explain` display SQL.
1201 tests pass (+8); clippy clean.
`create table … with pk` now parses the column-constraint
suffix; combined with the commit-1 db layer, a constrained
table works end to end.
- A shared constraint-suffix grammar fragment — `not null`,
`unique`, `default <literal>` — sits after each column's
`(type)` group; `build_create_table` walks the matched path
per column and folds the constraints into `ColumnSpec`.
- §9 redundancy check: every `with pk` column is a primary-key
column, so `not null` (any) and `unique` (single-column PK)
are rejected with a friendly error
(`parse.custom.constraint_redundant_on_pk`).
- `project.yaml` round-trip: `ColumnSchema` gains `not_null` /
`default`; the YAML reader/writer and `build_read_schema`
carry them, so `rebuild` / `export` / `import` preserve
constraints.
- ADR-0029 §2.1's example corrected — `create table` columns
are all PK columns, so its suffix is for `default` / `check`;
`docs/simple-mode-limitations.md` records that non-PK
columns at create time need advanced mode.
CHECK is deferred to the next commit. 1184 tests pass (+7);
clippy clean.
Designs the remaining C3 surface: the four column-level
constraints declared in the column-spec suffix at `create
table` / `add column`, and modified on existing columns via
`add constraint … to` / `drop constraint … from`.
- A pre-flight dry-run (the ADR-0017 ethos) scans a populated
column before applying NOT NULL / UNIQUE / CHECK and refuses
with a pretty-table of offending rows; no `--force`.
- CHECK reuses the ADR-0026 expression grammar via Subgrammar.
- `__rdbms_playground_columns` carries a new `check_expr`
column; the other three are recoverable from SQLite pragmas.
- README index updated.
ADR-0028 (query plans / `explain`) is fully implemented; the
handoff-16 design trio (ADR-0026 / 0027 / 0028) is now closed.
- handoff-21: session summary, the two deliberate deviations
from handoff-20's plan, test coverage, open clusters.
- requirements.md: QA1 / QA2 ticked.
- CLAUDE.md: the `EXPLAIN QUERY PLAN` deferred-items line
updated to "implemented per ADR-0028".
claude@clouddev1