Records the decision that advanced-mode SQL DML should stop handing literal
data values to the engine as text and instead parse/validate/bind them
through the DSL's proven path — closing the value-validation gap, the
hint/highlight gap, and the offending-value-in-errors gap together. Verbatim
text stays for expressions, WHERE, INSERT…SELECT, and SELECT (full SQL
surface preserved; ADR-0026's limited Expr not imposed). Narrows ADR-0030 §4
/ ADR-0033 §10 once accepted; SELECT half of §4 stands.
Includes a characterization test (tests/sql_insert.rs) proving the bind-layer
gap: the DSL rejects the malformed date 2025/01/15, advanced-mode SQL accepts
it. Forward-notes added to ADR-0030/0033; README index updated.
Status: Proposed (design + /runda done; pending go-ahead to implement).
Wire `insert`/`update`/`delete` as shared DSL/SQL entry words through the
category-grouped dispatcher (ADR-0033 Amendment 1): the Advanced SQL nodes
move off the dev words (`sqlinsert`/`sql_update`/`sql_delete`) to the real
keywords, registered alongside the Simple DSL nodes. Remove the dev-word
scaffold; collapse build_sql_{insert,update,delete} to source.trim();
de-duplicate the two REGISTRY entry-word listing sites.
Dispatch model (ADR-0033 Amendment 3, written this round):
- A command is the mode-rooted grammar-path outcome; identity is intrinsic.
Advanced mode tries SQL first, falling back to the Simple DSL command when
no SQL branch matches a token (`delete … --all-rows` falls back;
`update … --all-rows` does not — the SET expression absorbs it, harmless
since the engine treats `--all-rows` as a comment).
- Simple mode commits the DSL candidate for a shared word, surfacing the real
DSL error; bare "this is SQL" is reserved for SQL-only entry words
(`select`/`with`). A content rejection on the SQL candidate (internal
table) is committed, never masked by the DSL fallback.
Combined DSL-error + advanced-SQL pointer (ADR-0033 Amendment 3): a Simple-mode
definite DSL error that would run as SQL in advanced mode gains the
`advanced_mode.also_valid_sql` suffix — in the live hint (ambient_hint_in_mode)
and on submit (dispatch_dsl), via the shared advanced_alternative_note — so the
actionable DSL fix and the mode pointer coexist (submit covers constructs that
surface only on submit, e.g. `delete … returning`).
Internal-table rejection symmetrised (/runda finding B, ADR-0030 §6): the DSL
data-command target slots (insert/update/delete/show data/show table) gained
reject_internal_table, so `__rdbms_*` tables are refused in Simple mode too —
previously only the advanced SQL grammar rejected them.
Mode-awareness: classify_input_with_schema_in_mode and
invalid_ident_at_cursor_in_mode stop leaking the advanced SQL view into
simple-mode hints for shared words.
Tests: dev-word inputs migrated to the real words (advanced); DSL grammar /
completion / phase-D / db tests parse in Simple mode (the DSL surface); replay
keeps its advanced-mode model (one stale assertion fixed); dispatcher routing,
combined-pointer, and internal-table tests added. Suite 1626 pass / 0 fail /
1 ignored; clippy --all-targets -D warnings clean.
Defer M4 (execution-time mode side-channel; tracked in requirements.md) to its
own ADR.
on_conflict_clause on SQL_INSERT_SHAPE: optional (col,…) conflict
target (distinct conflict_target_column role so it never enters
listed_columns), DO NOTHING / DO UPDATE SET … [WHERE …]. `do` is
factored out of the action Choice so nothing/update disambiguate
without tripping the walk_seq/walk_choice shared-prefix trap
(ADR-0033 Amendment 1). Worker runs the UPSERT verbatim (SQLite
native); no new execution path.
build_sql_insert: row_source now stops before the FIRST trailing
clause — ON CONFLICT (3h) or RETURNING (3g) — and do_sql_insert's
shortid auto-fill rewrite re-appends the whole trailing tail, so an
auto-filled INSERT keeps its ON CONFLICT / RETURNING.
excluded pseudo-table (§9): resolves to the target's columns inside
the DO UPDATE action and completes at `excluded.|`, but stays flagged
as unknown_qualifier in VALUES / RETURNING / non-upsert statements.
Diagnostic pass scopes it by the DO UPDATE byte-range (update token →
RETURNING/end); completion resolves it against the INSERT target's
current_table_columns. NOTE: scoping uses byte-range rather than the
plan's prescribed from_scope TableBinding push — same behaviour, no
walker scope-frame change.
Tests (+13): grammar accept/reject; DO NOTHING / DO UPDATE-excluded /
no-target execution + persistence; auto-fill × ON CONFLICT with a
REAL unique conflict (proves the clause survives the rewrite, not a
no-op); excluded resolves in DO UPDATE SET + WHERE, flagged in VALUES
(incl. same statement), unknown column under excluded; excluded.|
completion; conflict-target not in listed_columns. 1576 pass / 0 fail
/ 1 ignored. Clippy clean. Dev sql_insert entry word still removed in
3j.
Known follow-up (tracked for 3i): UPSERT DO UPDATE bare column refs
(SET LHS / WHERE) are not schema-validated, unlike regular UPDATE —
the INSERT target isn't a diagnostic binding. Fits 3i's cross-cut
SET/WHERE validation scope.
Shared RETURNING_CLAUSE (reuses Phase-2 PROJECTION_LIST, now
pub(crate)) as an optional tail on all three SQL DML shapes.
`returning: bool` on the Command variants, set by the ast-builders
and threaded to the worker. run_returning collects the returned rows
as a DataResult (RETURNING mutates + yields in one pass), reusing
resolve_select_column_types for bare-column type recovery; computed
projections stay typeless. DeleteResult gains a `data` field rendered
alongside the cascade summary.
Follow-set fix: `returning` is added to the table-source and
projection bare-alias follow-sets so an INSERT … SELECT row source
stops before RETURNING instead of reading it as a table alias.
Auto-fill × RETURNING: build_sql_insert stops row_source before the
RETURNING token (keeping it preparable for shortid materialisation),
and plan_shortid_autofill re-appends the RETURNING tail so generated
shortids surface in RETURNING *.
Tests (+17): grammar accept on all three; INSERT/UPDATE/DELETE
RETURNING incl. *, aliases, multi-row, type recovery + computed-
typeless; auto-fill × RETURNING (single + multi-row distinct ids);
INSERT…SELECT…RETURNING execution; UPDATE…RETURNING zero-match;
DELETE…RETURNING cascade+rows; app-level render of both. Dev
sql_insert/sql_update/sql_delete entry words still removed in 3j.
1562 pass / 0 fail / 1 ignored. Clippy clean.
plan_shortid_autofill read exactly listed_columns.len() cells from
the materialised row source. When the row source produced a
different column count than the user's list, the extra columns were
silently dropped (wider → wrong data, insert succeeded) or read
out of range (narrower). Guard: if the materialised statement's
column_count differs from the listed-column count, skip auto-fill
and execute the verbatim statement so the engine reports the
mismatch — matching the non-auto-fill path. A friendly pre-flight
diagnostic remains sub-phase 3i.
Tests: VALUES with too many values; INSERT…SELECT with a wider and
a narrower projection — each rejected with nothing persisted.
When an INSERT's column list omits one or more shortid columns,
the worker now fills them. Command::SqlInsert gains listed_columns
and row_source, captured in build_sql_insert from the matched path
(the row source is located by the first values/select/with Word
token, so a string literal like 'select' can't be mistaken for the
keyword). do_sql_insert calls plan_shortid_autofill, which — per
the user-confirmed Option B — materialises the row source by
running it as a query, generates a distinct shortid per row via the
existing generate_shortid_batch (deduped against stored values),
and reconstructs a parameterised multi-row INSERT over the listed
columns plus the omitted shortid columns. Uniform for VALUES and
INSERT…SELECT, and handles multiple omitted shortids in one row
(each gets its own batch). No explicit list, no omitted shortid, or
a zero-row source → execute verbatim (the 3b path). serial stays
engine-filled via rowid. history.log keeps the original line, never
the rewrite (§11).
Tests: VALUES single/multi-row distinct; explicit override
honoured; INSERT…SELECT distinct fills; combined serial(engine) +
shortid(worker); two shortids (PK + non-PK) both fill; one provided
+ one omitted; compound-PK shortid member; mixed-case column name
(ADR-0009 DA gate); original-source-in-history on the rewrite path.
Still behind the dev `sqlinsert` entry word (3j). 1503 green,
clippy clean.
Make the INSERT row source a Choice between the VALUES clause and
Subgrammar(&sql_select::SQL_SELECT_COMPOUND). SQL_SELECT_COMPOUND
is itself a Choice that admits a leading WITH, so a WITH-prefixed
SELECT row source (R4) parses through it for free; the two
branches start on disjoint keywords (values vs select/with) so the
Choice never ambiguously commits. No worker change — do_sql_insert
already executes the validated SQL and re-persists, and the engine
handles insert-from-query.
Tests: grammar accept (plain / column-list+projection / WITH-
prefixed / trailing-semi) and reject (__rdbms_* on the SELECT's
FROM slot, incomplete select); integration parse-path lowering +
worker round-trip (rows land, CSV re-persisted) incl. R4 WITH end-
to-end; walker cross-cut that the Phase-2 unknown_column diagnostic
fires on the INSERT…SELECT projection; DA-gate test that a self-
sourced INSERT…SELECT runs as a plain insert (no cascade summary —
that is DELETE-only). Still behind the dev `sqlinsert` entry word
(shared `insert` is 3j). 1493 tests green, clippy clean.
SQL_INSERT_SHAPE (INTO <table> [(cols)] VALUES tuple(s)) with __rdbms_*
target rejection; Command::SqlInsert{sql,target_table}; Request::RunSqlInsert
+ do_sql_insert worker (tx-guarded: execute, then finalize_persistence for
CSV + history before commit, so failures roll back and don't re-persist).
Auto-show is best-effort via last_insert_rowid range.
Isolated behind a dev `sqlinsert` entry word (Advanced) so the SQL path is
testable without making `insert` a shared word yet (that's 3j, after 3d
auto-fill parity). Command::SqlInsert carries only sql+target_table; the
plan's listed_columns/returning land in 3d/3g where they're read.
6 grammar accept/reject tests + 8 integration tests (single/multi-row,
column-list, full-arity, history, rollback-on-failure, multi-row atomicity,
parse-path reconstruction, internal-table rejection). 1452 baseline green.
claude@clouddev1