diff --git a/CLAUDE.md b/CLAUDE.md index 75520ee..f4d72ca 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -89,8 +89,11 @@ Current decisions at a glance (each backed by an ADR): - **FK relationships:** declared via `add 1:n relationship [as ] from

. to . [on delete ] [on update ] [--create-fk]`. Implemented through the - rebuild-table primitive — the same machinery covers B2's - pending column drop/rename/type-change use cases (ADR-0013). + rebuild-table primitive — the same machinery backs B2's + column drop/rename/type-change operations (ADR-0013), which + are implemented in both simple mode (`drop column` / + `rename column` / `change column`) and advanced mode + (`ALTER TABLE`, ADR-0035 §4e/§4f). - **Data operations:** `insert / update / delete / show data` with required WHERE plus `--all-rows` opt-in for unfiltered ops; auto-show after writes shows just the affected rows; @@ -192,9 +195,6 @@ not yet implemented: 1–4 of ADR-0015). Pending pieces: `export` / `import` (Iter 5), `--resume` + persistent input history hydration + migration framework scaffold (Iter 6). -- **Column drops/renames/type changes** (B2 / C2 partial): the - rebuild-table primitive (ADR-0013) is in place; the grammar - and dispatch are pending. - **Modify relationship** (C3a): drop+add covers the use case today. - **m:n convenience** (C4): auto-generates a junction table diff --git a/docs/adr/0000-record-architecture-decisions.md b/docs/adr/0000-record-architecture-decisions.md index 810a424..deefd4e 100644 --- a/docs/adr/0000-record-architecture-decisions.md +++ b/docs/adr/0000-record-architecture-decisions.md @@ -38,6 +38,25 @@ The index lists ADRs in numerical order. Each entry shows the number, title, and — where relevant — status annotations such as "Superseded by ADR-NNNN" or "Deprecated". +## Out-of-scope discipline + +ADRs (and the plans they spawn) lean heavily on "out of scope" language. +The phrase carries two very different meanings, and conflating them +misleads a later reader: + +- **Deferred** — out of scope *for this plan / phase / step*, but a + reasonable thing to do later. A sequencing decision, effectively a + tracked TODO. Where possible, point at where it will be picked up. +- **Rejected** — considered and deliberately *not* done, on principle. + Durable. State the reason. + +When writing an out-of-scope item, **say which kind it is** — e.g. +`OOS (deferred)` / `OOS (rejected: )`, or the equivalent in prose +— so a future reader can tell a standing decision from a not-yet. A bare +"out of scope" is ambiguous and tends to read, wrongly, as permanent. +(Motivating example: ADR-0030 §13 OOS-2 was a deferred scope exclusion +that read as a permanent rejection until ADR-0039 lifted it.) + ## Consequences - New significant decisions require an ADR before or alongside the diff --git a/docs/adr/0030-advanced-mode-sql-surface.md b/docs/adr/0030-advanced-mode-sql-surface.md index 0df6535..98b2c61 100644 --- a/docs/adr/0030-advanced-mode-sql-surface.md +++ b/docs/adr/0030-advanced-mode-sql-surface.md @@ -301,6 +301,9 @@ until then. by the items panel's design (`S2`) but need their own model. - **OOS-2.** `EXPLAIN` of advanced-mode SQL queries. The DSL `explain` (ADR-0028) still works for what it already wraps. + **(Superseded by ADR-0039, 2026-05-27 — this was a *deferred* + scope exclusion, not a principled rejection; EXPLAIN over advanced + SQL is now in scope, as a deferred follow-up.)** - **OOS-3.** A function/expression allowlist for full expression-level engine neutrality (§7) — best-effort now. - **OOS-4.** Multi-statement batches and transaction control. diff --git a/docs/adr/0033-sql-dml-grammar.md b/docs/adr/0033-sql-dml-grammar.md index ce2759b..c48398b 100644 --- a/docs/adr/0033-sql-dml-grammar.md +++ b/docs/adr/0033-sql-dml-grammar.md @@ -1476,6 +1476,83 @@ forward-references that future ADR so the requirement is not lost. side-channel until then. No structure built in 3j assumes mode is irrelevant to execution. +## Amendment 4 — `update … --all-rows` falls back to the DSL; Amendment 3's counter-example was a bug (2026-05-27) + +Designing the DSL → SQL teaching echo (ADR-0038) re-examined Amendment 3's +**counter-example** — the claim that `update T set x = 42 --all-rows` in +Advanced mode does *not* fall back to the DSL because the SQL `UPDATE`'s +`SET ` "greedily consumes `--all-rows` as the expression +`42 - -all - rows` (with `all`/`rows` as column refs)." **That is a bug, +not correct behaviour**, and this amendment reverses it. Recorded with +explicit user approval (2026-05-27). + +### Why it is a bug + +- The walker has **no `--` comment support** (it lexes two minus + operators); the engine *does* treat `--` as a line comment. So the + walker's parse (`42 - -all - rows`) **diverges from execution** (which + runs `update T set x = 42`). +- Absent columns literally named `all` and `rows`, the walker is + **silently accepting a `SET` expression over non-existent columns** — + precisely the case ADR-0027 says to flag ("mark error if we know it + will fail at runtime"). It only *appears* to succeed because the + engine's comment leniency masks the divergence. +- **Trailing SQL comments are not a supported feature** of the surface, + so relying on the engine to treat `--all-rows` as one is wrong. + +So the SQL `UPDATE` shape should **not** match `… --all-rows`. + +### Decision + +The `--all-rows` token sequence makes the **SQL `UPDATE` shape fail**, so +dispatch **falls back to the DSL** `Command::Update { filter: AllRows }` +— restoring symmetry with `delete … --all-rows`, which already falls +back (the SQL `DELETE` has no slot to absorb the flag). Consequences: + +- `update … --all-rows` in Advanced mode is the **DSL** command, runs all + rows via the safe DSL path, and (ADR-0038) gains the teaching echo + `UPDATE T SET … ` with the `WHERE` omitted. +- **No `--` comment feature is introduced**; the playground still does + not support trailing SQL comments (consistent with the rest of the + surface, engine-neutral). + +### Mechanism — key on the **adjacent `--`** + +The DSL `--all-rows` is matched by `Node::Flag("all-rows")` via +`consume_flag` (`walker/lex_helpers.rs`), which requires an **adjacent +`--`**. The SQL expression grammar has no flag node, so it consumes the +same source as `- -all - rows` (two minus operators, `all`/`rows` as +column refs) and the SQL `UPDATE` shape wins (SQL-first). Crucially, +`--all-rows` and a legitimate `- -all` tokenise *identically* once split, +so the **only** robust discriminator is the adjacency of the two dashes: + +- **The fix:** the SQL expression treats an **adjacent `--`** as a + boundary it will not consume, so the SQL `UPDATE` shape **fails** there + and dispatch falls back to the DSL, whose `Node::Flag` matches + `--all-rows`. (Adjacency is what separates the DSL flag from real + arithmetic — see the preserved case below.) + +Verified consequences (probed against the current grammar), which the +build must lock down test-first: + +- `update T set x = 42 --all-rows` → DSL `Update { AllRows }` *(the goal; + inverts `sql_dml_e2e.rs::e2e_update_all_rows_in_advanced_does_not_fall_back_to_dsl`)*. +- `update T set x = 42 - -3` → **unchanged**, stays `SqlUpdate` (= 45): + the dashes are space-separated, not an adjacent `--`. This invariant is + non-negotiable and gets its own test. +- `update T set x = 42--3` and `update T set x = 42 --col` *(adjacent + `--` before a number or a real column)* → today they parse `Ok` as + `SqlUpdate`; after the fix the SQL expression stops at `--` and the DSL + flag grammar (which accepts only `all-rows`) rejects the rest, so they + become **parse errors**. This is a deliberate, acceptable behaviour + change for these contrived adjacent-dash inputs — the playground does + not support `--` line comments, so there is no valid reading of an + adjacent `--` here. *(User heads-up flagged 2026-05-27.)* + +The existing `delete … --all-rows` fall-back is unaffected. Folded into +the ADR-0038 echo effort (the fix that makes `update … --all-rows` +echoable). + ## See also - ADR-0005 — the ten-type vocabulary INSERT works with. diff --git a/docs/adr/0035-advanced-mode-sql-ddl.md b/docs/adr/0035-advanced-mode-sql-ddl.md index 3bc1dda..afdbee1 100644 --- a/docs/adr/0035-advanced-mode-sql-ddl.md +++ b/docs/adr/0035-advanced-mode-sql-ddl.md @@ -616,6 +616,117 @@ Single-column UNIQUE column drops are a **parallel** gap (different mechanism — ADR-0029 column-level `drop constraint`) and are **out of scope** here. +## Amendment 2 — Standard-first dialect + `ALTER COLUMN` constraint gap-fill (2026-05-27) + +Designing the **DSL → SQL teaching echo** (ADR-0030 §10, specified in +ADR-0038) surfaced two related things about this ADR's surface. First, a +**dialect drift**: ADR-0030 frames advanced mode as "the **standard-SQL** +surface," but §4f shipped the type-change verb as bare +`ALTER COLUMN … TYPE` — the **PostgreSQL shorthand** — and explicitly +declined the ISO `SET DATA TYPE` synonym (§4f, line "no `SET DATA TYPE` +synonym"). Second, **gaps**: advanced mode has no way to toggle `NOT NULL` +or `DEFAULT` on an existing column, though simple mode does +(ADR-0029 `add`/`drop constraint`), and the rebuild primitive that would +back them is already in place (it backs §4f type-change and §4g +constraint-add). This amendment records a **dialect stance** and **fills +the clean gaps** so the echo can emit portable SQL that is also runnable +in advanced mode. Recorded with explicit user approval (2026-05-27). + +### The dialect stance — standard-first (refines ADR-0030) + +Where ISO SQL provides a spelling, the **authored grammar's canonical +form is the ISO one**, and the **echo emits the ISO form**. A widely- +recognised vendor shorthand *may* be **accepted** as a synonym (so a +learner who knows it is not punished), but it is never the canonical or +emitted form. Where ISO provides **no** spelling for an operation the +playground teaches, **one** widely-recognised vendor spelling is adopted +as a **deliberate, documented extension** — not silently. This realigns +the surface with ADR-0030's stated posture and makes the divergence a +conscious, recorded choice rather than drift. + +The stance applies to the whole advanced surface going forward; this +amendment exercises it on the `ALTER COLUMN` family. + +### Type change: ISO `SET DATA TYPE` canonical, `TYPE` retained as a synonym + +Reverses §4f's "no `SET DATA TYPE` synonym." The grammar now accepts +**both** `ALTER COLUMN SET DATA TYPE ` (ISO; canonical) and +`ALTER COLUMN TYPE ` (PostgreSQL; accepted synonym, no +breakage for already-shipped usage). Both decompose to the same §4f +`AlterColumnType` action and the same `ChangeColumnMode::ForceConversion` +executor — semantics are **unchanged**; only the accepted spelling set +and the *canonical/echoed* form change. The echo (ADR-0038) emits +`SET DATA TYPE`. + +### New: `SET/DROP DEFAULT` (ISO) and `SET/DROP NOT NULL` (the one extension) + +Four new `AlterColumnType`-family actions under `ALTER COLUMN `: + +| Spelling | Standing | Decomposes to (ADR-0029 executor) | +|---|---|---| +| `SET DEFAULT ` | **ISO-standard** | `do_add_constraint(Default)` | +| `DROP DEFAULT` | **ISO-standard** | `do_drop_constraint(Default)` | +| `SET NOT NULL` | **documented extension** | `do_add_constraint(NotNull)` | +| `DROP NOT NULL` | **documented extension** | `do_drop_constraint(NotNull)` | + +`SET DEFAULT`/`DROP DEFAULT` are taken directly from the ISO +`` set. **`NOT NULL` toggling has no ISO spelling** +— in the standard `NOT NULL` is a column constraint, not an in-place +`ALTER COLUMN` verb, and the vendors diverge (PostgreSQL +`SET/DROP NOT NULL`; SQL Server `ALTER COLUMN NOT NULL`; +MySQL `MODIFY`; Oracle `MODIFY`). Per the stance, **one** spelling is +adopted as a deliberate extension: **PostgreSQL's `SET/DROP NOT NULL`**, +chosen because it is the only form that is type-independent (it does not +force the user to restate the column type), reads as plain English, and +composes uniformly with the ISO `SET/DROP DEFAULT` it sits beside. + +`SET DEFAULT`'s value slot reuses the §4a.2 / §4e **raw `sql_expr`** +default mechanism (`default_sql`), so a default may be any expression +the create-table `DEFAULT` accepts — one syntax, not a third +(ADR-0030 §11). + +### Execution — rebuild-backed, no new low-level op + +Each new action **runtime-decomposes to an existing ADR-0029 executor** +(`do_add_constraint` / `do_drop_constraint`), exactly as §4e/§4f +decompose their actions — the populated-column **pre-flight dry-run +guard** (ADR-0029 §5) and the internal-`__rdbms_*` guard come for free. +No new worker layer. The grammar discriminates the `ALTER COLUMN …` +tail by its leading keyword: `type` / `set data type` (type change), +`set not null` / `drop not null`, `set default` / `drop default` — the +`set`/`drop` lead is new alongside §4f's `type` lead. + +### Parity reached, and the one residual gap + +This brings advanced mode to **constraint-modification parity with +simple mode (ADR-0029) for `NOT NULL` and `DEFAULT`** — add and drop, +both directions. It closes the simple↔advanced asymmetry the echo design +flagged for those ops. + +**Residual gap (deliberately not closed here):** dropping a +**column-level `UNIQUE` or `CHECK`** (the single-column, *anonymous* +constraints simple mode adds via ADR-0029 `add constraint unique/check`). +`DROP CONSTRAINT ` (§4g) + the derived composite-UNIQUE name +(Amendment 1) resolve *table-level* / *named* constraints; a single- +column column-level `UNIQUE` lives as the column's `unique` flag and a +column-level `CHECK` is likewise anonymous, so neither has a portable +name to address. This is the same class Amendment 1 called a "parallel +gap … out of scope." Consequently ADR-0038's catalogue marks +`drop constraint unique/check from T.col` as **no headline echo** (a +residual gap), rather than inventing a name or a recipe. Flagged for the +user; closing it (e.g. extending the derived-name approach to single- +column UNIQUE) would be its own small follow-up. + +### Engine neutrality holds (the rebuild stays hidden) + +The chosen spellings are **portable SQL**, not engine features. The fact +that *this* engine satisfies `SET NOT NULL` / `SET DATA TYPE` via a table +rebuild (because it lacks in-place `ALTER`) is a **Category-1 engine +implementation detail** (ADR-0038's taxonomy) and stays **invisible** — +no recipe, no rebuild steps surfaced — exactly as §9 and ADR-0030 §7 +require. A learner sees the standard statement; the engine's means of +honouring it is not the lesson. + ## Consequences - Advanced mode reaches DDL parity with simple mode and adds diff --git a/docs/adr/0037-execution-time-mode-side-channel.md b/docs/adr/0037-execution-time-mode-side-channel.md new file mode 100644 index 0000000..0546563 --- /dev/null +++ b/docs/adr/0037-execution-time-mode-side-channel.md @@ -0,0 +1,182 @@ +# ADR-0037: Execution-time mode side-channel (the three-way submission mode) + +## Status + +Proposed (design agreed with the user 2026-05-27; pending `/runda` + +implementation). Redeems the follow-up **deferred by ADR-0033 +Amendment 3**, which named this very ADR and its motivating consumer. + +## Context + +ADR-0033 Amendment 3 ("Execution-time mode is side-channel — deferred +to its own ADR") recorded a requirement and deferred it: + +> every command should — at **execution time** — know which of three +> modes it ran under: `simple`, `advanced`, or `advanced-one-shot` +> (the `:` escape from Simple mode, ADR-0003), so execution can adjust +> **output** without changing **identity** (e.g. a Simple-mode +> `create table` echoing the generated SQL when run in Advanced mode, +> while staying silent in Simple mode). + +That echo is the **DSL → SQL teaching bridge** (ADR-0030 §10), specified +as its own Phase-5 ADR (ADR-0038). ADR-0038 is the motivating — and, at +time of writing, only — consumer of this side-channel. This ADR +establishes the channel; ADR-0038 consumes it. + +### What exists today + +- The persistent input mode `Mode` (`src/mode.rs`) is **two-way**: + `Simple` / `Advanced`. Its doc comment is explicit that the one-shot + `:` escape "is handled at submission time in `app::App::submit`, **not + as additional state here**." +- The one-shot `:` is therefore a **transient, per-submission** + property, collapsed to an effective mode at submit time. It is not + persistent state and was deliberately kept out of `Mode`. +- The only mode information that survives past submission is a + **rendering** side-channel: `OutputLine.mode_at_submission: Mode` + (the `[simple]`/`[advanced]` echo-line tag in `ui.rs`). It is two-way + and exists purely to label the input-echo line. +- Neither `Action::ExecuteDsl` nor the database worker + (ADR-0010) carries any mode. Execution is mode-agnostic. + +### Why the gap blocks the echo + +The teaching echo (ADR-0038) renders the equivalent SQL **beneath the +`[ok]` summary**. That summary is built in the App's outcome handler +(`App::update`, the `Dsl*Succeeded` arms) from the **worker's result +event** — not at dispatch time. At that point: + +- The typed `Command` is available (the success events already carry it). +- The submission mode is **not** available. `self.mode` is the *current* + persistent mode, which is unreliable for this purpose: it can change + between submission and outcome, and for a `:` one-shot it never + reflected the effective mode of that single line at all. + +So the echo needs the **effective submission mode delivered to +execution/outcome**. And several echo forms (ADR-0038 — auto-resolved +index / relationship names, generated `shortid` values, lossy-conversion +counts) are only knowable **after the worker has run**, so the echo's +data is fundamentally worker-produced. The natural place to gate and +build mode-dependent output is therefore the execution path, with the +mode threaded to it — exactly the side-channel Amendment 3 anticipated. + +## Decision + +Introduce a **three-way effective submission mode**, computed at submit +time and threaded through the `Action` → worker interface, available at +**execution time** to adjust **output only**. Command identity, dispatch, +and execution semantics are **unchanged** (ADR-0033 Amendment 3: identity +is intrinsic to the mode-rooted grammar path, not a flag). + +### 1. A new per-submission enum, distinct from persistent `Mode` + +```rust +/// The effective mode of a single submitted line, resolved at submit +/// time. Distinct from the persistent input `Mode` (which stays +/// two-way) because the one-shot `:` escape is a transient per-line +/// property, never persistent state (ADR-0003; `mode.rs`). +pub enum SubmissionMode { + Simple, + Advanced, + AdvancedOneShot, +} +``` + +**This refines Amendment 3's framing.** Amendment 3 sketched "widening +`Mode` to the three-way distinction." On implementation review, a +**separate enum** is cleaner: `Mode` models *persistent input state*, +and `mode.rs` deliberately keeps the one-shot out of it. Folding a +transient `AdvancedOneShot` into the persistent `Mode` would contradict +that design note and let a non-persistable value leak into persistent- +mode call sites. `SubmissionMode` carries the three-way distinction on +the per-submission channel where it belongs; `Mode` stays two-way and +untouched. The *requirement* Amendment 3 recorded is met; only the +**shape** is refined. (Flagged for `/runda`/user challenge — it is an +internal architecture choice with no user-facing effect.) + +The effective mode is resolved at submit time from the persistent +`Mode` plus whether the `:` sigil was used: + +- persistent `Simple`, no `:` → `Simple` +- persistent `Simple`, `:` prefix → `AdvancedOneShot` +- persistent `Advanced` → `Advanced` (the `:` is a no-op there) + +### 2. Threaded through `Action::ExecuteDsl` to the worker + +`Action::ExecuteDsl` gains a `submission_mode: SubmissionMode` field; +the worker request mirrors it. Execution thus knows, per command, the +effective mode it ran under. The value is **output-only**: no executor +branches its *effect* on it (that would be a behavioural mode dependency, +which ADR-0033 Amendment 3 forbids — identity and effect are intrinsic). + +### 3. The worker produces mode-dependent output; the App renders it + +For the first consumer (ADR-0038): when the command is a **DSL-form** +command (`Command::CreateTable`/`Insert`/… — *not* the `Sql*` variants) +and `submission_mode` is `Advanced` or `AdvancedOneShot`, the worker +builds the teaching echo (equivalent SQL + any category-3 expansion +data — ADR-0038) and returns it on the result event. In `Simple` mode, +or for a command typed as SQL, no echo is produced. The App renders the +returned echo as de-emphasised `OutputLine`(s) beneath `[ok]`. + +Co-locating echo construction with execution is deliberate: the echo's +harder forms (resolved auto-names, generated `shortid`s, conversion +counts) are facts the worker already computes. Gating on the threaded +mode means the work happens **only when an echo will be shown**. + +**Non-interactive re-execution does not echo.** `replay` (ADR-0034) +re-runs recorded commands through the dispatch pipeline in advanced mode +(ADR-0033 Amendment 3) — but a per-line teaching echo there would bury +the replay summary in noise. So replayed lines (and any future +programmatic re-execution) dispatch with a `SubmissionMode` that +**suppresses** mode-dependent output: command *identity* still parses in +advanced mode (Amendment 3, unchanged), but no echo fires. The mechanism +— a fourth non-echoing context, or an `interactive: bool` alongside the +mode — is a build choice; the contract fixed here is *replay is silent*. + +### 4. Scope of this ADR + +This ADR establishes the channel and the resolution rule **only**. The +echo renderer, its catalogue, and the `Value → SQL-literal` machinery +are ADR-0038. The advanced-mode `ALTER COLUMN` gap-fill the echo relies +on is the ADR-0035 amendment. No echo behaviour is specified here beyond +the gating contract in §3. + +## Alternatives considered + +- **Widen `Mode` to three-way (Amendment 3's literal sketch).** Avoids a + new type but conflates transient per-submission state with persistent + input state, against `mode.rs`'s explicit design note. Rejected for + §1's separate enum. +- **App-side gating, worker always returns echo data.** Keep the worker + mode-agnostic; have it return echo data on *every* result, and let the + App decide whether to render from `mode_at_submission` + command shape. + Rejected: it computes the echo unconditionally (including in Simple + mode, where it is never shown), does not generalise to other mode- + dependent output, and re-opens exactly the "the echo is purely render- + side" framing the user has twice ruled against — the settled direction + is execution-time mode awareness, per Amendment 3. + +## Consequences + +- Additive and small: a new enum, one field on `Action::ExecuteDsl` and + the worker request, and the submit-time resolution rule. No change to + parsing, dispatch, command identity, or any executor's effect. +- The persistent `Mode` enum and `OutputLine.mode_at_submission` are + unchanged. (ADR-0038 may enrich the render tag separately; not required + here.) +- Future mode-dependent **output** has a home. Anything touching command + *identity* or *effect* does not belong here (Amendment 3). +- Tests: submit-time resolution for all three cases (incl. `:` one-shot + and the Advanced-mode `:`-is-a-no-op case); the field survives the + `Action` → worker round-trip; a Simple-mode DSL command yields no echo + request while an Advanced / one-shot one does (the gating contract). + +## See also + +- ADR-0033 Amendment 3 — deferred this side-channel; defines the + intrinsic command-identity model this ADR must not disturb. +- ADR-0030 §10 — the DSL → SQL teaching bridge (the motivating consumer). +- ADR-0038 — the teaching echo; the consumer built on this channel. +- ADR-0003 — input modes and the one-shot `:` escape. +- ADR-0010 — the database worker thread this mode is threaded to. diff --git a/docs/adr/0038-dsl-to-sql-teaching-echo.md b/docs/adr/0038-dsl-to-sql-teaching-echo.md new file mode 100644 index 0000000..56a1e95 --- /dev/null +++ b/docs/adr/0038-dsl-to-sql-teaching-echo.md @@ -0,0 +1,299 @@ +# ADR-0038: The DSL → SQL teaching echo + +## Status + +Proposed (design agreed with the user 2026-05-27; pending `/runda` + +implementation). **Realises ADR-0030 §10** ("The DSL → SQL teaching +bridge") — the Phase-5 echo that **ADR-0035 §12 forward-referenced** as +"a separate ADR." Builds on **ADR-0037** (the execution-time mode +side-channel that gates it) and **ADR-0035 Amendment 2** (the standard- +first dialect + `ALTER COLUMN` gap-fill that makes its DDL echoes +runnable). + +## Context + +ADR-0030 §10 specified a teaching bridge: when a **DSL-form** command +runs **in advanced mode**, its output includes the equivalent SQL, so a +learner who knows the simple-mode form reads off how to spell it in SQL. +The §10 contract: fires only for DSL-entered commands in advanced mode +(a command already typed as SQL is not echoed; simple mode stays +uncluttered); renders as a de-emphasised `OutputLine` beneath the `[ok]` +summary (styled-runs, ADR-0028); app-level commands have no SQL form and +are not echoed; the reverse SQL → DSL echo is out of scope (§13 OOS-5). + +### The firing reality — this is a DDL + `show data` feature + +A consequence of ADR-0033 Amendment 3 sharpens the scope and must be +stated plainly, because it is easy to over-estimate coverage: **in +advanced mode the overlapping data commands route SQL-first.** A user who +types `insert into T values (…)` / `update T set …` / `delete from T +where …` in advanced mode produces `Command::SqlInsert` / `SqlUpdate` / +`SqlDelete` — they **already typed SQL**, so §10 explicitly does not echo +them. The echo is therefore meaningful only where the **DSL spelling +differs from the SQL spelling** — i.e. the DSL-*only* surface: + +- **DDL DSL spellings** with no winning SQL competitor — `create table … + with pk`, `add`/`drop`/`rename`/`change column`, `add`/`drop + constraint`, `add`/`drop index`, `add`/`drop relationship`. +- **`show data`** (DSL-only; SQL uses `SELECT`). +- The **`--all-rows` fall-throughs** — `delete … --all-rows` falls back + to the DSL `Delete` (the SQL `DELETE` has no slot for the flag), and + `update … --all-rows` likewise falls back to the DSL `Update` per + **ADR-0033 Amendment 4** (which reclassifies Amendment 3's non-fall- + back of `update … --all-rows` as a bug and reverses it). Plain / + `where`-filtered `insert`/`update`/`delete` stay SQL-first (`Sql*`). + +Everything a learner types that is *already SQL* needs no echo by +construction. So the catalogue below is dominated by DDL. + +### Dependencies + +- **ADR-0037** delivers the `SubmissionMode` to the worker, so a command + knows at execution time whether it ran under `Advanced` / + `AdvancedOneShot` (echo) or `Simple` (silent). +- **ADR-0035 Amendment 2** adds the `ALTER COLUMN SET/DROP NOT NULL`, + `SET/DROP DEFAULT`, and the ISO `SET DATA TYPE` canonical form, so the + constraint-modification echoes are runnable advanced-mode SQL, and + fixes the dialect stance the echoes emit in. + +## Decision + +### 1. The copy-paste contract — every echo is runnable advanced-mode SQL + +The defining invariant, stronger than "looks like SQL": **each echoed +line is exactly what the user could paste into advanced mode and run — +to the same effect, except where a Category-3 *caveat* line (§6) flags a +divergence the SQL surface cannot express** (today the sole such case is +`change column … --dont-convert`). This is testable as a **round-trip**: +parse the echo through the advanced-mode walker and assert it yields a +command with the same effect as the one that produced it (a caveated +line still parses and runs — it just runs differently, which the caveat +states). A planned one-shot "copy the +echo" UX affordance depends on this contract, so it is a hard +requirement, not a nicety. Multi-statement echoes (§6 category 2) hold +the contract per line. + +### 2. Type vocabulary — the playground's own keywords + +The echo emits the playground's `Type::keyword()` spelling (`serial`, +`shortid`, `decimal`, `bool`, `date`, `datetime`, …), **not** engine +storage types. This is sound because the advanced-mode type slot +(`Type::from_sql_name`) accepts those ten keywords, so the echo round- +trips (§1), and because the curated vocabulary is what the learner knows +from the DSL and what carries the teaching information (`serial`'s auto- +increment intent, `shortid`'s identity flavour) that `INTEGER`/`TEXT` +would erase. The **statement shape** follows the standard-first dialect +(ADR-0035 Amendment 2); the **type slots** carry playground keywords. + +### 3. Firing rule + +The echo fires **iff** the executed command is a **DSL-form** command — +*not* a `Sql*` variant and *not* a `Command::App(_)` — **and** its +`SubmissionMode` (ADR-0037) is `Advanced` or `AdvancedOneShot` **and** the +command is an interactive submission (**not** a `replay`ed line). Silent +in `Simple`; silent for SQL-entered commands (they are `Sql*`); silent +for app commands (no SQL form, §10); silent during `replay` (per-line +echoes would bury the replay summary — ADR-0037 §3). + +### 4. Where it is built and rendered + +The **worker builds** the echo (ADR-0037 §3) — it alone holds the facts +several echoes need: auto-resolved index / relationship names, generated +`shortid` values, and lossy-conversion counts. It returns the echo +payload on the result event. The **App renders** it as one or more +**de-emphasised** `OutputLine`s beneath the `[ok]` summary, using the +ADR-0028 styled-runs mechanism (a dimmed `Executing SQL:` prefix; the SQL +itself in a code-ish run). One statement per line (§6 category 2). + +### 5. `Value → SQL-literal` rendering + +DML echoes substitute the **actual literal values** (not `?` +placeholders — decision B1), so the line is runnable. A per-type renderer +produces SQL literals that **round-trip** (§1): + +| Type | Literal form | +|---|---| +| `text` / `shortid` | single-quoted, `'` doubled (`'O''Hara'`) | +| `int` / `serial` | bare integer | +| `real` / `decimal` | bare numeric (decimal as authored) | +| `bool` | `true` / `false` | +| `date` / `datetime` | quoted ISO-8601 (`'2026-05-27'`) | +| `NULL` | `NULL` | + +**`blob` has no literal:** neither the DSL nor advanced SQL has a blob- +literal syntax (the playground deliberately does not provide one), so a +DSL command **cannot carry a blob value** for the echo to render — this +is moot, not a gap. + +**Auto-generated columns are omitted** from an `INSERT` echo: `do_insert` +filters out PK `serial` (rowid alias), non-PK `serial` (MAX+1), and +`shortid` columns the user did not list, and advanced-mode SQL auto-fills +the same omissions (`requirements.md` X4). So the echo omits them too — +matching both the executed statement and the advanced-mode form a learner +would type. + +### 6. The three-category framework + +Everything that happens "beyond the literal SQL line" sorts into exactly +three categories; naming them makes the rule testable: + +- **Category 1 — engine-implementation-hiding. Never surfaced.** The + table rebuild that backs `ALTER COLUMN …` / `ADD CONSTRAINT` / + `RENAME TO` on this engine; the rowid alias behind a PK `serial`; the + `MAX(col)+1` behind a non-PK `serial`. These are how *this* engine + honours a standard operation — not the lesson (ADR-0030 §7, + ADR-0035 §9). The headline echo shows the clean statement; the means + stays invisible. (Non-PK `serial` auto-fill is **here, not category 3**: + auto-increment is an ordinary database feature; only the engine's + *means* of faking it on a non-PK column is hidden.) +- **Category 2 — decomposable into advanced-mode SQL. Shown as the + runnable multi-line sequence.** One statement per line; the lines + *are* the explanation, no prose. See `drop column --cascade` and + `add relationship --create-fk` below. +- **Category 3 — a playground type-system behaviour the SQL line cannot + express (no function, no clause exists to write it). Surfaced as a + de-emphasised prose line.** The note is one of two kinds: + *illuminating* — the headline echo *is* equivalent and the note merely + reveals a value-add the SQL does not show; or *caveat* — the headline + is the **nearest** SQL but **not** equivalent because a playground + flag's semantics have no SQL form (these caveats are the §1 contract's + only exceptions). Members: + - **`shortid` generation** *(illuminating)* — no `shortid()` function + exists, so the generation cannot be written in SQL: "generated unique + shortid(s) for ``". + - **type-conversion transform** *(illuminating)* — `change column type` + when cells are actually transformed; the playground auto-converts + where standard SQL would need a cast clause it deliberately omits + (the Postgres `USING`, ADR-0035 §12): "converted `` from + `` to `` [N values with loss]". + - **`change column … --dont-convert`** *(caveat)* — the headline + `ALTER TABLE T ALTER COLUMN c SET DATA TYPE ` converts, but + `--dont-convert` left the stored values as-is, so the line is not + equivalent: "`--dont-convert` kept the stored values as-is; standard + SQL always converts, so running the line above would transform them + instead." + + All are built from the worker's existing `client_side.*` result notes + (ADR-0017 §6 / ADR-0018 §9), not recomputed. + +### 7. The catalogue + +DSL-form commands that surface in advanced mode, with their echo. The +SQL uses the standard-first dialect (ADR-0035 Amendment 2) and playground +type keywords (§2). `` denotes a worker-resolved name (auto- +generated or positional → resolved at execution, "B1"). + +**Bucket A — single runnable statement.** + +| DSL command | Echoed SQL | Cat-3 expansion | +|---|---|---| +| `create table T with pk` | `CREATE TABLE T (id serial PRIMARY KEY)` | — | +| `create table T with pk a(int),b(int)` | `CREATE TABLE T (a int, b int, PRIMARY KEY (a, b))` | — | +| `add column to T: c () [not null] [unique] [default v] [check e]` | `ALTER TABLE T ADD COLUMN c [NOT NULL] [UNIQUE] [DEFAULT v] [CHECK (e)]` | shortid (if `` = shortid) | +| `drop column from T: c` *(no covering index)* | `ALTER TABLE T DROP COLUMN c` | — | +| `rename column in T: old to new` | `ALTER TABLE T RENAME COLUMN old TO new` | — | +| `change column in T: c ()` | `ALTER TABLE T ALTER COLUMN c SET DATA TYPE ` | conversion *(illum.)* if transformed (incl. → shortid); `--dont-convert` → *caveat* (§6) | +| `add constraint not null to T.c` | `ALTER TABLE T ALTER COLUMN c SET NOT NULL` | — | +| `add constraint default to T.c` | `ALTER TABLE T ALTER COLUMN c SET DEFAULT ` | — | +| `add constraint unique to T.c` | `ALTER TABLE T ADD UNIQUE (c)` | — | +| `add constraint check to T.c` | `ALTER TABLE T ADD CHECK (e)` | — | +| `drop constraint not null from T.c` | `ALTER TABLE T ALTER COLUMN c DROP NOT NULL` | — | +| `drop constraint default from T.c` | `ALTER TABLE T ALTER COLUMN c DROP DEFAULT` | — | +| `add index [as N] on T (cols)` | `CREATE INDEX ON T (cols)` | — | +| `show data T [where …] [limit n]` | `SELECT * FROM T [WHERE …] [ORDER BY LIMIT n]` | — | +| `delete from T --all-rows` | `DELETE FROM T` | — | +| `update T set … --all-rows` | `UPDATE T SET …` | — | + +**Bucket B — resolved-name and multi-line (one statement per line).** + +| DSL command | Echoed SQL | Notes | +|---|---|---| +| `drop index on T(cols)` *(positional)* | `DROP INDEX ` | name resolved at execution | +| `add 1:n relationship [as N] from P.pc to C.cc [on delete X] [on update Y]` | `ALTER TABLE C ADD CONSTRAINT FOREIGN KEY (cc) REFERENCES P (pc) [ON DELETE X] [ON UPDATE Y]` | name resolved | +| `drop relationship (from P.pc to C.cc \| N)` | `ALTER TABLE C DROP CONSTRAINT ` | name resolved | +| `add 1:n relationship … --create-fk` *(child col created)* | `ALTER TABLE C ADD COLUMN cc ` ⏎ `ALTER TABLE C ADD CONSTRAINT FOREIGN KEY (cc) REFERENCES P (pc) …` | category 2: two runnable lines (one if the column already existed) | +| `drop column T.c --cascade` *(drops covering indexes)* | `DROP INDEX ` ⏎ … ⏎ `ALTER TABLE T DROP COLUMN c` | category 2: index names resolved; plain `DROP COLUMN` would refuse an indexed column | + +**Bucket C — no echo.** + +| DSL command | Reason | +|---|---| +| `show table T` | structure display; no SQL spelling in the surface | +| `explain …` | `EXPLAIN` of advanced SQL is OOS-2 (ADR-0030 §13) | +| `replay ` | app-lifecycle; no SQL form | +| `drop constraint unique/check from T.c` *(column-level)* | residual gap (ADR-0035 Amendment 2): anonymous column constraint, no portable name | +| `Command::App(_)` (all app commands) | §10 — no SQL form | +| `Sql*` variants, `select`/`with` | already SQL-entered — not echoed by construction | +| `insert …`, `update …` / `update … where …`, `delete …` / `delete … where …` | route SQL-first in advanced mode (`Sql*`) — already SQL, nothing to echo (the two `--all-rows` forms fall back to DSL and *do* echo — Bucket A) | + +### 8. Coverage and phasing + +The renderer is one mechanism, so the natural unit is "Bucket A + B at +once." A reasonable phasing if a smaller first slice is wanted: + +- **Phase 1** — Bucket A single-statement DDL + `show data` (the bulk of + the teaching value; no resolved-name or multi-line machinery). +- **Phase 2** — Bucket B (worker-resolved names; multi-line sequences). +- **Phase 3** — the category-3 prose expansion. + +(Coverage/phasing is a sequencing choice for the user; the catalogue +itself is fixed.) + +**Build-order dependencies (verified against the current grammar).** Two +groups of echoes emit SQL the advanced surface does **not yet parse**, so +their round-trip tests (§1) are gated on their prerequisites landing +first: + +- The **constraint-modification** rows and the `change column` row — + `ALTER COLUMN … SET DATA TYPE / SET NOT NULL / DROP NOT NULL / + SET DEFAULT / DROP DEFAULT` — require **ADR-0035 Amendment 2**. Today + `alter table T alter column c …` accepts only `type ` (probed: + `set data type` / `set not null` / `set default` all error with + "expected `type`"). So Amendment 2 must build **before** these rows' + round-trip tests. (`change column` could echo the `TYPE` synonym in + the interim, but the canonical/emitted form is `SET DATA TYPE` per + Amendment 2.) +- The **`update … --all-rows`** row requires **ADR-0033 Amendment 4** + (today it is `SqlUpdate`, not the DSL `Update` the echo needs). + +All other Bucket A/B rows round-trip against the surface as it stands +today (probe-confirmed: `create table T (id serial primary key)`, +compound PK, `add unique (c)`, `add check (…)`, `create index … on …`, +`select * from …` all parse in advanced mode). + +### 9. Out of scope + +- **The reverse SQL → DSL echo** (ADR-0030 §13 OOS-5). +- **App commands**, `show table`, `explain`, `replay` (Bucket C). +- **A `blob` literal renderer** (§5 — moot; no blob literal exists). +- **Closing the column-level UNIQUE/CHECK drop gap** (ADR-0035 + Amendment 2 residual) — those stay Bucket C until that gap is closed. +- **Surfacing category-1 engine internals** (the rebuild, rowid, MAX+1) — + never (§6). + +## Consequences + +- A new `Command → SQL` renderer plus the `Value → SQL-literal` renderer, + worker-side (§4), gated by `SubmissionMode` (ADR-0037). Each Bucket A/B + row is a round-trip test (§1); each category-3 row asserts the prose + fires from the worker note. +- The teaching surface is honest: a learner reads runnable, portable SQL + in the playground's own type vocabulary; engine quirks stay hidden; + playground-only conveniences are named, not mysterious. +- The echo's coverage is **DDL-centric by construction** (the firing + reality, Context). This is correct, not a shortfall — overlapping DML + is already SQL in advanced mode. +- Adding a new DSL-form DDL command later must add its catalogue row + + round-trip test, or consciously place it in Bucket C. + +## See also + +- ADR-0030 §10 — the teaching bridge this realises; §13 OOS-5 (no reverse + echo). +- ADR-0037 — the `SubmissionMode` side-channel that gates firing. +- ADR-0035 Amendment 2 — the standard-first dialect + `ALTER COLUMN` + gap-fill the echoes emit and rely on. +- ADR-0033 Amendment 3 — the SQL-first dispatch that makes this a + DDL + `show data` feature. +- ADR-0028 — the `OutputLine` styled-runs the echo renders through. +- ADR-0017 / ADR-0018 — the `client_side.*` notes feeding category-3. diff --git a/docs/adr/0039-explain-over-advanced-sql.md b/docs/adr/0039-explain-over-advanced-sql.md new file mode 100644 index 0000000..5aca38f --- /dev/null +++ b/docs/adr/0039-explain-over-advanced-sql.md @@ -0,0 +1,73 @@ +# ADR-0039: EXPLAIN over advanced-mode SQL queries + +## Status + +**Accepted** — decision recorded 2026-05-27. **Implementation deferred** +as a follow-up to the ADR-0037/ADR-0038 teaching-echo effort: the +decision below is settled, but the full design has not been `/runda`'d +or built, and is *not* part of that pass. **Supersedes ADR-0030 §13 +OOS-2.** + +## Context + +ADR-0028 gave the DSL `explain` command: a prefix over `show data` / +`update` / `delete` that runs `EXPLAIN QUERY PLAN` and renders an +annotated, span-styled plan tree. ADR-0030 §13 **OOS-2** excluded +"`EXPLAIN` of advanced-mode SQL queries." + +On readback (2026-05-27) that exclusion is a **deferred** out-of-scope +item, not a **rejected** one (see ADR-0000's out-of-scope discipline): +its own wording — *"the DSL `explain` still works for what it already +wraps"* — shows it was "not included in this surface," never "undesirable +for teaching." There was no pedagogical argument against it; it simply +fell outside the Phase-4/5 SQL-surface scope. It surfaced while +characterising advanced-mode `explain` (briefly suspected a bug; it was +OOS-2 behaving exactly as written). + +Letting a learner see the plan for the SQL they *wrote* is a natural +extension of ADR-0028's intent, so OOS-2 is lifted. + +## Decision + +`explain` works over advanced-mode SQL queries — the SQL commands +`Select` / `SqlInsert` / `SqlUpdate` / `SqlDelete` — in addition to the +DSL `ShowData` / `Update` / `Delete` it already wraps (ADR-0028). It runs +`EXPLAIN QUERY PLAN` over the command's validated SQL text and renders +through the **existing ADR-0028 plan tree**. Advanced mode only (the SQL +commands are advanced-only); the DSL `explain` stays available in both +modes, unchanged. **Supersedes ADR-0030 §13 OOS-2.** + +## Design sketch (deferred to the build) + +- **Grammar.** The `explain` inner gains the SQL statement shapes in + advanced mode, alongside the DSL trio — mirroring how `explain` already + wraps the DSL nodes, here wrapping the SQL command shapes. +- **Execution.** Run `EXPLAIN QUERY PLAN` over the carried SQL text (the + `Sql*` / `Select` commands already hold validated text); reuse + ADR-0028's plan capture + renderer. `EXPLAIN QUERY PLAN` never executes + the statement, so explaining a destructive SQL command is safe — the + same property ADR-0028 already relies on. +- **Mode.** SQL inner only in advanced mode; DSL inner in both, unchanged. + +Built test-first when picked up. + +## Out of scope + +- **EXPLAIN of DDL** (`CREATE` / `ALTER` / `DROP`). `EXPLAIN QUERY PLAN` + applies to `SELECT` / `INSERT` / `UPDATE` / `DELETE`; DDL has no query + plan. *(Deferred — may be revisited if a useful rendering emerges; per + ADR-0000's out-of-scope discipline, this is deferred, not rejected.)* + +## Consequences + +- A self-contained feature, orthogonal to the DSL → SQL echo (ADR-0038): + the echo renders SQL *from* DSL commands; this explains SQL the user + *wrote*. They share nothing but the plan renderer's lineage. +- One OOS item in ADR-0030 §13 is now superseded; the rest stand. + +## See also + +- ADR-0028 — the DSL `explain` and the span-styled plan tree this reuses. +- ADR-0030 §13 — OOS-2, superseded here. +- ADR-0032 / ADR-0033 — the SQL `SELECT` / DML this explains. +- ADR-0000 — the out-of-scope discipline that reframed OOS-2 as deferred. diff --git a/docs/adr/README.md b/docs/adr/README.md index b8ed8f1..7f7d15e 100644 --- a/docs/adr/README.md +++ b/docs/adr/README.md @@ -35,10 +35,13 @@ This directory contains the project's ADRs, recorded per - [ADR-0027 — Input-field validity indicator](0027-input-validity-indicator.md) — **Accepted**, a debounced `[ERR]` / `[WRN]` marker at the input row's right edge, backed by a walker diagnostics-severity model (parse-outcome + schema-existence); advisory, never blocks submission (`S6`); Amendment 1 adds a `LIKE`-on-numeric-column WARNING - [ADR-0028 — Query plans (`EXPLAIN QUERY PLAN`)](0028-query-plans.md) — **Accepted**, an `explain` prefix command over `show data` / `update` / `delete`; an annotated, span-styled plan tree; introduces the `OutputLine` styled-runs mechanism (ADR-0016's deferred per-span styling) (`QA1` / `QA2`) - [ADR-0029 — Column constraints (NOT NULL / UNIQUE / CHECK / DEFAULT)](0029-column-constraints.md) — **Accepted**, the four column-level constraints declared in the column-spec suffix (`create table` / `add column`) and modified on existing columns via `add constraint …` / `drop constraint …`; a pre-flight dry-run guards populated columns; `CHECK` reuses the ADR-0026 expression grammar via `Subgrammar` (`C3`) -- [ADR-0030 — Advanced mode: the standard-SQL surface](0030-advanced-mode-sql-surface.md) — **Accepted**, SQL added as grammar *within the unified grammar tree* (ADR-0024), not a separate batch parser — so SQL gets the same completion / highlighting / hints / parse-errors as the DSL; mode gates the SQL forms; DDL routes through the typed `Command` executor (metadata + type vocabulary preserved), DML and `SELECT` execute as validated SQL; engine-neutral posture, the DSL→SQL teaching echo; supersedes ADR-0001's `sqlparser-rs` reservation; phased plan (`Q1` / `Q2` / `Q4`) +- [ADR-0030 — Advanced mode: the standard-SQL surface](0030-advanced-mode-sql-surface.md) — **Accepted**, SQL added as grammar *within the unified grammar tree* (ADR-0024), not a separate batch parser — so SQL gets the same completion / highlighting / hints / parse-errors as the DSL; mode gates the SQL forms; DDL routes through the typed `Command` executor (metadata + type vocabulary preserved), DML and `SELECT` execute as validated SQL; engine-neutral posture, the DSL→SQL teaching echo; supersedes ADR-0001's `sqlparser-rs` reservation; phased plan (`Q1` / `Q2` / `Q4`); **§13 OOS-2 (EXPLAIN of advanced SQL) superseded by ADR-0039** - [ADR-0031 — The SQL expression grammar](0031-sql-expression-grammar.md) — **Accepted**, the stratified SQL expression grammar fragment commissioned by ADR-0030 §3: a single precedence ladder (`OR`/`AND`/`NOT`, the comparison/`LIKE`/`IN`/`BETWEEN`/`IS NULL` predicate set, arithmetic incl. `||`, function calls, `CASE`) — the superset of ADR-0026's DSL `WHERE` grammar, authored as a parallel fragment so simple mode is untouched; pure validation, builds **no** AST (consumers run/store SQL as text per ADR-0030 §4/§6); reuses ADR-0026's `Subgrammar` recursion + depth cap unchanged; subquery expressions and qualified column refs deferred to ADR-0030 Phase 2 - [ADR-0032 — The full SQL `SELECT` grammar](0032-sql-select-grammar.md) — **Accepted**, the Phase-2 grammar commissioned by ADR-0030 §3: full `SELECT` with `INNER`/`LEFT`/`RIGHT`/`FULL OUTER`/`CROSS` joins, `GROUP BY`/`HAVING`, all four set ops (`UNION`/`UNION ALL`/`INTERSECT`/`EXCEPT`), `WITH` and `WITH RECURSIVE` CTEs, `LIMIT … OFFSET`, `DISTINCT`, `t.*`, and bare-alias projection (lifting Phase-1 §4.2); additive extensions to ADR-0031's `sql_expr` for scalar subqueries, `IN (SELECT …)`, `[NOT] EXISTS`, and qualified column refs (redeeming ADR-0031 §7 OOS-1/OOS-2); grammar-recursion via `Subgrammar(&SQL_SELECT_COMPOUND)` reuses ADR-0026's `MAX_SUBGRAMMAR_DEPTH = 64` cap unchanged; **softens ADR-0030 §8's "ambient assistance comes for free" claim**: completion scope needs new `WalkContext` accumulators (a `from_scope_stack` of `ScopeFrame`s holding `from_scope` / `cte_bindings` / `projection_aliases`), a **new walker node variant `Node::ScopedSubgrammar(&Node)`** as the push/pop trigger (existing `Node::Subgrammar` unchanged so DSL `Expr` and `sql_expr` recursion are unaffected), qualified-prefix completion narrowing, body-projection-derived CTE column resolution (so `SELECT *` and explicit-projection CTE bodies both yield real column completion past `cte_alias.|`), and a **post-walk fixup pass** that re-resolves projection-list identifier highlighting/validity once `FROM` is parsed (the projection-before-FROM problem); classifies every Phase-2 validation case against ADR-0027's ERROR/WARNING guideline (§11): five new `diagnostic.*` keys for parse-time-detectable cases (unknown qualifier, ambiguous column, projection-alias misplaced, CTE/compound arity mismatch) plus eight `engine.*` translation keys; a MatchedPath-walking predicate-warnings variant that closes the Phase-1 gap where SQL `WHERE` expressions emitted no `LIKE`-on-numeric / `= NULL` / type-mismatch warnings (ADR-0027 Amendment 1 finally extends to the SQL surface); adds a worker-side post-prepare type-resolution pass via engine column-origin metadata so bare column refs recover their playground type (partially lifting Phase-1 §4.5, the bool→0/1 case) — `Cargo.toml` gains `column_metadata` to rusqlite features (verified against pinned 0.39.0); `__rdbms_*` rejection extended to every new table-source slot; Amendment 1 narrows §12's resolution rule from a grammar-side structural classification to "trust the engine's column-origin metadata verbatim" after an empirical probe showed origin metadata follows through non-recursive CTEs, scalar subqueries, derived tables, set ops, and joins — the one structural exception is recursive CTE result columns, which return None and stay typeless; Amendment 2 records that §10.6's "rewrite the highlight class" prescription is realised via the two-pass schema-existence diagnostic + the renderer's diagnostic-overlay path (no separate per-byte rewrite step needed; no new HighlightClass variant), and that the projection-before-FROM completion narrowing has been improved by an `src/completion.rs` look-ahead probe when the leading walk's `from_scope` is empty but the full input parses -- [ADR-0033 — The full SQL DML grammar (`INSERT` / `UPDATE` / `DELETE`)](0033-sql-dml-grammar.md) — **Accepted** (implemented + verified through sub-phase 3k, 2026-05-23; phase-exit report `docs/handoff/20260523-phase-3-verification.md`), the Phase-3 grammar commissioned by ADR-0030 §3: single- and multi-row `INSERT` (incl. `INSERT … SELECT` recursing through ADR-0032's `SQL_SELECT_COMPOUND`), `UPDATE` with `SET` assignment list, `DELETE`, all three optionally followed by `RETURNING projection_list`, plus full `ON CONFLICT … DO NOTHING / DO UPDATE` UPSERT on INSERT; **fixes the DSL-vs-SQL dispatch architecture for shared entry words (`insert`/`update`/`delete`)**: SQL-first / DSL-fallback in Advanced mode via a `Choice(SQL_shape, DSL_shape)` per shape, gated by a new walker capability `Node::Guard(fn)` — a zero-byte-consumption gating node that fails the enclosing Seq with a `ValidationError`; carries `Command::SqlInsert` / `SqlUpdate` / `SqlDelete` variants and `do_sql_*` worker handlers each of which knows the target table (for re-persistence) and the `returning: bool` flag (for DataResult routing); `shortid` auto-fill mirrors the DSL `do_insert` mechanism via worker post-fill; SQL DELETE produces the same per-relationship cascade summary the DSL DELETE does (ADR-0014 parity); three new walker diagnostics (`insert_arity_mismatch` ERROR, `auto_column_overridden` WARNING, `not_null_missing` WARNING) with positive + negative tests each; OOS list explicitly carves out `DEFAULT VALUES` (the project's planned seed feature), SQLite-specific `OR REPLACE` / `OR IGNORE` / `OR ABORT` / `OR FAIL` / `OR ROLLBACK` prefixes, `UPDATE FROM` multi-table updates, and WITH-prefixed DML; the `excluded` keyword inside `ON CONFLICT DO UPDATE` is a deliberate carve-out from ADR-0030 §7's engine-neutral posture (no standard-SQL UPSERT spelling exists that SQLite and PostgreSQL share); eleven phased sub-phases each with explicit exit gates + written DA gate, opening with the dispatch mechanism before any DML grammar lands; initial DA review recorded seven critiques that were resolved before status moved to Proposed; **Amendment 1 supersedes §2's dispatch mechanism**: the originally-chosen `Node::Guard(fn)` + `Choice(SQL_shape, DSL_shape)` was found during 3a to be unworkable as framed (any guard-in-`Choice` mechanism forces a `walk_choice` change — `walk_choice` only falls through on `NoMatch`, so Simple-mode valid-DSL would wrongly surface "this is SQL", and `walk_seq` treats a `NoMatch` past `idx 0` as a hard `Failed`, breaking Advanced-mode DSL fall-through); replaced by **category-grouped, mode-aware dispatch** in `walker::walk` (each `REGISTRY` entry tagged `CommandCategory::{Simple, Advanced}`, generalising the existing whole-command `is_advanced_only` gate), shared entry words carrying a node in both groups, no `Node::Guard` and no `walk_choice`/`walk_seq` change, advanced-mode completion SQL-first with DSL as a full-line fallback; **Amendment 2 (sub-phase 3f) supersedes §7's cascade mechanism**: the WHERE-injected per-child pre-count rested on a premise that was factually wrong about the DSL handler (which detects cascades by before/after row-count diffing inside a transaction, not by `Expr`-derived pre-count subqueries) and would have broken the §2 parity promise by reporting `SET NULL` the DSL path doesn't; replaced by mirroring `do_delete`'s count-diff exactly (verbatim DELETE executes, child-count diff observes the cascade — `ON DELETE CASCADE` row removals only, SET NULL deferred for both paths to preserve parity), which shares the render-layer formatter for free via `CommandOutcome::Delete` and **withdraws risk R2** (no WHERE-byte extraction, no N+1 subquery); **Amendment 3 (sub-phase 3j) records the command-identity model and defers the execution-mode side-channel**: a command is the typed outcome of a *mode-rooted* grammar path and its identity is intrinsic (Advanced mode tries SQL first, falls back to the *Simple* DSL command when no SQL branch matches a token, e.g. `delete … --all-rows`; note `update … --all-rows` does *not* fall back — the SQL `SET` expression eats `--all-rows`, harmless since the engine treats it as a comment); **Simple mode commits the DSL candidate for shared words** so the *real* DSL error surfaces, and when that line would also run in advanced mode the rendering layer **combines** them — DSL error **plus** an `advanced_mode.also_valid_sql` pointer ("… (valid as SQL in advanced mode)") — keeping the actionable DSL fix while pointing at advanced mode; bare "this is SQL" is reserved for entry words with no DSL form (`select`/`with`); a fully-overlapping input (`insert … values …`) legitimately yields *two distinct commands* (`Command::Insert` typed-AST vs `Command::SqlInsert` validated-text) that do the same thing but execute differently (ADR-0030 §4), so each is tested in the mode that produces it; **corrects the plan's 3j exit-gate premise** that the DSL DML tests run in Simple mode (they call `parse_command`, which defaults to Advanced) — the real invariant is "Simple-mode behaviour unchanged, Advanced mode SQL-first, DSL grammar tested in Simple mode, both variants tested in their producing mode", with §6/§7 parity keeping the paths observably equivalent; and **defers to its own future ADR** the execution-time mode side-channel (three-way `Mode`: simple/advanced/advanced-one-shot threaded through `Action`→worker, for mode-dependent *output* like echoing generated SQL) — today only the *rendering* side-channel `OutputLine.mode_at_submission` exists, and the three-way distinction is not required for Phase 3 dispatch correctness +- [ADR-0033 — The full SQL DML grammar (`INSERT` / `UPDATE` / `DELETE`)](0033-sql-dml-grammar.md) — **Accepted** (implemented + verified through sub-phase 3k, 2026-05-23; phase-exit report `docs/handoff/20260523-phase-3-verification.md`), the Phase-3 grammar commissioned by ADR-0030 §3: single- and multi-row `INSERT` (incl. `INSERT … SELECT` recursing through ADR-0032's `SQL_SELECT_COMPOUND`), `UPDATE` with `SET` assignment list, `DELETE`, all three optionally followed by `RETURNING projection_list`, plus full `ON CONFLICT … DO NOTHING / DO UPDATE` UPSERT on INSERT; **fixes the DSL-vs-SQL dispatch architecture for shared entry words (`insert`/`update`/`delete`)**: SQL-first / DSL-fallback in Advanced mode via a `Choice(SQL_shape, DSL_shape)` per shape, gated by a new walker capability `Node::Guard(fn)` — a zero-byte-consumption gating node that fails the enclosing Seq with a `ValidationError`; carries `Command::SqlInsert` / `SqlUpdate` / `SqlDelete` variants and `do_sql_*` worker handlers each of which knows the target table (for re-persistence) and the `returning: bool` flag (for DataResult routing); `shortid` auto-fill mirrors the DSL `do_insert` mechanism via worker post-fill; SQL DELETE produces the same per-relationship cascade summary the DSL DELETE does (ADR-0014 parity); three new walker diagnostics (`insert_arity_mismatch` ERROR, `auto_column_overridden` WARNING, `not_null_missing` WARNING) with positive + negative tests each; OOS list explicitly carves out `DEFAULT VALUES` (the project's planned seed feature), SQLite-specific `OR REPLACE` / `OR IGNORE` / `OR ABORT` / `OR FAIL` / `OR ROLLBACK` prefixes, `UPDATE FROM` multi-table updates, and WITH-prefixed DML; the `excluded` keyword inside `ON CONFLICT DO UPDATE` is a deliberate carve-out from ADR-0030 §7's engine-neutral posture (no standard-SQL UPSERT spelling exists that SQLite and PostgreSQL share); eleven phased sub-phases each with explicit exit gates + written DA gate, opening with the dispatch mechanism before any DML grammar lands; initial DA review recorded seven critiques that were resolved before status moved to Proposed; **Amendment 1 supersedes §2's dispatch mechanism**: the originally-chosen `Node::Guard(fn)` + `Choice(SQL_shape, DSL_shape)` was found during 3a to be unworkable as framed (any guard-in-`Choice` mechanism forces a `walk_choice` change — `walk_choice` only falls through on `NoMatch`, so Simple-mode valid-DSL would wrongly surface "this is SQL", and `walk_seq` treats a `NoMatch` past `idx 0` as a hard `Failed`, breaking Advanced-mode DSL fall-through); replaced by **category-grouped, mode-aware dispatch** in `walker::walk` (each `REGISTRY` entry tagged `CommandCategory::{Simple, Advanced}`, generalising the existing whole-command `is_advanced_only` gate), shared entry words carrying a node in both groups, no `Node::Guard` and no `walk_choice`/`walk_seq` change, advanced-mode completion SQL-first with DSL as a full-line fallback; **Amendment 2 (sub-phase 3f) supersedes §7's cascade mechanism**: the WHERE-injected per-child pre-count rested on a premise that was factually wrong about the DSL handler (which detects cascades by before/after row-count diffing inside a transaction, not by `Expr`-derived pre-count subqueries) and would have broken the §2 parity promise by reporting `SET NULL` the DSL path doesn't; replaced by mirroring `do_delete`'s count-diff exactly (verbatim DELETE executes, child-count diff observes the cascade — `ON DELETE CASCADE` row removals only, SET NULL deferred for both paths to preserve parity), which shares the render-layer formatter for free via `CommandOutcome::Delete` and **withdraws risk R2** (no WHERE-byte extraction, no N+1 subquery); **Amendment 3 (sub-phase 3j) records the command-identity model and defers the execution-mode side-channel**: a command is the typed outcome of a *mode-rooted* grammar path and its identity is intrinsic (Advanced mode tries SQL first, falls back to the *Simple* DSL command when no SQL branch matches a token, e.g. `delete … --all-rows`; note `update … --all-rows` does *not* fall back — the SQL `SET` expression eats `--all-rows`, harmless since the engine treats it as a comment); **Simple mode commits the DSL candidate for shared words** so the *real* DSL error surfaces, and when that line would also run in advanced mode the rendering layer **combines** them — DSL error **plus** an `advanced_mode.also_valid_sql` pointer ("… (valid as SQL in advanced mode)") — keeping the actionable DSL fix while pointing at advanced mode; bare "this is SQL" is reserved for entry words with no DSL form (`select`/`with`); a fully-overlapping input (`insert … values …`) legitimately yields *two distinct commands* (`Command::Insert` typed-AST vs `Command::SqlInsert` validated-text) that do the same thing but execute differently (ADR-0030 §4), so each is tested in the mode that produces it; **corrects the plan's 3j exit-gate premise** that the DSL DML tests run in Simple mode (they call `parse_command`, which defaults to Advanced) — the real invariant is "Simple-mode behaviour unchanged, Advanced mode SQL-first, DSL grammar tested in Simple mode, both variants tested in their producing mode", with §6/§7 parity keeping the paths observably equivalent; and **defers to its own future ADR** the execution-time mode side-channel (three-way `Mode`: simple/advanced/advanced-one-shot threaded through `Action`→worker, for mode-dependent *output* like echoing generated SQL) — today only the *rendering* side-channel `OutputLine.mode_at_submission` exists, and the three-way distinction is not required for Phase 3 dispatch correctness; **Amendment 4, 2026-05-27** (design agreed, pending impl): **reverses Amendment 3's `update … --all-rows` counter-example as a bug** — surfaced by the ADR-0038 echo design. The walker has no `--` comment support (it lexes two minus operators) while the engine treats `--` as a comment, so `update T set x=42 --all-rows` was silently parsed as the expression `42 - -all - rows` over **non-existent columns** `all`/`rows` (an ADR-0027 "flag-if-it-will-fail" case) and matched `SqlUpdate`. Decision: the `--all-rows` sequence makes the SQL `UPDATE` shape **fail**, so dispatch **falls back to the DSL `Update { AllRows }`** — symmetry with `delete … --all-rows`; no `--` comment feature introduced (trailing comments stay unsupported). Inverts `sql_dml_e2e.rs::e2e_update_all_rows_in_advanced_does_not_fall_back_to_dsl`; mechanism settled test-first in the build; folded into the ADR-0038 effort (makes `update … --all-rows` echoable) - [ADR-0034 — `history.log` as a complete command journal; replay reads success-only](0034-history-journal-and-replay-filter.md) — **Accepted**, resolves a three-way tension in `history.log`'s roles found while implementing ADR-0033 3f: (1) the persistent log is success-only while the in-memory Up/Down recall ring records *every* submission (success or failure, "so users can recall and edit typo'd commands"), and the ring is re-seeded from the log on project open — so **failed commands are recallable within a session but silently lost across sessions**; (2) replay wants the state-building (successful) commands while recall wants everything typed, which one success-only file cannot serve; (3) `replay history.log` never actually worked — `run_replay` parses each whole line through the DSL parser with no understanding of the `||` record shape, so a real log fails on line 1, and **no test ever fed the pipe format to replay** (the `replay_history_log_records_subcommands_only` test only checks what replay *writes*, never replays the log as input). Decision: `history.log` becomes a **complete journal** — every submission recorded, tagged `ok`/`err` via the status field the format already reserved (ADR-0015 §5) — and **each consumer filters**: hydration reads all records (cross-session recall matches in-session), replay reads `ok` only (and learns the journal format, while still accepting bare-command `.commands` scripts; detection by the leading timestamp+status prefix so a `|` inside a bare command isn't misread). Successful commands stay journalled transactionally by the worker; failed commands are journalled `err` best-effort from the runtime/app error path (a parse failure never reaches the worker). Amends ADR-0006's "successfully executed" wording and ADR-0015 §5 ("status always `ok`") / §12 (hydration). Code deferred to two tracked test-first sub-tasks (journal-failures+filtering; replay-parses-journal-format); existing all-`ok` logs need no migration; **implemented 2026-05-24** (plan `docs/plans/20260524-adr-0034-history-journal.md`); **Amendment 1 (2026-05-24): replay filters out app-lifecycle commands** — a working `replay history.log` (the §3 fix) exposed that the journal also records `save as`/`load`/`new`/`export`/`import`/`rebuild`/`mode` (which would panic the worker dispatch or abort the replay), so replay now re-applies **only** schema/data write commands and **skips** every `Command::App` + nested `Command::Replay`; **all skips continue** (never abort — reversing the prior nested-`replay` refusal, so a journal containing a once-run `replay` needs no hand-editing, and the infinite-loop footgun is closed by construction), with a `[skip]` **warning** on `import` and nested-`replay` skips (their omission can leave replayed state incomplete) and silent skips for the rest; `replay.error_nested` removed, `replay.skipped_import`/`replay.skipped_replay` added, `ReplayCompleted` carries `warnings` -- [ADR-0035 — Advanced-mode SQL DDL](0035-advanced-mode-sql-ddl.md) — **Accepted** (design agreed 2026-05-24; validated end-to-end by sub-phases 4a/4a.2/4a.3/4b `CREATE TABLE` (incl. foreign keys) + 4c `DROP TABLE [IF EXISTS]` + 4d `CREATE [UNIQUE] INDEX` / `DROP INDEX [IF EXISTS]` + 4e `ALTER TABLE` add/drop/rename column + 4f `ALTER TABLE … ALTER COLUMN TYPE` + 4g `ALTER TABLE` add/drop constraint + add FK + 4h `ALTER TABLE … RENAME TO` + 4i verification sweep (completion merge + simple/advanced completion colour + describe of table-level constraints + self-ref FK indicator + CREATE-TABLE help/usage), implemented 2026-05-25/26 — **Phase 4 complete**; **Amendment 1, 2026-05-26**: drop a composite UNIQUE via a derived, engine-neutral `unique_` name that reuses the existing `DROP CONSTRAINT ` grammar — no new syntax, no metadata, §4g anonymity intact; `describe` shows the name; dropping a UNIQUE-covered *column* now refuses with that name + the drop command), **Phase 4** of the ADR-0030 roadmap (peer of 0031/0032/0033) and **clarifies ADR-0030 §4**. Advanced-mode `CREATE`/`DROP`/`ALTER TABLE` + `CREATE`/`DROP INDEX` get their **own per-statement commands** (`SqlCreateTable`/`SqlAlterTable`/`SqlDropTable`/`SqlCreateIndex`/`SqlDropIndex`), like DML's `Sql*` set — but unlike DML they **execute *structurally*, not verbatim** (raw execution would lose the playground's types, named relationships, and `STRICT`; "verbatim" was a DML convenience, not a rule). Handlers **reuse the low-level schema/metadata helpers** where the operation matches simple mode and **stand alone where the SQL surface is richer** (clarity over forced refactoring); simple mode is untouched (additive). Dispatch: `create`/`drop` reuse ADR-0033 Amendment 1's category-grouped mode-aware dispatch (SQL-first, simple fallback); `alter` is a new advanced-only entry word. Full surface (no pre-emptive cuts, `Q4`): `CREATE TABLE` with column + table constraints, single/compound `PRIMARY KEY`, inline + table-level `FOREIGN KEY` → **named relationships** (one statement = one command = **one undo step**, ADR-0006); `ALTER TABLE` add/drop/rename column, `ALTER COLUMN TYPE`, add/drop constraint, add FK, **`RENAME TO`** (advanced-only table rename — new low-level op renaming the table + its CSV + the relationship and table-CHECK metadata, closing the rename half of `C1`); `CREATE [UNIQUE] INDEX` / `DROP INDEX`. Type slot accepts the ten playground keywords **and** standard-SQL aliases (`integer`→`int`, `varchar`→`text`, `timestamp`→`datetime`, …; length args accepted-and-ignored; no engine type names in/out — ADR-0030 §5). `CHECK`/`DEFAULT` reuse ADR-0031 `sql_expr`. **Pre-implementation `/runda` refinements (2026-05-24, user-confirmed):** `CREATE TABLE`/`DROP TABLE` **admit `IF [NOT] EXISTS`** (no-op-that-succeeds-with-a-note — a near-universal cross-vendor idiom, reclassified *into* scope, not engine-specific); `INTEGER PRIMARY KEY` maps to a **plain `int`** PK, *not* auto-increment (`serial` stays the sole auto-increment type). **Column-type-conversion is unified** (ADR-0017 engine, mode-appropriate policy): clean auto-converts and incompatible/own-type-static cases refuse in both modes, but a **lossy** change refuses-by-default in simple mode (`--force-conversion` opts in) while advanced mode **performs it with a loss note** and relies on **`undo` as the safety net** — no force flag, no dropping to simple mode (a payoff of shipping ADR-0006 first). OOS: views/triggers/txn-control/PRAGMA/etc. (ADR-0030 §3), the Postgres `USING` clause, and the DSL→SQL teaching echo (ADR-0030 Phase 5). Sub-phases 4a–4i, plus **4a.2** (per-column `CHECK`/`DEFAULT` via raw `sql_expr` text — `sql_expr` is validate-only, no `Expr` AST — + composite `UNIQUE(a,b)`; no new internal table) and **4a.3** (table-level/multi-column `CHECK`, landed via the new `__rdbms_playground_table_checks` metadata table because SQLite has no PRAGMA for CHECK; the builder tells a table-level CHECK from a column-level one by element position) and **4b** (foreign keys — inline `REFERENCES` + table-level `FOREIGN KEY` → ADR-0013 named relationships in the create transaction, one undo step; self-references + bare `REFERENCES ` supported, user-confirmed) and **4c** (`DROP TABLE [IF EXISTS]` → `SqlDropTable`, reusing `do_drop_table`; `IF EXISTS` is a no-op-with-note via `DropOutcome::Skipped`) and **4d** (`CREATE [UNIQUE] INDEX [IF NOT EXISTS] [] ON (cols)` → `SqlCreateIndex` and `DROP INDEX [IF EXISTS] ` → `SqlDropIndex`, reusing `do_add_index`/`do_drop_index`; **`CREATE UNIQUE INDEX` admitted** — ADR-0025 **Amendment 1** — via an additive `IndexSchema.unique` flag that round-trips through `project.yaml` and rebuild, with `[unique]` markers in the structure view + items panel, while simple-mode `add unique index` stays deferred; `IF [NOT] EXISTS` reuses the 4c skip path; `create`/`drop` each gain a *second* advanced node, exercising the all-candidates dispatch) and **4e** (`ALTER TABLE` add/drop/rename column → `SqlAlterTable`; `alter` is a new advanced-**only** entry word, runtime-decomposed to the existing `do_add_column`/`do_drop_column`/`do_rename_column` — no new worker layer; `do_add_column` extended to consume raw `default_sql`/`check_sql` so ADD COLUMN reaches CREATE-TABLE constraint parity; drop/rename refuse a column any CHECK references (table-level AND column-level, incl. a column's own self-check on rename) — the 4a.3 deferral, via a raw-CHECK-text tokenizer in the shared executors, so it guards both surfaces and fixes a latent rename-drift bug; SQL DROP COLUMN refuses an index-covered column with no `--cascade` spelling; the column executors + `do_add_index` gained an internal-`__rdbms_*`-table guard — all user-confirmed) and **4f** (`ALTER TABLE … ALTER COLUMN TYPE` → a fourth `AlterTableAction`, runtime-decomposed to the existing `change_column_type` with `ChangeColumnMode::ForceConversion` — which **is** the §7 advanced policy: lossy converts *with a note* (no force flag), incompatible + ADR-0017 static refusals (`↔ blob`, same-type, `date ↔ datetime`, non-`int → serial`) still refuse, while **`int → serial` is allowed** (auto-fills nulls + UNIQUE, ADR-0018 §8 — the §7 "→serial refused" summary is looser than the code); the builder discriminates the fourth branch by the **`type` keyword** (unique — ADD COLUMN's type is an ident), the type slot reuses `SQL_TYPE`; the internal-`__rdbms_*` guard was folded into `do_change_column_type`, closing the simple `change column` exposure too — user-confirmed) and **4g** (`ALTER TABLE … ADD [CONSTRAINT ] (CHECK | UNIQUE | FOREIGN KEY)` + `DROP CONSTRAINT `; ADD = CHECK + composite UNIQUE + FK, with `ADD PRIMARY KEY` and a *named* UNIQUE refused — composite UNIQUE is anonymous in our model; each ADD reuses a low-level path (table-CHECK/UNIQUE rebuild with a dry-run guard; FK → `add_relationship`, bare `REFERENCES

` → parent single-PK), DROP CONSTRAINT resolves the name to a table-CHECK then a child-side FK; **named table-CHECKs round-trip** via a nullable `name` column on `__rdbms_playground_table_checks` (**rebuild-only** arrival — pre-4g projects gain it on `rebuild`, a named add on an un-upgraded project is refused with a friendly "rebuild first" message) *and* a `project.yaml` `check_constraints` extension to an `{expr, name}` mapping (the bare-string form still reads); the internal-`__rdbms_*` guard was folded into `do_add_constraint`/`do_add_relationship`, completing that guard class — all user-confirmed) and **4h** (`ALTER TABLE … RENAME TO` — the one genuinely new low-level op, `do_rename_table`: a native engine rename plus one-transaction reconciliation of every metadata row naming the table (`__rdbms_playground_columns`, **both ends** of `__rdbms_playground_relationships`, `__rdbms_playground_table_checks`), the CSV file (the existing rewrite+delete path — no new persistence method), and **CHECK text that qualifies a column with the old table name** (`T.age`→`U.age`, a planning-`/runda` finding — the engine rewrites the live CHECK but the stored text would drift and break a fresh rebuild; `rewrite_check_table_qualifier` keeps them in step); grammar splits the `rename` verb into one branch with an inner Choice on a distinct second keyword (`column` vs `to`), the new-name slot mirroring the `CREATE TABLE` name slot; refuses same-name / existing-target / `__rdbms_*` / non-existent, with **case-insensitive** collision checks behind an engine-neutral pre-check (a finished-slice `/runda` finding — the engine matches names case-insensitively); auto-named indexes *and* relationships keep their stale names (only table-name columns update — §6 scope); one undo step; advanced-only, closing the rename half of `C1` — all user-confirmed) and **4i** (the verification sweep that completes Phase 4: the shared-entry-word completion merge + the simple-vs-advanced completion colour-when-mixed with Both→Advanced→Simple block ordering; `describe` of table-level composite UNIQUE + table CHECK; the self-ref FK pre-submit indicator fix; and the CREATE-TABLE help/usage skeleton refresh). **All of Phase 4 (4a–4i) is shipped.** Each sub-phase has exit + DA gates +- [ADR-0035 — Advanced-mode SQL DDL](0035-advanced-mode-sql-ddl.md) — **Accepted** (design agreed 2026-05-24; validated end-to-end by sub-phases 4a/4a.2/4a.3/4b `CREATE TABLE` (incl. foreign keys) + 4c `DROP TABLE [IF EXISTS]` + 4d `CREATE [UNIQUE] INDEX` / `DROP INDEX [IF EXISTS]` + 4e `ALTER TABLE` add/drop/rename column + 4f `ALTER TABLE … ALTER COLUMN TYPE` + 4g `ALTER TABLE` add/drop constraint + add FK + 4h `ALTER TABLE … RENAME TO` + 4i verification sweep (completion merge + simple/advanced completion colour + describe of table-level constraints + self-ref FK indicator + CREATE-TABLE help/usage), implemented 2026-05-25/26 — **Phase 4 complete**; **Amendment 1, 2026-05-26**: drop a composite UNIQUE via a derived, engine-neutral `unique_` name that reuses the existing `DROP CONSTRAINT ` grammar — no new syntax, no metadata, §4g anonymity intact; `describe` shows the name; dropping a UNIQUE-covered *column* now refuses with that name + the drop command), **Phase 4** of the ADR-0030 roadmap (peer of 0031/0032/0033) and **clarifies ADR-0030 §4**. Advanced-mode `CREATE`/`DROP`/`ALTER TABLE` + `CREATE`/`DROP INDEX` get their **own per-statement commands** (`SqlCreateTable`/`SqlAlterTable`/`SqlDropTable`/`SqlCreateIndex`/`SqlDropIndex`), like DML's `Sql*` set — but unlike DML they **execute *structurally*, not verbatim** (raw execution would lose the playground's types, named relationships, and `STRICT`; "verbatim" was a DML convenience, not a rule). Handlers **reuse the low-level schema/metadata helpers** where the operation matches simple mode and **stand alone where the SQL surface is richer** (clarity over forced refactoring); simple mode is untouched (additive). Dispatch: `create`/`drop` reuse ADR-0033 Amendment 1's category-grouped mode-aware dispatch (SQL-first, simple fallback); `alter` is a new advanced-only entry word. Full surface (no pre-emptive cuts, `Q4`): `CREATE TABLE` with column + table constraints, single/compound `PRIMARY KEY`, inline + table-level `FOREIGN KEY` → **named relationships** (one statement = one command = **one undo step**, ADR-0006); `ALTER TABLE` add/drop/rename column, `ALTER COLUMN TYPE`, add/drop constraint, add FK, **`RENAME TO`** (advanced-only table rename — new low-level op renaming the table + its CSV + the relationship and table-CHECK metadata, closing the rename half of `C1`); `CREATE [UNIQUE] INDEX` / `DROP INDEX`. Type slot accepts the ten playground keywords **and** standard-SQL aliases (`integer`→`int`, `varchar`→`text`, `timestamp`→`datetime`, …; length args accepted-and-ignored; no engine type names in/out — ADR-0030 §5). `CHECK`/`DEFAULT` reuse ADR-0031 `sql_expr`. **Pre-implementation `/runda` refinements (2026-05-24, user-confirmed):** `CREATE TABLE`/`DROP TABLE` **admit `IF [NOT] EXISTS`** (no-op-that-succeeds-with-a-note — a near-universal cross-vendor idiom, reclassified *into* scope, not engine-specific); `INTEGER PRIMARY KEY` maps to a **plain `int`** PK, *not* auto-increment (`serial` stays the sole auto-increment type). **Column-type-conversion is unified** (ADR-0017 engine, mode-appropriate policy): clean auto-converts and incompatible/own-type-static cases refuse in both modes, but a **lossy** change refuses-by-default in simple mode (`--force-conversion` opts in) while advanced mode **performs it with a loss note** and relies on **`undo` as the safety net** — no force flag, no dropping to simple mode (a payoff of shipping ADR-0006 first). OOS: views/triggers/txn-control/PRAGMA/etc. (ADR-0030 §3), the Postgres `USING` clause, and the DSL→SQL teaching echo (ADR-0030 Phase 5). Sub-phases 4a–4i, plus **4a.2** (per-column `CHECK`/`DEFAULT` via raw `sql_expr` text — `sql_expr` is validate-only, no `Expr` AST — + composite `UNIQUE(a,b)`; no new internal table) and **4a.3** (table-level/multi-column `CHECK`, landed via the new `__rdbms_playground_table_checks` metadata table because SQLite has no PRAGMA for CHECK; the builder tells a table-level CHECK from a column-level one by element position) and **4b** (foreign keys — inline `REFERENCES` + table-level `FOREIGN KEY` → ADR-0013 named relationships in the create transaction, one undo step; self-references + bare `REFERENCES ` supported, user-confirmed) and **4c** (`DROP TABLE [IF EXISTS]` → `SqlDropTable`, reusing `do_drop_table`; `IF EXISTS` is a no-op-with-note via `DropOutcome::Skipped`) and **4d** (`CREATE [UNIQUE] INDEX [IF NOT EXISTS] [] ON (cols)` → `SqlCreateIndex` and `DROP INDEX [IF EXISTS] ` → `SqlDropIndex`, reusing `do_add_index`/`do_drop_index`; **`CREATE UNIQUE INDEX` admitted** — ADR-0025 **Amendment 1** — via an additive `IndexSchema.unique` flag that round-trips through `project.yaml` and rebuild, with `[unique]` markers in the structure view + items panel, while simple-mode `add unique index` stays deferred; `IF [NOT] EXISTS` reuses the 4c skip path; `create`/`drop` each gain a *second* advanced node, exercising the all-candidates dispatch) and **4e** (`ALTER TABLE` add/drop/rename column → `SqlAlterTable`; `alter` is a new advanced-**only** entry word, runtime-decomposed to the existing `do_add_column`/`do_drop_column`/`do_rename_column` — no new worker layer; `do_add_column` extended to consume raw `default_sql`/`check_sql` so ADD COLUMN reaches CREATE-TABLE constraint parity; drop/rename refuse a column any CHECK references (table-level AND column-level, incl. a column's own self-check on rename) — the 4a.3 deferral, via a raw-CHECK-text tokenizer in the shared executors, so it guards both surfaces and fixes a latent rename-drift bug; SQL DROP COLUMN refuses an index-covered column with no `--cascade` spelling; the column executors + `do_add_index` gained an internal-`__rdbms_*`-table guard — all user-confirmed) and **4f** (`ALTER TABLE … ALTER COLUMN TYPE` → a fourth `AlterTableAction`, runtime-decomposed to the existing `change_column_type` with `ChangeColumnMode::ForceConversion` — which **is** the §7 advanced policy: lossy converts *with a note* (no force flag), incompatible + ADR-0017 static refusals (`↔ blob`, same-type, `date ↔ datetime`, non-`int → serial`) still refuse, while **`int → serial` is allowed** (auto-fills nulls + UNIQUE, ADR-0018 §8 — the §7 "→serial refused" summary is looser than the code); the builder discriminates the fourth branch by the **`type` keyword** (unique — ADD COLUMN's type is an ident), the type slot reuses `SQL_TYPE`; the internal-`__rdbms_*` guard was folded into `do_change_column_type`, closing the simple `change column` exposure too — user-confirmed) and **4g** (`ALTER TABLE … ADD [CONSTRAINT ] (CHECK | UNIQUE | FOREIGN KEY)` + `DROP CONSTRAINT `; ADD = CHECK + composite UNIQUE + FK, with `ADD PRIMARY KEY` and a *named* UNIQUE refused — composite UNIQUE is anonymous in our model; each ADD reuses a low-level path (table-CHECK/UNIQUE rebuild with a dry-run guard; FK → `add_relationship`, bare `REFERENCES

` → parent single-PK), DROP CONSTRAINT resolves the name to a table-CHECK then a child-side FK; **named table-CHECKs round-trip** via a nullable `name` column on `__rdbms_playground_table_checks` (**rebuild-only** arrival — pre-4g projects gain it on `rebuild`, a named add on an un-upgraded project is refused with a friendly "rebuild first" message) *and* a `project.yaml` `check_constraints` extension to an `{expr, name}` mapping (the bare-string form still reads); the internal-`__rdbms_*` guard was folded into `do_add_constraint`/`do_add_relationship`, completing that guard class — all user-confirmed) and **4h** (`ALTER TABLE … RENAME TO` — the one genuinely new low-level op, `do_rename_table`: a native engine rename plus one-transaction reconciliation of every metadata row naming the table (`__rdbms_playground_columns`, **both ends** of `__rdbms_playground_relationships`, `__rdbms_playground_table_checks`), the CSV file (the existing rewrite+delete path — no new persistence method), and **CHECK text that qualifies a column with the old table name** (`T.age`→`U.age`, a planning-`/runda` finding — the engine rewrites the live CHECK but the stored text would drift and break a fresh rebuild; `rewrite_check_table_qualifier` keeps them in step); grammar splits the `rename` verb into one branch with an inner Choice on a distinct second keyword (`column` vs `to`), the new-name slot mirroring the `CREATE TABLE` name slot; refuses same-name / existing-target / `__rdbms_*` / non-existent, with **case-insensitive** collision checks behind an engine-neutral pre-check (a finished-slice `/runda` finding — the engine matches names case-insensitively); auto-named indexes *and* relationships keep their stale names (only table-name columns update — §6 scope); one undo step; advanced-only, closing the rename half of `C1` — all user-confirmed) and **4i** (the verification sweep that completes Phase 4: the shared-entry-word completion merge + the simple-vs-advanced completion colour-when-mixed with Both→Advanced→Simple block ordering; `describe` of table-level composite UNIQUE + table CHECK; the self-ref FK pre-submit indicator fix; and the CREATE-TABLE help/usage skeleton refresh). **All of Phase 4 (4a–4i) is shipped.** Each sub-phase has exit + DA gates; **Amendment 2, 2026-05-27** (design agreed, pending impl): a **standard-first dialect stance** (refines ADR-0030's "standard SQL" posture — ISO spelling is canonical + echoed where one exists; a vendor shorthand may be *accepted* but isn't canonical; where ISO offers none, *one* documented vendor spelling is a deliberate extension) + an `ALTER COLUMN` **constraint gap-fill** surfaced by the ADR-0038 echo design: makes ISO `ALTER COLUMN … SET DATA TYPE` the canonical type-change verb with `TYPE` retained as a synonym (**reverses §4f's "no `SET DATA TYPE`"**), and adds `SET/DROP DEFAULT` (ISO) + `SET/DROP NOT NULL` (the one documented extension — ISO has no in-place NOT-NULL verb; PostgreSQL's chosen for being type-independent), all **rebuild-backed via the existing ADR-0029 `do_add_constraint`/`do_drop_constraint` executors** (dry-run + internal-table guards free, no new worker layer), reaching simple↔advanced constraint-mod **parity for NOT NULL + DEFAULT**; the **rebuild stays hidden** (Category-1 engine detail, ADR-0038). Residual gap left open + flagged: dropping a **column-level (anonymous) UNIQUE/CHECK** (no portable name — same class as Am1's parallel gap), which ADR-0038's catalogue marks "no headline echo" - [ADR-0036 — Value validation for advanced-mode DML](0036-typed-dml-values-vs-verbatim.md) — **Accepted** (design agreed + `/runda`'d 2026-05-26; mechanism then **deliberately narrowed** from "bind literals via the DSL path" to surgical **"validate-and-retain, execute verbatim"** after the user resisted consolidating the modes and a concrete auto-fill difference confirmed even the single-row literal case isn't identical across modes; **Phases 1–2 implemented** 2026-05-26 — `INSERT … VALUES` and `UPDATE … SET` literal validation + offending-value retention, capture-at-parse with no grammar change; **Phase 3a implemented** 2026-05-26 — live typed-slot hints + numeric-shape highlighting for `UPDATE`/UPSERT `SET col = ` via a boundary-aware lookahead (Amendment 1 corrects this ADR's naive-`Choice` sketch); **Phase 3b implemented** 2026-05-27 — per-position typed slots for `INSERT … VALUES` (single/multi-row, Form A/B) via a new zero-width `Node::SetColumn` primitive + an arity-gating tuple lookahead that preserves the §8.1 arity diagnostic; **fully implemented**). **Augments — does NOT supersede — ADR-0030 §4 / ADR-0033 §10**: execution stays verbatim, ADR-0033 Amendment 3's two-command identity (`Insert` vs `SqlInsert`) **stands**. The problem (investigated 2026-05-26; characterization test `sql_insert.rs::sql_dml_skips_app_level_value_validation_that_the_dsl_enforces` proves it): advanced-mode SQL DML gets **none** of the DSL's value feedback — a malformed `date` like `2025/01/15` is silently written, and the offending value is missing from constraint errors — because literal values are spliced into text and discarded (only `STRICT` storage types check them). **Fix (surgical): validate each literal value against its column type before the verbatim insert, and retain it for error reporting — sharing only the per-type validators (`Value::bind_for_column`/`validate_date`/`shortid::validate`), nothing else.** No binding, no statement reconstruction, no auto-fill change, no command-identity collapse — because the two gaps are closed by validation + retention alone, and executing the user's own text is already safe. The literal set = `NULL`/boolean/string/**signed**-numeric; arithmetic/functions/subqueries/column-refs are expressions (skipped — the engine evaluates them). `WHERE` not validated (it's an expression in general; motivation met by `VALUES`/`SET`); `SELECT`/`INSERT … SELECT`/`RETURNING`/`ON CONFLICT` need no special handling since execution is untouched. Phased: **Phase 1** capture-at-parse + validate + retain for `INSERT … VALUES` (no grammar change, no reparse — closes both proven gaps); **Phase 2** `UPDATE … SET` literals; **Phase 3** completion hinting/highlighting (the only part needing a grammar change — a typed-literal slot vs `sql_expr` reusing the DSL `TypedValueSlot`s at `data.rs:141`/`189`/`269`, discriminated by a **boundary-aware lookahead** not a naive `Choice` per **Amendment 1**; split into **3a** `SET` (done) and **3b** `VALUES` (pending); supersedes only Phase 1/2's literal *detection*, not the validation/enrichment on top). Non-goals: binding/reconstruction, collapsing command identity (Am3 stands), changing `serial`/`shortid` auto-fill (`requirements.md` **X4**, a separate possible-bug), a structural `SELECT`, a full SQL-expression AST. Embodies `requirements.md` **X5** (share a *mechanic*, not a *command*); the neutral "that value" safety net (ADR-0035 Amendment 1) stays correct for genuinely-computed values +- [ADR-0037 — Execution-time mode side-channel (the three-way submission mode)](0037-execution-time-mode-side-channel.md) — **Proposed** (design agreed 2026-05-27; pending `/runda` + implementation), **redeems the follow-up deferred by ADR-0033 Amendment 3** (which named this ADR and its motivating consumer). Establishes the channel that lets a command know, **at execution time**, the effective mode it ran under — so execution can adjust **output** without touching **identity** (the motivating case: a DSL-form `create table` echoing the equivalent SQL when run in advanced mode, silent in simple — ADR-0030 §10, realised by ADR-0038). Introduces a **new per-submission enum `SubmissionMode` { Simple, Advanced, AdvancedOneShot }** — *refining* Amendment 3's "widen `Mode`" sketch: the persistent input `Mode` stays **two-way** (`mode.rs` keeps the one-shot `:` out of persistent state), and the three-way distinction lives on the per-submission channel where the transient `:` belongs. Resolved at submit time (Simple+`:` → `AdvancedOneShot`; Advanced `:` is a no-op), threaded through `Action::ExecuteDsl` → worker, **output-only** (no executor branches its *effect* on it — Amendment 3 forbids behavioural mode dependence). The worker builds the teaching echo (+ category-3 expansion data — ADR-0038) for DSL-form commands in advanced/one-shot mode and returns it; the App renders it beneath `[ok]`. Co-located with execution because the echo's harder forms (resolved auto-names, generated `shortid`s, conversion counts) are worker-computed facts, and gating on mode means the work happens only when shown. Alternatives weighed + rejected: widening `Mode` (conflates transient/persistent state); App-side gating with the worker always emitting echo data (computes unconditionally, doesn't generalise, re-opens the render-side framing ruled against). Scope: channel + resolution rule only — the renderer/catalogue/`Value → SQL-literal` are ADR-0038, the `ALTER COLUMN` gap-fill is the ADR-0035 amendment +- [ADR-0038 — The DSL → SQL teaching echo](0038-dsl-to-sql-teaching-echo.md) — **Proposed** (design agreed 2026-05-27; pending `/runda` + impl), **realises ADR-0030 §10** (the teaching bridge) — the Phase-5 echo **ADR-0035 §12 forward-referenced** — building on **ADR-0037** (the `SubmissionMode` gate) and **ADR-0035 Amendment 2** (standard-first dialect + `ALTER COLUMN` gap-fill). When a **DSL-form** command runs in advanced/one-shot mode, the worker emits the equivalent SQL beneath `[ok]` as a de-emphasised styled `OutputLine` (ADR-0028); the App renders it. **Defining invariant — the copy-paste contract:** every echoed line is *runnable advanced-mode SQL* (round-trip-tested: parse the echo → same-effect command; a planned "copy the echo" affordance depends on it). **Type vocabulary = the playground's own keywords** (`serial`/`shortid`/…, accepted by `from_sql_name`, decision (a)); **statement shape = the standard-first dialect** (Am2). **DML uses substituted literals, not `?`** (per-type `Value → SQL-literal`, round-trip-safe; `blob` moot — no literal syntax exists; auto-gen columns omitted to match `do_insert` + X4). **Firing reality — a DDL + `show data` feature:** in advanced mode `insert`/`update`/`delete … where` are SQL-first (`Sql*` = already SQL = nothing to echo per §10); only DSL-*only* spellings echo (DDL + `show data` + the `delete`/`update … --all-rows` fall-throughs — the latter via **ADR-0033 Amendment 4**, a bug-fix folded in here that reverses Amendment 3's `update … --all-rows` misparse). **Three-category framework** for "what happens beyond the literal SQL": **(1) engine-implementation-hiding** (the rebuild, rowid PK, non-PK `serial` MAX+1) — *never surfaced*; **(2) decomposable into advanced SQL** (`drop column --cascade`, `--create-fk` relationship) — *shown as the runnable multi-line sequence, one statement per line*; **(3) playground type-behaviour with no SQL-expressible form** (`shortid` generation — no `shortid()`; type-conversion transforms — no `USING`) — *de-emphasised prose expansion from the worker's `client_side.*` notes*. Carries the **full catalogue** (Buckets A single-statement / B resolved-name + multi-line / C no-echo) mapping every DSL-form command to its echo. OOS: reverse SQL→DSL echo (§13 OOS-5), app commands / `show table` / `explain` / `replay`, a `blob` literal, the column-level UNIQUE/CHECK drop residual (Bucket C until Am2's gap closes), and surfacing any category-1 engine internal +- [ADR-0039 — EXPLAIN over advanced-mode SQL queries](0039-explain-over-advanced-sql.md) — **Accepted** (decision recorded 2026-05-27; **implementation deferred** as a follow-up to the ADR-0037/0038 echo effort — not in that pass), **supersedes ADR-0030 §13 OOS-2**. Lets `explain` wrap the advanced SQL commands (`Select`/`SqlInsert`/`SqlUpdate`/`SqlDelete`) in addition to the DSL `ShowData`/`Update`/`Delete` it already covers (ADR-0028), running `EXPLAIN QUERY PLAN` over the validated SQL text through the existing ADR-0028 span-styled plan tree (advanced mode only; DSL `explain` unchanged in both modes). Reframes OOS-2 as a *deferred* scope exclusion (per ADR-0000's new out-of-scope discipline), not a principled rejection — surfaced 2026-05-27 while characterising advanced-mode explain (suspected a bug; it was OOS-2 as written). Self-contained, orthogonal to the echo (the echo renders SQL *from* DSL; this explains SQL the user *wrote*). OOS (deferred): EXPLAIN of DDL (no query plan exists). Built test-first when picked up