From 63b2927f105845be56cafaa5bdbef44beb9c1acb Mon Sep 17 00:00:00 2001 From: "claude@clouddev1" Date: Thu, 28 May 2026 12:24:07 +0000 Subject: [PATCH] =?UTF-8?q?docs:=20session=20handoff=2049=20=E2=80=94=20AD?= =?UTF-8?q?R-0038=20done,=20M4=20+=20ADR=20statuses=20in=20sync?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- docs/adr/0038-dsl-to-sql-teaching-echo.md | 22 ++- docs/adr/README.md | 4 +- docs/handoff/20260528-handoff-49.md | 171 ++++++++++++++++++++++ docs/requirements.md | 37 +++-- 4 files changed, 210 insertions(+), 24 deletions(-) create mode 100644 docs/handoff/20260528-handoff-49.md diff --git a/docs/adr/0038-dsl-to-sql-teaching-echo.md b/docs/adr/0038-dsl-to-sql-teaching-echo.md index 6fa65cb..6ef474f 100644 --- a/docs/adr/0038-dsl-to-sql-teaching-echo.md +++ b/docs/adr/0038-dsl-to-sql-teaching-echo.md @@ -2,11 +2,12 @@ ## Status -Accepted. Design agreed with the user (2026-05-27); **Phases 1–3 -implemented and verified** by round-tripping every catalogue row -(§7 Buckets A, B, plus the §6 category-3 prose) through the +Accepted. Design agreed with the user (2026-05-27); **fully +implemented and verified** — every catalogue row (§7 Buckets A, B, +plus the §6 category-3 prose) round-trips per line through the advanced-mode walker (the §1 copy-paste contract; §6 category 2 -holds it per line). Shipped across three feature commits: +holds it per line), and the §4 de-emphasised styled-runs rendering +polish (ADR-0028) is wired. Shipped across four feature commits: **Phase 1 — Bucket A single-statement** (handoff-46's `04c8e42` delivered the channel + create-table slice; handoff-47's `90479cb` expanded to the full Bucket A — `Value → SQL-literal` + `Expr → SQL`, @@ -31,8 +32,17 @@ already surfaced via the pre-existing `client_side.auto_fill_*` / `change column --dont-convert` **caveat** (the only Bucket A caveat; every other category-3 line is illuminating), gated on an advanced effective mode because it references "the line above". -**The §4 de-emphasised styled-runs rendering polish remains** — the -echo + caveat currently surface as plain `[system]` lines. +**Phase 4 styled-runs polish** (`2aab457`): the echoed SQL renders +as code — a new `OutputKind::TeachingEcho` line with a dimmed +`Executing SQL:` prefix followed by the SQL re-lexed in advanced +mode via `input_render::lex_to_runs_in_mode` for token-class +colouring (the same treatment the input echo gets); every +category-3 prose line (the new caveat plus the pre-existing +illuminating notes) renders dimmed via a new +`OutputStyleClass::Hint` styled-runs class resolving to +`theme.muted`. Scope went slightly beyond §4's "echo + caveat" to +cover the illuminating cat-3 notes too — user-confirmed, in service +of §6's "de-emphasised prose line" wording for every cat-3 line. **Realises ADR-0030 §10** ("The DSL → SQL teaching bridge") — the Phase-5 echo that **ADR-0035 §12 forward-referenced** as "a separate ADR." Builds on **ADR-0037** (the execution-time mode side-channel diff --git a/docs/adr/README.md b/docs/adr/README.md index 83a046c..c0ea708 100644 --- a/docs/adr/README.md +++ b/docs/adr/README.md @@ -42,6 +42,6 @@ This directory contains the project's ADRs, recorded per - [ADR-0034 — `history.log` as a complete command journal; replay reads success-only](0034-history-journal-and-replay-filter.md) — **Accepted**, resolves a three-way tension in `history.log`'s roles found while implementing ADR-0033 3f: (1) the persistent log is success-only while the in-memory Up/Down recall ring records *every* submission (success or failure, "so users can recall and edit typo'd commands"), and the ring is re-seeded from the log on project open — so **failed commands are recallable within a session but silently lost across sessions**; (2) replay wants the state-building (successful) commands while recall wants everything typed, which one success-only file cannot serve; (3) `replay history.log` never actually worked — `run_replay` parses each whole line through the DSL parser with no understanding of the `||` record shape, so a real log fails on line 1, and **no test ever fed the pipe format to replay** (the `replay_history_log_records_subcommands_only` test only checks what replay *writes*, never replays the log as input). Decision: `history.log` becomes a **complete journal** — every submission recorded, tagged `ok`/`err` via the status field the format already reserved (ADR-0015 §5) — and **each consumer filters**: hydration reads all records (cross-session recall matches in-session), replay reads `ok` only (and learns the journal format, while still accepting bare-command `.commands` scripts; detection by the leading timestamp+status prefix so a `|` inside a bare command isn't misread). Successful commands stay journalled transactionally by the worker; failed commands are journalled `err` best-effort from the runtime/app error path (a parse failure never reaches the worker). Amends ADR-0006's "successfully executed" wording and ADR-0015 §5 ("status always `ok`") / §12 (hydration). Code deferred to two tracked test-first sub-tasks (journal-failures+filtering; replay-parses-journal-format); existing all-`ok` logs need no migration; **implemented 2026-05-24** (plan `docs/plans/20260524-adr-0034-history-journal.md`); **Amendment 1 (2026-05-24): replay filters out app-lifecycle commands** — a working `replay history.log` (the §3 fix) exposed that the journal also records `save as`/`load`/`new`/`export`/`import`/`rebuild`/`mode` (which would panic the worker dispatch or abort the replay), so replay now re-applies **only** schema/data write commands and **skips** every `Command::App` + nested `Command::Replay`; **all skips continue** (never abort — reversing the prior nested-`replay` refusal, so a journal containing a once-run `replay` needs no hand-editing, and the infinite-loop footgun is closed by construction), with a `[skip]` **warning** on `import` and nested-`replay` skips (their omission can leave replayed state incomplete) and silent skips for the rest; `replay.error_nested` removed, `replay.skipped_import`/`replay.skipped_replay` added, `ReplayCompleted` carries `warnings` - [ADR-0035 — Advanced-mode SQL DDL](0035-advanced-mode-sql-ddl.md) — **Accepted** (design agreed 2026-05-24; validated end-to-end by sub-phases 4a/4a.2/4a.3/4b `CREATE TABLE` (incl. foreign keys) + 4c `DROP TABLE [IF EXISTS]` + 4d `CREATE [UNIQUE] INDEX` / `DROP INDEX [IF EXISTS]` + 4e `ALTER TABLE` add/drop/rename column + 4f `ALTER TABLE … ALTER COLUMN TYPE` + 4g `ALTER TABLE` add/drop constraint + add FK + 4h `ALTER TABLE … RENAME TO` + 4i verification sweep (completion merge + simple/advanced completion colour + describe of table-level constraints + self-ref FK indicator + CREATE-TABLE help/usage), implemented 2026-05-25/26 — **Phase 4 complete**; **Amendment 1, 2026-05-26**: drop a composite UNIQUE via a derived, engine-neutral `unique_` name that reuses the existing `DROP CONSTRAINT ` grammar — no new syntax, no metadata, §4g anonymity intact; `describe` shows the name; dropping a UNIQUE-covered *column* now refuses with that name + the drop command), **Phase 4** of the ADR-0030 roadmap (peer of 0031/0032/0033) and **clarifies ADR-0030 §4**. Advanced-mode `CREATE`/`DROP`/`ALTER TABLE` + `CREATE`/`DROP INDEX` get their **own per-statement commands** (`SqlCreateTable`/`SqlAlterTable`/`SqlDropTable`/`SqlCreateIndex`/`SqlDropIndex`), like DML's `Sql*` set — but unlike DML they **execute *structurally*, not verbatim** (raw execution would lose the playground's types, named relationships, and `STRICT`; "verbatim" was a DML convenience, not a rule). Handlers **reuse the low-level schema/metadata helpers** where the operation matches simple mode and **stand alone where the SQL surface is richer** (clarity over forced refactoring); simple mode is untouched (additive). Dispatch: `create`/`drop` reuse ADR-0033 Amendment 1's category-grouped mode-aware dispatch (SQL-first, simple fallback); `alter` is a new advanced-only entry word. Full surface (no pre-emptive cuts, `Q4`): `CREATE TABLE` with column + table constraints, single/compound `PRIMARY KEY`, inline + table-level `FOREIGN KEY` → **named relationships** (one statement = one command = **one undo step**, ADR-0006); `ALTER TABLE` add/drop/rename column, `ALTER COLUMN TYPE`, add/drop constraint, add FK, **`RENAME TO`** (advanced-only table rename — new low-level op renaming the table + its CSV + the relationship and table-CHECK metadata, closing the rename half of `C1`); `CREATE [UNIQUE] INDEX` / `DROP INDEX`. Type slot accepts the ten playground keywords **and** standard-SQL aliases (`integer`→`int`, `varchar`→`text`, `timestamp`→`datetime`, …; length args accepted-and-ignored; no engine type names in/out — ADR-0030 §5). `CHECK`/`DEFAULT` reuse ADR-0031 `sql_expr`. **Pre-implementation `/runda` refinements (2026-05-24, user-confirmed):** `CREATE TABLE`/`DROP TABLE` **admit `IF [NOT] EXISTS`** (no-op-that-succeeds-with-a-note — a near-universal cross-vendor idiom, reclassified *into* scope, not engine-specific); `INTEGER PRIMARY KEY` maps to a **plain `int`** PK, *not* auto-increment (`serial` stays the sole auto-increment type). **Column-type-conversion is unified** (ADR-0017 engine, mode-appropriate policy): clean auto-converts and incompatible/own-type-static cases refuse in both modes, but a **lossy** change refuses-by-default in simple mode (`--force-conversion` opts in) while advanced mode **performs it with a loss note** and relies on **`undo` as the safety net** — no force flag, no dropping to simple mode (a payoff of shipping ADR-0006 first). OOS: views/triggers/txn-control/PRAGMA/etc. (ADR-0030 §3), the Postgres `USING` clause, and the DSL→SQL teaching echo (ADR-0030 Phase 5). Sub-phases 4a–4i, plus **4a.2** (per-column `CHECK`/`DEFAULT` via raw `sql_expr` text — `sql_expr` is validate-only, no `Expr` AST — + composite `UNIQUE(a,b)`; no new internal table) and **4a.3** (table-level/multi-column `CHECK`, landed via the new `__rdbms_playground_table_checks` metadata table because SQLite has no PRAGMA for CHECK; the builder tells a table-level CHECK from a column-level one by element position) and **4b** (foreign keys — inline `REFERENCES` + table-level `FOREIGN KEY` → ADR-0013 named relationships in the create transaction, one undo step; self-references + bare `REFERENCES ` supported, user-confirmed) and **4c** (`DROP TABLE [IF EXISTS]` → `SqlDropTable`, reusing `do_drop_table`; `IF EXISTS` is a no-op-with-note via `DropOutcome::Skipped`) and **4d** (`CREATE [UNIQUE] INDEX [IF NOT EXISTS] [] ON (cols)` → `SqlCreateIndex` and `DROP INDEX [IF EXISTS] ` → `SqlDropIndex`, reusing `do_add_index`/`do_drop_index`; **`CREATE UNIQUE INDEX` admitted** — ADR-0025 **Amendment 1** — via an additive `IndexSchema.unique` flag that round-trips through `project.yaml` and rebuild, with `[unique]` markers in the structure view + items panel, while simple-mode `add unique index` stays deferred; `IF [NOT] EXISTS` reuses the 4c skip path; `create`/`drop` each gain a *second* advanced node, exercising the all-candidates dispatch) and **4e** (`ALTER TABLE` add/drop/rename column → `SqlAlterTable`; `alter` is a new advanced-**only** entry word, runtime-decomposed to the existing `do_add_column`/`do_drop_column`/`do_rename_column` — no new worker layer; `do_add_column` extended to consume raw `default_sql`/`check_sql` so ADD COLUMN reaches CREATE-TABLE constraint parity; drop/rename refuse a column any CHECK references (table-level AND column-level, incl. a column's own self-check on rename) — the 4a.3 deferral, via a raw-CHECK-text tokenizer in the shared executors, so it guards both surfaces and fixes a latent rename-drift bug; SQL DROP COLUMN refuses an index-covered column with no `--cascade` spelling; the column executors + `do_add_index` gained an internal-`__rdbms_*`-table guard — all user-confirmed) and **4f** (`ALTER TABLE … ALTER COLUMN TYPE` → a fourth `AlterTableAction`, runtime-decomposed to the existing `change_column_type` with `ChangeColumnMode::ForceConversion` — which **is** the §7 advanced policy: lossy converts *with a note* (no force flag), incompatible + ADR-0017 static refusals (`↔ blob`, same-type, `date ↔ datetime`, non-`int → serial`) still refuse, while **`int → serial` is allowed** (auto-fills nulls + UNIQUE, ADR-0018 §8 — the §7 "→serial refused" summary is looser than the code); the builder discriminates the fourth branch by the **`type` keyword** (unique — ADD COLUMN's type is an ident), the type slot reuses `SQL_TYPE`; the internal-`__rdbms_*` guard was folded into `do_change_column_type`, closing the simple `change column` exposure too — user-confirmed) and **4g** (`ALTER TABLE … ADD [CONSTRAINT ] (CHECK | UNIQUE | FOREIGN KEY)` + `DROP CONSTRAINT `; ADD = CHECK + composite UNIQUE + FK, with `ADD PRIMARY KEY` and a *named* UNIQUE refused — composite UNIQUE is anonymous in our model; each ADD reuses a low-level path (table-CHECK/UNIQUE rebuild with a dry-run guard; FK → `add_relationship`, bare `REFERENCES

` → parent single-PK), DROP CONSTRAINT resolves the name to a table-CHECK then a child-side FK; **named table-CHECKs round-trip** via a nullable `name` column on `__rdbms_playground_table_checks` (**rebuild-only** arrival — pre-4g projects gain it on `rebuild`, a named add on an un-upgraded project is refused with a friendly "rebuild first" message) *and* a `project.yaml` `check_constraints` extension to an `{expr, name}` mapping (the bare-string form still reads); the internal-`__rdbms_*` guard was folded into `do_add_constraint`/`do_add_relationship`, completing that guard class — all user-confirmed) and **4h** (`ALTER TABLE … RENAME TO` — the one genuinely new low-level op, `do_rename_table`: a native engine rename plus one-transaction reconciliation of every metadata row naming the table (`__rdbms_playground_columns`, **both ends** of `__rdbms_playground_relationships`, `__rdbms_playground_table_checks`), the CSV file (the existing rewrite+delete path — no new persistence method), and **CHECK text that qualifies a column with the old table name** (`T.age`→`U.age`, a planning-`/runda` finding — the engine rewrites the live CHECK but the stored text would drift and break a fresh rebuild; `rewrite_check_table_qualifier` keeps them in step); grammar splits the `rename` verb into one branch with an inner Choice on a distinct second keyword (`column` vs `to`), the new-name slot mirroring the `CREATE TABLE` name slot; refuses same-name / existing-target / `__rdbms_*` / non-existent, with **case-insensitive** collision checks behind an engine-neutral pre-check (a finished-slice `/runda` finding — the engine matches names case-insensitively); auto-named indexes *and* relationships keep their stale names (only table-name columns update — §6 scope); one undo step; advanced-only, closing the rename half of `C1` — all user-confirmed) and **4i** (the verification sweep that completes Phase 4: the shared-entry-word completion merge + the simple-vs-advanced completion colour-when-mixed with Both→Advanced→Simple block ordering; `describe` of table-level composite UNIQUE + table CHECK; the self-ref FK pre-submit indicator fix; and the CREATE-TABLE help/usage skeleton refresh). **All of Phase 4 (4a–4i) is shipped.** Each sub-phase has exit + DA gates; **Amendment 2, 2026-05-27** (design agreed, pending impl): a **standard-first dialect stance** (refines ADR-0030's "standard SQL" posture — ISO spelling is canonical + echoed where one exists; a vendor shorthand may be *accepted* but isn't canonical; where ISO offers none, *one* documented vendor spelling is a deliberate extension) + an `ALTER COLUMN` **constraint gap-fill** surfaced by the ADR-0038 echo design: makes ISO `ALTER COLUMN … SET DATA TYPE` the canonical type-change verb with `TYPE` retained as a synonym (**reverses §4f's "no `SET DATA TYPE`"**), and adds `SET/DROP DEFAULT` (ISO) + `SET/DROP NOT NULL` (the one documented extension — ISO has no in-place NOT-NULL verb; PostgreSQL's chosen for being type-independent), all **rebuild-backed via the existing ADR-0029 `do_add_constraint`/`do_drop_constraint` executors** (dry-run + internal-table guards free, no new worker layer), reaching simple↔advanced constraint-mod **parity for NOT NULL + DEFAULT**; the **rebuild stays hidden** (Category-1 engine detail, ADR-0038). Residual gap left open + flagged: dropping a **column-level (anonymous) UNIQUE/CHECK** (no portable name — same class as Am1's parallel gap), which ADR-0038's catalogue marks "no headline echo" - [ADR-0036 — Value validation for advanced-mode DML](0036-typed-dml-values-vs-verbatim.md) — **Accepted** (design agreed + `/runda`'d 2026-05-26; mechanism then **deliberately narrowed** from "bind literals via the DSL path" to surgical **"validate-and-retain, execute verbatim"** after the user resisted consolidating the modes and a concrete auto-fill difference confirmed even the single-row literal case isn't identical across modes; **Phases 1–2 implemented** 2026-05-26 — `INSERT … VALUES` and `UPDATE … SET` literal validation + offending-value retention, capture-at-parse with no grammar change; **Phase 3a implemented** 2026-05-26 — live typed-slot hints + numeric-shape highlighting for `UPDATE`/UPSERT `SET col = ` via a boundary-aware lookahead (Amendment 1 corrects this ADR's naive-`Choice` sketch); **Phase 3b implemented** 2026-05-27 — per-position typed slots for `INSERT … VALUES` (single/multi-row, Form A/B) via a new zero-width `Node::SetColumn` primitive + an arity-gating tuple lookahead that preserves the §8.1 arity diagnostic; **fully implemented**). **Augments — does NOT supersede — ADR-0030 §4 / ADR-0033 §10**: execution stays verbatim, ADR-0033 Amendment 3's two-command identity (`Insert` vs `SqlInsert`) **stands**. The problem (investigated 2026-05-26; characterization test `sql_insert.rs::sql_dml_skips_app_level_value_validation_that_the_dsl_enforces` proves it): advanced-mode SQL DML gets **none** of the DSL's value feedback — a malformed `date` like `2025/01/15` is silently written, and the offending value is missing from constraint errors — because literal values are spliced into text and discarded (only `STRICT` storage types check them). **Fix (surgical): validate each literal value against its column type before the verbatim insert, and retain it for error reporting — sharing only the per-type validators (`Value::bind_for_column`/`validate_date`/`shortid::validate`), nothing else.** No binding, no statement reconstruction, no auto-fill change, no command-identity collapse — because the two gaps are closed by validation + retention alone, and executing the user's own text is already safe. The literal set = `NULL`/boolean/string/**signed**-numeric; arithmetic/functions/subqueries/column-refs are expressions (skipped — the engine evaluates them). `WHERE` not validated (it's an expression in general; motivation met by `VALUES`/`SET`); `SELECT`/`INSERT … SELECT`/`RETURNING`/`ON CONFLICT` need no special handling since execution is untouched. Phased: **Phase 1** capture-at-parse + validate + retain for `INSERT … VALUES` (no grammar change, no reparse — closes both proven gaps); **Phase 2** `UPDATE … SET` literals; **Phase 3** completion hinting/highlighting (the only part needing a grammar change — a typed-literal slot vs `sql_expr` reusing the DSL `TypedValueSlot`s at `data.rs:141`/`189`/`269`, discriminated by a **boundary-aware lookahead** not a naive `Choice` per **Amendment 1**; split into **3a** `SET` (done) and **3b** `VALUES` (pending); supersedes only Phase 1/2's literal *detection*, not the validation/enrichment on top). Non-goals: binding/reconstruction, collapsing command identity (Am3 stands), changing `serial`/`shortid` auto-fill (`requirements.md` **X4**, a separate possible-bug), a structural `SELECT`, a full SQL-expression AST. Embodies `requirements.md` **X5** (share a *mechanic*, not a *command*); the neutral "that value" safety net (ADR-0035 Amendment 1) stays correct for genuinely-computed values -- [ADR-0037 — Execution-time mode side-channel (the three-way submission mode)](0037-execution-time-mode-side-channel.md) — **Accepted** (design agreed 2026-05-27; channel **implemented + verified end-to-end** by its motivating consumer — ADR-0038's DSL → SQL teaching echo — across handoff-46 `04c8e42` (channel + first echo slice), handoff-47 `90479cb` (full Bucket A), `275c726` (Bucket B resolved-name + multi-statement renderers), and `e6ad1ae` (the category-3 `--dont-convert` caveat — gated on this channel too)), **redeems the follow-up deferred by ADR-0033 Amendment 3** (which named this ADR and its motivating consumer). Establishes the channel that lets a command know, **at execution time**, the effective mode it ran under — so execution can adjust **output** without touching **identity** (the motivating case: a DSL-form `create table` echoing the equivalent SQL when run in advanced mode, silent in simple — ADR-0030 §10, realised by ADR-0038). Introduces a **new per-submission enum `SubmissionMode` { Simple, Advanced, AdvancedOneShot }** — *refining* Amendment 3's "widen `Mode`" sketch: the persistent input `Mode` stays **two-way** (`mode.rs` keeps the one-shot `:` out of persistent state), and the three-way distinction lives on the per-submission channel where the transient `:` belongs. Resolved at submit time (Simple+`:` → `AdvancedOneShot`; Advanced `:` is a no-op), threaded through `Action::ExecuteDsl` → worker, **output-only** (no executor branches its *effect* on it — Amendment 3 forbids behavioural mode dependence). The worker builds the teaching echo (+ category-3 expansion data — ADR-0038) for DSL-form commands in advanced/one-shot mode and returns it; the App renders it beneath `[ok]`. Co-located with execution because the echo's harder forms (resolved auto-names, generated `shortid`s, conversion counts) are worker-computed facts, and gating on mode means the work happens only when shown. Alternatives weighed + rejected: widening `Mode` (conflates transient/persistent state); App-side gating with the worker always emitting echo data (computes unconditionally, doesn't generalise, re-opens the render-side framing ruled against). Scope: channel + resolution rule only — the renderer/catalogue/`Value → SQL-literal` are ADR-0038, the `ALTER COLUMN` gap-fill is the ADR-0035 amendment -- [ADR-0038 — The DSL → SQL teaching echo](0038-dsl-to-sql-teaching-echo.md) — **Accepted** (design agreed 2026-05-27; **Phases 1–3 implemented + verified** — every catalogue row in §7 Buckets A + B and the §6 category-3 prose round-trips per line through the advanced walker per §1: handoff-46 `04c8e42` shipped the channel + create-table slice, handoff-47 `90479cb` the full Bucket A expansion + a skeleton contract-gap fix (dropped per-column `DEFAULT`/`CHECK`), `275c726` the Bucket B resolved-name + multi-statement renderers (auto- and user-named `add index`, positional `drop index`, `add`/`drop relationship` in both selector forms, `drop column --cascade`, `add relationship --create-fk`), and `e6ad1ae` the only missing category-3 line — the `change column --dont-convert` *caveat* (shortid + transform notes were already surfaced via pre-existing `client_side.*` keys). **Only the §4 de-emphasised styled-runs rendering polish remains** — the echo currently surfaces as plain `[system]` lines), **realises ADR-0030 §10** (the teaching bridge) — the Phase-5 echo **ADR-0035 §12 forward-referenced** — building on **ADR-0037** (the `SubmissionMode` gate) and **ADR-0035 Amendment 2** (standard-first dialect + `ALTER COLUMN` gap-fill). When a **DSL-form** command runs in advanced/one-shot mode, the worker emits the equivalent SQL beneath `[ok]` as a de-emphasised styled `OutputLine` (ADR-0028); the App renders it. **Defining invariant — the copy-paste contract:** every echoed line is *runnable advanced-mode SQL* (round-trip-tested: parse the echo → same-effect command; a planned "copy the echo" affordance depends on it). **Type vocabulary = the playground's own keywords** (`serial`/`shortid`/…, accepted by `from_sql_name`, decision (a)); **statement shape = the standard-first dialect** (Am2). **DML uses substituted literals, not `?`** (per-type `Value → SQL-literal`, round-trip-safe; `blob` moot — no literal syntax exists; auto-gen columns omitted to match `do_insert` + X4). **Firing reality — a DDL + `show data` feature:** in advanced mode `insert`/`update`/`delete … where` are SQL-first (`Sql*` = already SQL = nothing to echo per §10); only DSL-*only* spellings echo (DDL + `show data` + the `delete`/`update … --all-rows` fall-throughs — the latter via **ADR-0033 Amendment 4**, a bug-fix folded in here that reverses Amendment 3's `update … --all-rows` misparse). **Three-category framework** for "what happens beyond the literal SQL": **(1) engine-implementation-hiding** (the rebuild, rowid PK, non-PK `serial` MAX+1) — *never surfaced*; **(2) decomposable into advanced SQL** (`drop column --cascade`, `--create-fk` relationship) — *shown as the runnable multi-line sequence, one statement per line*; **(3) playground type-behaviour with no SQL-expressible form** (`shortid` generation — no `shortid()`; type-conversion transforms — no `USING`) — *de-emphasised prose expansion from the worker's `client_side.*` notes*. Carries the **full catalogue** (Buckets A single-statement / B resolved-name + multi-line / C no-echo) mapping every DSL-form command to its echo. OOS: reverse SQL→DSL echo (§13 OOS-5), app commands / `show table` / `explain` / `replay`, a `blob` literal, the column-level UNIQUE/CHECK drop residual (Bucket C until Am2's gap closes), and surfacing any category-1 engine internal +- [ADR-0037 — Execution-time mode side-channel (the three-way submission mode)](0037-execution-time-mode-side-channel.md) — **Accepted** (design agreed 2026-05-27; channel **implemented + verified end-to-end** by its motivating consumer — ADR-0038's fully-shipped DSL → SQL teaching echo — across handoff-46 `04c8e42` (channel + first echo slice), handoff-47 `90479cb` (full Bucket A), `275c726` (Bucket B resolved-name + multi-statement renderers), `e6ad1ae` (the category-3 `--dont-convert` caveat — gated on this channel too), and `2aab457` (the §4 styled-runs rendering polish)), **redeems the follow-up deferred by ADR-0033 Amendment 3** (which named this ADR and its motivating consumer). Establishes the channel that lets a command know, **at execution time**, the effective mode it ran under — so execution can adjust **output** without touching **identity** (the motivating case: a DSL-form `create table` echoing the equivalent SQL when run in advanced mode, silent in simple — ADR-0030 §10, realised by ADR-0038). Introduces a **new per-submission enum `SubmissionMode` { Simple, Advanced, AdvancedOneShot }** — *refining* Amendment 3's "widen `Mode`" sketch: the persistent input `Mode` stays **two-way** (`mode.rs` keeps the one-shot `:` out of persistent state), and the three-way distinction lives on the per-submission channel where the transient `:` belongs. Resolved at submit time (Simple+`:` → `AdvancedOneShot`; Advanced `:` is a no-op), threaded through `Action::ExecuteDsl` → worker, **output-only** (no executor branches its *effect* on it — Amendment 3 forbids behavioural mode dependence). The worker builds the teaching echo (+ category-3 expansion data — ADR-0038) for DSL-form commands in advanced/one-shot mode and returns it; the App renders it beneath `[ok]`. Co-located with execution because the echo's harder forms (resolved auto-names, generated `shortid`s, conversion counts) are worker-computed facts, and gating on mode means the work happens only when shown. Alternatives weighed + rejected: widening `Mode` (conflates transient/persistent state); App-side gating with the worker always emitting echo data (computes unconditionally, doesn't generalise, re-opens the render-side framing ruled against). Scope: channel + resolution rule only — the renderer/catalogue/`Value → SQL-literal` are ADR-0038, the `ALTER COLUMN` gap-fill is the ADR-0035 amendment +- [ADR-0038 — The DSL → SQL teaching echo](0038-dsl-to-sql-teaching-echo.md) — **Accepted** (design agreed 2026-05-27; **fully implemented + verified** — every catalogue row in §7 Buckets A + B and the §6 category-3 prose round-trips per line through the advanced walker per §1, and the §4 de-emphasised styled-runs polish is wired: handoff-46 `04c8e42` shipped the channel + create-table slice, handoff-47 `90479cb` the full Bucket A expansion + a skeleton contract-gap fix (dropped per-column `DEFAULT`/`CHECK`), `275c726` the Bucket B resolved-name + multi-statement renderers (auto- and user-named `add index`, positional `drop index`, `add`/`drop relationship` in both selector forms, `drop column --cascade`, `add relationship --create-fk`), `e6ad1ae` the last category-3 line — the `change column --dont-convert` *caveat* (shortid + transform notes were already surfaced via pre-existing `client_side.*` keys), and `2aab457` the §4 styled-runs polish: a new `OutputKind::TeachingEcho` custom rendering branch (dimmed `Executing SQL:` prefix + the SQL re-lexed in advanced mode for token-class colouring, same as the input echo) plus a new `OutputStyleClass::Hint` for every cat-3 prose line — caveat *and* the existing illuminating notes, user-confirmed broader scope), **realises ADR-0030 §10** (the teaching bridge) — the Phase-5 echo **ADR-0035 §12 forward-referenced** — building on **ADR-0037** (the `SubmissionMode` gate) and **ADR-0035 Amendment 2** (standard-first dialect + `ALTER COLUMN` gap-fill). When a **DSL-form** command runs in advanced/one-shot mode, the worker emits the equivalent SQL beneath `[ok]` as a de-emphasised styled `OutputLine` (ADR-0028); the App renders it. **Defining invariant — the copy-paste contract:** every echoed line is *runnable advanced-mode SQL* (round-trip-tested: parse the echo → same-effect command; a planned "copy the echo" affordance depends on it). **Type vocabulary = the playground's own keywords** (`serial`/`shortid`/…, accepted by `from_sql_name`, decision (a)); **statement shape = the standard-first dialect** (Am2). **DML uses substituted literals, not `?`** (per-type `Value → SQL-literal`, round-trip-safe; `blob` moot — no literal syntax exists; auto-gen columns omitted to match `do_insert` + X4). **Firing reality — a DDL + `show data` feature:** in advanced mode `insert`/`update`/`delete … where` are SQL-first (`Sql*` = already SQL = nothing to echo per §10); only DSL-*only* spellings echo (DDL + `show data` + the `delete`/`update … --all-rows` fall-throughs — the latter via **ADR-0033 Amendment 4**, a bug-fix folded in here that reverses Amendment 3's `update … --all-rows` misparse). **Three-category framework** for "what happens beyond the literal SQL": **(1) engine-implementation-hiding** (the rebuild, rowid PK, non-PK `serial` MAX+1) — *never surfaced*; **(2) decomposable into advanced SQL** (`drop column --cascade`, `--create-fk` relationship) — *shown as the runnable multi-line sequence, one statement per line*; **(3) playground type-behaviour with no SQL-expressible form** (`shortid` generation — no `shortid()`; type-conversion transforms — no `USING`) — *de-emphasised prose expansion from the worker's `client_side.*` notes*. Carries the **full catalogue** (Buckets A single-statement / B resolved-name + multi-line / C no-echo) mapping every DSL-form command to its echo. OOS: reverse SQL→DSL echo (§13 OOS-5), app commands / `show table` / `explain` / `replay`, a `blob` literal, the column-level UNIQUE/CHECK drop residual (Bucket C until Am2's gap closes), and surfacing any category-1 engine internal - [ADR-0039 — EXPLAIN over advanced-mode SQL queries](0039-explain-over-advanced-sql.md) — **Accepted** (decision recorded 2026-05-27; **implementation deferred** as a follow-up to the ADR-0037/0038 echo effort — not in that pass), **supersedes ADR-0030 §13 OOS-2**. Lets `explain` wrap the advanced SQL commands (`Select`/`SqlInsert`/`SqlUpdate`/`SqlDelete`) in addition to the DSL `ShowData`/`Update`/`Delete` it already covers (ADR-0028), running `EXPLAIN QUERY PLAN` over the validated SQL text through the existing ADR-0028 span-styled plan tree (advanced mode only; DSL `explain` unchanged in both modes). Reframes OOS-2 as a *deferred* scope exclusion (per ADR-0000's new out-of-scope discipline), not a principled rejection — surfaced 2026-05-27 while characterising advanced-mode explain (suspected a bug; it was OOS-2 as written). Self-contained, orthogonal to the echo (the echo renders SQL *from* DSL; this explains SQL the user *wrote*). OOS (deferred): EXPLAIN of DDL (no query plan exists). Built test-first when picked up diff --git a/docs/handoff/20260528-handoff-49.md b/docs/handoff/20260528-handoff-49.md new file mode 100644 index 0000000..30e90b5 --- /dev/null +++ b/docs/handoff/20260528-handoff-49.md @@ -0,0 +1,171 @@ +# Session handoff — 2026-05-28 (49) + +Forty-ninth handover. **ADR-0038 is done.** Every catalogue row of the +DSL → SQL teaching echo is implemented, every echo round-trips per +line per the §1 copy-paste contract, every §6 category-3 line surfaces, +and the §4 de-emphasised styled-runs rendering polish landed in +`2aab457`. Docs in sync. If you continue, pick a fresh feature; if +you're triaging, ADR-0038 won't surprise you. + +## §1. State at handoff + +**Branch:** `main`. **HEAD `2aab457`.** **Tests: 2019 passing, 0 +failing, 0 skipped, 1 ignored.** **Clippy: clean** +(`--all-targets -D warnings`, nursery). + +Commits since handoff-48's `df5c4e2`: + +``` +2aab457 feat: DSL→SQL teaching echo — §4 styled-runs polish (ADR-0038) +``` + +(Plus this handoff + lockstep doc updates landing alongside it.) + +## §2. ADR-0038 — done + +The feature shipped across **five** commits, the last of which is the +§4 polish: + +| Commit | Phase | What | +|---|---|---| +| handoff-46 `04c8e42` | 1 (start) | channel + create-table slice | +| `90479cb` | 1 (rest) | Bucket A renderer — every single-statement DDL row + `show data` + `--all-rows` fall-throughs; create-table contract-gap fix | +| `275c726` | 2 | Bucket B — resolved-name + multi-statement (auto- and user-named `add index`, positional `drop index`, `add`/`drop relationship` in both selector forms, `drop column --cascade`, `add relationship --create-fk`) | +| `e6ad1ae` | 3 | category-3 prose — `change column --dont-convert` *caveat* (the other two cat-3 lines were already in place via `client_side.*` keys) | +| `2aab457` | 4 (polish) | §4 styled-runs rendering — `OutputKind::TeachingEcho` with dimmed prefix + advanced-mode lex of the SQL; `OutputStyleClass::Hint` for every cat-3 prose line | + +Plus the /runda DA cleanup at `5cb105b` (doc-drift + Bucket C +explicit tests). + +ADR-0037 and ADR-0038 both **Accepted** with implementation notes +referencing every shipping commit. + +## §3. What the polish (Phase 4) does, in shape + +Per ADR-0038 §4 + §6, the de-emphasised styled-runs treatment landed +in `2aab457`. The user-confirmed scope (per session discussion) went +slightly beyond §4's "echo + caveat" to cover the existing +illuminating cat-3 notes too — visual consistency for every cat-3 +prose line, matching §6's "de-emphasised prose line" wording. + +**The polish in code:** + +- **`OutputKind::TeachingEcho`** (new). Lines with this kind go through + a custom `ui::render_output_line` branch that mirrors the + `OutputKind::Echo` simple-mode input-echo path: strip the canonical + `crate::echo::TEACHING_ECHO_LABEL` ("Executing SQL: "), render the + prefix dimmed (`theme.muted`), then lex the rest in `Mode::Advanced` + via `input_render::lex_to_runs_in_mode` and emit one span per token. + Tag stays `[system]` for visual consistency with other system + output. +- **`OutputStyleClass::Hint`** (new), resolved to `theme.muted` by + `output_span_style`. Carried on category-3 prose lines as a single + styled-runs span covering the whole text (so the body renders + dimmed; the `[system]` tag keeps its kind tint). +- **`crate::echo::TEACHING_ECHO_LABEL`** (new pub const) — the fixed + byte boundary the ui.rs branch needs. The label moved out of the + `echo.executing_sql` i18n key (now retired in en-US.yaml + keys.rs; + a comment in en-US.yaml points future locales at re-introducing + it). +- **App-side helpers**: `App::push_teaching_echo(sql)` builds the + TeachingEcho line; `App::push_category_three_prose(text)` builds a + System line with a whole-text Hint span. `note_ok_summary` and + `handle_dsl_change_column_success` / `handle_dsl_add_column_success` + use these instead of plain `note_system` for the echo, the + DontConvert caveat, and the illuminating client-side notes. + +**Where each rendering rule lives:** + +| Line | Built by | Renders as | +|---|---|---| +| `[ok] verb subject` | `note_ok_summary` via `note_system` | `[system]` tag + `system` green body (unchanged) | +| `Executing SQL: ` (one per echo line) | `push_teaching_echo` | `[system]` tag + dimmed prefix + token-coloured SQL | +| `[client-side] N row(s) were transformed …` | `push_category_three_prose` (auto_fill_*, transformed*) | `[system]` tag + whole-body dim | +| `[client-side] --dont-convert kept …` | `push_category_three_prose` (caveat path) | `[system]` tag + whole-body dim | +| structure render, row counts, etc. | `note_system` | unchanged | + +**Coverage:** four new tests pin the polish — `ui::tests:: +teaching_echo_line_renders_dim_prefix_and_lexed_sql` (asserts the +dim prefix span + keyword-coloured SQL spans confirm advanced-mode +lex), `ui::tests::category_three_prose_line_renders_all_dim` +(whole-text Hint coverage), `ui::tests:: +hint_class_resolves_to_muted_foreground` (theme resolution across +both palettes), `app::tests:: +polished_echo_carries_teaching_echo_kind_and_caveat_a_hint_span` +(App-side wiring kinds + styled_runs shape). The pre-existing echo +tests pass unchanged — the text content is identical, only styling +changed. + +## §4. Notable observations carried over from handoff-48 + +These are the same ones flagged before the polish — none became +blockers, the polish didn't introduce new ones. + +- **Unquoted-identifier round-trip caveat.** Echo style is bare + identifiers (matching the original create-table echo aesthetic). A + user-created column or table named after a SQL keyword (e.g. + `ORDER`, `SELECT`) would produce an echo whose round-trip is + fragile. Pre-existing limitation; the §1 contract is only formally + violated for this edge case. Fix is identifier-quoting if it ever + bites a learner. +- **No full-spawn integration test for the echo pipeline.** Coverage + is layered (pure renderers + runtime-helper unit tests with real + Database + App-level rendering tests + ui.rs styling tests). The + unwired glue is small; a Tier-3 end-to-end test would complete the + picture, not blocking. +- **Pure-Command Schema commands double-compute echo.** `echo_for` is + called for every command; the Schema arm recomputes via + `build_schema_echo` (whose catch-all redelegates to + `command_to_sql`). Minor inefficiency, no behaviour bug. + +## §5. What's next + +Up to you. ADR-0038 won't pull on any more rope. Candidate features: + +- **ADR-0039 — EXPLAIN over advanced-mode SQL queries** (`Accepted`, + implementation deferred). Self-contained: lets `explain` wrap the + advanced SQL commands (`Select`/`SqlInsert`/`SqlUpdate`/ + `SqlDelete`) in addition to the DSL `ShowData`/`Update`/`Delete` it + already covers (ADR-0028). Test-first when picked up. +- **Other tracks** from `requirements.md` — see the file for the + pending list. Likely candidates: track 2 (project storage Iter 5/6 + — export/import + `--resume` + persistent input history + migration + framework), `C3a` (modify relationship), `C4` (m:n convenience), + `H1`/`H1a` (friendly error layer + syntax-help in parse errors), + the tutorial/lesson system, V4 session log + Markdown export, the + remaining input UX items (`I1`/`I1b` multi-line + readline + shortcuts, `I3` tab completion polish, `I4` syntax highlighting + beyond input echo). +- **Polish-spawned ideas** from this session — none currently + outstanding; the OutputStyleClass vocabulary now has `Hint` and + could grow further if another consumer needs it. + +## §6. Process pins (unchanged) + +- **Confirm every commit** (propose message, wait). **No AI attribution.** +- **Test-first**; green + clippy-clean is the only acceptable end + state; current baseline **2019 / 0 / 1**. +- **Keep docs lockstep**: ADR + `README.md` index + `requirements.md`. + This session's polish commit ships alongside this handoff and the + three doc-update edits (M4 / ADR-0038 Status / ADR-0037 + ADR-0038 + README index entries). +- **Round-trip every catalogue row** (the §1 copy-paste contract; for + multi-statement, *per line*). + +## §7. How to take over + +1. **Read, in order:** this file → `requirements.md` for the open + tracks → the relevant ADR for whatever feature you pick up. + ADR-0038 / ADR-0037 are done and well-documented; touch them only + to amend if a real correction comes up. +2. **Baseline:** `cargo test` (2019 / 0 / 1) + `cargo clippy + --all-targets -- -D warnings` (clean). +3. **For a fresh feature**: write the ADR (or extend an existing + one), run `/runda` over the design before building, then build + test-first. +4. **The ADR-0028 styled-runs vocabulary** is now richer: `Neutral`, + `Efficient`, `Expensive`, `AutomaticIndex`, `Hint`. Adding a new + consumer? Pattern is one variant per semantic class, theme-resolved + in `ui::output_span_style`, used via `OutputLine::styled` or a + dedicated kind + custom render branch (as `TeachingEcho` does for + the dim-prefix + lex-rest case). diff --git a/docs/requirements.md b/docs/requirements.md index e09ea2c..a00546b 100644 --- a/docs/requirements.md +++ b/docs/requirements.md @@ -132,22 +132,27 @@ handoff-14 cleanup; 449 after B2/C2.) ADR-0030 §10 teaching echo on it: a DSL-form command run in advanced/one-shot mode renders the equivalent advanced-mode SQL beneath `[ok]`; simple-mode and SQL-entered submissions stay - silent. Echo coverage: **Buckets A and B complete, plus - category-3 prose** — every catalogue row in ADR-0038 §7 - round-trips per line through the advanced walker (the §1 - copy-paste contract; §6 category 2 holds it per line). Shipped - across three commits: **Phase 1** Bucket A — single-statement DDL - + `show data` + `--all-rows` fall-throughs (`90479cb`); **Phase 2** - Bucket B — resolved-name + multi-line echoes (`add index` auto- and - user-named, positional `drop index`, `add`/`drop relationship` in - both selector forms, `drop column --cascade`, `add relationship - --create-fk`) (`275c726`); **Phase 3** the category-3 prose — - `shortid` generation and type-conversion transforms via the - pre-existing `client_side.auto_fill_*` / `client_side.transformed*` - notes, plus the new `change column --dont-convert` caveat - (`e6ad1ae`). The only remaining ADR-0038 item is the §4 - de-emphasised **styled-runs rendering polish** (ADR-0028) — the - echo + caveat currently surface as plain `[system]` lines. + silent. Echo coverage: **ADR-0038 is feature-complete** — every + catalogue row in §7 round-trips per line through the advanced + walker (the §1 copy-paste contract; §6 category 2 holds it per + line), every §6 category-3 line surfaces, and the §4 + de-emphasised styled-runs rendering polish (ADR-0028) is wired. + Shipped across four feature commits: **Phase 1** Bucket A — + single-statement DDL + `show data` + `--all-rows` fall-throughs + (`90479cb`); **Phase 2** Bucket B — resolved-name + multi-line + echoes (`add index` auto- and user-named, positional + `drop index`, `add`/`drop relationship` in both selector forms, + `drop column --cascade`, `add relationship --create-fk`) + (`275c726`); **Phase 3** category-3 prose — `shortid` generation + and type-conversion transforms via the pre-existing + `client_side.auto_fill_*` / `client_side.transformed*` notes, plus + the new `change column --dont-convert` caveat (`e6ad1ae`); **Phase + 4 styled-runs polish** — `OutputKind::TeachingEcho` custom + rendering branch (dimmed `Executing SQL:` prefix + the SQL re-lexed + via `input_render::lex_to_runs_in_mode(Advanced)` for token + highlighting, same as the input echo), `OutputStyleClass::Hint` + for every cat-3 prose line (caveat + the existing illuminating + notes — broader scope, visually consistent) (`2aab457`). ## App-level commands (per ADR-0003)