When an INSERT's column list omits one or more shortid columns,
the worker now fills them. Command::SqlInsert gains listed_columns
and row_source, captured in build_sql_insert from the matched path
(the row source is located by the first values/select/with Word
token, so a string literal like 'select' can't be mistaken for the
keyword). do_sql_insert calls plan_shortid_autofill, which — per
the user-confirmed Option B — materialises the row source by
running it as a query, generates a distinct shortid per row via the
existing generate_shortid_batch (deduped against stored values),
and reconstructs a parameterised multi-row INSERT over the listed
columns plus the omitted shortid columns. Uniform for VALUES and
INSERT…SELECT, and handles multiple omitted shortids in one row
(each gets its own batch). No explicit list, no omitted shortid, or
a zero-row source → execute verbatim (the 3b path). serial stays
engine-filled via rowid. history.log keeps the original line, never
the rewrite (§11).
Tests: VALUES single/multi-row distinct; explicit override
honoured; INSERT…SELECT distinct fills; combined serial(engine) +
shortid(worker); two shortids (PK + non-PK) both fill; one provided
+ one omitted; compound-PK shortid member; mixed-case column name
(ADR-0009 DA gate); original-source-in-history on the rewrite path.
Still behind the dev `sqlinsert` entry word (3j). 1503 green,
clippy clean.
Make the INSERT row source a Choice between the VALUES clause and
Subgrammar(&sql_select::SQL_SELECT_COMPOUND). SQL_SELECT_COMPOUND
is itself a Choice that admits a leading WITH, so a WITH-prefixed
SELECT row source (R4) parses through it for free; the two
branches start on disjoint keywords (values vs select/with) so the
Choice never ambiguously commits. No worker change — do_sql_insert
already executes the validated SQL and re-persists, and the engine
handles insert-from-query.
Tests: grammar accept (plain / column-list+projection / WITH-
prefixed / trailing-semi) and reject (__rdbms_* on the SELECT's
FROM slot, incomplete select); integration parse-path lowering +
worker round-trip (rows land, CSV re-persisted) incl. R4 WITH end-
to-end; walker cross-cut that the Phase-2 unknown_column diagnostic
fires on the INSERT…SELECT projection; DA-gate test that a self-
sourced INSERT…SELECT runs as a plain insert (no cascade summary —
that is DELETE-only). Still behind the dev `sqlinsert` entry word
(shared `insert` is 3j). 1493 tests green, clippy clean.
walk_repeated discarded the last matched item's trailing-optional
expectations at a clean item boundary, so a comma-separated list
offered no continuation after a complete item: `order by Name `
gave no asc/desc, `select Name ` no `as`, `create table …
Code(text) ` no not/unique/default/check. Capture the last item's
skipped set and surface it when the list ends at an item boundary
(the separator `,` itself is deliberately not surfaced).
That fix made expression-position candidate lists long, which
exposed a visibility problem: the hint panel's candidate line is
single-row and window-scrolls on overflow, centring on item 0 when
nothing is selected — so with keywords-first, schema identifiers
scrolled off behind the `>` marker. Reverse the ordering: schema
identifiers (table/column/relationship names) now sort before
keywords, since a name the user would have to look up is the
highest-value completion and must stay visible (keywords are
learned over time; the tok_identifier/tok_keyword colour split
marks the boundary). This reverses the handoff-14 keywords-first
call, now recorded in ADR-0022 Amendment 2.
Tests: walker expected-set + completion-layer regressions for the
trailing-optionals and the ordering; candidate_ordering.rs header
invariant inverted; ~20 typing-surface snapshots re-baselined; a
two-line hint box recorded as a deferred follow-up.
SQL_INSERT_SHAPE (INTO <table> [(cols)] VALUES tuple(s)) with __rdbms_*
target rejection; Command::SqlInsert{sql,target_table}; Request::RunSqlInsert
+ do_sql_insert worker (tx-guarded: execute, then finalize_persistence for
CSV + history before commit, so failures roll back and don't re-persist).
Auto-show is best-effort via last_insert_rowid range.
Isolated behind a dev `sqlinsert` entry word (Advanced) so the SQL path is
testable without making `insert` a shared word yet (that's 3j, after 3d
auto-fill parity). Command::SqlInsert carries only sql+target_table; the
plan's listed_columns/returning land in 3d/3g where they're read.
6 grammar accept/reject tests + 8 integration tests (single/multi-row,
column-list, full-arity, history, rollback-on-failure, multi-row atomicity,
parse-path reconstruction, internal-table rejection). 1452 baseline green.
Three DA critiques from the Phase-2 verification flagged real gaps;
this commit closes them.
1. Type recovery row-independence (critique #1). The all-10-types
test left col_blob NULL because the DSL Value enum has no Blob
variant. The DA flagged this as a potential row-dependence gap.
Added `database_run_select_type_recovery_works_on_empty_table`
that proves column-origin metadata works on Text AND Blob
columns with zero rows, pinning the invariant. The all-types
test now carries an explicit comment referencing it.
2. Engine.* pattern matching against real SQLite output (critique
#2). The pre-rework tests fed `translate_generic` hand-coded
strings; never verified that the pinned SQLite version actually
produces those wordings. Added three engine-routing tests in
`tests/sql_select.rs` that produce real engine errors via
`run_select` and assert catalog routing. Aggregate-in-WHERE
confirms end-to-end. GROUP-BY-required and scalar-subquery
are SQLite-permissive (no real error on the natural triggers),
so those tests verify the matcher doesn't false-positive on
benign queries + that synthetic messages route correctly.
3. Manual TUI verification (critique #3) surfaced an additional
gap: `App::input_validity_verdict()` was hard-coded silent in
Advanced mode, so SQL predicate warnings emitted but never
reached the [WRN] indicator. Wired the verdict through to the
active effective mode; updated two pre-existing tests that
pinned the now-superseded "silent in Advanced" behavior; added
one new test confirming a SQL `LIKE`-on-numeric warning fires
the indicator. Launched the TUI, typed a representative
warning-triggering SELECT, confirmed SELECT/FROM/WHERE/LIKE
highlight as keyword colour AND the [WRN] indicator appears.
Test totals: 1441 → 1446 passing (+5). Clippy clean.
Cross-cut verification matrix for ADR-0032 Phase 2 is now fully
populated with concrete test references — every row green. Filling
the matrix surfaced three real gaps that this commit closes.
1. Advanced-mode syntax highlighting (ADR-0030 §8 matrix row).
The `ui.rs` Advanced branch routed through `plain_input_spans`,
bypassing the highlight walker entirely. In production SQL
keywords past the entry word rendered as plain identifiers.
Fix: mode-aware variants of `highlight_runs`,
`render_input_runs`, `lex_to_runs`, and `input_diagnostics`;
the Advanced render path now uses the highlighted form with
`Mode::Advanced`. `plain_input_spans` removed (unused).
2. Engine.* key wiring (ADR-0032 §11.4 / §13 matrix rows + handoff
§3.3 follow-up). The four Phase-2 engine.* catalog entries
were authored in 2d but never reached: `translate_generic`
discarded the engine message and returned a vague catalog
entry. Fix: pattern-match the engine message text for the four
Phase-2 categories (aggregate misuse, group-by required,
compound arity mismatch fallback, scalar-subquery cardinality)
inside `translate_generic`, routing each to its engine-neutral
catalog entry.
3. Matrix-coverage tests. Thirteen new tests covering the rows
that had no explicit coverage:
- 3 SQL keyword/operator/CASE highlight tests
- 4 engine.* engine-message tests
- 3 sql_expr column-completion tests (WHERE, HAVING)
- 3 predicate-warning slot tests (CASE, ORDER BY, projection)
- 1 all-10-playground-types recovery test (tests/sql_select.rs)
Plan document (docs/plans/20260520-adr-0032-phase-2.md) updated:
every (TBD) row in the cross-cut matrix replaced with a concrete
test file::function reference and a green status marker.
Test totals: 1428 → 1441 passing (+13 new). Clippy clean.
`Cargo.toml`: add `column_metadata` to rusqlite's feature list.
This pulls in the SQLite `SQLITE_ENABLE_COLUMN_METADATA`
compile flag and surfaces `sqlite3_column_table_name` /
`sqlite3_column_origin_name` on prepared statements via
rusqlite's `Statement::columns_with_metadata()`.
`do_run_select` in db.rs now calls a new
`resolve_select_column_types(conn, stmt)` helper after
`prepare`. The helper walks each result-column's origin
metadata; when both `table_name` and `origin_name` come back
populated (the result column traces back to a base-table
column), it looks up the playground type in
`__rdbms_playground_columns`. The per-column types thread
through to `format_cell(value, ty)` so the data-table
renderer (ADR-0016) gets the same per-type rendering it
applies to `show data` results.
Effect: ADR-0030 Phase-1 §4.5 (bool SELECT results render as
`0` / `1`) is lifted for any bare-column reference whose
origin the engine carries through — per ADR-0032 Amendment 1
(2026-05-20 empirical probe), that means all non-recursive
CTE bodies, scalar subqueries (aliased or not), derived
tables, set ops, and JOINs. Computed projections and
recursive-CTE result columns remain typeless (the engine
populates no origin), which the renderer handles via neutral
alignment.
The lookup is engine-driven verbatim — no grammar-side
structural classification (ADR-0032 Amendment 1 replaces
§12's original "structurally a single column reference" rule
with "trust column_table_name / column_origin_name").
Tests (3 new in `tests/sql_select.rs`, all green):
- `database_run_select_recovers_bool_column_type` — the
Phase-1 §4.5 case: `SELECT Active FROM Products` returns
`column_types = [Some(Bool)]` and rows render as `true` /
`false`.
- `database_run_select_recovers_text_type_through_alias` —
`SELECT Name AS n FROM Users` remaps the result column
name to `n` but the origin metadata still resolves the
playground type to `Some(Text)`.
- `database_run_select_computed_expression_stays_typeless`
— `SELECT Score + 1 FROM T` keeps `column_types[0] =
None`, the documented Amendment-1 exception.
The CTE pass-through, scalar subquery, set-op, and JOIN
cases all work for free given the empirical findings;
their behaviour is asserted by the Amendment-1 probe
results recorded in the ADR, so no per-case integration
tests are duplicated here.
Test totals: 1382 → 1385 passing (+3), 0 failed, 1 ignored.
Clippy clean.
`tests/sql_select.rs` covers the full advanced-mode SELECT path
end to end (ADR-0030 Phase 1, ADR-0031):
App-level dispatch
- `advanced_mode_select_dispatches_as_command_select`: an
advanced-mode `select 1` produces exactly one
`Action::ExecuteDsl { command: Command::Select { sql }, .. }`
carrying the validated SQL text.
- `simple_mode_select_yields_sql_hint_and_does_not_dispatch`:
a simple-mode `select` produces no dispatch action and the
error output contains the SQL hint naming both recovery
paths (`mode advanced` / the `:` one-shot).
- `colon_one_shot_from_simple_mode_dispatches_select`:
`:select 1` keeps the persistent mode as `Simple` while
dispatching `Command::Select` with the `:` stripped.
- `advanced_mode_select_from_internal_table_is_rejected`:
a SELECT against `__rdbms_playground_columns` is refused by
the grammar's `reject_internal_table` validator.
Worker round-trip
- `database_run_select_constant_returns_a_single_row`:
`select 1` runs through `Database::run_select` and returns
a `DataResult` with one row whose only cell is `1`; all
`column_types` are `None` (ADR-0030 §6).
- `database_run_select_from_user_table_returns_inserted_rows`:
create-table → insert → `select Name from T` round-trips
the inserted row through the worker.
- `database_run_select_appends_to_history_when_source_present`:
the literal source line lands in `history.log` so replay
re-runs it (ADR-0030 §11).
The first cut of advanced-mode SQL: a `select` line in advanced
mode parses, runs against the database, and renders its rows
through the existing data-table renderer; the same line in
simple mode lights up the precise "this is SQL" hint instead of
running.
Walker mode gate (ADR-0030 §2)
------------------------------
- `WalkContext` gains a `mode: Mode` field; `Mode` derives
`Default` (= `Simple`, matching the app's startup mode).
- `grammar::is_advanced_only` keys an advanced-only entry-word
set (Phase 1: just `select`). When the walker matches an
advanced-only entry word with `ctx.mode == Simple`, it
short-circuits to a `WalkOutcome::ValidationFailed` carrying
the `advanced_mode.sql_in_simple` catalog key — the input
highlights as a keyword, the validity indicator goes ERROR,
and the parse-error layer renders the "switch with `mode
advanced`, or prefix the line with `:`" hint.
- `parser::parse_command_with_schema_in_mode` (and the
schemaless `parse_command_in_mode`) threads the mode into
`WalkContext`; existing `parse_command*` entry points default
to `Mode::Advanced` (most permissive) so back-compat callers
see the full grammar.
- `App::submit` is unified: both modes route through
`dispatch_dsl(&effective_input, effective_mode)`, which now
parses with the line's effective mode. The placeholder
advanced-mode echo branch is gone.
Builder signature sweep (ADR-0031 §2)
-------------------------------------
- `CommandNode.ast_builder` gains a `source: &str` parameter,
forwarded by the walker. `build_select` reads it to put the
validated SQL text into `Command::Select`; the 21 existing
builders accept it as `_source`.
SQL `SELECT` (ADR-0030 §6, ADR-0031)
-------------------------------------
- New `Command::Select { sql: String }` variant. Every
exhaustive `match Command` updated (`verb`, `target_table`,
`build_translate_context`, `execute_command_typed`,
`typing_surface`'s label).
- `grammar::data::SELECT` `CommandNode`: projection (`*` or
`expr [as alias]` list), optional `FROM <table>`, optional
`WHERE`/`ORDER BY`/`LIMIT`, optional trailing `;`. The
expression slots reference the ADR-0031 fragment through
`Subgrammar(&sql_expr::SQL_OR_EXPR)`. The `FROM` table-name
slot carries a `reject_internal_table` validator that
refuses `__rdbms_*` references at parse time.
- The `FROM` clause is optional — `select 1`, `select upper('x')`
(zero-table constant/function-call SELECTs) work alongside
the single-table form. Standard SQL admits them and they are
the canonical learner probe.
- Implicit projection aliasing (`select a x`) is deliberately
unsupported — `from` is a keyword, the bare alias would be
ambiguous; only `select a as x` is admitted.
Worker / runtime
----------------
- `Request::RunSelect { sql, source, reply }` + a new
`Database::run_select` method. `do_run_select_request` runs
the prepared statement, collects rows into a `DataResult`
with `column_types: Vec<None>` (Phase-1 SELECT result columns
carry no playground type per ADR-0030 §6), and appends the
literal source line to `history.log` so replay re-runs it
(ADR-0030 §11).
- `runtime::execute_command_typed` gains a `Command::Select`
arm that calls `database.run_select(sql, src)` and maps to
`CommandOutcome::Query`, which flows into the existing
`AppEvent::DslDataSucceeded` → `render_data_table` path.
Catalog (ADR-0019)
------------------
- `advanced_mode.sql_in_simple` — the walker's gate message.
- `select.internal_table` — the `__rdbms_*` rejection.
- `parse.usage.select` — the parse-error usage template.
Tests
-----
Two `app::tests` cases that pinned the pre-ADR-0030 placeholder
echo are updated to pin the new dispatch contract — both verify
that the advanced-mode `select` (one persistent, one via the
`:` one-shot) produces `ExecuteDsl(Command::Select)` with the
submission's effective mode tagged on the echo. The matching
walking-skeleton test is updated likewise.
A separate follow-up commit lands the ambient mode-threading
(completion / live overlay / validity indicator) so simple-mode
users do not see SQL surfaced through Tab or the live error
overlay either — the dispatch-layer gate landed here is the
behavioural foundation that follow-up builds on. Integration
tests for the full end-to-end land in a third commit.
Completes ADR-0029's implementation: the friendly-error layer
now names the rule a CHECK violation broke, and the
typing-surface matrix covers the whole constraint grammar.
CHECK-violation friendly error (ADR-0029 §10):
- enrich_dsl_failure gains a CHECK branch — it reads the column
from the engine's `CHECK constraint failed: <column>`
message, then resolves the table, the offending value, and
the column's compiled CHECK expression.
- FailureContext / TranslateContext carry the resolved
check_rule; translate_check renders "the value <v> breaks the
rule `<rule>`" when it is known, falling back to the plain
hint otherwise.
Typing-surface matrix: a new `constraints` submodule, 14 cells
covering the create-table / add-column constraint suffix and
the add-constraint / drop-constraint commands (174 → 188).
16 tests added (1 translate unit, 1 enrichment integration, 14
matrix cells).
Adds the two commands for modifying a column's constraints after
creation, completing ADR-0029's §2.2 surface.
Grammar (dsl/grammar/ddl.rs): `add constraint <constraint> to
<T>.<col>` reuses the §2.1 COLUMN_CONSTRAINT choice; `drop
constraint <kind> from <T>.<col>` names only the kind. Both join
the `add` / `drop` choices, discriminated by the `constraint`
form word.
AST (dsl/command.rs): `Command::AddConstraint` / `DropConstraint`
plus the `Constraint` / `ConstraintKind` enums.
Worker (db.rs): `do_add_constraint` / `do_drop_constraint` apply
the change through the rebuild-table primitive. `add` runs the §5
dry-run first — `not null` / `unique` / `check` against a
populated column are refused, before any write, with a
pretty-printed table of offending rows. §9 redundant-on-PK
declarations and §6 `default` on an auto-generated column are
friendly refusals; dropping a constraint the column does not
carry is likewise refused.
Also fixes schema_to_ddl, which suppressed UNIQUE for every PK
column — a compound-PK member is not individually unique, so an
explicit UNIQUE on it must survive the rebuild.
23 tests added (6 grammar, 17 worker); 3 completion-test and 3
matrix snapshots updated for the new `constraint` subcommand.
The fourth constraint. `check ( <expr> )` reuses the ADR-0026
WHERE-expression grammar via `Subgrammar`, so a check is
written in the same language as a `where` filter.
- Grammar: a `CHECK_CONSTRAINT` arm joins the shared
constraint-suffix Choice; `consume_check_expr` extracts the
parenthesised expression (paren-depth aware) into
`ColumnSpec.check` / `Command::AddColumn.check`.
- Storage: the parsed `Expr` is compiled once to inline SQL
(`compile_check_sql` — `compile_expr` + ADR-0028's
param-inliner) and stored in that form everywhere — a new
`check_expr` column in `__rdbms_playground_columns`,
`project.yaml`'s `ColumnSchema.check`, and the column DDL
emitted by `do_create_table` / `schema_to_ddl`.
- `add column … check` routes through the rebuild primitive
(SQLite's `ALTER … ADD COLUMN` cannot carry it); a CHECK on
a serial/shortid column is create-table-only and refused at
add-column with a friendly message.
- `describe` surfaces the CHECK. ADR-0029 §7/§8 updated to the
SQL-form decision — double-quoted identifiers, consistent
with ADR-0028's `explain` display SQL.
1201 tests pass (+8); clippy clean.
The database layer now honours the ColumnSpec constraint
fields end to end, ahead of the grammar that lets users type
them.
- `do_create_table` emits ` NOT NULL` / ` UNIQUE` / ` DEFAULT
<literal>` per column via the new `column_constraints_sql`
helper (the default literal bound against the column's type).
- `ReadColumn` gains `default_sql`, read from
`pragma_table_info.dflt_value`; `schema_to_ddl` emits it, so
the rebuild-table primitive preserves DEFAULT — it already
preserved NOT NULL / UNIQUE.
- `ColumnDescription` gains `unique` / `default`;
`do_describe_table` now sources columns from `read_schema`
(one source of per-column truth) and `constraints_display`
lists PK / NOT NULL / UNIQUE / DEFAULT.
No user-facing change yet — no grammar produces constrained
columns. Tests exercise creation, enforcement, describe, and
rebuild-preservation programmatically.
1177 tests pass (+5); clippy clean.
Expand ColumnSpec and Command::AddColumn with the four
ADR-0029 constraint slots (not_null, unique, default, check),
all defaulting off; `Database::add_column` now takes a
ColumnSpec. No behaviour change — the grammar to set the
fields and the DDL to enforce them land in the following
commits. Isolated here so those commits stay readable.
Adds ColumnSpec::new for the unconstrained case; 110 call
sites updated. 1172 tests pass; clippy clean.
13 matrix cells for the `explain` prefix across all three
wrapped commands — `explain show data` / `explain update` /
`explain delete` — covering each typing position (after the
prefix, the inner entry word, the table, the filter clause)
plus the three complete forms. The cells confirm `explain`
plugs into the inner query grammars cleanly: candidates, hints
and column scoping match the standalone commands, and the
complete forms parse as `Command::Explain`.
Also adds a worker test pinning the display SQL's `<>`
rendering of inequality (ADR-0028 §3).
Matrix: 161 -> 174 cells. 1172 tests pass; clippy clean.
Add the `explain` prefix command — `explain show data`,
`explain update`, `explain delete` — from grammar through to a
rendered plan tree.
- Grammar: an `EXPLAIN` CommandNode whose shape is a Choice over
the three explainable query shapes, referenced (not
duplicated) through `Subgrammar`. `Command::Explain { query:
Box<Self> }`; `build_show_data` is extracted so the role-based
builders serve both standalone and explain-wrapped commands.
- Worker: SQL construction is split out of do_query_data /
do_update / do_delete into `build_*_sql`, so EXPLAIN QUERY
PLAN runs the exact same statement. `Request::ExplainPlan` /
`do_explain_plan` capture the plan; `QueryPlan` / `ExplainRow`
carry it back. EXPLAIN QUERY PLAN never executes, so
explaining update/delete changes nothing.
- Display SQL: the executed statement with `?N` parameters
inlined as standard-SQL literals via a quote-aware scan.
- Render: `render_explain_plan` draws the box-drawing plan tree
(plain output; ADR-0028 step 4 adds the styled tree).
- Catalog: `parse.usage.explain` and the `help.data.explain`
entry, so `explain` shows up in the in-app `help` listing.
1151 tests pass (+18); clippy clean.
ambient_hint now reads the walker's schema-aware diagnostics.
input_diagnostics is non-empty only for a command that
structurally parses — so a non-empty result means "complete
and submittable, but wrong or dubious". That is checked early
(right after the Tab-cycle memo), ahead of slot hints and
completions: a command that parses but is flawed no longer
gets the misleading "Submit with Enter" prose, it gets the
diagnostic's why. pick_hint_diagnostic prefers the diagnostic
under the cursor, else the most severe.
The cursor-local invalid-ident hint is kept for genuinely
incomplete commands (no Match → no diagnostics).
5 ambient_hint tests (unknown table, type-mismatch over
submit-prose, LIKE-numeric, clean command still submittable,
cursor-following). The complex_and_or matrix cell referenced a
non-existent column `t`; fixed to a real column so it tests a
valid expression as intended. 1118 passing, clippy clean.
Adds the `[ERR]` / `[WRN]` validity indicator to the input
row. `App` gains `input_indicator: Option<Severity>` (the
runtime owns its timing — step E) and a pure
`input_validity_verdict()` query that runs `input_verdict`
in simple mode only (advanced mode is raw SQL, ADR-0027 §7).
`render_input_panel` reserves the rightmost six columns of
the input row unconditionally (ADR-0027 §4) — a five-column
label plus a one-column gap — so the typed command never
shifts sideways when the indicator appears or hides. The
label renders only when `input_indicator` is set: `[ERR]` in
`theme.error`, `[WRN]` in the new amber `theme.warning`
(defined for both light and dark themes).
The indicator is not yet wired live — `input_indicator`
stays `None` until the debounce lands (step E). Covered by a
render test and the theme contrast test; the input-panel
snapshot is updated for the six-column reservation.
Adds tests/typing_surface/where_expression.rs — 9 matrix
cells for the complex WHERE / show-data limit typing surface:
operator candidates after an operand, AND / OR after a
predicate, NOT, BETWEEN / IN bounds, and `show data`
where / limit.
Writing the cells surfaced a grammar bug. `predicate_tail`'s
`[NOT] negatable` branch started with `Optional(not)`, and an
Optional-first `Seq` always "commits" — so on an incomplete
input the walker's `Choice` returned that branch's
`Incomplete` early and discarded every sibling branch's
expected set, dropping `is` and the comparison operators from
completion after a column. Fixed by splitting it into
explicit `NOT negatable` and bare `negatable` branches — no
`predicate_tail` branch starts with an `Optional` now. The
matched terminal sequence is unchanged, so `build_expr` is
untouched.
Docs: ADR-0026 gains an "As-built notes" section recording
the option-1 builder realization, its two deviations from the
§3 sketch, and the deferral of §7 diagnostic flagging to
ADR-0027. requirements.md C5a -> [x] (steps 1-4) with the
test baseline refreshed to 1079; CLAUDE.md's deferred list
reconciled (C5a implemented; the QA1/QA2 note now points at
ADR-0028).
Wires the stratified WHERE-expression fragment into the three
filter commands and compiles the resulting Expr to SQL.
Grammar (data.rs): the `update` / `delete` `where` clause is
now the expression fragment (`Subgrammar(&expr::OR_EXPR)`) in
place of the single `col = val` slot; `show data` gains an
optional `where <expr>` and an optional `limit <n>` (a
non-negative integer, validated at parse time). The
expression's right-hand operands are a schema-aware
`DynamicSubgrammar` so the hint panel still narrows to the
left column's type (ADR-0026 §8) — but the inner grammar is
permissive: a type-mismatched literal still parses (§7).
AST: `RowFilter::Where{column,value}` -> `RowFilter::Where(Expr)`;
`ShowData` gains `filter: Option<Expr>` and `limit: Option<u64>`.
A `RowFilter::eq` convenience constructor keeps simple-equality
call sites and tests readable.
SQL (db.rs): `compile_expr` lowers an `Expr` to a
parameterised WHERE — every literal a `?` placeholder,
identifiers `quote_ident`-quoted, `<>` for inequality. A
literal compared against a column binds through that column's
type where compatible and falls back to its syntactic shape on
a mismatch (§7 — permissive). `show data ... limit n` emits
`LIMIT ?` with an implicit primary-key `ORDER BY`, so it is a
stable "first n by primary key".
completion.rs: `invalid_ident_at_cursor` no longer mis-flags a
digit-led literal (`1`) as an unknown column now that the
WHERE operand slot also accepts a column reference; a
`ProseOnly` slot suppresses keyword candidates even when the
expected set also carries a column ident.
11 db integration tests cover AND / OR / NOT, BETWEEN, IN,
LIKE, filtered `show data`, and limit ordering; walker and
expr unit tests cover the parse surface. Type-mismatch /
`= NULL` diagnostic flagging (§7 highlight + hint) is the
remaining ADR-0026 piece.
`create table … with pk` parsed column types as `name:type`,
while `add column` uses `name(type)`. Unify on the parens
form so column-type syntax is consistent across the DSL:
create table T with pk id(serial), name(text)
Only `COL_SPEC` changes (`:` → `( … )`); `build_create_table`
reads columns by role, so it is unaffected. The `:` that
separates table from column in `add column` / `drop column`
is unchanged. Sweeps the test suite, the typing-surface
matrix (two `after_colon` cells renamed to `after_paren`,
4 snapshots regenerated), the friendly catalog's usage
templates, ADR-0009's example, and requirements.md.
1039 passing / 0 failing / 1 ignored; clippy clean.
Implement ADR-0025 — indexes as a DSL DDL feature.
- Grammar: `add index [as <name>] on <T> (<cols>)`, `drop index
<name>` / `drop index on <T> (<cols>)`, plus a `--cascade`
flag on `drop column`.
- db.rs: index operations over the engine's native index
catalog (no metadata table). The rebuild-table primitive now
captures and recreates indexes, so `change column` and the
relationship operations no longer silently drop them.
- `drop column` refuses an indexed column unless `--cascade`,
which drops the covering indexes and reports each.
- Persistence: additive `indexes:` list in `project.yaml`
(version unchanged); round-trips through rebuild/export/import.
- Display: an `Indexes:` section in the structure view and a
nested tables/indexes items panel (S2).
Reconciles requirements.md (C3 index portion, S2 satisfied)
and CLAUDE.md. 1038 tests passing (+31), clippy clean.
run_replay parsed each line with the schemaless parse_command, so
Phase D typed-slot rejections (wrong-count value lists, wrong-type
column values) fired only at bind time during replay — inconsistent
with the interactive path (handoff-12 §2.1).
run_replay now re-snapshots the schema per line (the schema mutates
as replayed create-table / add-column commands run) and parses with
parse_command_with_schema. Extracted build_schema_cache, shared with
the interactive refresh_schema_cache.
Added a replay integration test asserting a typed-slot violation is
caught at parse time (through the replay.error_parse wrapper).
Form C (`insert into T (vals)`) shared the `(` opener with Form A,
so its paren was an untyped Repeated(Choice(literal, ident)) — values
weren't type- or count-checked at parse time (handoff-12 §2.2).
New Node::Lookahead variant: a factory that peeks the source. The
insert first-paren factory inspects the first token — a value literal
routes the contents through the typed column_value_list (Form B
dispatch contract: per-non-auto-column typed slots); an identifier or
empty paren routes to a Form A column-name list. So Form C now gets
the same per-column typed slots, hints, and parse-time type/count
checking Form B has.
The explicit-Choice-branch split is impossible here (committed-choice
semantics commit after `(` matches); lookahead is the only route, and
DynamicSubgrammar factories couldn't see the source. Node::Lookahead
is not memoized — its output depends on source — but it returns only
a small node (a Repeated, or a thin DynamicSubgrammar wrapper that
delegates to the memoized column_value_list).
`insert into T (` now cleanly shows Form A column candidates instead
of mixed Form-A/C suggestions. Form C matrix tests updated for the
type-aware behaviour.
Handoff-12 §2.2: Form B `insert into T values (…)` silently skips
auto-generated columns from the value list, so a user who wants to
set a serial/shortid column explicitly could only discover Form A by
reading help. Now the hint at the first Form B value slot appends a
note naming the skipped column(s) and pointing at the explicit-column
form.
hint_resolution_at_input derives the skipped columns from the
post-walk WalkContext (Form B = no user_listed_columns + table has
serial/shortid columns) and reports them on HintResolution; the note
fires only at the first slot so it doesn't repeat at every comma.
ambient_hint composes it onto the per-column prose.
8 tests covering completion-candidate order: connective keywords in
reading order (`to`/`from`/`in` before `table`), and command-part
keywords before schema identifiers. The ordering already held via
declaration-order preservation + keywords-first sectioning in
candidates_at_cursor; nothing pinned it until now, so a future
grammar or sort change could silently break the hint panel's
left-to-right reading.
55 tests covering create table, drop column, drop relationship
(endpoints + by-name), add relationship, rename/change column, and
all app-lifecycle commands. The drop-column and relationship tests
drove the §2.2 writes_table fix in the previous commit.
Documents one UX wrinkle as a flagged finding: partial entry words
(`qu`) classify as DefiniteErrorAt — same as unknown commands —
because the walker only engages on a complete entry word.
859 baseline -> 989 passing; 1 ignored (pre-existing doc-test).
34 new tests covering:
- Form C bare-value-list (happy path + Form-A-recovery + type-unaware grammar limitation per handoff §2.2)
- update with WHERE (column-narrowing invariant per handoff §1 bug E1; typed-slot prose for assignments and where filters)
- update --all-rows (filter-clause requirement per ADR-0014)
- delete with WHERE (column-narrowing; typed-slot prose for where filters)
- delete --all-rows
859 baseline -> 931 passing. No bugs surfaced — the data-mutation
command family was already well-shaped post-Phase-D.
12 tests across schema_serial_pk / text_pk / multi_table / every_type.
Pins (a) Form B skips auto-gen columns from the slot list (regression
for handoff-12 §B fix); (b) wrong-count value lists are now flagged
at typing time, not only at submit (the previous commit's fix); and
(c) per-type slot prose advances correctly through every Type variant.
Per docs/handoff/20260515-handoff-12.md §1. Systematic per-position
coverage of (state, hint, completion, parse_result) across canonical
schema shapes; submodule per command family. Insert Form A covers 23
cursor positions across serial-PK, text-PK, multi-table, and
every-Type schemas. Both bugs fixed in the previous commit were
surfaced by these tests.
Shared helpers under tests/typing_surface/mod.rs: 5 canonical schema
shapes, assess() helper, property-assertion shortcuts, and a snap!
macro that wraps insta with a stable per-cell suffix.
859 -> 885 tests passing; 1 ignored (pre-existing doc-test).
New `dsl::usage` module: registry pairing each command's
entry-keyword with a `parse.usage.*` catalog key.
`matched_entry()` resolves the entry keyword from the
consumed token prefix; multi-entry families (add, drop,
show) return all matching keys.
Catalog: new `parse.usage.<command>` keys (one per command),
`parse.token.{keyword,punct,...}` vocabulary (one per
Keyword/Punct variant + token-class labels + LexError
kinds), and `parse.available_commands` for the no-prefix
fallback. Catalog grows ~60 entries.
Validator: extended KEYS_AND_PLACEHOLDERS; new completeness
test asserts every Keyword and Punct variant has its
`parse.token.*` entry.
`app::dispatch_dsl` rewritten to compose three blocks per
ADR-0021 §2: caret + structural/custom error + usage block
(or available-commands fallback per §5). Caret math fixed
to use original-input byte position rather than
trimmed-input position (the lexer no longer trims before
lexing). Three pre-existing app tests adjusted to look
across all error lines instead of `output.back()` (the
usage block is now the last line).
`dsl::usage::matched_entry` uses `<=` rather than `<` for
position comparison so custom errors raised by `try_map`
(whose span starts at the first consumed token) still
resolve to the entry keyword.
Tests: 668 passing, 0 failing, 1 ignored (650 baseline →
+18: 8 usage + 1 token-vocab completeness + 9 new
integration tests in tests/parse_error_pedagogy.rs
covering create/add/drop/show/frobulate/update/insert
cases). Clippy clean.
Final pass of the i18n migration sweep. Every user-visible
string in `src/` now flows through the catalog via `t!()`.
## Categories migrated in this commit
- **help.cli_banner** — the entire `cli::HELP_TEXT` const,
formerly a 40-line `&'static str`, is now a YAML block in
the catalog. The const is replaced by a thin
`cli::help_text() -> String` wrapper that performs the
catalog lookup. `main.rs` calls `help_text()` for both
`--help` output and the args-parse error path. The two
integration tests that referenced `HELP_TEXT` directly are
updated.
- **help.in_app_body** — the in-app `help` command's body is
one YAML block; `note_help` becomes 5 lines that iterate
the lines and emit each as its own output row (preserving
the renderer's "one logical line = one display row"
invariant for accurate scroll math).
- **modal.*** — load picker, rebuild confirm, and save-as
path-entry strings: rebuild_cancelled, load_cancelled,
generic_cancelled, load_picker_nothing,
path_entry_empty_name, path_entry_empty_path.
- **dsl.failed** — the `"<verb> <subject>" failed: <rendered>`
wrapper around the friendly-error layer's translated
message.
- **dsl.running** — the `running: <input>` echo line shown
above each command's response. (Note: the en-US prefix
"running: " is hardcoded in the parse-error caret-padding
calculation. Translators changing the prefix must keep the
width consistent — documented inline.)
- **advanced_mode.not_implemented** — the placeholder echo
shown when SQL hits the unimplemented advanced-mode path
(Q1 territory).
- **fatal.persistence** — the FATAL banner for
PersistenceFatal events (ADR-0015 §8).
- **project.{load_path_missing,saveas_target_exists,**
**import_zip_missing}** — runtime-side project-switch
validation errors that surface via ProjectSwitchFailed.
## Catalog start-up ordering
`main.rs` now calls `friendly::catalog()` at the very top
(before args parsing) so `help_text()` works in both the
success path and the args-error path. A corrupted build
artefact still fails loudly with a useful panic; the
practical risk is essentially zero since the catalog is
`include_str!`'d at compile time and validated by the unit
test before shipping.
## Remaining literals
The only `note_*` calls in `src/` that still pass plain
strings are inside `#[cfg(test)]` modules — synthetic test
fixtures, not user-visible. The codebase passes the "every
user-visible string flows through the catalog" bar.
## Tally
610 tests passing (no change in count — pure refactor).
Clippy clean with nursery lints.
## What this closes
ADR-0019 §9 (migration sweep) — done.
ADR-0019 itself is now fully implemented:
- §1-§5: catalog + translator + voice + verbosity ✓ (`eac7e5b`)
- §6: row pinpointing + schema enrichment ✓ (`431645a`)
- §9: migration sweep ✓ (this + `aff528a`)
- §10: anchor phrases preserved throughout ✓
- The five "Out of scope" items remain explicitly bounded
to future ADRs (advanced-mode SQL, settings persistence,
pluralisation, runtime locale, value formatting,
constraint management).
Closes the placeholder-substitution gap reported during manual
testing: FK violations were rendering `<value>` and `<column>`
literally because the App had no schema awareness. With this
change the runtime resolves the schema-dependent facts before
the App ever sees the failure.
## Architecture
- **Database** gains two public methods backed by new worker
Request variants:
- `read_relationships(table)` → (outbound, inbound) FK list
(lifts the previously-private `read_relationships_*` pair
into the public surface, behind a `RelationshipsReply`
type alias).
- `find_rows_matching(table, column, value, limit)` →
`DataResult` for row pinpoint queries.
- **friendly module** gets:
- New `FailureContext` struct: schema-resolved facts the
runtime builds (table, column, value, parent_table,
parent_column, child_table, optional diagnostic_table).
- `TranslateContext` loses its lifetime parameter and gains
`parent_table` / `parent_column` fields. All string fields
are now `Option<String>` for ownership simplicity.
- `TranslateContext::from_facts(operation, verbosity, facts)`
helper.
- Translator's FK paths now use `ctx.parent_table` /
`ctx.parent_column` for child-side wording; FK Update gets
a dedicated `fk_child_side_update` arm.
- FK dispatch is enrichment-driven first
(`parent_table` set → child-side; `child_table` set →
parent-side), with operation as the tiebreaker.
- The translator forwards `ctx.diagnostic_table` onto the
`FriendlyError` so pinpointed rows render through the
existing ADR-0017 §7 bordered renderer.
- **Event** `DslFailed` carries `(command, error, facts)`.
The runtime populates `facts` via `enrich_dsl_failure`
before posting the event.
- **Runtime** `enrich_dsl_failure(database, command, error)`
classifies and resolves:
- UNIQUE INSERT/UPDATE: parses `T.col` from engine message,
finds the user's attempted value (with schema fallback
for natural-order multi-value INSERT — including the
serial/shortid auto-skip rule from `do_insert`), pinpoints
the existing conflicting row(s) via `find_rows_matching`
and renders as a `DiagnosticTable`.
- NOT NULL INSERT/UPDATE: parses `T.col`; no value
(definitionally null) and no pinpoint (engine doesn't
identify the row).
- FK INSERT/UPDATE: outbound relationship lookup picks the
FK column the user is touching; resolves
`parent_table`/`parent_column`/`value`. UPDATE falls back
to inbound (parent-side) when no outbound match.
- FK DELETE: inbound relationship lookup picks a child_table
that references this row.
- **App** drops its old `attempted_value_for` /
`column_from_qualified_target` helpers (their work moved to
runtime where the Database is in scope).
`build_translate_context` combines the runtime-supplied
facts with the operation derived from the Command and the
App's verbosity.
## Manual-test fixes folded in
Two issues surfaced during manual testing of the initial
implementation, both fixed:
1. Natural-order multi-value INSERT
(`insert into Orders values (4, 11.99)`) skipped FK
enrichment because `user_value_for_column` only knew the
single-value short form. The schema-aware lookup
(`user_value_for_column_with_schema`) now mirrors
`do_insert`'s position-mapping rule (auto-generated
columns skipped), so positional INSERTs onto tables with
serial/shortid PKs resolve correctly. Regression test:
`enrich_fk_insert_natural_order_multi_value_resolves_via_schema`.
2. The arity error on INSERT now lists the columns it
expected — `expected 3 value(s) for (id, Name, Email), got 2`
instead of the bare count. Surfaces what the user needs
to fix without making them go check the schema.
## Tests
`tests/friendly_enrichment.rs` (+8 integration tests):
- UNIQUE INSERT with explicit columns: facts.{table, column,
value, diagnostic_table} all resolved; pinpoint shows
conflicting row.
- UNIQUE INSERT natural-order short form: schema fallback
resolves the value.
- UNIQUE UPDATE: value pulled from assignments.
- NOT NULL INSERT: table+column resolved, value None
(correct), no pinpoint.
- FK INSERT: parent_table, parent_column, value all resolved
via outbound relationship lookup.
- FK INSERT natural-order multi-value: schema-aware lookup
with auto-skip resolves correctly (regression for the
manual-test bug).
- FK DELETE: child_table resolved via inbound relationship
lookup.
- DbError::Unsupported: enrichment returns default
FailureContext (no false positives).
App-level tests updated to populate `FailureContext` directly
(simulating runtime enrichment) for the verbosity / threading
checks.
## Tally
610 tests passing (was 603: +8 enrichment integration tests
minus 1 obsolete App-side helper test that the runtime
absorbed). Clippy clean with nursery lints. Release builds.
All eight implementation steps from ADR-0019's §"Order of
operations":
Step 1 — `src/friendly/` module skeleton; `t!()` macro; YAML
catalog loader (`include_str!` + `serde_yml`); `{name}`
substitution helper that rejects format specifiers per §8.4.
Step 2 — `error.*` catalog populated for UNIQUE / FK /
NOT NULL / CHECK / type-mismatch / not_found / already_exists /
generic / invalid_value, with verbose hints per
pedagogical-voice rule (§5). Anchor phrases (§10) preserved
verbatim.
Step 3 — `FriendlyError { headline, hint, diagnostic_table }`
+ renderer composing the three blocks per §7.
Step 4 — `translate(&DbError, &TranslateContext) → FriendlyError`.
Classifies by `SqliteErrorKind` first, then by message text
for the constraint family. `change column` failures route to
the type-mismatch headline, subsuming the previous
`friendly_change_column_engine_error` helper.
Step 5 — `DbError::friendly_message()` delegates to the
translator with default context. Removed
`friendly_change_column_engine_error` (absorbed) and
`enrich_fk_message` (FK list moves to the deferred re-query
step). One test rewritten to assert on the engine-classified
payload rather than the removed enrichment text.
Step 6 — `messages (short|verbose)` app-level command parallel
to `mode`. `App::messages_verbosity` (default verbose)
threaded into `TranslateContext` via
`App::build_translate_context`. `AppEvent::DslFailed` now
carries the structured `DbError`, plus the App extracts the
user's attempted value from `Command::Insert` / `Update`
to fill the `{value}` placeholder for UNIQUE / NOT NULL.
Step 7 — Catalog validator (§8.6) checks for missing keys,
unused/undeclared placeholders, format specifiers, and
forbidden engine vocabulary. `main.rs` parses the embedded
catalog at startup so a corrupted build artefact fails
loudly there rather than at the first `t!()` call.
Step 8 — Anchor phrases (§10) held: existing tests asserting
on "no such table", "already exists", "cannot be converted",
etc. all pass without rewording.
## Tally
603 tests passing (was 561: +42 net). Clippy clean with
nursery lints. Release binary 7.7 MB.
## Deliberately deferred
- Schema-aware enrichment for FK violations (parent_table /
parent_column / child_table) and the multi-value
natural-order INSERT case for UNIQUE. Both need the
Database handle in scope at translation time, so they
bundle naturally with the row-pinpoint re-query work
(ADR-0019 §6) — that follow-on adds runtime-side
enrichment via a `Database` lookup and a structured
failure-context carried on `DslFailed`. Until then,
unfilled placeholders render as their `{name}` form for
visual consistency with the catalog.
- Migration sweep (§9). Only `error.*` is catalog-driven so
far; `help.*`, `ok.*`, `client_side.*`, `replay.*`,
`parse.*`, modal labels, etc. migrate per-PR.
- Settings persistence for `messages`. In-session state for
now; waits on the future settings ADR.
Implements the U4 replay command per handoff §A3:
replay <path>
Reads <path> and dispatches each non-blank, non-`#`-comment
line through the same DSL pipeline as interactive input.
Aborts at the first per-line failure (parse or runtime),
reporting the line number; previously dispatched commands
stay applied (no rollback) — matches the "I'm replaying my
history" mental model where partial replay is a recoverable
state.
Architecture choices and why:
- **Parsed by the DSL parser** (Command::Replay), not as an
app-level command alongside `import` / `export`. The
handoff's implementation sketch was explicit and the
parsed-AST shape gives us a clean test surface for the
path-lexing rules. A new `path_literal` parser terminal
accepts either a single-quoted string (escape rules
mirror `string_literal` — `''` for a literal quote) or a
bare run of non-whitespace, with explicit refusal of `'`,
`(`, `)`, `;` in bare form. Empty paths fail at parse
time so file-system-layer errors aren't shadowed by
silly inputs.
- **Routed away from the worker thread.** Command::Replay
is intercepted in `App::dispatch_dsl` and emitted as
`Action::Replay` rather than `Action::ExecuteDsl`. Two
reasons: (1) the worker has no filesystem context, and
(2) the replay invocation must NOT land in
`history.log` — otherwise `replay history.log` would
re-trigger itself recursively. Only the individual
sub-commands write to history.log via the normal
per-command persistence path.
- **Inner loop separated from spawn.** `runtime::spawn_replay`
is a thin tokio::spawn wrapper around `runtime::run_replay`,
which is `pub` and returns a Vec<AppEvent>. The inner
function is what tests exercise, sidestepping mpsc plumbing.
- **Relative paths resolve under the project root** so
`replay history.log` works without ceremony from inside
any project. Absolute paths pass through unchanged.
- **Nested `replay` is refused.** Allowing `replay foo` from
inside a replay file invites infinite-loop footguns and
opens design questions (transitive composition, ordering)
we'd rather not answer right now. Refusal is explicit.
New plumbing:
- `Command::Replay { path }` AST variant + verb/target_table.
- `Action::Replay { path }` runtime action.
- `AppEvent::ReplayCompleted { path, count }` and
`AppEvent::ReplayFailed { path, line_number, command, error }`.
- `runtime::run_replay` (public) and `runtime::spawn_replay`.
- App handlers render success as
`[ok] replay <path> — N command(s) run` and failures as
`replay <path> failed at line N: <error>` with a
` > <command>` echo line for line context. Line 0 is the
"file open failed" signal — header reads
`replay <path> failed: <error>` and the echo line is
suppressed.
- In-app `help` lists the new command with a continuation
describing comment/blank handling and the relative-path
rule.
Tests (+20):
- 7 parser tests covering bare/quoted/escaped paths,
case-insensitive keyword, and refusal cases (no path,
empty quoted path).
- 9 integration tests in `tests/replay_command.rs`:
- happy 3-line replay → 3 commands run, state mutated;
- blank lines + `#` comments skipped;
- empty file + only-comments file → count 0;
- missing file → ReplayFailed line_number 0;
- parse failure mid-replay → reports correct line +
leaves earlier commands applied + does NOT run later
lines;
- runtime failure mid-replay (refers to nonexistent
table) → reports correct line;
- nested replay refused;
- history.log contains per-command entries but NOT the
`replay …` invocation itself.
- 4 App-level tests: Action::Replay dispatch (not
ExecuteDsl); ReplayCompleted rendering; ReplayFailed
rendering with and without line-number context.
541 -> 561 passing, clippy clean with nursery lints,
release build successful.
A future ADR on the parser-as-source-of-truth direction
(handoff §"Pending §3") would bring richer error reporting
for replay parse failures (currently uses the same
single-line wording as interactive parse failures, which is
adequate but not great when a script has many lines around
the failing one).
Verifies the user-facing posture in ADR-0002 §"User-facing
posture": no occurrence of SQLite, sqlite, rusqlite, STRICT,
or PRAGMA may appear in any user-reachable string.
The audit's mechanical sweep of `src/` confirmed the
codebase already conforms — every appearance of those
tokens is in either:
- code comments / module-level docstrings (allowed by
ADR-0002 explicitly),
- DDL strings sent to the engine (not displayed to the
user),
- internal field/function names like `sqlite_type` /
`sqlite_strict_type` (code identifiers, not user-visible).
The previous session removed the last known leak in
`do_add_column`. To stop a future change from quietly
re-introducing one, this commit adds a regression test
file covering a representative set of user surfaces:
- `cli::HELP_TEXT` (`--help` banner).
- The in-app `help` command output.
- DSL parse errors for a battery of failing inputs
(column-name-first typo, unknown type token, mutually
exclusive flags, missing clause, garbage).
- `DbError::friendly_message()` for realistic Sqlite,
Unsupported, and InvalidValue payloads — the surface the
runtime forwards via `AppEvent::DslFailed`.
The forbidden-token list lives in one place
(`engine_vocabulary_audit.rs::FORBIDDEN`) so future audits
can extend it. Failure messages name the leaking token and
its byte offset so a regressing edit pinpoints itself.
Out of scope (and called out in the handoff for separate
work): the H1 friendly-error layer that translates the
remaining engine error wording into pedagogical English.
That needs its own ADR. The local
`friendly_change_column_engine_error` wrapper (db.rs §2354)
is the prototype.
537 -> 541 passing (4 new), clippy clean.
ADR-0017 added --force-conversion / --dont-convert as
opt-in flags on `change column`; the help text already
mentioned the flags but didn't explain when they apply.
ADR-0018 generalised serial beyond PK and added auto-fill
on `add column ... (serial|shortid)` for non-empty tables;
none of that was reflected in user-visible help.
This commit:
- Annotates the `add column` line with a continuation note
that adding serial/shortid to a non-empty table
auto-fills existing rows.
- Annotates the `change column` line with a continuation
note that converting to serial/shortid auto-fills null
cells.
- Appends an "Auto-generated types" section explaining
serial and shortid: how they auto-fill, that they imply
UNIQUE outside a PK (serial) or always (shortid), and
that adding/converting-to either type on a non-empty
table auto-fills existing/null cells.
The new test
`help_describes_auto_generated_type_behaviour` pins these
phrases so a future help-text edit can't silently drop the
pedagogical lines. The existing
`help_command_lists_supported_commands` and
`help_lists_export_and_import` tests still pass — they
only assert substring presence.
No engine vocabulary leaks (ADR-0002 posture preserved).
536 -> 537 passing, clippy clean.
Replaces the placeholder pipe-and-dash output with Unicode
box-drawing tables for both data results and table-structure
listings, per ADR-0016.
* New `src/output_render.rs` module with `render_data_table`
and `render_structure`. Hand-rolled to match the project's
existing CSV/YAML pattern; ~300 lines.
* Header-only outer-frame border style: outer ┌─┐│└─┘ box +
├─┤ header underline, no per-row separators. NULL renders
as `(null)`; cell newlines/tabs/control chars become
`↵`/`→`/`·` as display-only substitutions.
* Type-aware column alignment: numeric types right-aligned,
everything else left. `DataResult` gains a `column_types:
Vec<Option<Type>>` field, populated from the existing
metadata lookup at the two query sites in db.rs (no new
query paths).
* Structure view shows Name | Type | Constraints columns;
References / Referenced-by sections retain plain-text
format, leaving room for the future relationship-rendering
ADR.
* 18 new unit tests in output_render.rs (plus 4 insta
snapshots for the canonical layouts). Existing assertions
in app.rs and walking_skeleton.rs updated to match the new
format.
Total: 426 passing, 0 failing, 0 skipped (up from 408).
Clippy clean.
Closes out track 2's ADR-0015 backlog.
* `--resume` CLI flag (L1a, ADR-0015 §7) opens the most-
recently-used project, tracked in <data-root>/last_project.
Mutually exclusive with a positional <project-path>; errors
cleanly to stderr (above the shell prompt) on missing file
or stale recorded path. last_project is rewritten on every
successful project open (startup, load, new, save as,
import).
* Persistent input history (I2-persist, ADR-0015 §12). On
project open, the in-memory navigable history is hydrated
from the tail of history.log (capped at the in-memory cap).
ProjectSwitched gains a `history_entries` payload field;
App::seed_history is the entry point. Pipes inside source
text round-trip via splitn(3); unknown escape sequences are
passed through literally.
* Migration framework scaffold (F3, ADR-0015 §9). New
persistence::migrations module with MigratorRegistry +
migrate_to_latest + ensure_project_yaml_migrated. Empty
in v1 (production registry has no migrators); the loader
runs through it on every project open and is exercised by
tests with a fake v1→v2 migrator. Writes
project.yaml.v<N>.bak before any migrator runs; verifies
each step bumps the version field.
Refreshes docs/requirements.md (A1 / I2 / F3 / E1 / L1a /
test baseline) and adds docs/handoff/20260508-handoff-3.md
covering both Iter 5 and Iter 6.
Total tests: 408 passing, 0 failing, 0 skipped (up from 345
at handoff-2). Clippy clean.
Implements the `export` and `import` app-level commands per
ADR-0015 §11 + ADR-0007 amendment 1.
- `export [<path>]` writes a zip of project.yaml + data/ to
<data-root>/YYYYMMDD-<projectname>-export-NN.zip by default,
preserving the project's directory name as the single
top-level folder inside the archive.
- `import <zip> [as <target>]` extracts an exported zip into
a new named project and switches to it. Target name is
derived from the zip's top-level folder by default; on
collision the destination auto-suffixes -02, -03, ... up
to -99 instead of refusing (deviates from §2's refuse-on-
collision rule for save/save as; recorded as an amendment
to ADR-0015 §11).
- Excludes playground.db and history.log from the zip.
- Path-traversal protection via zip::enclosed_name + post-
resolution check that the extraction path stays inside
the target directory.
Adds the zip = "5" dep with default-features = false +
features = ["deflate"] to keep the binary-size cost modest.
Test baseline: 370 passing, 0 failing, 0 skipped.
The previous remove_dir_all on a path returned by Project::path()
was too trusting: an unusual CLI argument or a hand-edited
project.yaml could in principle have steered cleanup into
deleting the wrong directory. Replace it with
safely_delete_temp_project, which refuses unless every one of
the following passes:
1. Path is not a symlink (checked before canonicalize so a
symlink can't smuggle a different target through).
2. Path is a directory.
3. Canonical path is under <active-data-root>/projects/
(canonical-prefix containment).
4. Directory basename contains the literal `[temp]` marker.
5. Direct children are exclusively well-known project
artefacts (project.yaml, data/, history.log,
playground.db, .gitignore, lock file) plus migration .bak
files and atomic-write .tmp files. Any stranger file
(notes.md, .git/, screenshots, etc.) makes the helper
refuse.
is_unmodified_temp now also requires data/ to be empty, in
addition to project.yaml's tables and relationships being
empty. A hand-edited yaml that drops the schema list but
leaves CSVs in data/ no longer passes.
Failure to delete is non-fatal -- the helper returns
SafeDeleteError, the runtime logs a tracing::warn!, and the
project stays on disk. Leaving an unexpected directory alone
is always preferable to a wrong delete.
Tests: 345 passing (272 lib + 9 + 5 + 6 + 27 + 9 + 17),
0 failing, 0 skipped. 7 new tests covering each guard,
including a unix-only symlink-rejection test.
Four post-Iteration-4 polish items surfaced by manual testing.
1. `--help` / `-h` CLI flag prints a usage banner (options +
app-level commands + DSL grammar reference) and exits. Parse
errors also print the banner to stderr.
2. `help` app-level command notes the same list of supported
commands to the output panel -- a simple stand-in for the
richer H3 help system, kept in sync with what's actually
wired up.
3. The silent rebuild that runs when playground.db is missing
now surfaces a system message in the output panel ("[ok]
rebuild -- N tables, M rows reconstructed; ...") via a new
initial_events plumbing. The user no longer wonders whether
the .db was magically restored or whether anything happened
on launch.
4. Unmodified empty temp projects (kind=Temp, project.yaml has
tables: [] and relationships: []) are now auto-deleted when
the user switches away (load / new / save as) or quits. This
addresses the "launch app, load existing project, quit"
pattern that was leaving an empty temp directory behind
every time. Modified temps (with any user-created tables or
relationships) are never auto-deleted; corrupted projects
are also never auto-deleted (defensive default-to-false on
yaml read/parse errors).
Tests: 338 passing (272 lib + 9 + 5 + 6 + 20 + 9 + 17),
0 failing, 0 skipped. Clippy clean.
Adds the rest of the track-2 lifecycle commands (ADR-0015 §11)
and the project-switching machinery they need at runtime.
Temp vs named distinction: replaced the fragile naming heuristic
with an explicit `[temp]` marker in the directory pattern
(`<YYYYMMDD>-[temp]-<word>-<word>-<word>`). validate_user_name
already rejects brackets, so user-typed names can never collide
with a temp marker. The status bar shows `[TEMP] <Display Name>`
for temp projects; the prettifier strips both the date and the
marker so display names are clean.
save / save as: temp project's `save` opens a path-entry modal
(acts as save as); named project's `save` reports "already
auto-saved; use `save as`". `save as` always prompts. Relative
names resolve under <data-root>/projects/; absolute paths used
as-is. Copy excludes the per-process lock file; everything else
(.db, yaml, csvs, history.log) is copied.
new: closes current project, creates a fresh auto-named temp,
switches.
load: opens a picker. List sub-mode shows projects in the active
data root, sorted newest-first by project.yaml mtime; arrow keys
navigate, Enter loads, `b` switches to a path-entry sub-mode for
projects elsewhere, Esc cancels. Empty data root jumps straight
to path entry.
Runtime: `Session` holds Option<Project> + Option<Database> so
project switches can drop old (releasing lock + stopping worker)
before opening new -- required for the "load my own current
project" case. `perform_switch` handles Load / SaveAs / NewTemp
uniformly.
Tests: 332 passing (270 lib + 9 + 5 + 6 + 16 new + 9 + 17),
0 failing, 0 skipped. Clippy clean.
Adds the explicit `rebuild` app-level command (ADR-0015 §7, §11)
and a modal UI infrastructure to host its confirmation dialog.
Typing `rebuild` emits Action::PrepareRebuild; the runtime reads
project.yaml + data/ to compute a summary ("3 tables and 47 rows
will be reconstructed; the existing playground.db will be
replaced") and posts AppEvent::RebuildPrepared, which opens the
modal. Y confirms, N/Esc cancels. While the modal is open,
normal input is gated.
The worker's do_rebuild_from_text now wipes existing user tables
and metadata before reloading from text, so it works on both
fresh and populated databases. Source text is plumbed through
rebuild_from_text so the explicit rebuild logs to history.log
while the silent on-load rebuild from Iteration 3 stays silent.
Modal infrastructure (App.modal field + key routing + centered
overlay rendering + word-wrap) is reused by Iteration 4b's save
/ save as / load / new flows.
Tests: 314 passing (268 lib + 9 + 5 + 6 new + 9 + 17),
0 failing, 0 skipped. Clippy clean.
When the runtime opens a project whose playground.db is missing,
it now rebuilds the database from project.yaml + data/<table>.csv
per ADR-0015 §7. The rebuild path:
1. Parses project.yaml (serde_yml). Unknown versions / types /
actions surface as PersistenceFatal.
2. Recreates each user table with FK constraints inline
(PRAGMA foreign_keys=OFF), then populates the column-type,
relationship, and project metadata tables.
3. Loads each table's CSV via a hand-rolled reader that
preserves the NULL-vs-empty distinction (the csv crate
doesn't expose whether a field was quoted; ours does).
4. Runs PRAGMA foreign_key_check before commit; any violation
aborts.
5. Restores foreign_keys=ON regardless of success.
Row-level failures get DbError::RebuildRowFailed with row
number, file, table, and a friendly per-type detail. They land
in the runtime as a fatal stderr message ("unable to load row N
from `data/T.csv` into table `T`: ...") before the alternate
screen is entered.
created_at from project.yaml overwrites the configure-time
placeholder so timestamps round-trip stably.
Tests: 307 passing (267 lib + 9 + 5 new + 9 + 17), 0 failing,
0 skipped. Clippy clean with nursery lints.
The Iteration-2 rule wrote a header-only CSV for every existing
table, which surprised users who created a table and saw a file
appear before any data went in. Tighten the rule: a CSV exists
iff the table has rows. Persistence::write_table_data now
delegates to delete_table_data when the snapshot is empty,
removing any prior CSV. The schema-only invariant (YAML knows the
table; CSV knows its rows) is preserved.
The cascade-delete integration test was rewritten to assert the
CSVs vanish; two new tests pin the rule (create -> no CSV;
delete --all-rows -> CSV removed).
Tests: 291 passing (256 lib + 9 + 9 + 17), 0 failing, 0 skipped.
Every successful user command now persists through to YAML, the
affected CSVs, and history.log inside the same SQLite transaction,
with the commit-db-last ordering from ADR-0015 §6: validate ->
mutate -> stage text + fsync -> atomic rename -> append history ->
commit. A failure in any text-write step rolls back the SQLite tx,
so disk state is unchanged on failure. Persistence failures are
routed through a new AppEvent::PersistenceFatal which sets a
fatal_message on the App, emits Action::Quit, and is printed to
stderr after terminal teardown so the banner remains above the
shell prompt (ADR-0015 §8).
New persistence module owns the file formats: hand-rolled YAML
schema writer, per-type CSV encoder (RFC 4180, NULL distinct from
empty string, base64 blobs), append-only history.log with ISO-8601
timestamps and successful-only entries. Atomic per-file writes via
tmp + fsync + rename.
The db worker holds an Option<Persistence>; tests still use
Database::open(":memory:") with no persistence. Action::ExecuteDsl
gains a source field carrying the user-typed text, threaded
through to history.log.
Tests: 289 passing (256 lib + 7 new integration + 9 lifecycle + 17
walking-skeleton), 0 failing, 0 skipped. Clippy clean with nursery
lints.
Replaces the in-memory database with an on-disk project. Startup either
opens a project at the positional CLI path (L1) or creates an auto-named
temp project (<YYYYMMDD>-<word>-<word>-<word>) under the OS-standard
data directory or a --data-dir override. The new project::Project type
owns the directory skeleton and a PID+hostname lock file with
stale-lock takeover via sysinfo. The status bar now shows
"Project: <Display Name>", derived by a small kebab/snake/camel
prettifier. Per-command persistence to YAML/CSV/history.log is NOT
yet wired -- that's Iteration 2; for now playground.db carries the
state across quits.
Tests: 257 passing (231 lib + 9 new integration + 17 existing),
0 failing, 0 skipped. Clippy clean with nursery lints.
DSL data operations (ADR-0014):
- insert into T [(cols)] values (vals); short form
insert into T (vals) omits values keyword for friendlier
syntax.
- update T set ... where col=val | --all-rows; delete from T
where col=val | --all-rows; show data T.
- Value AST (Number/Text/Bool/Null) with per-column-type
validation in the executor: int/real/decimal/bool/date/
datetime/shortid each accept a documented literal shape
and produce friendly format errors naming the column.
- INSERT short form fills non-auto-generated columns in
schema order; auto-fills serial via SQLite and shortid
via the new generator (T2).
- `add column [to table] T: c (type)` -- `to table` now
optional.
Database:
- insert/update/delete via prepared statements with bound
rusqlite::types::Value parameters.
- InsertResult/UpdateResult/DeleteResult: writes return
rows_affected plus the affected row(s) only (not the whole
table), so users see exactly what changed.
- INSERT shows the just-inserted row via last_insert_rowid.
- UPDATE captures matching rowids up-front and fetches them
post-update -- works even if the UPDATE changed the WHERE
column.
- DELETE reports per-relationship cascade effects by row-
count diffing inbound child tables; UPDATE-side cascades
are not yet detected (would need value diffing).
- query_data formats cells (booleans true/false, NULLs as
None).
FK error enrichment:
- Now lists both outbound (INSERT/UPDATE relevance) and
inbound (DELETE/UPDATE on parent relevance) FKs from the
metadata, so RESTRICT errors point at the children
blocking the delete.
- RelationshipSelector has a proper Display impl -- "no
such relationship" reads cleanly.
Relationship display:
- target_table for AddRelationship/DropRelationship now
returns the parent (1-side); structure rendering after
add/drop shows that side's "Referenced by:" entry,
matching the `from <Parent>` direction of the command.
- [ok] summary uses display_subject so relationship
commands show both endpoints (`from P.col to C.col`)
rather than a single misleading table name.
- Auto-name format `<Parent>_<pcol>_to_<Child>_<ccol>`
(matches the from..to direction).
Output rendering and scrolling:
- Wrap-aware scroll: renderer reports both visible-row
count and total wrapped-row count to App; scroll math
caps against actual displayable rows. Long lines wrap;
the bottom line is always reachable; PageUp/PageDown work
correctly even after paging past the buffer top.
- Multi-line messages (FK error enrichment, cascade summary)
split into single-line OutputLines at creation time so
wrap/scroll math agree.
Runtime / events:
- New AppEvent variants for Insert/Update/Delete success
carrying typed result structs; DslDataSucceeded reserved
for show-data queries.
Docs:
- ADR-0014 covers data-op grammar, value model, --all-rows
safety, auto-show.
- requirements.md: C5 done, T2 done, V2 partial (basic data
view), V5 partial (show data added). New entries: C5a
complex WHERE expressions; H1 progress note for FK
enrichment; H1a (strong syntax-help in parse errors).
Tests: 200 passing (183 lib + 17 integration), 0 skipped.
Includes parser, type-validation, DB write/read, FK-failure
enrichment, cascade-delete propagation, focused-auto-show
behaviour, scroll-cap invariants. Clippy clean with nursery
enabled.
claude@clouddev1