Completes the X1 full sweep started in a8ad0c6 (db.rs). Closes X1 -> [x].
- persistence/mod.rs: debug! on every yaml/CSV/history write -- the
silent-failure-prone disk paths (write_schema, write_table_data incl.
the empty->delete branch, append_history/_failure).
- runtime.rs: debug! on execute_command_typed dispatch (one per executed
command, complements the db.rs executor logs).
- app.rs: debug! on submit (route + submission mode), dispatch_app_command,
and the ADR-0044 diagram-vs-prose render-mode choice.
- dsl/parser.rs: trace! on parse begin/outcome at the parse_command_inner
choke point -- trace, not debug, because the live overlay/completion
re-parse per keystroke (hot path).
- logging.rs: documented level discipline (error/warn/info/debug/trace) so
the convention survives across sessions.
Levels verified end-to-end through the real worker thread + logging::init.
~75 -> 135 tracing sites total. Tests: 2207 pass / 0 fail / 1 ignored.
Clippy clean.
serde_yml (RUSTSEC-2025-0068) and its libyml backend
(RUSTSEC-2025-0067) are archived, unsound, and unmaintained with no
patched version. Swap to serde_norway, the maintained serde_yaml fork
on unsafe-libyaml-norway — a drop-in for our from_str / to_string /
Value usage across persistence, undo, and the catalog parser.
Clears both advisories (cargo audit / osv-scanner / grype all clean;
serde_yml + libyml gone from the tree). No behaviour change; full
suite 2151/0/1.
The input mode always started in simple; a learner who quit in advanced
had to re-toggle every launch. Store the mode per-project in project.yaml
(project.mode:, optional, default simple) and restore it on every open.
Mode is live UI state, not schema: the worker stamps the current mode
into project.yaml on every write, so a later command rewrites the live
value rather than clobbering it — no db round-trip needed. The mode is
persisted on unload (quit + project switch) so the mode you leave a
project in is always what reopens; the `mode` command also persists
immediately. A switch saves the outgoing mode, then restores the
incoming project's stored mode.
New --mode simple|advanced CLI flag (precedence --mode > stored >
simple; combines with --resume). A teacher can ship a project that
opens in advanced mode and export it to students (the mode travels in
the zip).
ADR-0015 Amendment 1; ADR-0003 note; help banner; requirements L1b.
ALTER TABLE <T> ADD [CONSTRAINT <name>] (CHECK | UNIQUE | FOREIGN KEY)
and DROP CONSTRAINT <name>. ADD = table-CHECK + composite UNIQUE + FK
(ADD PRIMARY KEY and a named UNIQUE refused — composite UNIQUE is
anonymous in our model). Each ADD reuses a low-level path with a dry-run
guard (table-CHECK/UNIQUE rebuild; FK -> add_relationship, bare
REFERENCES -> parent single PK). DROP CONSTRAINT resolves the name to a
named table-CHECK then a child-side FK, else refuses. One undo step each.
Named table-CHECKs round-trip: a nullable `name` column on
__rdbms_playground_table_checks (rebuild-only arrival; a named add on a
pre-4g project is refused with a "rebuild first" hint) plus a project.yaml
check_constraints {expr, name} extension (bare-string form still reads).
The internal-__rdbms_* guard was folded into do_add_constraint /
do_add_relationship, completing that guard class.
Grammar: the action Choice keeps one branch per verb (add/drop/rename/
alter) with an inner Choice fanning out on the distinct second keyword,
since the walker's Choice does not backtrack between same-led branches.
Tests: 7 Tier-1 parse + 2 yaml round-trip + 1 internal-guard + 9 Tier-3
e2e. Help/usage refreshed; ADR-0035 §13 4g + README + requirements.md in
lockstep.
Advanced-mode SQL CREATE [UNIQUE] INDEX [IF NOT EXISTS] [<name>] ON
<T> (cols) -> SqlCreateIndex and DROP INDEX [IF EXISTS] <name> ->
SqlDropIndex, both reusing the ADR-0025 executors (do_add_index /
do_drop_index), like 4c reused do_drop_table.
- CREATE UNIQUE INDEX admitted in advanced mode (ADR-0025 Amendment 1):
ADR-0025 deferred UNIQUE indexes for the simple-mode DSL, but advanced
mode trusts the user like SQL does. Adds an additive IndexSchema.unique
flag (project.yaml, serde-default, version stays 1); rebuild re-emits
CREATE UNIQUE INDEX; the redundant-set guard keys on (columns, unique).
Simple-mode `add unique index` stays deferred.
- IF [NOT] EXISTS on both forms reuses the 4c no-op-with-note skip
(journalled, not snapshotted) via CreateIndexOutcome / DropIndexOutcome.
- Unnamed CREATE INDEX auto-named (ADR-0025 convention); the [UNIQUE]
prefix is a concrete-keyword Choice and the optional name an on-led-first
selector (the drop-index selector precedent) — trap-safe.
- create/drop each gain a second advanced node; the existing all-candidates
dispatch handles it (locked by parse tests).
- Unique indexes marked [unique] in the structure view and items panel.
- do_add_index refuses internal __rdbms_* tables as "no such table",
closing a latent exposure on both the simple `add index` and the new
SQL CREATE INDEX surfaces (ADR-0025 Amendment 1).
Docs: ADR-0035 status + §13 4d + 4i; ADR-0025 Amendment 1; ADR README;
requirements.md Q1/C3. Plan: docs/plans/20260525-adr-0035-sql-ddl-4d.md.
Tests: 1834 passing / 0 failing / 0 skipped / 1 ignored; clippy clean.
Add table-level CHECK (e.g. `CREATE TABLE t (a int, b int, CHECK (a < b))`)
to advanced-mode SQL CREATE TABLE. Since SQLite exposes no PRAGMA for CHECK
constraints, a table-level CHECK cannot be read back from the engine and
becomes the source of truth in a new internal metadata table
`__rdbms_playground_table_checks (table_name, seq, check_expr)`.
- Grammar: new TABLE_CHECK element in ELEMENT_CHOICES.
- Builder: distinguishes a table-level CHECK from a column-level one by
element position (no column-def open in the element), using depth-aware
boundary tracking so a length-arg comma (`numeric(10,2)`) or a
table-PRIMARY KEY's inner comma is not mistaken for an element separator.
- Worker: do_create_table emits the CHECK clauses and writes the metadata
rows in its transaction; schema_to_ddl emits them identically on rebuild;
read_schema / read_schema_snapshot read them from the metadata table;
do_drop_table clears them.
- Persistence: TableSchema.check_constraints round-trips through project.yaml
(#[serde(default)], optional on read), mirroring unique_constraints.
- Composite UNIQUE deliberately stays PRAGMA-detected (engine-reportable,
unlike CHECK) — user-confirmed.
DA/runda round added cross-cutting tests and a forward-looking doc fix:
- table CHECK survives a rebuild triggered by `add column`, and a later
rebuild_from_text (the ADR-0013 rebuild primitive uses a raw DROP, so the
metadata rows keyed on the final name are preserved);
- dropping a column a table CHECK references fails cleanly (rollback, table
intact); detection is 4e, friendly wording is H1;
- dropping a table clears its CHECK metadata (no orphan rows on re-create);
- amended ADR §6 so 4h's RENAME also updates the new metadata table.
20 Tier-3 + 9 grammar/builder + 2 YAML tests. Docs: ADR-0035 Status/§13/§6,
README index, requirements.md Q1. Help/usage skeleton + describe display of
table-level constraints deferred to 4i (symmetric with 4a.2).
Tests: 1769 passing, 0 failing, 1 ignored. Clippy clean.
Advanced-mode SQL CREATE TABLE gains the constraints that need no new
internal table (the 4a.2 slice):
- Grammar (sql_create_table.rs): column-level DEFAULT/CHECK and
table-level UNIQUE(cols). DEFAULT is a literal or a *parenthesised*
expression (standard SQL) — a bare sql_expr greedily eats a following
NOT (NOT IN/LIKE/BETWEEN), breaking `DEFAULT 0 NOT NULL`; the parens
bound it. CHECK is paren-bounded already.
- Builder (ddl.rs): captures CHECK/DEFAULT raw SQL text by byte span
(sql_expr builds no AST) via capture_parenthesised_span /
capture_expr_span; routes single-column table UNIQUE into the
column's flag and composite UNIQUE into unique_constraints.
- Command/worker: ColumnSpec gains check_sql/default_sql (raw, preferred
over the typed Expr/Value); Command::SqlCreateTable + Request +
do_create_table gain unique_constraints; do_create_table emits raw
CHECK/DEFAULT and composite UNIQUE clauses.
- Round-trip (part D): ReadSchema/TableSchema gain unique_constraints;
read_schema detects composite UNIQUE via PRAGMA index_list origin 'u'
(single-column still folds to the column flag); schema_to_ddl emits
them; YAML RawTable/write_table round-trips (optional-on-read).
CHECK round-trips via __rdbms_playground_columns.check_expr, DEFAULT
via PRAGMA table_info — no new metadata table.
Table-level/multi-column CHECK remains 4a.3 (rejected "not yet
supported"); FK is 4b.
Tests: +7 builder (raw-text capture incl. the DEFAULT 0 NOT NULL
boundary the fix was found by; single/composite UNIQUE routing) and +4
Tier-3 (CHECK enforced, DEFAULT applied, composite UNIQUE enforced, and
all three survive a rebuild — the part-D round-trip). 1752 pass / 0 fail
/ 1 ignored; clippy clean. Plan + requirements.md updated.
Replay (§3): run_replay parses <ts>|<status>|<source> journal records — runs ok, skips non-ok — while still accepting bare .commands scripts (prefix-detected so a | inside a bare command isn't misread). Fixes replay history.log, which died on line 1.
Journal failures (§1/§2): failed commands are recorded err via a new Action::JournalFailure, emitted by the pure-sync App for both parse failures and worker-execution failures (runtime appends best-effort, never fatal). Hydration reads all records so typo'd/rejected commands are recallable across sessions.
Amendment 1 — replay filters app-lifecycle commands: a working replay history.log exposed that the journal also records save as/load/new/export/import/rebuild/mode (which would panic the worker dispatch or abort replay). Replay now re-applies only schema/data writes and skips every app-lifecycle command + nested replay, classified by entry word so modal/incomplete forms (save as, bare mode) and quit skip uniformly rather than aborting. All skips continue (reversing the nested-replay refusal); import and nested replay warn. replay.error_nested removed; replay.skipped_import/_replay added; ReplayCompleted carries warnings. requirements.md U3/U4 updated; app-command runtime-failure journalling tracked as a follow-up.
1659 passing / 0 failing / 0 skipped / 1 ignored. Clippy clean.
The fourth constraint. `check ( <expr> )` reuses the ADR-0026
WHERE-expression grammar via `Subgrammar`, so a check is
written in the same language as a `where` filter.
- Grammar: a `CHECK_CONSTRAINT` arm joins the shared
constraint-suffix Choice; `consume_check_expr` extracts the
parenthesised expression (paren-depth aware) into
`ColumnSpec.check` / `Command::AddColumn.check`.
- Storage: the parsed `Expr` is compiled once to inline SQL
(`compile_check_sql` — `compile_expr` + ADR-0028's
param-inliner) and stored in that form everywhere — a new
`check_expr` column in `__rdbms_playground_columns`,
`project.yaml`'s `ColumnSchema.check`, and the column DDL
emitted by `do_create_table` / `schema_to_ddl`.
- `add column … check` routes through the rebuild primitive
(SQLite's `ALTER … ADD COLUMN` cannot carry it); a CHECK on
a serial/shortid column is create-table-only and refused at
add-column with a friendly message.
- `describe` surfaces the CHECK. ADR-0029 §7/§8 updated to the
SQL-form decision — double-quoted identifiers, consistent
with ADR-0028's `explain` display SQL.
1201 tests pass (+8); clippy clean.
`create table … with pk` now parses the column-constraint
suffix; combined with the commit-1 db layer, a constrained
table works end to end.
- A shared constraint-suffix grammar fragment — `not null`,
`unique`, `default <literal>` — sits after each column's
`(type)` group; `build_create_table` walks the matched path
per column and folds the constraints into `ColumnSpec`.
- §9 redundancy check: every `with pk` column is a primary-key
column, so `not null` (any) and `unique` (single-column PK)
are rejected with a friendly error
(`parse.custom.constraint_redundant_on_pk`).
- `project.yaml` round-trip: `ColumnSchema` gains `not_null` /
`default`; the YAML reader/writer and `build_read_schema`
carry them, so `rebuild` / `export` / `import` preserve
constraints.
- ADR-0029 §2.1's example corrected — `create table` columns
are all PK columns, so its suffix is for `default` / `check`;
`docs/simple-mode-limitations.md` records that non-PK
columns at create time need advanced mode.
CHECK is deferred to the next commit. 1184 tests pass (+7);
clippy clean.
`create table … with pk` parsed column types as `name:type`,
while `add column` uses `name(type)`. Unify on the parens
form so column-type syntax is consistent across the DSL:
create table T with pk id(serial), name(text)
Only `COL_SPEC` changes (`:` → `( … )`); `build_create_table`
reads columns by role, so it is unaffected. The `:` that
separates table from column in `add column` / `drop column`
is unchanged. Sweeps the test suite, the typing-surface
matrix (two `after_colon` cells renamed to `after_paren`,
4 snapshots regenerated), the friendly catalog's usage
templates, ADR-0009's example, and requirements.md.
1039 passing / 0 failing / 1 ignored; clippy clean.
Implement ADR-0025 — indexes as a DSL DDL feature.
- Grammar: `add index [as <name>] on <T> (<cols>)`, `drop index
<name>` / `drop index on <T> (<cols>)`, plus a `--cascade`
flag on `drop column`.
- db.rs: index operations over the engine's native index
catalog (no metadata table). The rebuild-table primitive now
captures and recreates indexes, so `change column` and the
relationship operations no longer silently drop them.
- `drop column` refuses an indexed column unless `--cascade`,
which drops the covering indexes and reports each.
- Persistence: additive `indexes:` list in `project.yaml`
(version unchanged); round-trips through rebuild/export/import.
- Display: an `Indexes:` section in the structure view and a
nested tables/indexes items panel (S2).
Reconciles requirements.md (C3 index portion, S2 satisfied)
and CLAUDE.md. 1038 tests passing (+31), clippy clean.
Completes the i18n sweep started in the previous commit. All
remaining hand-rolled user-facing English strings inside
thiserror #[error(...)] attributes have been moved into the
catalog. Drops the thiserror dependency entirely.
Twelve error types migrated:
- dsl::action::UnknownAction → parse.custom.unknown_action
- dsl::parser::ParseError → parse.error_wrapper + parse.empty
- dsl::value::ValueError → value.{type_mismatch,format}
- persistence::csv_io::CsvError → persistence.csv.*
- persistence::mod::PersistenceError → persistence.{io,encode}
- persistence::yaml::YamlError → persistence.yaml.*
- persistence::migrations::MigrateError → persistence.migrate.*
- project::lock::LockError → project.lock.*
- project::naming::NamingError → project.naming.*
- project::naming::UserNameError → project.user_name.*
- project::mod::ProjectError → project.{path_not_found,...}
- project::mod::SafeDeleteError → project.safe_delete.*
- archive::ArchiveError → archive.*
- cli::ArgsError → cli.*
- db::DbError → db.error.*
Pattern per type: drop thiserror::Error derive, write manual
Display calling crate::t!(), keep #[from] semantics via
explicit From impls, override Error::source() where applicable
so #[source]-style chaining is preserved.
Why this matters (user rationale): "fine to have fallbacks for
errors that are purely technical, but lift the output to a
place where it can be localized later and where an adjustment
with friendly text is easily possible if any of them become
part of the happy path." All surface strings now live in
en-US.yaml and can be reworded or localized without touching
Rust source.
Tests: 769 passing, 0 failed, 1 ignored. Clippy clean with
-D warnings. Cargo.toml: drop thiserror = "2.0.18".
Generalises serial and shortid beyond their previous restricted
forms:
- `serial` is no longer restricted to single-column PK. Non-PK
serial columns get an emitted UNIQUE constraint and use
application-side MAX(col)+1 at INSERT time (rowid alias still
drives the PK case for free; per ADR-0010 worker-thread
serialisation, the read-then-insert sequence is safe).
- `shortid` columns auto-fill existing null cells when the
column is materialised — `add column T: x (shortid)` on a
non-empty table no longer leaves rows in a not-really-valid
NULL state.
- `int -> serial` joins the type-change matrix as always-clean
identity (closes the asymmetry vs `text -> shortid`); other
sources are refused with a route-via-int hint.
- `change column T: x (serial|shortid)` fills null source
cells with sequence / generated values in the same rebuild
transaction.
Internal infrastructure:
- ReadColumn gains `unique: bool`; read_schema detects single-
column UNIQUE indexes via pragma_index_list /
pragma_index_info; schema_to_ddl emits inline UNIQUE for
non-PK columns.
- ColumnSchema (persistence) gains `unique: bool` so the flag
survives YAML round-trip and rebuild-from-text reconstructs
it faithfully — preserves the "serial -> int leaves UNIQUE
in place" promise across save/load cycles.
- ChangeColumnTypeResult.client_side now carries `auto_filled`
+ `auto_fill_kind` alongside `transformed` + `lossy`; the
app handler renders separate note lines when both apply.
- AddColumnResult is a new return type carrying pre-rendered
[client-side] note lines for the auto-fill paths.
Tests: 519 -> 534 (+15). Clippy clean.
Closes out track 2's ADR-0015 backlog.
* `--resume` CLI flag (L1a, ADR-0015 §7) opens the most-
recently-used project, tracked in <data-root>/last_project.
Mutually exclusive with a positional <project-path>; errors
cleanly to stderr (above the shell prompt) on missing file
or stale recorded path. last_project is rewritten on every
successful project open (startup, load, new, save as,
import).
* Persistent input history (I2-persist, ADR-0015 §12). On
project open, the in-memory navigable history is hydrated
from the tail of history.log (capped at the in-memory cap).
ProjectSwitched gains a `history_entries` payload field;
App::seed_history is the entry point. Pipes inside source
text round-trip via splitn(3); unknown escape sequences are
passed through literally.
* Migration framework scaffold (F3, ADR-0015 §9). New
persistence::migrations module with MigratorRegistry +
migrate_to_latest + ensure_project_yaml_migrated. Empty
in v1 (production registry has no migrators); the loader
runs through it on every project open and is exercised by
tests with a fake v1→v2 migrator. Writes
project.yaml.v<N>.bak before any migrator runs; verifies
each step bumps the version field.
Refreshes docs/requirements.md (A1 / I2 / F3 / E1 / L1a /
test baseline) and adds docs/handoff/20260508-handoff-3.md
covering both Iter 5 and Iter 6.
Total tests: 408 passing, 0 failing, 0 skipped (up from 345
at handoff-2). Clippy clean.
When the runtime opens a project whose playground.db is missing,
it now rebuilds the database from project.yaml + data/<table>.csv
per ADR-0015 §7. The rebuild path:
1. Parses project.yaml (serde_yml). Unknown versions / types /
actions surface as PersistenceFatal.
2. Recreates each user table with FK constraints inline
(PRAGMA foreign_keys=OFF), then populates the column-type,
relationship, and project metadata tables.
3. Loads each table's CSV via a hand-rolled reader that
preserves the NULL-vs-empty distinction (the csv crate
doesn't expose whether a field was quoted; ours does).
4. Runs PRAGMA foreign_key_check before commit; any violation
aborts.
5. Restores foreign_keys=ON regardless of success.
Row-level failures get DbError::RebuildRowFailed with row
number, file, table, and a friendly per-type detail. They land
in the runtime as a fatal stderr message ("unable to load row N
from `data/T.csv` into table `T`: ...") before the alternate
screen is entered.
created_at from project.yaml overwrites the configure-time
placeholder so timestamps round-trip stably.
Tests: 307 passing (267 lib + 9 + 5 new + 9 + 17), 0 failing,
0 skipped. Clippy clean with nursery lints.
The Iteration-2 rule wrote a header-only CSV for every existing
table, which surprised users who created a table and saw a file
appear before any data went in. Tighten the rule: a CSV exists
iff the table has rows. Persistence::write_table_data now
delegates to delete_table_data when the snapshot is empty,
removing any prior CSV. The schema-only invariant (YAML knows the
table; CSV knows its rows) is preserved.
The cascade-delete integration test was rewritten to assert the
CSVs vanish; two new tests pin the rule (create -> no CSV;
delete --all-rows -> CSV removed).
Tests: 291 passing (256 lib + 9 + 9 + 17), 0 failing, 0 skipped.
Every successful user command now persists through to YAML, the
affected CSVs, and history.log inside the same SQLite transaction,
with the commit-db-last ordering from ADR-0015 §6: validate ->
mutate -> stage text + fsync -> atomic rename -> append history ->
commit. A failure in any text-write step rolls back the SQLite tx,
so disk state is unchanged on failure. Persistence failures are
routed through a new AppEvent::PersistenceFatal which sets a
fatal_message on the App, emits Action::Quit, and is printed to
stderr after terminal teardown so the banner remains above the
shell prompt (ADR-0015 §8).
New persistence module owns the file formats: hand-rolled YAML
schema writer, per-type CSV encoder (RFC 4180, NULL distinct from
empty string, base64 blobs), append-only history.log with ISO-8601
timestamps and successful-only entries. Atomic per-file writes via
tmp + fsync + rename.
The db worker holds an Option<Persistence>; tests still use
Database::open(":memory:") with no persistence. Action::ExecuteDsl
gains a source field carrying the user-typed text, threaded
through to history.log.
Tests: 289 passing (256 lib + 7 new integration + 9 lifecycle + 17
walking-skeleton), 0 failing, 0 skipped. Clippy clean with nursery
lints.
claude@clouddev1