Handoff doc for end of 2026-05-08 (#5)

This commit is contained in:
claude@clouddev1
2026-05-08 14:39:54 +00:00
parent 5bb0a147f0
commit dcfeef5d3c
+692
View File
@@ -0,0 +1,692 @@
# Session handoff — 2026-05-08 (5)
Fifth handover for what's been a long day. The previous
session (handoff-4) shipped pretty-table rendering, the
B2/C2 column ops, and **designed** ADR-0017. This session
implemented ADR-0017 in full, drafted and implemented a
new ADR-0018 covering auto-fill semantics for `serial` and
`shortid`, and landed a small parser-error tiny-win along
the way. The user is busy for a while; the next agent
session can pick up several well-scoped tasks listed in
§"Independent work" without further input.
## State at handoff
**Branch:** `main`. Working tree clean. **4 commits ahead
of `origin/main`** (the 1 ADR design commit from
handoff-4's last action plus 3 from this session). Push
remains the user's call.
Commits since handoff-4:
```
5bb0a14 ADR-0018 implementation: auto-fill contracts for serial
and shortid
7dfa718 parser: structural error rendering, source echo, and
caret pointer
00947b9 ADR-0017 implementation: per-cell type-change with
override flags
545cbf5 Handoff doc for end of 2026-05-08 (#4)
c3e5f90 ADR-0017 + ADR-0002 amendment: type-change compatibility
+ engine-agnostic posture
```
**Tests:** **534 passing, 0 failing, 0 skipped** (up from
449 at handoff-4's baseline; +85 over this session). Test
counts per phase:
- ADR-0017 implementation: +68 (449 → 517)
- Parser tiny-win: +2 (517 → 519)
- ADR-0018 implementation: +15 (519 → 534)
**Clippy:** clean with `nursery` lints enabled.
**Release build:** ~7.2 MB single binary (up ~100 KB from
handoff-4's 7.1 MB; the increase is the type_change
matrix module and ADR-0018 auto-fill paths).
## What's implemented (delta vs. handoff-4)
The previous handoff covered: Iter 16 of track 2,
pretty-table rendering, B2/C2 column ops, optional
`to`/`table` parser polish, and silent-rebuild banner
suppression. This session adds:
### ADR-0017 (column type-change compatibility) — implemented
Replaces the placeholder "trust STRICT" body of
`do_change_column_type` with the full per-cell transformer
matrix from ADR-0017. New module `src/type_change.rs`
(~620 lines + 56 unit tests) carrying:
- `CellOutcome { Clean(Value) | Lossy { new, reason } |
Incompatible { reason } }` plus `transform_cell` covering
every entry in ADR-0017 §3.
- `static_refusal` for same-type / blob / date↔datetime /
cross-domain refusals.
`change column [in] [table] <T>: <col> (<newtype>)` now
accepts `--force-conversion` (accept lossy) and
`--dont-convert` (skip the entire client-side layer; let
the engine's STRICT typing decide). Mutually exclusive at
parse time.
Refusal preconditions per ADR-0017 §4:
- Outbound FK (column is a child-side FK): refused outright.
- Inbound FK (column is parent-side / referenced): refused
only when `old_ty.fk_target_type() != new_ty.fk_target_type()`.
- Post-transformation uniqueness check for any column that
carries a UNIQUE constraint in the new schema (PK + ADR-
0018's added serial/shortid).
Diagnostic refusals render through ADR-0016's pretty-table
renderer — bordered, capped at 100 rows with `… and N more`
inside the box, identifying rows by their PK value(s) per
the ADR-0017 §7 amendment we added (PK identifiers replace
positional row indices, since SQLite returns rows
unordered).
`[client-side]` success note (§6) fires when any cell was
non-identity transformed; lossy variant adds the lossy
count under `--force-conversion`.
ADR-0017 §3 was amended in place to add `serial → int` as
an always-clean matrix entry (it was missing despite §4.1
treating it as the canonical PK conversion).
### Parser: structural error rendering + source echo + caret
The old `humanise()` rendered chumsky's terse default
("found 'i' expected ':' (near `i`)") as-is and added a
not-helpful `(near `X`)` suffix. Now `humanise()` reads the
structured `RichReason::ExpectedFound`, lists the
`expected` patterns in plain prose, prefixes the consumed
context, and produces messages like:
```
parse error: after `change column Rich`, expected `:`,
found `in`
```
`dispatch_dsl` additionally echoes the source line on
parse failure (matching the success path's "running: …")
and prints a `^` caret under the failure position.
**Known limit captured for future work**: chumsky
combinators in `keyword_ci` emit `Rich::custom` errors on
mismatch, which are opaque to chumsky's choice-aggregation
machinery. Result: errors like "expected `data` or `table`"
(bison-equivalent) aren't yet possible — only one
alternative shows up. A structural fix to keyword_ci
would aggregate properly. Deferred to a future "parser-as-
source-of-truth" ADR (covered in §"Pending" below).
### ADR-0018 (auto-fill contracts for serial and shortid) — designed and implemented
User noticed three asymmetric gaps during ADR-0017 testing:
1. `serial` was restricted to single-column PK. Other RDBMS
(PostgreSQL `SEQUENCE`, MySQL `AUTO_INCREMENT`) don't
have this restriction; ours was an artefact of SQLite's
only free auto-increment mechanism (`INTEGER PRIMARY KEY`
rowid alias) leaking into the user-facing surface.
2. `text → shortid` round-trip worked end-to-end (per
ADR-0017's matrix); `int → serial` was statically
refused.
3. `add column T: x (shortid)` on a non-empty table left
existing rows NULL — violating the design contract that
shortids are unique non-null identifiers.
ADR-0018 generalises both auto-generated types with the
unifying principle: *auto-generated column types honour
their generation contract on every path that creates or
transitions the column.* Concretely:
- **`serial` is no longer PK-restricted.** Non-PK serial
columns get an emitted UNIQUE constraint and use
application-side `MAX(col) + 1` at INSERT time. PK case
unchanged (rowid alias). Implementation switch hidden
per ADR-0002.
- **`shortid` auto-fill at column-materialisation time.**
`add column T: x (shortid)` on a non-empty table now
generates fresh shortids for existing rows in the same
rebuild transaction. `change column → shortid` does the
same for null cells.
- **`int → serial` joins the matrix** as always-clean
identity. Other source types refused with a route-via-
int hint.
- **`change column → serial` auto-fills null cells** with
sequence values continuing from `MAX + 1`.
- **UNIQUE story**: non-PK serial / shortid gain UNIQUE on
creation/conversion. Reverse direction (`serial → int`,
`shortid → text`) leaves UNIQUE in place — user can drop
it later when the constraint-management surface lands
(C3-track work, deferred).
ADR-0018 implementation pulled C3 partially forward:
`schema_to_ddl` gains UNIQUE-clause emission, `read_schema`
gains UNIQUE detection via `pragma_index_list` /
`pragma_index_info`, and `ColumnSchema` (persistence)
gains a `unique: bool` field that survives the YAML round-
trip. The user-facing constraint surface (`add unique`
syntax, drop/rename UNIQUE, multi-column UNIQUE) stays
deferred — only the internal infrastructure required by
the auto-generated type contracts landed.
`[client-side]` notes extended: when both ADR-0017
transformation AND ADR-0018 auto-fill apply in the same
operation, two distinct note lines emit (e.g.,
`change column T: x (shortid)` from text where some cells
had to be validated and others auto-filled).
`AddColumnResult` is a new return type carrying pre-
rendered `[client-side]` note lines for the new auto-fill
paths.
### Engine-vocabulary cleanup
While in `do_add_column`, fixed an existing user-visible
string that named "SQLite's ALTER TABLE" — an ADR-0002
posture violation that pre-dated this session. The
refusal it lived in was being lifted anyway as part of
ADR-0018, so the leak went with it. A broader engine-name
sweep is listed in §"Independent work" below.
## ADR index (read these before touching the related areas)
```
0000 Record architecture decisions (process)
0001 Language and TUI framework (Rust + Ratatui)
0002 Database engine
— User-facing posture (no engine name in user-visible
strings; amended in handoff-4's session)
0003 Input modes and command dispatch
0004 Project file format
— amended by 0015
0005 Column type vocabulary
— definition of `serial` generalised by ADR-0018 (no
longer restricted to PK; implementation hidden)
0006 Undo snapshots and replay log (deferred)
0007 Sharing and export
— amended by 0015 amendment 1
0008 Testing approach
0009 DSL command syntax conventions
0010 Database access via worker thread
— load-bearing for ADR-0018 §5's MAX+1 INSERT path
safety
0011 FK column type compatibility
0012 Internal metadata for user-facing column types
0013 Relationships, naming, and rebuild-table strategy
— primitive carries every auto-fill-on-rebuild case
0014 Data operations, value literals, and auto-show
— INSERT-time auto-fill amended by ADR-0018 §5
0015 Project storage runtime
— ColumnSchema gained `unique: bool` for ADR-0018's
round-trip (no migration needed; older project
files default to `unique: false`)
0016 Pretty table rendering for data and structure views
— used by ADR-0017's diagnostic tables
0017 Column type-change compatibility
— IMPLEMENTED (this session). §3 + §7 amended in place
for serial→int matrix entry and PK-based row
identifiers. §3 + §4.3 further amended by ADR-0018
for int→serial entry and uniqueness-check extension.
0018 Auto-fill contracts for serial and shortid columns
— IMPLEMENTED (this session). Generalises serial
beyond PK; tightens shortid contract; pulls forward
internal UNIQUE infrastructure.
```
## Pending — proposed next moves (in order)
### 1. Independent work for next session — see dedicated section below
This is the substantive output for an unattended agent
session. Three Tier-A and two Tier-B items are detailed
in §"Independent work for next session".
### 2. Friendly error layer (H1) — needs a small ADR first
ADR-0002's user-facing posture commits to never exposing
engine error text verbatim. The current friendly-message
helper just calls `Display`. ADR-0017's `--dont-convert`
path has a tiny local wrapper
(`friendly_change_column_engine_error`) that recognises
common kinds — when H1 lands, that helper folds into the
broader translator. ADR scope: defining the translation
mapping (which engine error patterns map to which user-
facing wording), how to surface FK / NOT NULL / type-
mismatch errors symmetrically. Probably 200 lines of code
+ tests once the ADR settles.
### 3. Parser-as-source-of-truth ADR
Discussed in this session: chumsky gives us structural
information (expected sets, span-tagged AST, partial
parses on failure) we're not extracting. That feeds H1a
(syntax help in parse errors), I3 (tab completion), I4
(syntax highlighting), and on-the-fly error squiggles.
The parser tiny-win this session was a down payment;
the broader ADR maps out what we extract from one source
(chumsky's parse output) to drive each affordance.
The specific keyword_ci structural-error rework (so
"expected `data` or `table`"-style messages aggregate
across choice alternatives) is the load-bearing piece.
### 4. Query DSL ADR + implementation
Biggest remaining design piece. Earlier discussion
landed on extending `show data` into a SELECT-style
command with WHERE / projection / order; expose
generated SQL as a pedagogical hook; bundle C5a's
complex WHERE into one coherent feature. Then QA1
(EXPLAIN QUERY PLAN) becomes meaningful.
### 5. Bigger UX projects
- V4 (session log + Markdown export).
- V1/V2 pretty-rendering refinements (relationship
rendering ADR — the "two structures + arrow" view).
- V3 (ER diagram export).
## Independent work for next session
These are well-scoped tasks an agent can pick up and
finish without user input. Each is sized to fit in one
session.
### A1. CI workflow (TT5)
**Scope:** single GitHub Actions YAML at
`.github/workflows/ci.yml`. Cross-platform Linux / macOS /
Windows; `cargo test` + `cargo clippy --all-targets -- -D
warnings`. Locks in the 534-test green baseline.
**Why independent:** no design questions, no codebase
integration. Standard Rust CI template adapted to this
project's nursery-clippy posture.
**Done when:** workflow file exists, syntax-validated,
runs on the next push to `main`. Local verification not
strictly required but `act` (if installed) can simulate.
**Watch out for:** the `bundled` feature on `rusqlite`
means SQLite is statically linked; no system-package
install step needed. `tokio` works on all three
platforms unchanged.
**Estimated:** 12 hours.
### A2. Engine-name audit (ADR-0002 posture sweep)
**Scope:** grep error messages and other user-facing
strings across `src/` for "SQLite", "STRICT", "PRAGMA",
"rusqlite", "ALTER TABLE", "CAST" (selectively — `CAST`
is a legitimate SQL keyword users will encounter, only a
problem when prescriptive). Replace with abstract
"the database" / "the engine" phrasing per ADR-0002.
**Why independent:** mechanical, well-defined. ADR-0002's
"User-facing posture" section is the spec.
**Where to look:**
- `DbError` variants — `Sqlite { message }` carries
engine-vocabulary; check whether `friendly_message()`
needs upgrading.
- Help text in `app.rs:1100-1200` area.
- Error messages constructed via `format!` with `Err(...)`
/ `DbError::Unsupported(...)` — search for these.
- Unsupported-feature refusals.
**Done when:** zero matches for "SQLite" / "STRICT" /
"PRAGMA" / "rusqlite" in user-reachable strings, AND the
test suite still green. Code comments and ADR prose are
fair game (they explicitly may name the engine — see
ADR-0002).
**Watch out for:** `rusqlite::Error::*` variant names that
appear in formatted error messages — those leak the crate
name. Replace with a switch on the error kind.
**Estimated:** 12 hours.
### A3. `replay` command (U4)
**Scope:** new DSL command `replay <path>` that reads a
file (typically `history.log` or a `.commands` file) and
dispatches each non-comment, non-blank line through the
existing DSL pipeline. On a per-line failure, abort the
replay and report `replay failed at line N: <error>`. On
success, report `replay complete: N command(s)`.
**Why independent:** small, well-bounded. The DSL pipeline
already exists; this just feeds it lines from a file.
**Implementation sketch:**
1. Parser: `replay` keyword followed by a quoted or bare
path. The path lexing might need a small new helper
(current parser doesn't have a "file path" terminal).
2. Command AST: `Command::Replay { path: String }`.
3. Runtime: read file, iterate lines, parse-and-execute
each, abort on first failure. Probably best kept
transactional at the file level (no individual command
commits if any later one fails) — but that's a design
question worth flagging in the implementation.
**Default to "stop on first error, report line number,
don't roll back"**: matches the "I'm replaying my
history" mental model where partial replay is a
recoverable state.
4. AppEvent + handler for replay outcome.
5. Tests: happy path (3-line replay), failure-mid-replay
(reports line number + stops), empty file, blank lines
skipped, comment lines (`# ...`) skipped.
**Watch out for:** ADR-0015's history.log format — entries
are append-only DSL command lines. `replay history.log`
on a project should reproduce its current state if started
from an empty database. That's the implicit invariant the
test suite should prove.
**Estimated:** 34 hours.
### B1. Update help text for ADR-0017 + ADR-0018 features
**Scope:** the in-app `help` command's output (in `app.rs`,
the `do_help` or similar function around line 11001200)
shows DSL command shapes. ADR-0017 added `--force-
conversion` and `--dont-convert` flags (already added to
help). ADR-0018 changed semantics of `add column ...
(serial|shortid)` on non-empty tables (now auto-fills
existing rows + emits UNIQUE) — this isn't called out
anywhere user-facing.
**Why independent:** the ADRs spell out the behaviour; the
help text just needs to surface it.
**Suggested additions:**
- `add column ... (serial|shortid)` line gains a sub-line:
` (existing rows auto-filled with sequence/generated values)`.
- `change column ... (serial|shortid)` similarly.
- New section "Auto-generated types" explaining serial
and shortid in 3-4 lines.
**Done when:** the help output describes the behaviour
matching ADR-0018 + ADR-0017. Existing help-output tests
pass (some may need string-matching updates).
**Estimated:** 30 min.
### B2. Test gap: change_column → bool from int 0/1
**Scope:** the type_change matrix has `(Int, Bool)` per-
cell-classified (clean for 0/1, incompatible otherwise).
This is well-tested at the matrix unit-test level. But
there's no end-to-end test in `db.rs` exercising
`change column T: x (bool)` from an int column. Trivial
coverage gap to fill.
**Why independent:** identical pattern to existing change-
column tests; just a different type pair.
**Suggested test:**
- `change_column_type_int_to_bool_with_zero_one_succeeds`:
rows with values 0, 1, 0 → success, no client-side note
expected (storage class doesn't change).
- `change_column_type_int_to_bool_refuses_other_values`:
row with value 2 → incompatible refusal.
**Done when:** 2 new tests pass; total 536.
**Estimated:** 30 min.
## Sharp edges and subtleties (delta vs. handoff-4)
Carried-over edges still apply (sync `update`, worker
thread, metadata transactions, rebuild-table primitive,
modal infrastructure, project-switch lock dance, `[temp]`
cleanup guards, persistence ordering, `DataResult` carries
`column_types`, `output_render` is the only place
tabular output should originate). New ones this session:
- **`Type::Serial` no longer implies PK at the type
system level.** ADR-0018 generalised serial. Existing
references to "serial" in code comments may say "PK
type" — those are stale. The non-PK serial path is
active and tested.
- **`add column` returns `AddColumnResult`, not
`TableDescription`.** Tests that called
`db.add_column(...).await.unwrap()` and used the result
as a description directly need `.description` indirection.
Five existing tests were updated; new tests should follow
the new shape.
- **`ChangeColumnTypeResult.client_side` is now
`Option<ClientSideNote>` where `ClientSideNote` carries
`transformed`, `lossy`, `auto_filled`, `auto_fill_kind`.**
When auto-fill happens (target is serial/shortid + null
cells), the note fires even though `transformed` is 0.
The filter `note.transformed > 0 || note.auto_filled > 0`
is the canonical "should we emit a note" test.
- **Non-PK serial INSERT auto-fill happens via `MAX(col)+1`.**
Per ADR-0010, the worker-thread serialisation makes this
safe without explicit locking. If you ever extract the
worker thread or change the connection model, this is
one of the things that breaks.
- **`schema_to_ddl` emits inline `UNIQUE` for non-PK
columns flagged unique.** PK columns aren't separately
marked unique in `ReadColumn` (PK already implies it);
the schema_to_ddl filter `unique && !primary_key`
matters.
- **`read_schema` reads UNIQUE via `pragma_index_list`
filtered to `origin = 'u'`.** Compound UNIQUE constraints
are deliberately ignored (ADR-0018 OOS-6 / future C3).
If you ever add multi-column UNIQUE support, the
detection logic needs extending.
- **Parse-error messages now show grammar-derived
expected/found and a consumed-context prefix.** Existing
tests that asserted on the old message shape may have
needed updates — none did, since the structural-error
tests assert on substrings (the consumed context, the
expected token).
## Repository layout (delta vs. handoff-4)
```
src/
type_change.rs — new (ADR-0017)
db.rs — many additions:
AddColumnResult, ChangeColumn­
TypeResult, ClientSideNote,
AutoFillKind, ReadColumn.unique,
read_unique_columns,
schema_to_ddl UNIQUE emission,
do_add_plain_column / do_add_auto_
generated_column,
do_change_column_type rewrite,
run_change_column_with_dry_run +
fill_auto_generated_cells,
generate_shortid_batch,
format_auto_fill_add_note,
diagnostic helpers (lossy /
incompatible / collision)
dsl/
parser.rs — change_column flag parsing,
RichPattern-aware humanise,
identifier .labelled,
consumed-context rendering
command.rs — ChangeColumnMode enum
value.rs — validate_date / validate_datetime
made pub(crate) so type_change
can consume them
app.rs — handle_dsl_change_column_success,
handle_dsl_add_column_success,
source-echo + caret on parse fail
event.rs — DslChangeColumnSucceeded,
DslAddColumnSucceeded
output_render.rs — render_diagnostic_table public,
Alignment public,
numeric_alignment_for public
persistence/
mod.rs — ColumnSchema.unique
yaml.rs — write_column emits unique flag,
RawColumn parses it
csv_io.rs — test fixture updated
runtime.rs — CommandOutcome::ChangeColumn
+ AddColumn variants
docs/
adr/
0017-column-type-change-compatibility.md
— §3 (serial→int row), §7 (PK
identifiers) amended
0018-auto-fill-contracts-for-serial-and-shortid.md
— new (this session)
README.md — indexed
handoff/
20260508-handoff-5.md — this file
```
## How to take over
1. Read this file.
2. Read `CLAUDE.md` for the working-style rules.
3. Read `docs/requirements.md` for granular progress.
4. **If picking up an Independent work item (§A1B2)**:
read just that item plus the listed ADR section it
refers to. The items are scoped to be independently
tackleable.
5. **If working on H1 / Query DSL / Parser-as-source-of-
truth**: start with an ADR draft. Don't implement
without one — those touch enough code to warrant the
discipline.
6. Run `cargo test` to confirm the 534-test green baseline.
7. `cargo clippy --all-targets` to confirm clippy-clean.
8. `cargo run --release` to see the UI.
### End-to-end smoke test (current state)
Demonstrates ADR-0017 + ADR-0018 features. Replaces the
handoff-4 recipe (which is now stale — `change column`
under ADR-0017 emits `[client-side]` notes the previous
recipe didn't show).
```
$ rm -rf /tmp/handoff5-smoke
$ rdbms-playground --data-dir /tmp/handoff5-smoke
# Inside the app:
help -- help text
(B1: extend with
ADR-0018 wording)
create table Customers with pk id:serial
add column Customers: Name (text)
add column Customers: Score (int)
insert into Customers ('Alice', 10)
insert into Customers ('Bob', 20)
insert into Customers ('Carol', 30)
show data Customers -- pretty-table render
# ADR-0017 type-change with [client-side] note:
change column Customers: Score (real)
-- emits:
-- [client-side] 3 row(s)
-- were transformed before
-- being stored. ...
# ADR-0017 lossy refusal:
change column Customers: Score (int)
-- emits a bordered
-- diagnostic table
-- listing the lossy rows
-- by PK; suggests
-- --force-conversion.
change column Customers: Score (int) --force-conversion
-- succeeds with both
-- "transformed" and
-- "lossy" counts in note.
# ADR-0018 add column auto-fill:
add column Customers: Tag (shortid) -- emits:
-- [client-side] 3 row(s)
-- given auto-generated
-- shortid values. ...
show data Customers -- Tag column populated
# ADR-0018 non-PK serial INSERT auto-fill:
add column Customers: Seq (serial) -- emits another
-- [client-side] note
insert into Customers ('Dave', 40) -- Seq auto-fills 4
-- (MAX of existing
-- 1,2,3 plus 1)
# ADR-0018 int -> serial round-trip:
add column Customers: Counter (int)
update Customers set Counter=1 where id=1
update Customers set Counter=2 where id=2
update Customers set Counter=3 where id=3
update Customers set Counter=4 where id=4
change column Customers: Counter (serial)
-- succeeds (no auto-fill
-- needed since values
-- are unique non-null)
# ADR-0017 PK FK-cascade refinement:
add column Customers: Email (text)
update Customers set Email='alice@example.com' where id=1
update Customers set Email='bob@example.com' where id=2
update Customers set Email='carol@example.com' where id=3
update Customers set Email='dave@example.com' where id=4
change column Customers: id (int) -- serial -> int on PK,
-- no inbound FK ->
-- allowed.
change column Customers: id (serial) -- int -> serial round
-- trip succeeds.
# Parser tiny-win demo:
change column Tag in Customers: Tag (text)
-- typo: column-name-
-- first. Error now reads
-- "after `change column
-- Tag`, expected `:`,
-- found `in`" with caret
-- under the offending
-- character.
quit
```
### Manual spot-checks worth running
- `--help` lists all column ops (drop / rename / change)
with their flags.
- Pretty rendering kicks in for `show data` AND every
schema-mutating command's auto-show.
- `change column T: c (real)` succeeds and emits the
`[client-side]` note for any non-empty table where the
source values differ in storage class from the target.
- `change column T: c (real) --force-conversion` accepts
fractional → int truncation; the note carries both
counts.
- `change column T: c (real) --dont-convert` bypasses the
client-side layer entirely (no `[client-side]` note,
even if all cells transformed cleanly).
- `add column T: x (shortid)` on a non-empty table fills
every existing row's `x` with a generated shortid.
- `add column T: x (serial)` on a non-empty table fills
with 1..N. Subsequent inserts get N+1, N+2…
- Non-PK serial UNIQUE: `update T set Seq=1 --all-rows`
→ engine refuses with a unique-violation diagnostic.
- Save/load round-trip: create a non-PK serial column,
quit, re-open. Read back: column is still UNIQUE.
- `change column id (int)` on a `serial` PK with no
inbound FKs → allowed (per ADR-0017 §4.1 refinement).
- `change column id (text)` on a `serial` PK with an
inbound FK → refused (per ADR-0017 §4.1 — fk_target_type
would change).