Concrete specification for the direction in ADR-0023, landed
during the round-6 design pass. Resolves all four rounds of
open design questions: walker as single source of truth,
scannerless terminal vocabulary (~8 building blocks), typed
value slots with content validators, WalkContext for schema-
aware narrowing from day one, WalkOutcome multi-purpose
return, HintMode per-node, ranker as separate layer, static
+ dynamic sub-grammars, aliases as Word annotations,
IdentSource taxonomy, six-phase per-command migration with
chumsky and walker side-by-side during the transition.
Key shifts from ADR-0023's sketch:
- Lexer dissolves entirely. Walker operates on bytes directly.
dsl/lexer.rs, dsl/keyword.rs go away in Phase F.
- Schema-aware parse from day one (not phased). Typed value
slots reject mis-shaped input at parse time with localised
wording. Completion narrows per column type.
- Sub-grammars: static (fn() -> Node) for composition;
dynamic (fn(&WalkContext) -> Node) for schema-dependent
expansion. No global named registry.
- Path-bearing commands: BarePath becomes a routine
non-whitespace terminal. Paths with spaces require quoting
via StringLit (UX simplification, aligns with standard CLI
convention).
- 13-node taxonomy: Word, Punct, Ident, NumberLit, StringLit,
BlobLit, Flag, BarePath, Choice, Seq, Optional, Repeated,
DynamicSubgrammar.
Migration plan: Phase A (walker scaffolding + app-lifecycle
commands), Phase B (DDL without value literals), Phase C
(create table), Phase D (data commands with full schema
awareness -- the design's central claim landing), Phase E
(replay), Phase F (delete chumsky + lexer + legacy parser
modules, simplify catalog). Estimated ~4 sessions total.
Also: rename ADR-0023 from 0023-proposed-unified-grammar-tree.md
to 0023-unified-grammar-tree.md (git mv preserves history)
and update its status to reflect the direction-accepted-but-
superseded-for-execution-detail relationship with ADR-0024.
Index updated.
Captures the architectural critique surfaced during round-5
manual testing — that adding a keyword or command currently
requires edits in 7-10 files across parser, completion, usage
registry, catalog, and tests — and the proposed direction: a
single declarative trie registry that drives parse, completion,
highlight, and usage rendering from one source.
Status: Proposed. Not yet accepted. Filename carries the
`-proposed-` segment so status is visible at directory-listing
time; rename to `0023-unified-grammar-tree.md` on acceptance.
Estimated cost: ~4 sessions, per-command migration. Why not
now: feature backlog and bearable scatter cost. Right moment
to execute when backlog quiets or scatter cost becomes
visibly painful.
Replaces the originally-planned separate ADRs for syntax
highlighting (I4) and tab completion (I3) with a single
unified design. The framing: colour, hint panel, and Tab
are three answers to the same question — what does the user
need to know mid-typing? — and planning them separately
produces three loose pieces that drift apart.
Three mid-typing states (valid-so-far / definite-error /
incomplete-but-plausible) drive four layered channels:
token-class colour and parse-error overlay (silent, always
on), hint panel ambient and Tab-triggered completion mode
(verbose, in the existing hint panel — no floating popups).
Schema-aware from day one via an IdentSlot taxonomy in the
parser (NewName / TableName / ColumnIn(TableRef::Earlier(N))
/ RelationshipName); every existing ident() call gets
audited and tagged. Completion candidates come from chumsky's
expected-token-set for keyword slots and from a new worker
request (ListNamesFor) for identifier slots.
Implementation lands in 8 green-after-each commits: theme
colours; input panel highlighting; echo line highlighting;
render-time parse + error overlay; hint panel ambient;
identifier-slot taxonomy + parser audit; schema query
plumbing; completion mode + key bindings. Estimated
1500-2500 lines across the eight stages.
Out of scope (deliberately): inline ghost text (could return
as a "most-likely" affordance later — fish-shell style),
fuzzy matching, punctuation completion, user-customisable
keybindings, SQL highlighting in advanced mode (waits on Q4).
ADR-0020 amends ADR-0001 with a two-phase parse: a lexer
producing a span-tagged token stream, then chumsky over
&[Token]. Single source of truth for keywords and punct via
a define_keywords!/define_punct! macro pattern. Parser
contract committed for I3 (queryable expected-token-set)
and I4 (lexer always succeeds, Error tokens for invalid
input). Includes an honest history note: the no-lexer shape
in dsl/parser.rs arose incrementally without ADR-level
deliberation against the known H1a/I3/I4 requirements; this
ADR corrects that.
ADR-0021 builds on ADR-0020 to close the H1a gap: a
per-command UsageEntry registry keyed off entry-keyword,
with parse errors rendered as caret + structural error +
matching usage template(s). Multi-entry families (add,
drop, show) render together. New catalog sections under
parse.usage.* (per-command grammar) and parse.token.*
(single-token vocabulary). Zero-prefix case ("frobulate
Customers") falls back to an "available commands:" framing.
Anchor-phrase compliance preserved.
Settles the design we discussed across this session's
follow-up to the engine-vocabulary audit:
- A central `friendly` module owns translation; the existing
ad-hoc helpers (`friendly_change_column_engine_error`,
`enrich_fk_message`) absorb into it.
- Initial catalog covers UNIQUE / FK / NOT NULL / CHECK /
type-mismatch errors with operation-tailored,
pedagogically-voiced wording in verbose and short variants.
- New `messages (short|verbose)` app-level command lets
advanced learners shrink the output. In-session state for
now; persisted later when settings persistence lands.
- Row pinpointing via post-failure re-query, rendered through
ADR-0017 §7's bordered diagnostic-table renderer.
`FriendlyError` is a structured payload (headline + hint +
optional table); `output_render` composes it.
- i18n foundation: hierarchical YAML catalog, embedded via
`include_str!`, fixed locale (en-US) for now, no external
files. `{name}` plain substitution; format specifiers
explicitly rejected so a translator cannot reformat values.
Value formats stay invariant across all locales (ISO 8601
dates, `.` decimals, `true`/`false`, `NULL`) — explicitly
not a translatable concern.
- Migration sweep is required follow-on but separable: a
`t!()` macro marks call sites and lets per-category PRs
land incrementally. Anchor-phrase list (§10) limits test
churn for the most common substring assertions.
Out of scope and explicitly deferred: advanced-mode SQL
error sanitisation (waits on Q1), settings persistence for
the messages command, plural-form rules per locale, runtime
locale selection, locale-aware value formatting (rejected,
not deferred), constraint-management surface (C3 territory).
README index updated.
Generalises serial and shortid beyond their previous restricted
forms:
- `serial` is no longer restricted to single-column PK. Non-PK
serial columns get an emitted UNIQUE constraint and use
application-side MAX(col)+1 at INSERT time (rowid alias still
drives the PK case for free; per ADR-0010 worker-thread
serialisation, the read-then-insert sequence is safe).
- `shortid` columns auto-fill existing null cells when the
column is materialised — `add column T: x (shortid)` on a
non-empty table no longer leaves rows in a not-really-valid
NULL state.
- `int -> serial` joins the type-change matrix as always-clean
identity (closes the asymmetry vs `text -> shortid`); other
sources are refused with a route-via-int hint.
- `change column T: x (serial|shortid)` fills null source
cells with sequence / generated values in the same rebuild
transaction.
Internal infrastructure:
- ReadColumn gains `unique: bool`; read_schema detects single-
column UNIQUE indexes via pragma_index_list /
pragma_index_info; schema_to_ddl emits inline UNIQUE for
non-PK columns.
- ColumnSchema (persistence) gains `unique: bool` so the flag
survives YAML round-trip and rebuild-from-text reconstructs
it faithfully — preserves the "serial -> int leaves UNIQUE
in place" promise across save/load cycles.
- ChangeColumnTypeResult.client_side now carries `auto_filled`
+ `auto_fill_kind` alongside `transformed` + `lossy`; the
app handler renders separate note lines when both apply.
- AddColumnResult is a new return type carrying pre-rendered
[client-side] note lines for the auto-fill paths.
Tests: 519 -> 534 (+15). Clippy clean.
Specifies the curated per-cell classification (clean /
lossy / incompatible) for column type changes, the static
transformer matrix (numeric chains, text↔structured types,
always-clean stringifications), and the PK / shortid /
uniqueness-bearing handling. Replaces the B2/C2
placeholder of "rely on engine STRICT and surface its
errors" with a learner-friendly model that:
* refuses incompatibles up-front,
* refuses lossy conversions by default with a re-run-with-
--force-conversion hint,
* refines the PK refusal: an inbound-FK PK is only refused
when the new type would change the FK target type
(so `serial → int` and `shortid → text` on FK-referenced
PKs are allowed; `int → text` etc. still refuse),
* adds a post-transformation uniqueness check for PK and
shortid columns,
* uses the pretty-table renderer (ADR-0016) for all
diagnostic row lists,
* emits a `[client-side] …` note in the success summary
whenever the transformer rewrote any cell.
`--force-conversion` accepts loss; `--dont-convert` skips
the client-side layer entirely; mutually exclusive.
Forward-look: a future iteration may add resolution flags
(`--default 0`, `--on-incompatible '<value>'`).
Also amends ADR-0002 with a new "User-facing posture"
section cementing that the database engine choice is an
implementation detail and is never named in user-visible
strings. Adds a corresponding bullet to CLAUDE.md's
working-style rules so every session picks it up.
Implementation lands as a follow-up.
Replaces the placeholder pipe-and-dash output with Unicode
box-drawing tables for both data results and table-structure
listings, per ADR-0016.
* New `src/output_render.rs` module with `render_data_table`
and `render_structure`. Hand-rolled to match the project's
existing CSV/YAML pattern; ~300 lines.
* Header-only outer-frame border style: outer ┌─┐│└─┘ box +
├─┤ header underline, no per-row separators. NULL renders
as `(null)`; cell newlines/tabs/control chars become
`↵`/`→`/`·` as display-only substitutions.
* Type-aware column alignment: numeric types right-aligned,
everything else left. `DataResult` gains a `column_types:
Vec<Option<Type>>` field, populated from the existing
metadata lookup at the two query sites in db.rs (no new
query paths).
* Structure view shows Name | Type | Constraints columns;
References / Referenced-by sections retain plain-text
format, leaving room for the future relationship-rendering
ADR.
* 18 new unit tests in output_render.rs (plus 4 insta
snapshots for the canonical layouts). Existing assertions
in app.rs and walking_skeleton.rs updated to match the new
format.
Total: 426 passing, 0 failing, 0 skipped (up from 408).
Clippy clean.
Designs track-2 lifecycle and persistence end-to-end: per-command
write-through to db+yaml+csv+history.log gated by the combined db
persistence logic with commit-db-last ordering; existence-only load
with explicit rebuild command; --resume CLI flag backed by
<data-root>/last_project; in-TUI list-with-browse picker; lock file
for single-instance enforcement; fatal-banner-then-quit failure
model (with --resume making restart cheap); fatal CSV row-load
errors with full diagnosis; YYYYMMDD-word-word-word temp naming
with display-name prettifier; collision-checked names for both
temp and user-supplied projects. Project name lives only on the
filesystem (not duplicated in YAML). ADR-0004 and ADR-0007 amended
in place. requirements.md and CLAUDE.md updated; OOS-6 (global
rolling history) tracked as deferred.
DSL data operations (ADR-0014):
- insert into T [(cols)] values (vals); short form
insert into T (vals) omits values keyword for friendlier
syntax.
- update T set ... where col=val | --all-rows; delete from T
where col=val | --all-rows; show data T.
- Value AST (Number/Text/Bool/Null) with per-column-type
validation in the executor: int/real/decimal/bool/date/
datetime/shortid each accept a documented literal shape
and produce friendly format errors naming the column.
- INSERT short form fills non-auto-generated columns in
schema order; auto-fills serial via SQLite and shortid
via the new generator (T2).
- `add column [to table] T: c (type)` -- `to table` now
optional.
Database:
- insert/update/delete via prepared statements with bound
rusqlite::types::Value parameters.
- InsertResult/UpdateResult/DeleteResult: writes return
rows_affected plus the affected row(s) only (not the whole
table), so users see exactly what changed.
- INSERT shows the just-inserted row via last_insert_rowid.
- UPDATE captures matching rowids up-front and fetches them
post-update -- works even if the UPDATE changed the WHERE
column.
- DELETE reports per-relationship cascade effects by row-
count diffing inbound child tables; UPDATE-side cascades
are not yet detected (would need value diffing).
- query_data formats cells (booleans true/false, NULLs as
None).
FK error enrichment:
- Now lists both outbound (INSERT/UPDATE relevance) and
inbound (DELETE/UPDATE on parent relevance) FKs from the
metadata, so RESTRICT errors point at the children
blocking the delete.
- RelationshipSelector has a proper Display impl -- "no
such relationship" reads cleanly.
Relationship display:
- target_table for AddRelationship/DropRelationship now
returns the parent (1-side); structure rendering after
add/drop shows that side's "Referenced by:" entry,
matching the `from <Parent>` direction of the command.
- [ok] summary uses display_subject so relationship
commands show both endpoints (`from P.col to C.col`)
rather than a single misleading table name.
- Auto-name format `<Parent>_<pcol>_to_<Child>_<ccol>`
(matches the from..to direction).
Output rendering and scrolling:
- Wrap-aware scroll: renderer reports both visible-row
count and total wrapped-row count to App; scroll math
caps against actual displayable rows. Long lines wrap;
the bottom line is always reachable; PageUp/PageDown work
correctly even after paging past the buffer top.
- Multi-line messages (FK error enrichment, cascade summary)
split into single-line OutputLines at creation time so
wrap/scroll math agree.
Runtime / events:
- New AppEvent variants for Insert/Update/Delete success
carrying typed result structs; DslDataSucceeded reserved
for show-data queries.
Docs:
- ADR-0014 covers data-op grammar, value model, --all-rows
safety, auto-show.
- requirements.md: C5 done, T2 done, V2 partial (basic data
view), V5 partial (show data added). New entries: C5a
complex WHERE expressions; H1 progress note for FK
enrichment; H1a (strong syntax-help in parse errors).
Tests: 200 passing (183 lib + 17 integration), 0 skipped.
Includes parser, type-validation, DB write/read, FK-failure
enrichment, cascade-delete propagation, focused-auto-show
behaviour, scroll-cap invariants. Clippy clean with nursery
enabled.
DSL:
- add 1:n relationship [as <name>] from <P>.<col> to <C>.<col>
[on delete <action>] [on update <action>] [--create-fk]
- drop relationship <name> | from <P>.<col> to <C>.<col>
- show table <name> for re-displaying a structure on demand
Database (ADR-0013):
- Rebuild-table primitive following SQLite's
ALTER-via-rebuild recipe (foreign_keys=OFF outside tx,
copy-by-name, foreign_key_check before commit). Reusable for
B2 (column drops/renames/type changes).
- ReferentialAction enum (no action / restrict / set null /
cascade); SET DEFAULT awaits column DEFAULTs.
- __rdbms_playground_relationships metadata table -- names,
auto-generated as <Parent>_<pcol>_to_<Child>_<ccol>.
- Type::fk_target_type() validation at declaration; friendly
errors for type mismatch, non-PK target, missing column,
duplicate name.
- describe_table populates symmetric outbound + inbound
relationship lists. drop_table refuses while inbound
references exist; outbound metadata cleaned up alongside drop.
App / UI:
- In-line cursor editing in the input field: Left, Right,
Home, End, Delete, Backspace honoring UTF-8 boundaries.
- PageUp / PageDown scrolls the output buffer; viewport row
count fed back from the renderer via App::note_output_viewport
so scroll is capped against the actual visible area
(regression-tested) and snaps to the bottom on new output.
- Failure messages quote the command portion ("verb target"
failed: ...) for visual clarity; RelationshipSelector has a
proper Display impl so "no such relationship" reads cleanly.
- Structure rendering shows References / Referenced by sections.
Docs:
- ADR-0013 covers naming, metadata table, symmetric view, and
the rebuild-table strategy.
- requirements.md updates: C3 (FK done), B2 (primitive in),
T3 (compound-PK FK still pending). New entries: I1a (cursor
editing -- landed), I1b (Ctrl-A/E and readline shortcuts --
pending), V4 partial scroll, V5 (show family), C3a (modify
relationship -- deferred).
Tests: 154 passing (140 lib + 14 integration), 0 skipped.
Clippy clean with nursery enabled.
Captures up-front design decisions for RDBMS Playground:
stack (Rust + Ratatui + SQLite), input modes, project file
format, type vocabulary, undo snapshots and replay log,
sharing/export, and testing approach. ADR-0000 establishes
the ADR practice itself and mandates index upkeep alongside
any ADR change.
claude@clouddev1