`create table … with pk` now parses the column-constraint
suffix; combined with the commit-1 db layer, a constrained
table works end to end.
- A shared constraint-suffix grammar fragment — `not null`,
`unique`, `default <literal>` — sits after each column's
`(type)` group; `build_create_table` walks the matched path
per column and folds the constraints into `ColumnSpec`.
- §9 redundancy check: every `with pk` column is a primary-key
column, so `not null` (any) and `unique` (single-column PK)
are rejected with a friendly error
(`parse.custom.constraint_redundant_on_pk`).
- `project.yaml` round-trip: `ColumnSchema` gains `not_null` /
`default`; the YAML reader/writer and `build_read_schema`
carry them, so `rebuild` / `export` / `import` preserve
constraints.
- ADR-0029 §2.1's example corrected — `create table` columns
are all PK columns, so its suffix is for `default` / `check`;
`docs/simple-mode-limitations.md` records that non-PK
columns at create time need advanced mode.
CHECK is deferred to the next commit. 1184 tests pass (+7);
clippy clean.
Expand ColumnSpec and Command::AddColumn with the four
ADR-0029 constraint slots (not_null, unique, default, check),
all defaulting off; `Database::add_column` now takes a
ColumnSpec. No behaviour change — the grammar to set the
fields and the DDL to enforce them land in the following
commits. Isolated here so those commits stay readable.
Adds ColumnSpec::new for the unconstrained case; 110 call
sites updated. 1172 tests pass; clippy clean.
`MatchedKind::Ident` now carries its `IdentSource`. A
post-walk pass over a structurally-valid parse flags a
matched `Tables` ident that is absent from the schema, or a
`Columns` ident absent from the table in scope, as an ERROR
diagnostic — the command parses but would fail at execution
(ADR-0027 §2). New behaviour: an unknown table / column used
to parse cleanly and fail only when run.
Column scope is resolved by one left-to-right pass over the
matched path (every command places its table ident before
the columns that belong to it); an unknown table clears the
scope, so its columns are not cascaded into a second
diagnostic. New catalog keys `diagnostic.unknown_table` /
`diagnostic.unknown_column`.
`create table … with pk` parsed column types as `name:type`,
while `add column` uses `name(type)`. Unify on the parens
form so column-type syntax is consistent across the DSL:
create table T with pk id(serial), name(text)
Only `COL_SPEC` changes (`:` → `( … )`); `build_create_table`
reads columns by role, so it is unaffected. The `:` that
separates table from column in `add column` / `drop column`
is unchanged. Sweeps the test suite, the typing-surface
matrix (two `after_colon` cells renamed to `after_paren`,
4 snapshots regenerated), the friendly catalog's usage
templates, ADR-0009's example, and requirements.md.
1039 passing / 0 failing / 1 ignored; clippy clean.
Implement ADR-0025 — indexes as a DSL DDL feature.
- Grammar: `add index [as <name>] on <T> (<cols>)`, `drop index
<name>` / `drop index on <T> (<cols>)`, plus a `--cascade`
flag on `drop column`.
- db.rs: index operations over the engine's native index
catalog (no metadata table). The rebuild-table primitive now
captures and recreates indexes, so `change column` and the
relationship operations no longer silently drop them.
- `drop column` refuses an indexed column unless `--cascade`,
which drops the covering indexes and reports each.
- Persistence: additive `indexes:` list in `project.yaml`
(version unchanged); round-trips through rebuild/export/import.
- Display: an `Indexes:` section in the structure view and a
nested tables/indexes items panel (S2).
Reconciles requirements.md (C3 index portion, S2 satisfied)
and CLAUDE.md. 1038 tests passing (+31), clippy clean.
HintMode became per-node (Node::Hinted) in the node-attached refactor;
the per-command hint_mode field was never the mechanism and is now
read by nothing. Removed the field and its 20 `None` initialisers.
Replaces the hint resolver's signature-matching (does the expected set
contain all five literal forms? an Ident{NewName}?) with a grammar-
declared annotation. New Node::Hinted { mode, inner } wrapper; the
walker records the mode in WalkContext::pending_hint_mode on entry and
clears it on any successful match (cursor moved past the slot — this
also undoes the leak where a failed Hinted branch of a Choice would
otherwise strand a stale mode). The resolver reads pending_hint_mode
directly.
Value-literal fallback slots carry ProseOnly; NewName ident slots carry
ForceProse. hint_mode_at_input_inner now delegates to
hint_resolution_at_input — one resolution path, no duplicated logic.
No behaviour change; the typing-surface matrix guards it.
Handoff-12 §2.2: the DDL TABLE_NAME_EXISTING slot and the
relationship-endpoint table idents didn't set writes_table, so
column-name slots downstream (drop/rename/change column; relationship
qualified columns) couldn't narrow to the active table — candidates
leaked from every table. Set writes_table: true on TABLE_NAME_EXISTING
and on DR_PARENT/DR_CHILD/AR_PARENT/AR_CHILD table idents. The
deliberately-documenting completion test now asserts per-table
narrowing.
Bug: hint at \`insert into Customers values (\` for a Customers
table with id:serial PK suggested typing an integer for \`id\`,
but the dispatch path (\`db::do_insert\`) deliberately doesn't
accept user-supplied values for auto-generated columns in
Form B. The grammar prompted for a value the dispatch would
refuse.
The fix aligns Phase D's \`column_value_list\` dynamic sub-grammar
with do_insert's three forms (ADR-0014 + ADR-0018 §3):
- **Form A** \`insert into <T> (col1, col2, …) values (…)\` —
user explicitly lists columns. Slot list mirrors that
selection; serial / shortid columns CAN appear if the user
lists them.
- **Form B** \`insert into <T> values (…)\` — bare values. Slot
list = non-auto-generated columns of the table in
declaration order. Serial / shortid get auto-filled by the
dispatch; the grammar doesn't prompt for them.
- **Form C** \`insert into <T> (v1, v2, …)\` — bare value list.
Not affected by this change (column_value_list isn't on this
path; Form C's literals route through the schemaless
INSERT_PAREN_LIST).
Implementation:
\`WalkContext.user_listed_columns: Option<Vec<String>>\` — when
\`Some\`, signals Form A; \`None\` is Form B. Populated by walking
the first paren's column-list idents.
\`Node::Ident.writes_user_listed_column: bool\` — new field;
\`true\` on the INSERT_PAREN_ITEM's Ident child. When the
walker matches that ident in Form A, it appends the
schema-canonical column name (case-corrected against the
schema) to user_listed_columns.
\`column_value_list\` factory:
- If user_listed_columns is Some → resolve each name from the
schema; one typed slot per listed column.
- Else → filter current_table_columns to non-auto-generated;
one typed slot per remaining column.
- Empty result → fall back to the schemaless value-literal
list (a serial-only table in Form B has nothing for the
user to type).
Tests:
- New \`phase_d_insert_form_b_skips_serial_column\` confirms the
bug: \`insert into Customers values (1, 'Alice')\` against a
Customers with serial id rejects at parse time (Form B
expects 1 value for Name, not 2).
- New \`phase_d_insert_form_a_accepts_serial_when_listed\`
confirms \`insert into Customers (id, Name) values (1, 'Alice')\`
works.
- New \`phase_d_insert_form_a_filters_to_user_listed_columns\`
confirms partial Form A (\`(Name) values ('Alice')\`).
- Updated \`phase_d_insert_with_schema_accepts_typed_values_per_column\`
to match the new Form B contract (2 user-typed values, not 3).
- Updated typed-hint test matrix split into form-B (8 types)
and form-A (serial / shortid).
- New \`typed_hint_form_b_skips_serial_column_to_generic_or_text_neighbor\`
pins the fallback behavior for a serial-only table.
For the user: \`insert into Customers values (\` for a Customers
with \`(id:serial, Name:text, Email:text)\` now hints
\`for \`Name\`: Type a quoted string …\` (skipping id entirely)
and accepts exactly 2 values. To set the serial explicitly,
use Form A: \`insert into Customers (id, Name, Email) values
(1, 'Alice', 'a@b.c')\`.
Tests: 851 passing, 0 failing, 1 ignored. Clippy clean.
Schema-aware typed value slots — the central design claim of
ADR-0024 §Phase D. Insert / update / delete value slots now
dispatch on the user-facing column type at parse time, rejecting
mis-shaped input with localised wording instead of waiting for
the bind-time error.
What changed:
**SchemaCache extension** (`src/completion.rs`):
- New `TableColumn { name, user_type }` for per-table column
metadata.
- `SchemaCache.table_columns: HashMap<String, Vec<TableColumn>>`.
- `SchemaCache::columns_for_table(name)` — case-insensitive
lookup, mirrors the walker's case-insensitive entry-word
resolution.
**WalkContext schema plumbing** (`src/dsl/walker/context.rs`):
- `WalkContext<'a>` gains a lifetime and a `schema: Option<&'a
SchemaCache>`. `WalkContext::new()` keeps the schemaless
default; `with_schema(s)` is the new schema-aware constructor.
**Parser entry point** (`src/dsl/parser.rs`):
- `parse_command_with_schema(input, schema)` is the new public
schema-aware variant. `parse_command(input)` becomes a thin
wrapper that delegates with `None` for back-compat.
- Internal `try_walker_route` accepts an `Option<&SchemaCache>`
and threads it into the WalkContext.
**Node::Ident writes_table/writes_column** (`src/dsl/grammar/mod.rs`):
- Two new fields on `Node::Ident`. When `writes_table: true` and
`source: Tables`, the walker writes the matched ident's name
into `current_table` and resolves `current_table_columns`
against the schema cache. When `writes_column: true` and
`source: Columns`, the walker writes the resolved
`TableColumn` into `current_column`.
**Walker driver DynamicSubgrammar dispatch** (`src/dsl/walker/driver.rs`):
- The `Node::DynamicSubgrammar(factory)` branch now resolves the
factory at walk time and `Box::leak`s the result so its inner
static-slice fields (Choice/Seq) have the lifetime the walker
expects (per ADR-0024 §sub-grammars). The leak is bounded by
command-shape complexity per walk; per-walk arena is a future
optimisation.
- `walk_ident` extends to perform the schema writes when the
flags are set.
**Typed value slot factories + dynamic sub-grammars** (`src/dsl/grammar/shared.rs`):
- `int_slot` / `real_slot` / `decimal_slot` / `bool_slot` /
`text_slot` / `date_slot` / `datetime_slot` / `blob_slot` —
one per `Type`. Each accepts the appropriate literal kind plus
`null`; integer-only validator rejects `3.14` at int columns;
decimal validator pins numeric shape.
- `slot_for_type(ty) -> Node` is the dispatcher.
- `current_column_value(ctx) -> Node` is the dynamic sub-grammar
for `set col = …` and `where col = …` values; reads
`current_column` and dispatches via `slot_for_type`.
- `column_value_list(ctx) -> Node` is the dynamic sub-grammar
for `insert into T values (…)`; reads `current_table_columns`
and unfolds a Seq of typed slots separated by commas.
- Both fall back to the schemaless `VALUE_LITERAL` choice when
the context lacks the schema-resolved entries — keeps
schemaless `parse_command` callers (tests, replay path)
working.
**Data-command grammar wires the new types** (`src/dsl/grammar/data.rs`):
- `TABLE_NAME_INSERT` / `TABLE_NAME_WRITES` (new): table-name
slots that set `writes_table: true`. Used by insert / update /
delete to populate `current_table_columns`.
- `SET_COLUMN` / `FILTER_COLUMN` (new): column-name slots in
`set col=…` / `where col=…` set `writes_column: true`.
- `INSERT_VALUES_LIST` becomes `DynamicSubgrammar(column_value_list)`.
- `UPDATE_ASSIGNMENT` and `WHERE_CLAUSE` use
`PER_COLUMN_VALUE = DynamicSubgrammar(current_column_value)`.
**Runtime plumbs schema-with-types** (`src/runtime.rs`):
- `refresh_schema_cache` calls `describe_table` for each table
and populates `SchemaCache::table_columns` with
`TableColumn { name, user_type }` entries. Best-effort: a
`describe_table` miss leaves that table unpopulated and the
walker falls back to schemaless dispatch.
**App dispatches with schema** (`src/app.rs`):
- `dispatch_dsl` routes through `parse_command_with_schema(&self
.schema_cache, …)` so live typing/dispatch sees the typed
slots. The replay path stays schemaless (deferred — replay
bind-time errors still catch type mismatches).
**Catalog** (`src/friendly/strings/en-US.yaml`, `src/friendly/keys.rs`):
- New `parse.custom.bind_type_mismatch` entry with `{found}` and
`{expected}` placeholders. Surfaced by the int_slot /
decimal_slot validators.
Tests:
- 11 new walker-side Phase D tests cover insert / update /
delete with schemas — typed acceptance per column, decimal
rejection at int columns, null acceptance at any slot,
multi-assignment per-column dispatch, schemaless fallback.
- The pre-existing `parse_command(input)` test suite (no
schema) still passes — the fallback path is behaviour-
preserving.
- 828 passing total, 0 failing, 1 ignored. Clippy clean.
Migrates parse-error usage-block rendering from the legacy
`dsl::usage::matched_entry` (which scanned a `Vec<Token>` for the
first matched Keyword) to walker-side lookup driven by each
`CommandNode`'s `usage_ids` slice.
`CommandNode.usage_id: Option<&'static str>` becomes
`usage_ids: &'static [&'static str]`. Multi-form families
(`drop`, `add`, `show`) carry every variant — `drop` lists
table/column/relationship templates; `add` lists column /
relationship; `show` lists data / table. The single-shape
commands carry their single catalog key.
App-lifecycle CommandNodes had pointed at non-existent
`parse.usage.app.*` keys (never noticed because the field was
unused); they now point at the real catalog entries
(`parse.usage.quit`, `parse.usage.help`, …).
New helpers in `dsl::grammar`:
- `usage_keys_for_input(source) -> Option<(entry_word, usage_ids)>`
resolves the first identifier-shape token to a CommandNode and
returns its usage_ids list. Used by `app::render_usage_block`
and `input_render::ambient_hint`.
- `entry_words_alphabetised() -> Vec<&'static str>` replaces
`dsl::usage::entry_keywords_alphabetised`.
`dsl::usage` is deleted. The "available commands:" fallback in
`render_usage_block` now formats entry words as `` `<word>` ``
directly (matching the `parse.token.keyword.*` catalog renders);
the per-keyword catalog wrappers will collapse in the next step
(ADR-0024 §cleanup-pass §F).
`parse_command` and `parse_tokens` slim down:
- `parse_command(input)` no longer pre-lexes — the walker scans
source bytes directly.
- `parse_tokens` (internal-only `pub` for "future I3/I4 work")
is removed; its body folded into `parse_command`.
- `unknown_command_error` reads the walker registry directly.
Touched modules also drop their `crate::dsl::lexer::lex` and
`crate::dsl::usage` imports: `app.rs`, `input_render.rs`,
`completion.rs`.
Tests: 852 passing, 0 failing, 1 ignored (down from 860 because
the 8 `dsl::usage::tests::*` tests are gone with the module).
Migrate `create table <Name> [with pk [<col>:<type>[, ...]]]`
to the walker. Exercises Repeated{separator: Some(Punct(','))}
for the first time — the with-pk column-spec list.
Walker behaviour changes:
- Optional now backtracks on partial-match failure (Incomplete
or Failed-Mismatch from a Seq mid-shape). Path / per-byte
state rolls back to before the partial attempt; the inner's
expected-set propagates as `skipped` so callers see "what
would have completed it". Matches chumsky's `or_not`
semantics. ValidationFailed (content errors) does NOT
backtrack — the user means to fix those.
- Bridge: ValidationFailed errors now classify as
`at_eof = true`, mirroring the chumsky-side custom-error
convention. This is what lets `create table Customers`
classify as IncompleteAtEof rather than DefiniteErrorAt
(the user can still continue typing `with pk …`).
Grammar:
- src/dsl/grammar/ddl.rs gains CREATE: shape is
Seq(Word("table"), Ident{NewName,table_name}, Optional(WITH_PK))
where WITH_PK = Seq(Word("with"), Word("pk"),
Optional(Repeated{COL_SPEC, separator: Punct(','), min:1})).
AST builder enforces `with pk needs at least one column`
with the existing parse.custom.create_table_needs_pk catalog
wording; `with pk` alone defaults to id:serial.
Tests:
- 6 new walker-specific tests for create_table: with-pk
default, named typed PK, compound PK, whitespace tolerance
around `:` and `,`, bare-create-table-errors-with-with-pk-
hint, case-insensitive keywords.
- Total: 825 passed, 0 failed, 1 ignored (was 819 / 1).
- cargo clippy --all-targets -- -D warnings clean.
Migrate the five DDL commands at four entry words: drop (drop
table / drop column / drop relationship), add (add column /
add 1:n relationship), rename (rename column), change (change
column). The walker route now owns these end-to-end; chumsky
declarations remain unreachable for these inputs but stay
until Phase F.
Walker extensions:
- New node kinds: NumberLit (with optional content validator)
and Literal(&str) (verbatim byte sequence with word-boundary
lookahead — used for the `1` in `add 1:n …` so it surfaces
as `\`1\`` in the expected-set, matching the existing
parse_error_pedagogy contract).
- Flag (--name) terminal — Phase A stubbed; now wired to the
walker driver with consume_flag() in lex_helpers.
- Repeated combinator with optional separator and `min` floor.
Used by referential clauses (0..2 `on <delete|update>` runs)
and change-column flags (0..N --force-conversion /
--dont-convert; AST builder enforces mutual exclusion).
- Optional now propagates its inner's expectations as a
`skipped` field on the Matched result. Seq accumulates these
across children so the next failure's expected-set surfaces
the full union — closes the keyword-completion regression
(`add column ` must offer `to`, `table`, plus the table-name
identifier slot).
- Expectation::Ident gained a `source: IdentSource` field; the
parser-side bridge maps Tables/Columns/Relationships/Types
to the IdentSlot::expected_label strings ("table name",
"column name", …) so the existing completion engine's
schema-cache lookup still resolves.
- Walker error wording now includes "after `<consumed>`,
expected …" framing — matches the chumsky-side test
contract for structural errors mid-shape.
- AST-builder validation errors now propagate as
WalkOutcome::ValidationFailed (not the generic "AST builder
failed" fallback), so `change column … --force-conversion
--dont-convert` and repeated `on delete` clauses surface
their friendly catalog wording verbatim.
Grammar additions:
- src/dsl/grammar/shared.rs: type-name validator (TYPE_VALIDATOR
uses Type::from_str via parse.custom.unknown_type catalog),
qualified_column sub-grammar, referential action keyword
(`cascade`/`restrict`/`set null`/`no action`), repeated
on-clauses.
- src/dsl/grammar/ddl.rs: drop/add/rename/change CommandNodes
with inline shapes (per-use-site `role` annotations let the
AST builder discriminate parent vs child columns, etc.).
The four entry words each have one CommandNode whose `shape`
is a Choice across sub-forms.
Tests:
- 14 new walker-specific tests covering all DDL forms (bare
drop table, drop column with optional connectives, drop
relationship by name and by endpoints, add column with type
validator, rename column, change column with each flag form
+ mutual-exclusion check, add 1:n relationship minimal /
full, repeated-clause-twice rejection).
- Total: 819 passed, 0 failed, 1 ignored (was 805 / 1).
- cargo clippy --all-targets -- -D warnings clean.
claude@clouddev1