Adds `Severity` (Error / Warning, ordered so Error > Warning)
and `Diagnostic { severity, span, message }` in
`walker::outcome`, plus a `diagnostics` field on `WalkResult`
— the schema-aware findings layered on a structurally-valid
parse (ADR-0027 §2).
`input_verdict(source, schema)` is the validity-indicator
entry point: `None` when the input would run clean (and for
empty input), `Some(Error)` for a parse failure or unknown
command, `Some(Warning)` for the ADR-0026 expression flags.
The verdict is the highest severity across the parse outcome
and the diagnostics set.
`diagnostics` is empty at this step — the schema-existence
(ERROR) and expression (WARNING) passes that fill it land
next. Covered by `input_verdict` unit tests.
Pre-Phase-D, `save ` parsed as a complete `save` command, so
the completion engine had nothing to mine: `as` never surfaced
as a Tab candidate. This is the round-5 gap the handoffs have
been tracking.
`WalkResult` gains a `tail_expected: Vec<Expectation>` field.
The walker's top-level `Matched` branch copies the outer
shape's skipped-Optional expectations into it. `expected_at_input`
returns `tail_expected` on `Match` so the completion engine
sees the optional-suffix continuations and offers them as Tab
candidates.
`hint_mode_at_input` deliberately does NOT consume
`tail_expected` — surfacing prose like "Type a name" at the
end of a valid command would be misleading. A new private
`expected_for_hint` returns empty on `Match` to preserve this.
The split distinguishes "valid + could continue" (completion
helps) from "invalid + must continue" (hint resolver helps).
Tests:
- `save ` Tab → `as` (new test, the original round-5 gap).
- `messages ` Tab → `short` and `verbose` (same shape).
- Existing `hint_mode_none_for_complete_command` stays green
because hint resolver ignores tail_expected.
- 830 total passing, 0 failing, 1 ignored. Clippy clean.
Replaces the lex()-driven `base_runs` span builder in
`input_render.rs` with `walker::highlight_runs`. The new
walker-side `dsl::walker::highlight` module returns per-byte
`HighlightClass` assignments for every token shape in the source:
- For commands the walker engages on, `WalkResult::per_byte_class`
is the authoritative source (keyword / identifier / number /
string / punct / flag).
- Trailing junk past a partial match — and inputs the walker
doesn't engage on at all (no registered entry word) — fall
through to a byte-shape scanner over `lex_helpers` so unknown
command words, stray punctuation, and unterminated strings
still highlight sensibly.
`Theme::highlight_class_color` is the walker-side analogue of
`token_color(&TokenKind)`; the renderer reads `walker::highlight_runs`
output and looks up colours through it. `token_color` and the
`lex()` pre-pass remain in place for now — the lexer module is
still consumed by usage rendering and completion until the
remaining Phase F steps land.
`HighlightClass`'s and `WalkResult::per_byte_class`'s
`#[allow(dead_code)]` annotations come off — they're now part of
the production highlight path.
Tests:
- 16 new tests under `dsl::walker::highlight` cover end-to-end
walks, byte-shape fallbacks (unknown commands, bare flags,
numbers, punctuation), UTF-8 codepoint advance, and trailing-
token handling after partial walks.
- Existing `input_render` tests pass unchanged.
- 860 total tests passing (727 lib + 133 integration), 1 ignored.
Clippy clean with `nursery` lints + `-D warnings`.
Migrate the five DDL commands at four entry words: drop (drop
table / drop column / drop relationship), add (add column /
add 1:n relationship), rename (rename column), change (change
column). The walker route now owns these end-to-end; chumsky
declarations remain unreachable for these inputs but stay
until Phase F.
Walker extensions:
- New node kinds: NumberLit (with optional content validator)
and Literal(&str) (verbatim byte sequence with word-boundary
lookahead — used for the `1` in `add 1:n …` so it surfaces
as `\`1\`` in the expected-set, matching the existing
parse_error_pedagogy contract).
- Flag (--name) terminal — Phase A stubbed; now wired to the
walker driver with consume_flag() in lex_helpers.
- Repeated combinator with optional separator and `min` floor.
Used by referential clauses (0..2 `on <delete|update>` runs)
and change-column flags (0..N --force-conversion /
--dont-convert; AST builder enforces mutual exclusion).
- Optional now propagates its inner's expectations as a
`skipped` field on the Matched result. Seq accumulates these
across children so the next failure's expected-set surfaces
the full union — closes the keyword-completion regression
(`add column ` must offer `to`, `table`, plus the table-name
identifier slot).
- Expectation::Ident gained a `source: IdentSource` field; the
parser-side bridge maps Tables/Columns/Relationships/Types
to the IdentSlot::expected_label strings ("table name",
"column name", …) so the existing completion engine's
schema-cache lookup still resolves.
- Walker error wording now includes "after `<consumed>`,
expected …" framing — matches the chumsky-side test
contract for structural errors mid-shape.
- AST-builder validation errors now propagate as
WalkOutcome::ValidationFailed (not the generic "AST builder
failed" fallback), so `change column … --force-conversion
--dont-convert` and repeated `on delete` clauses surface
their friendly catalog wording verbatim.
Grammar additions:
- src/dsl/grammar/shared.rs: type-name validator (TYPE_VALIDATOR
uses Type::from_str via parse.custom.unknown_type catalog),
qualified_column sub-grammar, referential action keyword
(`cascade`/`restrict`/`set null`/`no action`), repeated
on-clauses.
- src/dsl/grammar/ddl.rs: drop/add/rename/change CommandNodes
with inline shapes (per-use-site `role` annotations let the
AST builder discriminate parent vs child columns, etc.).
The four entry words each have one CommandNode whose `shape`
is a Choice across sub-forms.
Tests:
- 14 new walker-specific tests covering all DDL forms (bare
drop table, drop column with optional connectives, drop
relationship by name and by endpoints, add column with type
validator, rename column, change column with each flag form
+ mutual-exclusion check, add 1:n relationship minimal /
full, repeated-clause-twice rejection).
- Total: 819 passed, 0 failed, 1 ignored (was 805 / 1).
- cargo clippy --all-targets -- -D warnings clean.
Stand up the unified-grammar tree walker alongside the existing
chumsky parser and migrate the eleven app-lifecycle commands
(quit, help, rebuild, save / save as, new, load, export, import,
mode, messages) end-to-end. The router in parse_tokens consults
the walker first; non-migrated commands still fall through to
chumsky.
Scope:
- src/dsl/grammar/{mod,app}.rs: Node enum (13 kinds), Word /
IdentSource / HintMode / HighlightClass / ValidationError /
CommandNode types, REGISTRY of the eleven app commands.
- src/dsl/walker/{mod,driver,context,outcome,lex_helpers}.rs:
scannerless byte-level walker, per-node-kind dispatch with
Choice/Seq/Optional backtracking, WalkContext (Phase B-D
schema fields stubbed), WalkOutcome with Match/Incomplete/
Mismatch/ValidationFailed.
- src/dsl/parser.rs: try_walker_route() runs first in
parse_tokens; bridge converts WalkOutcome to ParseError
preserving catalog wording (mode.unknown / messages.unknown
surface verbatim via friendly::translate). Legacy
try_parse_app_path_command deleted; chumsky's bare-keyword
app branches remain unreachable until Phase F sweep.
Walker design choices worth noting:
- mode <value> / messages <value> use Choice(Word, Word, Ident)
so known keywords appear in the expected-set; the trailing
Ident catch-all funnels unknown values into the friendly
validator that always errors with the catalog wording.
- save / save as is one CommandNode (Optional(Word("as"))) -
closes the round-5 "save Tab can't offer as" limitation
structurally.
- Path-bearing UX shipped per ADR-0024: BarePath terminates at
whitespace; paths with spaces use the (not-yet-wired) quoted
form. Existing tests pass on the new shape.
Tests:
- 28 new walker-specific tests in dsl::walker::tests covering
every app-lifecycle command, friendly-error wording for
mode/messages unknown values, trailing-garbage detection,
whitespace tolerance, and routing fall-through.
- Total: 805 passed, 0 failed, 1 ignored (was 777 / 1).
- cargo clippy --all-targets -- -D warnings clean.
claude@clouddev1