ADR-0020 implementation: lexer + parser refactor over &[Token]

New `dsl::keyword` module: macro-driven Keyword and Punct
enums (single source of truth — enum, lex-side mapping,
catalog-key derivation generated from one declaration).

New `dsl::lexer` module: tokenizer producing a span-tagged
Vec<Token>. Always succeeds; lex-shape errors (unterminated
string, unrecognised character, malformed flag) embed as
TokenKind::Error tokens so I4 can highlight invalid input
uniformly.

Parser refactored from `Parser<'a, &'a str, ...>` to
`Parser<'a, &'a [Token], ...>`. All 50+ existing parser
unit tests ported and passing; aggregation across `choice`
now works as designed (e.g. `add` → "expected `1` or
`column`", `drop` → "expected `column`, `relationship`, or
`table`", `frobulate Customers` lists all ten command-entry
keywords). Custom `try_map` content errors (unknown type,
mutually-exclusive flags, "with pk needs at least one
column", "specified twice") preserved.

`replay` bare-path UX kept via the source-slice special
case from ADR-0020 §6 (~10 lines, documented inline).

Tests: 650 passing, 0 failing, 1 ignored (610 baseline + 40
new lexer/keyword tests). Clippy clean.
This commit is contained in:
claude@clouddev1
2026-05-10 09:22:13 +00:00
parent 857ee753f2
commit fdaf7e3e0e
4 changed files with 1353 additions and 527 deletions
+2
View File
@@ -11,6 +11,8 @@
pub mod action;
pub mod command;
pub mod keyword;
pub mod lexer;
pub mod parser;
pub mod shortid;
pub mod types;