ADR-0024 Phase A: walker framework + app-lifecycle commands
Stand up the unified-grammar tree walker alongside the existing
chumsky parser and migrate the eleven app-lifecycle commands
(quit, help, rebuild, save / save as, new, load, export, import,
mode, messages) end-to-end. The router in parse_tokens consults
the walker first; non-migrated commands still fall through to
chumsky.
Scope:
- src/dsl/grammar/{mod,app}.rs: Node enum (13 kinds), Word /
IdentSource / HintMode / HighlightClass / ValidationError /
CommandNode types, REGISTRY of the eleven app commands.
- src/dsl/walker/{mod,driver,context,outcome,lex_helpers}.rs:
scannerless byte-level walker, per-node-kind dispatch with
Choice/Seq/Optional backtracking, WalkContext (Phase B-D
schema fields stubbed), WalkOutcome with Match/Incomplete/
Mismatch/ValidationFailed.
- src/dsl/parser.rs: try_walker_route() runs first in
parse_tokens; bridge converts WalkOutcome to ParseError
preserving catalog wording (mode.unknown / messages.unknown
surface verbatim via friendly::translate). Legacy
try_parse_app_path_command deleted; chumsky's bare-keyword
app branches remain unreachable until Phase F sweep.
Walker design choices worth noting:
- mode <value> / messages <value> use Choice(Word, Word, Ident)
so known keywords appear in the expected-set; the trailing
Ident catch-all funnels unknown values into the friendly
validator that always errors with the catalog wording.
- save / save as is one CommandNode (Optional(Word("as"))) -
closes the round-5 "save Tab can't offer as" limitation
structurally.
- Path-bearing UX shipped per ADR-0024: BarePath terminates at
whitespace; paths with spaces use the (not-yet-wired) quoted
form. Existing tests pass on the new shape.
Tests:
- 28 new walker-specific tests in dsl::walker::tests covering
every app-lifecycle command, friendly-error wording for
mode/messages unknown values, trailing-garbage detection,
whitespace tolerance, and routing fall-through.
- Total: 805 passed, 0 failed, 1 ignored (was 777 / 1).
- cargo clippy --all-targets -- -D warnings clean.
This commit is contained in:
claude@clouddev1
@@ -0,0 +1,43 @@
|
||||
//! `WalkContext` — per-walk mutable state that flows through the
|
||||
//! walker (ADR-0024 §WalkContext).
|
||||
//!
|
||||
//! Phase A keeps this minimal: app-lifecycle commands have no
|
||||
//! schema dependency. The `current_table`, `current_table_columns`,
|
||||
//! and schema-cache pointer become populated as Phase B-D land
|
||||
//! the schema-aware DDL/data commands.
|
||||
|
||||
/// Per-walk state. Cheap to construct; `default()` is the right
|
||||
/// shape for app-lifecycle commands.
|
||||
#[derive(Debug, Default)]
|
||||
pub struct WalkContext {
|
||||
/// Table whose name an `Ident { source: Tables, writes_table:
|
||||
/// true }` matched earlier in the walk. Phase B+ writes this.
|
||||
pub current_table: Option<String>,
|
||||
|
||||
/// Columns of `current_table`, resolved against the schema
|
||||
/// cache when the table identifier matched. Phase D+ uses
|
||||
/// this to drive the dynamic `column_value_list` sub-grammar.
|
||||
#[allow(dead_code)]
|
||||
pub current_table_columns: Option<Vec<ColumnInfo>>,
|
||||
|
||||
/// For `set col=…` and `where col=…`, the column whose value
|
||||
/// is about to be consumed. Phase D+ writes this so the value
|
||||
/// slot picks the right typed sub-grammar.
|
||||
#[allow(dead_code)]
|
||||
pub current_column: Option<ColumnInfo>,
|
||||
}
|
||||
|
||||
impl WalkContext {
|
||||
pub fn new() -> Self {
|
||||
Self::default()
|
||||
}
|
||||
}
|
||||
|
||||
/// Schema info for a single column. Phase D+ populates this from
|
||||
/// the schema cache; Phase A leaves it unused.
|
||||
#[derive(Debug, Clone)]
|
||||
#[allow(dead_code)]
|
||||
pub struct ColumnInfo {
|
||||
pub name: String,
|
||||
pub user_type: crate::dsl::types::Type,
|
||||
}
|
||||
@@ -0,0 +1,330 @@
|
||||
//! Per-node-kind walk dispatch (ADR-0024 §architecture).
|
||||
//!
|
||||
//! `walk_node` is the recursive workhorse that the public
|
||||
//! `walk()` entry calls into for a `CommandNode`'s `shape`. It
|
||||
//! tries to match `node` starting at `position`, mutating
|
||||
//! `path` (matched terminals collected in declaration order) and
|
||||
//! `per_byte` (highlight class assignments) as it goes.
|
||||
//!
|
||||
//! The return value distinguishes four cases:
|
||||
//!
|
||||
//! - `Matched { end }` — full match, walker consumed up to `end`.
|
||||
//! - `NoMatch { … }` — node didn't engage at this position. For
|
||||
//! `Optional` and `Choice` callers this is benign (try the
|
||||
//! next branch / skip the optional); for `Seq` it's only
|
||||
//! benign on the first child.
|
||||
//! - `Incomplete { … }` — node committed (consumed at least one
|
||||
//! terminal) but ran out of input. Surfaces as
|
||||
//! `WalkOutcome::Incomplete` at the top level.
|
||||
//! - `Failed { … }` — node committed and a content validator
|
||||
//! rejected the value, or a hard structural failure occurred
|
||||
//! mid-shape. Surfaces as `WalkOutcome::Mismatch` or
|
||||
//! `WalkOutcome::ValidationFailed` at the top level.
|
||||
|
||||
use crate::dsl::grammar::{HighlightClass, Node, ValidationError};
|
||||
use crate::dsl::walker::context::WalkContext;
|
||||
use crate::dsl::walker::lex_helpers::{consume_bare_path, consume_ident, skip_whitespace};
|
||||
use crate::dsl::walker::outcome::{
|
||||
ByteClass, Expectation, MatchedItem, MatchedKind, MatchedPath,
|
||||
};
|
||||
|
||||
#[derive(Debug, Clone)]
|
||||
pub enum NodeWalkResult {
|
||||
Matched {
|
||||
end: usize,
|
||||
},
|
||||
/// Did not engage at this position. Caller decides whether
|
||||
/// this is benign (Optional, Choice fallthrough) or a hard
|
||||
/// failure (Seq mid-shape).
|
||||
NoMatch {
|
||||
position: usize,
|
||||
expected: Vec<Expectation>,
|
||||
},
|
||||
/// Committed and ran out of input.
|
||||
Incomplete {
|
||||
position: usize,
|
||||
expected: Vec<Expectation>,
|
||||
},
|
||||
/// Committed and hit a hard mismatch or validator failure.
|
||||
Failed {
|
||||
position: usize,
|
||||
kind: FailureKind,
|
||||
},
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone)]
|
||||
pub enum FailureKind {
|
||||
Mismatch { expected: Vec<Expectation> },
|
||||
Validation(ValidationError),
|
||||
}
|
||||
|
||||
pub fn walk_node(
|
||||
source: &str,
|
||||
position: usize,
|
||||
node: &Node,
|
||||
ctx: &mut WalkContext,
|
||||
path: &mut MatchedPath,
|
||||
per_byte: &mut Vec<ByteClass>,
|
||||
) -> NodeWalkResult {
|
||||
let pos = skip_whitespace(source, position);
|
||||
match node {
|
||||
Node::Word(word) => walk_word(source, pos, word, path, per_byte),
|
||||
Node::Punct(ch) => walk_punct(source, pos, *ch, path, per_byte),
|
||||
Node::Ident {
|
||||
source: src,
|
||||
role,
|
||||
validator,
|
||||
highlight_override: _,
|
||||
} => walk_ident(source, pos, *src, role, *validator, path, per_byte),
|
||||
Node::NumberLit
|
||||
| Node::StringLit
|
||||
| Node::BlobLit
|
||||
| Node::Flag(_)
|
||||
| Node::Repeated { .. }
|
||||
| Node::DynamicSubgrammar(_) => {
|
||||
// Phase A: not exercised by app-lifecycle commands.
|
||||
// Reaching this branch means a Phase B+ grammar got
|
||||
// declared without the walker support landing yet —
|
||||
// surface as a hard failure so the test suite catches
|
||||
// it loudly instead of silently mis-parsing.
|
||||
NodeWalkResult::Failed {
|
||||
position: pos,
|
||||
kind: FailureKind::Mismatch { expected: vec![] },
|
||||
}
|
||||
}
|
||||
Node::BarePath => walk_bare_path(source, pos, path, per_byte),
|
||||
Node::Choice(children) => walk_choice(source, pos, children, ctx, path, per_byte),
|
||||
Node::Seq(children) => walk_seq(source, pos, children, ctx, path, per_byte),
|
||||
Node::Optional(child) => walk_optional(source, pos, child, ctx, path, per_byte),
|
||||
}
|
||||
}
|
||||
|
||||
fn walk_word(
|
||||
source: &str,
|
||||
position: usize,
|
||||
word: &crate::dsl::grammar::Word,
|
||||
path: &mut MatchedPath,
|
||||
per_byte: &mut Vec<ByteClass>,
|
||||
) -> NodeWalkResult {
|
||||
// First scan an identifier-shape token at `position`; if
|
||||
// none, we definitely don't have this keyword. If one, check
|
||||
// it against the word's primary + aliases.
|
||||
let Some((start, end)) = consume_ident(source, position) else {
|
||||
return NodeWalkResult::NoMatch {
|
||||
position,
|
||||
expected: vec![Expectation::Word(word.primary)],
|
||||
};
|
||||
};
|
||||
let candidate = &source[start..end];
|
||||
if word.matches(candidate) {
|
||||
path.push(MatchedItem {
|
||||
kind: MatchedKind::Word(word.primary),
|
||||
text: candidate.to_string(),
|
||||
span: (start, end),
|
||||
});
|
||||
per_byte.push(ByteClass {
|
||||
start,
|
||||
end,
|
||||
class: HighlightClass::Keyword,
|
||||
});
|
||||
NodeWalkResult::Matched { end }
|
||||
} else {
|
||||
NodeWalkResult::NoMatch {
|
||||
position,
|
||||
expected: vec![Expectation::Word(word.primary)],
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fn walk_punct(
|
||||
source: &str,
|
||||
position: usize,
|
||||
ch: char,
|
||||
path: &mut MatchedPath,
|
||||
per_byte: &mut Vec<ByteClass>,
|
||||
) -> NodeWalkResult {
|
||||
let bytes = source.as_bytes();
|
||||
if position < bytes.len() && bytes[position] == ch as u8 {
|
||||
path.push(MatchedItem {
|
||||
kind: MatchedKind::Punct(ch),
|
||||
text: ch.to_string(),
|
||||
span: (position, position + 1),
|
||||
});
|
||||
per_byte.push(ByteClass {
|
||||
start: position,
|
||||
end: position + 1,
|
||||
class: HighlightClass::Punct,
|
||||
});
|
||||
NodeWalkResult::Matched {
|
||||
end: position + 1,
|
||||
}
|
||||
} else {
|
||||
NodeWalkResult::NoMatch {
|
||||
position,
|
||||
expected: vec![Expectation::Punct(ch)],
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fn walk_ident(
|
||||
source: &str,
|
||||
position: usize,
|
||||
_src: crate::dsl::grammar::IdentSource,
|
||||
role: &'static str,
|
||||
validator: Option<crate::dsl::grammar::IdentValidator>,
|
||||
path: &mut MatchedPath,
|
||||
per_byte: &mut Vec<ByteClass>,
|
||||
) -> NodeWalkResult {
|
||||
let Some((start, end)) = consume_ident(source, position) else {
|
||||
return NodeWalkResult::NoMatch {
|
||||
position,
|
||||
expected: vec![Expectation::Ident { role }],
|
||||
};
|
||||
};
|
||||
let text = source[start..end].to_string();
|
||||
if let Some(v) = validator
|
||||
&& let Err(err) = v(&text)
|
||||
{
|
||||
return NodeWalkResult::Failed {
|
||||
position: start,
|
||||
kind: FailureKind::Validation(err),
|
||||
};
|
||||
}
|
||||
path.push(MatchedItem {
|
||||
kind: MatchedKind::Ident { role },
|
||||
text,
|
||||
span: (start, end),
|
||||
});
|
||||
per_byte.push(ByteClass {
|
||||
start,
|
||||
end,
|
||||
class: HighlightClass::Identifier,
|
||||
});
|
||||
NodeWalkResult::Matched { end }
|
||||
}
|
||||
|
||||
fn walk_bare_path(
|
||||
source: &str,
|
||||
position: usize,
|
||||
path: &mut MatchedPath,
|
||||
per_byte: &mut Vec<ByteClass>,
|
||||
) -> NodeWalkResult {
|
||||
let Some((start, end)) = consume_bare_path(source, position) else {
|
||||
return NodeWalkResult::NoMatch {
|
||||
position,
|
||||
expected: vec![Expectation::BarePath],
|
||||
};
|
||||
};
|
||||
let text = source[start..end].to_string();
|
||||
path.push(MatchedItem {
|
||||
kind: MatchedKind::BarePath,
|
||||
text,
|
||||
span: (start, end),
|
||||
});
|
||||
per_byte.push(ByteClass {
|
||||
start,
|
||||
end,
|
||||
class: HighlightClass::String,
|
||||
});
|
||||
NodeWalkResult::Matched { end }
|
||||
}
|
||||
|
||||
fn walk_choice(
|
||||
source: &str,
|
||||
position: usize,
|
||||
children: &[Node],
|
||||
ctx: &mut WalkContext,
|
||||
path: &mut MatchedPath,
|
||||
per_byte: &mut Vec<ByteClass>,
|
||||
) -> NodeWalkResult {
|
||||
let mut all_expected: Vec<Expectation> = Vec::new();
|
||||
for child in children {
|
||||
let saved_path_len = path.items.len();
|
||||
let saved_byte_len = per_byte.len();
|
||||
match walk_node(source, position, child, ctx, path, per_byte) {
|
||||
NodeWalkResult::Matched { end } => return NodeWalkResult::Matched { end },
|
||||
NodeWalkResult::NoMatch { expected, .. } => {
|
||||
path.items.truncate(saved_path_len);
|
||||
per_byte.truncate(saved_byte_len);
|
||||
merge_expected(&mut all_expected, expected);
|
||||
}
|
||||
// Once a choice branch commits, propagate its outcome.
|
||||
other => return other,
|
||||
}
|
||||
}
|
||||
NodeWalkResult::NoMatch {
|
||||
position,
|
||||
expected: all_expected,
|
||||
}
|
||||
}
|
||||
|
||||
fn walk_seq(
|
||||
source: &str,
|
||||
position: usize,
|
||||
children: &[Node],
|
||||
ctx: &mut WalkContext,
|
||||
path: &mut MatchedPath,
|
||||
per_byte: &mut Vec<ByteClass>,
|
||||
) -> NodeWalkResult {
|
||||
let mut cur = position;
|
||||
let mut idx = 0;
|
||||
for child in children {
|
||||
match walk_node(source, cur, child, ctx, path, per_byte) {
|
||||
NodeWalkResult::Matched { end } => {
|
||||
cur = end;
|
||||
idx += 1;
|
||||
}
|
||||
NodeWalkResult::NoMatch { position, expected } => {
|
||||
if idx == 0 {
|
||||
// Seq didn't even start.
|
||||
return NodeWalkResult::NoMatch { position, expected };
|
||||
}
|
||||
// Mid-shape: did we run out of input or hit a
|
||||
// wrong token?
|
||||
let post_ws = skip_whitespace(source, position);
|
||||
let kind = if post_ws >= source.len() {
|
||||
return NodeWalkResult::Incomplete { position: post_ws, expected };
|
||||
} else {
|
||||
FailureKind::Mismatch { expected }
|
||||
};
|
||||
return NodeWalkResult::Failed { position: post_ws, kind };
|
||||
}
|
||||
NodeWalkResult::Incomplete { position, expected } => {
|
||||
return NodeWalkResult::Incomplete { position, expected };
|
||||
}
|
||||
NodeWalkResult::Failed { position, kind } => {
|
||||
return NodeWalkResult::Failed { position, kind };
|
||||
}
|
||||
}
|
||||
}
|
||||
NodeWalkResult::Matched { end: cur }
|
||||
}
|
||||
|
||||
fn walk_optional(
|
||||
source: &str,
|
||||
position: usize,
|
||||
child: &Node,
|
||||
ctx: &mut WalkContext,
|
||||
path: &mut MatchedPath,
|
||||
per_byte: &mut Vec<ByteClass>,
|
||||
) -> NodeWalkResult {
|
||||
let saved_path_len = path.items.len();
|
||||
let saved_byte_len = per_byte.len();
|
||||
match walk_node(source, position, child, ctx, path, per_byte) {
|
||||
NodeWalkResult::Matched { end } => NodeWalkResult::Matched { end },
|
||||
NodeWalkResult::NoMatch { .. } => {
|
||||
path.items.truncate(saved_path_len);
|
||||
per_byte.truncate(saved_byte_len);
|
||||
NodeWalkResult::Matched { end: position }
|
||||
}
|
||||
other => other,
|
||||
}
|
||||
}
|
||||
|
||||
fn merge_expected(dst: &mut Vec<Expectation>, src: Vec<Expectation>) {
|
||||
for e in src {
|
||||
if !dst.contains(&e) {
|
||||
dst.push(e);
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,99 @@
|
||||
//! Byte-level helpers for the scannerless walker (ADR-0024
|
||||
//! §scannerless).
|
||||
//!
|
||||
//! Each helper takes the source string and a byte position,
|
||||
//! returns either `Some(end_position)` (matched, post-token end)
|
||||
//! or `None` (didn't match here). Helpers are pure and span-
|
||||
//! exact; multi-byte UTF-8 within identifiers and string
|
||||
//! literals is handled byte-correctly.
|
||||
//!
|
||||
//! These helpers internally mirror the logic of the legacy
|
||||
//! `dsl::lexer` module but are invoked per-position by the
|
||||
//! walker rather than as a pre-pass.
|
||||
|
||||
/// Return the byte index of the first non-whitespace byte at or
|
||||
/// after `start`. If the rest is all whitespace, returns
|
||||
/// `source.len()`.
|
||||
pub fn skip_whitespace(source: &str, start: usize) -> usize {
|
||||
let bytes = source.as_bytes();
|
||||
let mut i = start;
|
||||
while i < bytes.len() && bytes[i].is_ascii_whitespace() {
|
||||
i += 1;
|
||||
}
|
||||
i
|
||||
}
|
||||
|
||||
/// Identifier shape: ASCII letter or `_` to start, then ASCII
|
||||
/// alphanumeric or `_`. Returns `Some((start, end))` on match.
|
||||
pub fn consume_ident(source: &str, start: usize) -> Option<(usize, usize)> {
|
||||
let bytes = source.as_bytes();
|
||||
let first = *bytes.get(start)?;
|
||||
if !(first.is_ascii_alphabetic() || first == b'_') {
|
||||
return None;
|
||||
}
|
||||
let mut i = start + 1;
|
||||
while i < bytes.len() {
|
||||
let b = bytes[i];
|
||||
if b.is_ascii_alphanumeric() || b == b'_' {
|
||||
i += 1;
|
||||
} else {
|
||||
break;
|
||||
}
|
||||
}
|
||||
Some((start, i))
|
||||
}
|
||||
|
||||
/// Try to match `keyword` at `position` case-insensitively.
|
||||
///
|
||||
/// The match must end at a non-identifier byte (or end-of-input)
|
||||
/// so that `save` doesn't half-match the prefix of `saved`.
|
||||
/// Returns the end byte index on match.
|
||||
pub fn match_keyword(source: &str, position: usize, keyword: &str) -> Option<usize> {
|
||||
let bytes = source.as_bytes();
|
||||
let kw_bytes = keyword.as_bytes();
|
||||
if position + kw_bytes.len() > bytes.len() {
|
||||
return None;
|
||||
}
|
||||
for (offset, &kb) in kw_bytes.iter().enumerate() {
|
||||
let sb = bytes[position + offset];
|
||||
if !sb.eq_ignore_ascii_case(&kb) {
|
||||
return None;
|
||||
}
|
||||
}
|
||||
let end = position + kw_bytes.len();
|
||||
if end < bytes.len() {
|
||||
let next = bytes[end];
|
||||
if next.is_ascii_alphanumeric() || next == b'_' {
|
||||
return None;
|
||||
}
|
||||
}
|
||||
Some(end)
|
||||
}
|
||||
|
||||
/// Bare-path token: a non-whitespace run.
|
||||
///
|
||||
/// Per ADR-0024 the path-bearing UX dropped the "spaces don't
|
||||
/// need quoting" feature; paths with spaces use `StringLit`.
|
||||
/// Phase A's `import` / `export` slots use this.
|
||||
pub fn consume_bare_path(source: &str, start: usize) -> Option<(usize, usize)> {
|
||||
let bytes = source.as_bytes();
|
||||
if start >= bytes.len() || bytes[start].is_ascii_whitespace() {
|
||||
return None;
|
||||
}
|
||||
let mut i = start;
|
||||
while i < bytes.len() && !bytes[i].is_ascii_whitespace() {
|
||||
i += 1;
|
||||
}
|
||||
Some((start, i))
|
||||
}
|
||||
|
||||
/// Match a single punctuation character at `position`.
|
||||
#[allow(dead_code)]
|
||||
pub fn match_punct(source: &str, position: usize, ch: char) -> Option<usize> {
|
||||
let bytes = source.as_bytes();
|
||||
if position < bytes.len() && bytes[position] == ch as u8 {
|
||||
Some(position + 1)
|
||||
} else {
|
||||
None
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,432 @@
|
||||
//! Walker entry point (ADR-0024 §architecture).
|
||||
//!
|
||||
//! The walker is the single source of truth for the migrated
|
||||
//! commands. Phase A wires the parse consumer; completion +
|
||||
//! highlighting still flow through the chumsky path until
|
||||
//! Phase D / F.
|
||||
//!
|
||||
//! Routing rule (ADR-0024 §migration): the input's first
|
||||
//! identifier-shape token decides whether the walker owns this
|
||||
//! command. If it matches a registered entry word, the walker
|
||||
//! takes over end-to-end (success or failure). Otherwise, the
|
||||
//! router falls through to the chumsky parser, which still
|
||||
//! carries every non-migrated command's grammar through Phase F.
|
||||
|
||||
pub mod context;
|
||||
pub mod driver;
|
||||
pub mod lex_helpers;
|
||||
pub mod outcome;
|
||||
|
||||
use crate::dsl::command::Command;
|
||||
use crate::dsl::grammar;
|
||||
use crate::dsl::walker::context::WalkContext;
|
||||
use crate::dsl::walker::driver::{FailureKind, NodeWalkResult, walk_node};
|
||||
use crate::dsl::walker::lex_helpers::{consume_ident, skip_whitespace};
|
||||
use crate::dsl::walker::outcome::{
|
||||
Expectation, MatchedPath, WalkBound, WalkOutcome, WalkResult,
|
||||
};
|
||||
|
||||
pub use context::ColumnInfo;
|
||||
|
||||
/// Public walk entry. `bound` is `EndOfInput` for parse;
|
||||
/// `Position(cursor)` for completion / hint (Phase A: not yet
|
||||
/// wired).
|
||||
///
|
||||
/// Returns:
|
||||
/// - `(Some(WalkResult), Some(Command))` on full match — the
|
||||
/// AST builder produced a typed Command.
|
||||
/// - `(Some(WalkResult), None)` on failure where the walker
|
||||
/// committed (matched the entry word). Caller surfaces the
|
||||
/// walker's error.
|
||||
/// - `(None, None)` when the entry word doesn't match any
|
||||
/// registered command — the router falls through to chumsky.
|
||||
pub fn walk(
|
||||
source: &str,
|
||||
bound: WalkBound,
|
||||
ctx: &mut WalkContext,
|
||||
) -> (Option<WalkResult>, Option<Command>) {
|
||||
// Phase A only consumes EndOfInput; Position would slice
|
||||
// the source, which is the same operation.
|
||||
let effective_source: &str = match bound {
|
||||
WalkBound::EndOfInput => source,
|
||||
WalkBound::Position(end) => &source[..end.min(source.len())],
|
||||
};
|
||||
|
||||
let start = skip_whitespace(effective_source, 0);
|
||||
if start >= effective_source.len() {
|
||||
return (None, None);
|
||||
}
|
||||
|
||||
// Identify the command by its entry word. If the first
|
||||
// identifier-shape token isn't a registered entry, the
|
||||
// walker yields to chumsky.
|
||||
let Some((kw_start, kw_end)) = consume_ident(effective_source, start) else {
|
||||
return (None, None);
|
||||
};
|
||||
let entry_text = &effective_source[kw_start..kw_end];
|
||||
let Some((command_idx, command_node)) = grammar::command_for_entry_word(entry_text)
|
||||
else {
|
||||
return (None, None);
|
||||
};
|
||||
|
||||
let mut path = MatchedPath::new();
|
||||
let mut per_byte = Vec::new();
|
||||
|
||||
// Record the entry-word match.
|
||||
path.push(crate::dsl::walker::outcome::MatchedItem {
|
||||
kind: crate::dsl::walker::outcome::MatchedKind::Word(command_node.entry.primary),
|
||||
text: entry_text.to_string(),
|
||||
span: (kw_start, kw_end),
|
||||
});
|
||||
per_byte.push(crate::dsl::walker::outcome::ByteClass {
|
||||
start: kw_start,
|
||||
end: kw_end,
|
||||
class: grammar::HighlightClass::Keyword,
|
||||
});
|
||||
|
||||
let outcome = match walk_node(
|
||||
effective_source,
|
||||
kw_end,
|
||||
&command_node.shape,
|
||||
ctx,
|
||||
&mut path,
|
||||
&mut per_byte,
|
||||
) {
|
||||
NodeWalkResult::Matched { end } => {
|
||||
let trailing = skip_whitespace(effective_source, end);
|
||||
if trailing < effective_source.len() {
|
||||
WalkOutcome::Mismatch {
|
||||
position: trailing,
|
||||
expected: vec![Expectation::EndOfInput],
|
||||
}
|
||||
} else {
|
||||
WalkOutcome::Match { command_idx }
|
||||
}
|
||||
}
|
||||
NodeWalkResult::NoMatch { position, expected } => {
|
||||
// The shape required content the user hasn't typed.
|
||||
// (Optional/empty-Seq shapes always return Matched
|
||||
// even when skipped, so reaching NoMatch here means
|
||||
// the command really wanted something more.)
|
||||
let post = skip_whitespace(effective_source, position);
|
||||
if post >= effective_source.len() {
|
||||
WalkOutcome::Incomplete { position: post, expected }
|
||||
} else {
|
||||
WalkOutcome::Mismatch { position: post, expected }
|
||||
}
|
||||
}
|
||||
NodeWalkResult::Incomplete { position, expected } => {
|
||||
WalkOutcome::Incomplete { position, expected }
|
||||
}
|
||||
NodeWalkResult::Failed { position, kind } => match kind {
|
||||
FailureKind::Mismatch { expected } => {
|
||||
WalkOutcome::Mismatch { position, expected }
|
||||
}
|
||||
FailureKind::Validation(error) => {
|
||||
WalkOutcome::ValidationFailed { position, error }
|
||||
}
|
||||
},
|
||||
};
|
||||
|
||||
let cmd = if matches!(outcome, WalkOutcome::Match { .. }) {
|
||||
(command_node.ast_builder)(&path).ok()
|
||||
} else {
|
||||
None
|
||||
};
|
||||
|
||||
let result = WalkResult {
|
||||
outcome,
|
||||
matched_path: path,
|
||||
per_byte_class: per_byte,
|
||||
};
|
||||
(Some(result), cmd)
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
//! Walker behaviour tests — Phase A (ADR-0024 §migration).
|
||||
//!
|
||||
//! These cover every app-lifecycle command the walker now
|
||||
//! owns. Each input is paired with its expected `Command`
|
||||
//! output (the differential-against-chumsky check
|
||||
//! materialised as hand-curated expectations — same role
|
||||
//! the differential test scaffolding plays per ADR-0024
|
||||
//! §test-discipline).
|
||||
//!
|
||||
//! The handoff document lists these tests as "walker-
|
||||
//! specific tests for trie-only features" — they pin down
|
||||
//! the walker's contract for the migrated commands so
|
||||
//! Phase B-F migrations can refactor without regression.
|
||||
use crate::dsl::command::{AppCommand, Command, MessagesValue, ModeValue};
|
||||
use crate::dsl::parser::parse_command;
|
||||
|
||||
fn parse(input: &str) -> Result<Command, crate::dsl::ParseError> {
|
||||
parse_command(input)
|
||||
}
|
||||
|
||||
// ---- Bare no-arg commands ---------------------------------
|
||||
|
||||
#[test]
|
||||
fn walker_parses_quit() {
|
||||
assert_eq!(parse("quit").unwrap(), Command::App(AppCommand::Quit));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn walker_parses_help() {
|
||||
assert_eq!(parse("help").unwrap(), Command::App(AppCommand::Help));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn walker_parses_rebuild() {
|
||||
assert_eq!(parse("rebuild").unwrap(), Command::App(AppCommand::Rebuild));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn walker_parses_new() {
|
||||
assert_eq!(parse("new").unwrap(), Command::App(AppCommand::New));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn walker_parses_load() {
|
||||
assert_eq!(parse("load").unwrap(), Command::App(AppCommand::Load));
|
||||
}
|
||||
|
||||
// ---- Save / save as ---------------------------------------
|
||||
|
||||
#[test]
|
||||
fn walker_parses_save() {
|
||||
assert_eq!(parse("save").unwrap(), Command::App(AppCommand::Save));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn walker_parses_save_as() {
|
||||
assert_eq!(parse("save as").unwrap(), Command::App(AppCommand::SaveAs));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn walker_save_keywords_case_insensitive() {
|
||||
assert_eq!(parse("SAVE").unwrap(), Command::App(AppCommand::Save));
|
||||
assert_eq!(parse("Save AS").unwrap(), Command::App(AppCommand::SaveAs));
|
||||
}
|
||||
|
||||
// ---- Mode -------------------------------------------------
|
||||
|
||||
#[test]
|
||||
fn walker_parses_mode_simple() {
|
||||
assert_eq!(
|
||||
parse("mode simple").unwrap(),
|
||||
Command::App(AppCommand::Mode {
|
||||
value: ModeValue::Simple,
|
||||
})
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn walker_parses_mode_advanced() {
|
||||
assert_eq!(
|
||||
parse("mode advanced").unwrap(),
|
||||
Command::App(AppCommand::Mode {
|
||||
value: ModeValue::Advanced,
|
||||
})
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn walker_mode_unknown_value_emits_friendly_error() {
|
||||
let err = parse("mode foo").unwrap_err();
|
||||
match err {
|
||||
crate::dsl::ParseError::Invalid { message, .. } => {
|
||||
// The catalog wording for `mode.unknown` carries
|
||||
// the user's value verbatim.
|
||||
assert!(message.contains("foo"), "got: {message}");
|
||||
}
|
||||
other => panic!("expected Invalid, got {other:?}"),
|
||||
}
|
||||
}
|
||||
|
||||
// ---- Messages ---------------------------------------------
|
||||
|
||||
#[test]
|
||||
fn walker_parses_messages_bare() {
|
||||
assert_eq!(
|
||||
parse("messages").unwrap(),
|
||||
Command::App(AppCommand::Messages { value: None })
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn walker_parses_messages_short() {
|
||||
assert_eq!(
|
||||
parse("messages short").unwrap(),
|
||||
Command::App(AppCommand::Messages {
|
||||
value: Some(MessagesValue::Short),
|
||||
})
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn walker_parses_messages_verbose() {
|
||||
assert_eq!(
|
||||
parse("messages verbose").unwrap(),
|
||||
Command::App(AppCommand::Messages {
|
||||
value: Some(MessagesValue::Verbose),
|
||||
})
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn walker_messages_unknown_value_emits_friendly_error() {
|
||||
let err = parse("messages bogus").unwrap_err();
|
||||
match err {
|
||||
crate::dsl::ParseError::Invalid { message, .. } => {
|
||||
assert!(message.contains("bogus"), "got: {message}");
|
||||
}
|
||||
other => panic!("expected Invalid, got {other:?}"),
|
||||
}
|
||||
}
|
||||
|
||||
// ---- Export -----------------------------------------------
|
||||
|
||||
#[test]
|
||||
fn walker_parses_export_bare() {
|
||||
assert_eq!(
|
||||
parse("export").unwrap(),
|
||||
Command::App(AppCommand::Export { path: None })
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn walker_parses_export_with_path() {
|
||||
assert_eq!(
|
||||
parse("export backups/MyExport.zip").unwrap(),
|
||||
Command::App(AppCommand::Export {
|
||||
path: Some("backups/MyExport.zip".to_string()),
|
||||
})
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn walker_export_trims_trailing_whitespace() {
|
||||
// Pre-migration the source-slice helper trimmed; the
|
||||
// walker treats " " after `export` as zero BarePath
|
||||
// matches and produces the bare form.
|
||||
assert_eq!(
|
||||
parse("export ").unwrap(),
|
||||
Command::App(AppCommand::Export { path: None })
|
||||
);
|
||||
}
|
||||
|
||||
// ---- Import -----------------------------------------------
|
||||
|
||||
#[test]
|
||||
fn walker_parses_import_bare() {
|
||||
assert_eq!(
|
||||
parse("import").unwrap(),
|
||||
Command::App(AppCommand::Import {
|
||||
path: String::new(),
|
||||
target: None,
|
||||
})
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn walker_parses_import_with_path() {
|
||||
assert_eq!(
|
||||
parse("import some/file.zip").unwrap(),
|
||||
Command::App(AppCommand::Import {
|
||||
path: "some/file.zip".to_string(),
|
||||
target: None,
|
||||
})
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn walker_parses_import_with_path_and_target() {
|
||||
assert_eq!(
|
||||
parse("import some/file.zip as MyImported").unwrap(),
|
||||
Command::App(AppCommand::Import {
|
||||
path: "some/file.zip".to_string(),
|
||||
target: Some("MyImported".to_string()),
|
||||
})
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn walker_import_keeps_as_inside_path() {
|
||||
// The lexer-free walker terminates `BarePath` at the
|
||||
// first whitespace byte. `path/asfile.zip` is one
|
||||
// token; the `as` *inside* it stays part of the path.
|
||||
assert_eq!(
|
||||
parse("import path/asfile.zip").unwrap(),
|
||||
Command::App(AppCommand::Import {
|
||||
path: "path/asfile.zip".to_string(),
|
||||
target: None,
|
||||
})
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn walker_import_trailing_as_without_target_errors() {
|
||||
let err = parse("import foo.zip as ").unwrap_err();
|
||||
match err {
|
||||
crate::dsl::ParseError::Invalid { message, expected, .. } => {
|
||||
// Phase A: the friendly `project.import_empty_target`
|
||||
// wording moves out of the parser; the walker's
|
||||
// structural error names the `target` slot.
|
||||
assert!(
|
||||
message.contains("target") || expected.iter().any(|e| e == "target"),
|
||||
"expected mention of target slot; got message={message:?}, expected={expected:?}"
|
||||
);
|
||||
}
|
||||
other => panic!("expected Invalid, got {other:?}"),
|
||||
}
|
||||
}
|
||||
|
||||
// ---- Routing fall-through ---------------------------------
|
||||
|
||||
#[test]
|
||||
fn walker_does_not_engage_for_non_app_keywords() {
|
||||
// The router falls through to the chumsky path. The
|
||||
// existing chumsky parser produces this Command.
|
||||
assert!(matches!(
|
||||
parse("drop table Customers").unwrap(),
|
||||
Command::DropTable { .. }
|
||||
));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn walker_does_not_engage_for_unknown_first_token() {
|
||||
// Not an entry word — chumsky yields its usual
|
||||
// unknown-command error.
|
||||
assert!(parse("frobulate").is_err());
|
||||
}
|
||||
|
||||
// ---- Trailing-garbage detection ---------------------------
|
||||
|
||||
#[test]
|
||||
fn walker_quit_with_trailing_garbage_errors() {
|
||||
assert!(parse("quit nonsense").is_err());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn walker_save_with_trailing_garbage_errors() {
|
||||
assert!(parse("save Customers").is_err());
|
||||
}
|
||||
|
||||
// ---- Whitespace tolerance ---------------------------------
|
||||
|
||||
#[test]
|
||||
fn walker_tolerates_leading_and_internal_whitespace() {
|
||||
assert_eq!(parse(" quit ").unwrap(), Command::App(AppCommand::Quit));
|
||||
assert_eq!(
|
||||
parse("save as").unwrap(),
|
||||
Command::App(AppCommand::SaveAs)
|
||||
);
|
||||
assert_eq!(
|
||||
parse("mode\tadvanced").unwrap(),
|
||||
Command::App(AppCommand::Mode {
|
||||
value: ModeValue::Advanced,
|
||||
})
|
||||
);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,159 @@
|
||||
//! Walker output types (ADR-0024 §architecture).
|
||||
//!
|
||||
//! `WalkResult` carries everything a consumer (parse / completion /
|
||||
//! highlight / hint) needs from a single walk: the outcome
|
||||
//! (matched, incomplete, mismatched, validation-failed), the
|
||||
//! matched-node path the AST builder reads, and the per-byte
|
||||
//! highlight class assignments collected as terminals matched.
|
||||
//!
|
||||
//! Phase A note: only the parse consumer is wired today. The
|
||||
//! `per_byte_class` field is populated but unused outside
|
||||
//! tests; completion + highlighting still flow through the
|
||||
//! chumsky path until Phase D / F.
|
||||
|
||||
use crate::dsl::grammar::{HighlightClass, ValidationError};
|
||||
|
||||
/// How far into the input the walker should consume.
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||
pub enum WalkBound {
|
||||
/// Consume all input. Trailing whitespace OK; trailing tokens
|
||||
/// fail the walk. Used by the parse consumer.
|
||||
EndOfInput,
|
||||
/// Consume up to (but not including) the given byte position.
|
||||
/// Used by completion / hint to ask "what was expected at the
|
||||
/// cursor?".
|
||||
#[allow(dead_code)]
|
||||
Position(usize),
|
||||
}
|
||||
|
||||
/// Closed shape describing what could legally have continued the
|
||||
/// walk at its stopping position. Phase A keeps this minimal —
|
||||
/// only what the router needs to render a parse error.
|
||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||
pub enum Expectation {
|
||||
/// The walker expected one of these literal keywords.
|
||||
Word(&'static str),
|
||||
/// The walker expected an identifier of the given role.
|
||||
Ident { role: &'static str },
|
||||
/// The walker expected this exact punctuation character.
|
||||
Punct(char),
|
||||
/// The walker expected a number literal.
|
||||
NumberLit,
|
||||
/// The walker expected a string literal.
|
||||
StringLit,
|
||||
/// The walker expected a blob literal.
|
||||
#[allow(dead_code)]
|
||||
BlobLit,
|
||||
/// The walker expected a flag with this name (without `--`).
|
||||
#[allow(dead_code)]
|
||||
Flag(&'static str),
|
||||
/// The walker expected a bare-path argument (non-whitespace
|
||||
/// run).
|
||||
BarePath,
|
||||
/// The walker expected end of input.
|
||||
EndOfInput,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||
pub enum WalkOutcome {
|
||||
/// Input fully matched a command. `command_idx` is the
|
||||
/// position of the matched command in the active registry.
|
||||
Match { command_idx: usize },
|
||||
/// Input matched a prefix; more input would have continued
|
||||
/// the parse. `position` is the byte offset where input ran
|
||||
/// out (post-whitespace).
|
||||
Incomplete {
|
||||
position: usize,
|
||||
expected: Vec<Expectation>,
|
||||
},
|
||||
/// Input had a token at `position` that no expected node
|
||||
/// accepts.
|
||||
Mismatch {
|
||||
position: usize,
|
||||
expected: Vec<Expectation>,
|
||||
},
|
||||
/// The walker matched a terminal but a content validator
|
||||
/// rejected the value (e.g. `mode foo` matched the value
|
||||
/// slot's identifier shape, then the validator fired
|
||||
/// `mode.unknown`).
|
||||
ValidationFailed {
|
||||
position: usize,
|
||||
error: ValidationError,
|
||||
},
|
||||
}
|
||||
|
||||
/// One terminal-node match. Combinators (Seq / Choice / Optional /
|
||||
/// Repeated) shape the order; the AST builder reads the items in
|
||||
/// declaration order.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct MatchedItem {
|
||||
pub kind: MatchedKind,
|
||||
pub text: String,
|
||||
pub span: (usize, usize),
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||
pub enum MatchedKind {
|
||||
/// A `Word` node matched. Carries the primary literal (not
|
||||
/// the alias actually typed) so the AST builder can match
|
||||
/// on it canonically.
|
||||
Word(&'static str),
|
||||
Punct(char),
|
||||
/// An `Ident` matched. The role identifies which slot.
|
||||
Ident { role: &'static str },
|
||||
NumberLit,
|
||||
StringLit,
|
||||
BlobLit,
|
||||
Flag(&'static str),
|
||||
BarePath,
|
||||
}
|
||||
|
||||
/// The path of matched terminals, in order. Optional / Repeated
|
||||
/// nodes that produced no match contribute nothing.
|
||||
#[derive(Debug, Clone, Default)]
|
||||
pub struct MatchedPath {
|
||||
pub items: Vec<MatchedItem>,
|
||||
}
|
||||
|
||||
impl MatchedPath {
|
||||
pub fn new() -> Self {
|
||||
Self::default()
|
||||
}
|
||||
|
||||
pub fn push(&mut self, item: MatchedItem) {
|
||||
self.items.push(item);
|
||||
}
|
||||
|
||||
/// Convenience: find the first item matching the predicate.
|
||||
pub fn find<F: Fn(&MatchedItem) -> bool>(&self, pred: F) -> Option<&MatchedItem> {
|
||||
self.items.iter().find(|i| pred(i))
|
||||
}
|
||||
|
||||
/// Convenience: did any item match this exact word literal
|
||||
/// (by primary)? Used by Optional-keyword discrimination
|
||||
/// (e.g., `save` vs `save as`).
|
||||
pub fn contains_word(&self, primary: &'static str) -> bool {
|
||||
self.items
|
||||
.iter()
|
||||
.any(|i| matches!(&i.kind, MatchedKind::Word(p) if *p == primary))
|
||||
}
|
||||
}
|
||||
|
||||
/// Per-byte highlight class assignment, collected as terminals
|
||||
/// match. Phase A keeps this for future consumers; not yet used
|
||||
/// outside walker-internal tests.
|
||||
#[derive(Debug, Clone)]
|
||||
#[allow(dead_code)]
|
||||
pub struct ByteClass {
|
||||
pub start: usize,
|
||||
pub end: usize,
|
||||
pub class: HighlightClass,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct WalkResult {
|
||||
pub outcome: WalkOutcome,
|
||||
pub matched_path: MatchedPath,
|
||||
#[allow(dead_code)]
|
||||
pub per_byte_class: Vec<ByteClass>,
|
||||
}
|
||||
Reference in New Issue
Block a user