ADR-0024 Phase A: walker framework + app-lifecycle commands
Stand up the unified-grammar tree walker alongside the existing
chumsky parser and migrate the eleven app-lifecycle commands
(quit, help, rebuild, save / save as, new, load, export, import,
mode, messages) end-to-end. The router in parse_tokens consults
the walker first; non-migrated commands still fall through to
chumsky.
Scope:
- src/dsl/grammar/{mod,app}.rs: Node enum (13 kinds), Word /
IdentSource / HintMode / HighlightClass / ValidationError /
CommandNode types, REGISTRY of the eleven app commands.
- src/dsl/walker/{mod,driver,context,outcome,lex_helpers}.rs:
scannerless byte-level walker, per-node-kind dispatch with
Choice/Seq/Optional backtracking, WalkContext (Phase B-D
schema fields stubbed), WalkOutcome with Match/Incomplete/
Mismatch/ValidationFailed.
- src/dsl/parser.rs: try_walker_route() runs first in
parse_tokens; bridge converts WalkOutcome to ParseError
preserving catalog wording (mode.unknown / messages.unknown
surface verbatim via friendly::translate). Legacy
try_parse_app_path_command deleted; chumsky's bare-keyword
app branches remain unreachable until Phase F sweep.
Walker design choices worth noting:
- mode <value> / messages <value> use Choice(Word, Word, Ident)
so known keywords appear in the expected-set; the trailing
Ident catch-all funnels unknown values into the friendly
validator that always errors with the catalog wording.
- save / save as is one CommandNode (Optional(Word("as"))) -
closes the round-5 "save Tab can't offer as" limitation
structurally.
- Path-bearing UX shipped per ADR-0024: BarePath terminates at
whitespace; paths with spaces use the (not-yet-wired) quoted
form. Existing tests pass on the new shape.
Tests:
- 28 new walker-specific tests in dsl::walker::tests covering
every app-lifecycle command, friendly-error wording for
mode/messages unknown values, trailing-garbage detection,
whitespace tolerance, and routing fall-through.
- Total: 805 passed, 0 failed, 1 ignored (was 777 / 1).
- cargo clippy --all-targets -- -D warnings clean.
This commit is contained in:
@@ -0,0 +1,330 @@
|
||||
//! Per-node-kind walk dispatch (ADR-0024 §architecture).
|
||||
//!
|
||||
//! `walk_node` is the recursive workhorse that the public
|
||||
//! `walk()` entry calls into for a `CommandNode`'s `shape`. It
|
||||
//! tries to match `node` starting at `position`, mutating
|
||||
//! `path` (matched terminals collected in declaration order) and
|
||||
//! `per_byte` (highlight class assignments) as it goes.
|
||||
//!
|
||||
//! The return value distinguishes four cases:
|
||||
//!
|
||||
//! - `Matched { end }` — full match, walker consumed up to `end`.
|
||||
//! - `NoMatch { … }` — node didn't engage at this position. For
|
||||
//! `Optional` and `Choice` callers this is benign (try the
|
||||
//! next branch / skip the optional); for `Seq` it's only
|
||||
//! benign on the first child.
|
||||
//! - `Incomplete { … }` — node committed (consumed at least one
|
||||
//! terminal) but ran out of input. Surfaces as
|
||||
//! `WalkOutcome::Incomplete` at the top level.
|
||||
//! - `Failed { … }` — node committed and a content validator
|
||||
//! rejected the value, or a hard structural failure occurred
|
||||
//! mid-shape. Surfaces as `WalkOutcome::Mismatch` or
|
||||
//! `WalkOutcome::ValidationFailed` at the top level.
|
||||
|
||||
use crate::dsl::grammar::{HighlightClass, Node, ValidationError};
|
||||
use crate::dsl::walker::context::WalkContext;
|
||||
use crate::dsl::walker::lex_helpers::{consume_bare_path, consume_ident, skip_whitespace};
|
||||
use crate::dsl::walker::outcome::{
|
||||
ByteClass, Expectation, MatchedItem, MatchedKind, MatchedPath,
|
||||
};
|
||||
|
||||
#[derive(Debug, Clone)]
|
||||
pub enum NodeWalkResult {
|
||||
Matched {
|
||||
end: usize,
|
||||
},
|
||||
/// Did not engage at this position. Caller decides whether
|
||||
/// this is benign (Optional, Choice fallthrough) or a hard
|
||||
/// failure (Seq mid-shape).
|
||||
NoMatch {
|
||||
position: usize,
|
||||
expected: Vec<Expectation>,
|
||||
},
|
||||
/// Committed and ran out of input.
|
||||
Incomplete {
|
||||
position: usize,
|
||||
expected: Vec<Expectation>,
|
||||
},
|
||||
/// Committed and hit a hard mismatch or validator failure.
|
||||
Failed {
|
||||
position: usize,
|
||||
kind: FailureKind,
|
||||
},
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone)]
|
||||
pub enum FailureKind {
|
||||
Mismatch { expected: Vec<Expectation> },
|
||||
Validation(ValidationError),
|
||||
}
|
||||
|
||||
pub fn walk_node(
|
||||
source: &str,
|
||||
position: usize,
|
||||
node: &Node,
|
||||
ctx: &mut WalkContext,
|
||||
path: &mut MatchedPath,
|
||||
per_byte: &mut Vec<ByteClass>,
|
||||
) -> NodeWalkResult {
|
||||
let pos = skip_whitespace(source, position);
|
||||
match node {
|
||||
Node::Word(word) => walk_word(source, pos, word, path, per_byte),
|
||||
Node::Punct(ch) => walk_punct(source, pos, *ch, path, per_byte),
|
||||
Node::Ident {
|
||||
source: src,
|
||||
role,
|
||||
validator,
|
||||
highlight_override: _,
|
||||
} => walk_ident(source, pos, *src, role, *validator, path, per_byte),
|
||||
Node::NumberLit
|
||||
| Node::StringLit
|
||||
| Node::BlobLit
|
||||
| Node::Flag(_)
|
||||
| Node::Repeated { .. }
|
||||
| Node::DynamicSubgrammar(_) => {
|
||||
// Phase A: not exercised by app-lifecycle commands.
|
||||
// Reaching this branch means a Phase B+ grammar got
|
||||
// declared without the walker support landing yet —
|
||||
// surface as a hard failure so the test suite catches
|
||||
// it loudly instead of silently mis-parsing.
|
||||
NodeWalkResult::Failed {
|
||||
position: pos,
|
||||
kind: FailureKind::Mismatch { expected: vec![] },
|
||||
}
|
||||
}
|
||||
Node::BarePath => walk_bare_path(source, pos, path, per_byte),
|
||||
Node::Choice(children) => walk_choice(source, pos, children, ctx, path, per_byte),
|
||||
Node::Seq(children) => walk_seq(source, pos, children, ctx, path, per_byte),
|
||||
Node::Optional(child) => walk_optional(source, pos, child, ctx, path, per_byte),
|
||||
}
|
||||
}
|
||||
|
||||
fn walk_word(
|
||||
source: &str,
|
||||
position: usize,
|
||||
word: &crate::dsl::grammar::Word,
|
||||
path: &mut MatchedPath,
|
||||
per_byte: &mut Vec<ByteClass>,
|
||||
) -> NodeWalkResult {
|
||||
// First scan an identifier-shape token at `position`; if
|
||||
// none, we definitely don't have this keyword. If one, check
|
||||
// it against the word's primary + aliases.
|
||||
let Some((start, end)) = consume_ident(source, position) else {
|
||||
return NodeWalkResult::NoMatch {
|
||||
position,
|
||||
expected: vec![Expectation::Word(word.primary)],
|
||||
};
|
||||
};
|
||||
let candidate = &source[start..end];
|
||||
if word.matches(candidate) {
|
||||
path.push(MatchedItem {
|
||||
kind: MatchedKind::Word(word.primary),
|
||||
text: candidate.to_string(),
|
||||
span: (start, end),
|
||||
});
|
||||
per_byte.push(ByteClass {
|
||||
start,
|
||||
end,
|
||||
class: HighlightClass::Keyword,
|
||||
});
|
||||
NodeWalkResult::Matched { end }
|
||||
} else {
|
||||
NodeWalkResult::NoMatch {
|
||||
position,
|
||||
expected: vec![Expectation::Word(word.primary)],
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fn walk_punct(
|
||||
source: &str,
|
||||
position: usize,
|
||||
ch: char,
|
||||
path: &mut MatchedPath,
|
||||
per_byte: &mut Vec<ByteClass>,
|
||||
) -> NodeWalkResult {
|
||||
let bytes = source.as_bytes();
|
||||
if position < bytes.len() && bytes[position] == ch as u8 {
|
||||
path.push(MatchedItem {
|
||||
kind: MatchedKind::Punct(ch),
|
||||
text: ch.to_string(),
|
||||
span: (position, position + 1),
|
||||
});
|
||||
per_byte.push(ByteClass {
|
||||
start: position,
|
||||
end: position + 1,
|
||||
class: HighlightClass::Punct,
|
||||
});
|
||||
NodeWalkResult::Matched {
|
||||
end: position + 1,
|
||||
}
|
||||
} else {
|
||||
NodeWalkResult::NoMatch {
|
||||
position,
|
||||
expected: vec![Expectation::Punct(ch)],
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fn walk_ident(
|
||||
source: &str,
|
||||
position: usize,
|
||||
_src: crate::dsl::grammar::IdentSource,
|
||||
role: &'static str,
|
||||
validator: Option<crate::dsl::grammar::IdentValidator>,
|
||||
path: &mut MatchedPath,
|
||||
per_byte: &mut Vec<ByteClass>,
|
||||
) -> NodeWalkResult {
|
||||
let Some((start, end)) = consume_ident(source, position) else {
|
||||
return NodeWalkResult::NoMatch {
|
||||
position,
|
||||
expected: vec![Expectation::Ident { role }],
|
||||
};
|
||||
};
|
||||
let text = source[start..end].to_string();
|
||||
if let Some(v) = validator
|
||||
&& let Err(err) = v(&text)
|
||||
{
|
||||
return NodeWalkResult::Failed {
|
||||
position: start,
|
||||
kind: FailureKind::Validation(err),
|
||||
};
|
||||
}
|
||||
path.push(MatchedItem {
|
||||
kind: MatchedKind::Ident { role },
|
||||
text,
|
||||
span: (start, end),
|
||||
});
|
||||
per_byte.push(ByteClass {
|
||||
start,
|
||||
end,
|
||||
class: HighlightClass::Identifier,
|
||||
});
|
||||
NodeWalkResult::Matched { end }
|
||||
}
|
||||
|
||||
fn walk_bare_path(
|
||||
source: &str,
|
||||
position: usize,
|
||||
path: &mut MatchedPath,
|
||||
per_byte: &mut Vec<ByteClass>,
|
||||
) -> NodeWalkResult {
|
||||
let Some((start, end)) = consume_bare_path(source, position) else {
|
||||
return NodeWalkResult::NoMatch {
|
||||
position,
|
||||
expected: vec![Expectation::BarePath],
|
||||
};
|
||||
};
|
||||
let text = source[start..end].to_string();
|
||||
path.push(MatchedItem {
|
||||
kind: MatchedKind::BarePath,
|
||||
text,
|
||||
span: (start, end),
|
||||
});
|
||||
per_byte.push(ByteClass {
|
||||
start,
|
||||
end,
|
||||
class: HighlightClass::String,
|
||||
});
|
||||
NodeWalkResult::Matched { end }
|
||||
}
|
||||
|
||||
fn walk_choice(
|
||||
source: &str,
|
||||
position: usize,
|
||||
children: &[Node],
|
||||
ctx: &mut WalkContext,
|
||||
path: &mut MatchedPath,
|
||||
per_byte: &mut Vec<ByteClass>,
|
||||
) -> NodeWalkResult {
|
||||
let mut all_expected: Vec<Expectation> = Vec::new();
|
||||
for child in children {
|
||||
let saved_path_len = path.items.len();
|
||||
let saved_byte_len = per_byte.len();
|
||||
match walk_node(source, position, child, ctx, path, per_byte) {
|
||||
NodeWalkResult::Matched { end } => return NodeWalkResult::Matched { end },
|
||||
NodeWalkResult::NoMatch { expected, .. } => {
|
||||
path.items.truncate(saved_path_len);
|
||||
per_byte.truncate(saved_byte_len);
|
||||
merge_expected(&mut all_expected, expected);
|
||||
}
|
||||
// Once a choice branch commits, propagate its outcome.
|
||||
other => return other,
|
||||
}
|
||||
}
|
||||
NodeWalkResult::NoMatch {
|
||||
position,
|
||||
expected: all_expected,
|
||||
}
|
||||
}
|
||||
|
||||
fn walk_seq(
|
||||
source: &str,
|
||||
position: usize,
|
||||
children: &[Node],
|
||||
ctx: &mut WalkContext,
|
||||
path: &mut MatchedPath,
|
||||
per_byte: &mut Vec<ByteClass>,
|
||||
) -> NodeWalkResult {
|
||||
let mut cur = position;
|
||||
let mut idx = 0;
|
||||
for child in children {
|
||||
match walk_node(source, cur, child, ctx, path, per_byte) {
|
||||
NodeWalkResult::Matched { end } => {
|
||||
cur = end;
|
||||
idx += 1;
|
||||
}
|
||||
NodeWalkResult::NoMatch { position, expected } => {
|
||||
if idx == 0 {
|
||||
// Seq didn't even start.
|
||||
return NodeWalkResult::NoMatch { position, expected };
|
||||
}
|
||||
// Mid-shape: did we run out of input or hit a
|
||||
// wrong token?
|
||||
let post_ws = skip_whitespace(source, position);
|
||||
let kind = if post_ws >= source.len() {
|
||||
return NodeWalkResult::Incomplete { position: post_ws, expected };
|
||||
} else {
|
||||
FailureKind::Mismatch { expected }
|
||||
};
|
||||
return NodeWalkResult::Failed { position: post_ws, kind };
|
||||
}
|
||||
NodeWalkResult::Incomplete { position, expected } => {
|
||||
return NodeWalkResult::Incomplete { position, expected };
|
||||
}
|
||||
NodeWalkResult::Failed { position, kind } => {
|
||||
return NodeWalkResult::Failed { position, kind };
|
||||
}
|
||||
}
|
||||
}
|
||||
NodeWalkResult::Matched { end: cur }
|
||||
}
|
||||
|
||||
fn walk_optional(
|
||||
source: &str,
|
||||
position: usize,
|
||||
child: &Node,
|
||||
ctx: &mut WalkContext,
|
||||
path: &mut MatchedPath,
|
||||
per_byte: &mut Vec<ByteClass>,
|
||||
) -> NodeWalkResult {
|
||||
let saved_path_len = path.items.len();
|
||||
let saved_byte_len = per_byte.len();
|
||||
match walk_node(source, position, child, ctx, path, per_byte) {
|
||||
NodeWalkResult::Matched { end } => NodeWalkResult::Matched { end },
|
||||
NodeWalkResult::NoMatch { .. } => {
|
||||
path.items.truncate(saved_path_len);
|
||||
per_byte.truncate(saved_byte_len);
|
||||
NodeWalkResult::Matched { end: position }
|
||||
}
|
||||
other => other,
|
||||
}
|
||||
}
|
||||
|
||||
fn merge_expected(dst: &mut Vec<Expectation>, src: Vec<Expectation>) {
|
||||
for e in src {
|
||||
if !dst.contains(&e) {
|
||||
dst.push(e);
|
||||
}
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user