ADR-0024 Phase A: walker framework + app-lifecycle commands

Stand up the unified-grammar tree walker alongside the existing
chumsky parser and migrate the eleven app-lifecycle commands
(quit, help, rebuild, save / save as, new, load, export, import,
mode, messages) end-to-end. The router in parse_tokens consults
the walker first; non-migrated commands still fall through to
chumsky.

Scope:
- src/dsl/grammar/{mod,app}.rs: Node enum (13 kinds), Word /
  IdentSource / HintMode / HighlightClass / ValidationError /
  CommandNode types, REGISTRY of the eleven app commands.
- src/dsl/walker/{mod,driver,context,outcome,lex_helpers}.rs:
  scannerless byte-level walker, per-node-kind dispatch with
  Choice/Seq/Optional backtracking, WalkContext (Phase B-D
  schema fields stubbed), WalkOutcome with Match/Incomplete/
  Mismatch/ValidationFailed.
- src/dsl/parser.rs: try_walker_route() runs first in
  parse_tokens; bridge converts WalkOutcome to ParseError
  preserving catalog wording (mode.unknown / messages.unknown
  surface verbatim via friendly::translate). Legacy
  try_parse_app_path_command deleted; chumsky's bare-keyword
  app branches remain unreachable until Phase F sweep.

Walker design choices worth noting:
- mode <value> / messages <value> use Choice(Word, Word, Ident)
  so known keywords appear in the expected-set; the trailing
  Ident catch-all funnels unknown values into the friendly
  validator that always errors with the catalog wording.
- save / save as is one CommandNode (Optional(Word("as"))) -
  closes the round-5 "save Tab can't offer as" limitation
  structurally.
- Path-bearing UX shipped per ADR-0024: BarePath terminates at
  whitespace; paths with spaces use the (not-yet-wired) quoted
  form. Existing tests pass on the new shape.

Tests:
- 28 new walker-specific tests in dsl::walker::tests covering
  every app-lifecycle command, friendly-error wording for
  mode/messages unknown values, trailing-garbage detection,
  whitespace tolerance, and routing fall-through.
- Total: 805 passed, 0 failed, 1 ignored (was 777 / 1).
- cargo clippy --all-targets -- -D warnings clean.
This commit is contained in:
claude@clouddev1
2026-05-15 06:39:29 +00:00
parent 3e1ff83f26
commit 50b3542050
9 changed files with 1696 additions and 60 deletions
+330
View File
@@ -0,0 +1,330 @@
//! Per-node-kind walk dispatch (ADR-0024 §architecture).
//!
//! `walk_node` is the recursive workhorse that the public
//! `walk()` entry calls into for a `CommandNode`'s `shape`. It
//! tries to match `node` starting at `position`, mutating
//! `path` (matched terminals collected in declaration order) and
//! `per_byte` (highlight class assignments) as it goes.
//!
//! The return value distinguishes four cases:
//!
//! - `Matched { end }` — full match, walker consumed up to `end`.
//! - `NoMatch { … }` — node didn't engage at this position. For
//! `Optional` and `Choice` callers this is benign (try the
//! next branch / skip the optional); for `Seq` it's only
//! benign on the first child.
//! - `Incomplete { … }` — node committed (consumed at least one
//! terminal) but ran out of input. Surfaces as
//! `WalkOutcome::Incomplete` at the top level.
//! - `Failed { … }` — node committed and a content validator
//! rejected the value, or a hard structural failure occurred
//! mid-shape. Surfaces as `WalkOutcome::Mismatch` or
//! `WalkOutcome::ValidationFailed` at the top level.
use crate::dsl::grammar::{HighlightClass, Node, ValidationError};
use crate::dsl::walker::context::WalkContext;
use crate::dsl::walker::lex_helpers::{consume_bare_path, consume_ident, skip_whitespace};
use crate::dsl::walker::outcome::{
ByteClass, Expectation, MatchedItem, MatchedKind, MatchedPath,
};
#[derive(Debug, Clone)]
pub enum NodeWalkResult {
Matched {
end: usize,
},
/// Did not engage at this position. Caller decides whether
/// this is benign (Optional, Choice fallthrough) or a hard
/// failure (Seq mid-shape).
NoMatch {
position: usize,
expected: Vec<Expectation>,
},
/// Committed and ran out of input.
Incomplete {
position: usize,
expected: Vec<Expectation>,
},
/// Committed and hit a hard mismatch or validator failure.
Failed {
position: usize,
kind: FailureKind,
},
}
#[derive(Debug, Clone)]
pub enum FailureKind {
Mismatch { expected: Vec<Expectation> },
Validation(ValidationError),
}
pub fn walk_node(
source: &str,
position: usize,
node: &Node,
ctx: &mut WalkContext,
path: &mut MatchedPath,
per_byte: &mut Vec<ByteClass>,
) -> NodeWalkResult {
let pos = skip_whitespace(source, position);
match node {
Node::Word(word) => walk_word(source, pos, word, path, per_byte),
Node::Punct(ch) => walk_punct(source, pos, *ch, path, per_byte),
Node::Ident {
source: src,
role,
validator,
highlight_override: _,
} => walk_ident(source, pos, *src, role, *validator, path, per_byte),
Node::NumberLit
| Node::StringLit
| Node::BlobLit
| Node::Flag(_)
| Node::Repeated { .. }
| Node::DynamicSubgrammar(_) => {
// Phase A: not exercised by app-lifecycle commands.
// Reaching this branch means a Phase B+ grammar got
// declared without the walker support landing yet —
// surface as a hard failure so the test suite catches
// it loudly instead of silently mis-parsing.
NodeWalkResult::Failed {
position: pos,
kind: FailureKind::Mismatch { expected: vec![] },
}
}
Node::BarePath => walk_bare_path(source, pos, path, per_byte),
Node::Choice(children) => walk_choice(source, pos, children, ctx, path, per_byte),
Node::Seq(children) => walk_seq(source, pos, children, ctx, path, per_byte),
Node::Optional(child) => walk_optional(source, pos, child, ctx, path, per_byte),
}
}
fn walk_word(
source: &str,
position: usize,
word: &crate::dsl::grammar::Word,
path: &mut MatchedPath,
per_byte: &mut Vec<ByteClass>,
) -> NodeWalkResult {
// First scan an identifier-shape token at `position`; if
// none, we definitely don't have this keyword. If one, check
// it against the word's primary + aliases.
let Some((start, end)) = consume_ident(source, position) else {
return NodeWalkResult::NoMatch {
position,
expected: vec![Expectation::Word(word.primary)],
};
};
let candidate = &source[start..end];
if word.matches(candidate) {
path.push(MatchedItem {
kind: MatchedKind::Word(word.primary),
text: candidate.to_string(),
span: (start, end),
});
per_byte.push(ByteClass {
start,
end,
class: HighlightClass::Keyword,
});
NodeWalkResult::Matched { end }
} else {
NodeWalkResult::NoMatch {
position,
expected: vec![Expectation::Word(word.primary)],
}
}
}
fn walk_punct(
source: &str,
position: usize,
ch: char,
path: &mut MatchedPath,
per_byte: &mut Vec<ByteClass>,
) -> NodeWalkResult {
let bytes = source.as_bytes();
if position < bytes.len() && bytes[position] == ch as u8 {
path.push(MatchedItem {
kind: MatchedKind::Punct(ch),
text: ch.to_string(),
span: (position, position + 1),
});
per_byte.push(ByteClass {
start: position,
end: position + 1,
class: HighlightClass::Punct,
});
NodeWalkResult::Matched {
end: position + 1,
}
} else {
NodeWalkResult::NoMatch {
position,
expected: vec![Expectation::Punct(ch)],
}
}
}
fn walk_ident(
source: &str,
position: usize,
_src: crate::dsl::grammar::IdentSource,
role: &'static str,
validator: Option<crate::dsl::grammar::IdentValidator>,
path: &mut MatchedPath,
per_byte: &mut Vec<ByteClass>,
) -> NodeWalkResult {
let Some((start, end)) = consume_ident(source, position) else {
return NodeWalkResult::NoMatch {
position,
expected: vec![Expectation::Ident { role }],
};
};
let text = source[start..end].to_string();
if let Some(v) = validator
&& let Err(err) = v(&text)
{
return NodeWalkResult::Failed {
position: start,
kind: FailureKind::Validation(err),
};
}
path.push(MatchedItem {
kind: MatchedKind::Ident { role },
text,
span: (start, end),
});
per_byte.push(ByteClass {
start,
end,
class: HighlightClass::Identifier,
});
NodeWalkResult::Matched { end }
}
fn walk_bare_path(
source: &str,
position: usize,
path: &mut MatchedPath,
per_byte: &mut Vec<ByteClass>,
) -> NodeWalkResult {
let Some((start, end)) = consume_bare_path(source, position) else {
return NodeWalkResult::NoMatch {
position,
expected: vec![Expectation::BarePath],
};
};
let text = source[start..end].to_string();
path.push(MatchedItem {
kind: MatchedKind::BarePath,
text,
span: (start, end),
});
per_byte.push(ByteClass {
start,
end,
class: HighlightClass::String,
});
NodeWalkResult::Matched { end }
}
fn walk_choice(
source: &str,
position: usize,
children: &[Node],
ctx: &mut WalkContext,
path: &mut MatchedPath,
per_byte: &mut Vec<ByteClass>,
) -> NodeWalkResult {
let mut all_expected: Vec<Expectation> = Vec::new();
for child in children {
let saved_path_len = path.items.len();
let saved_byte_len = per_byte.len();
match walk_node(source, position, child, ctx, path, per_byte) {
NodeWalkResult::Matched { end } => return NodeWalkResult::Matched { end },
NodeWalkResult::NoMatch { expected, .. } => {
path.items.truncate(saved_path_len);
per_byte.truncate(saved_byte_len);
merge_expected(&mut all_expected, expected);
}
// Once a choice branch commits, propagate its outcome.
other => return other,
}
}
NodeWalkResult::NoMatch {
position,
expected: all_expected,
}
}
fn walk_seq(
source: &str,
position: usize,
children: &[Node],
ctx: &mut WalkContext,
path: &mut MatchedPath,
per_byte: &mut Vec<ByteClass>,
) -> NodeWalkResult {
let mut cur = position;
let mut idx = 0;
for child in children {
match walk_node(source, cur, child, ctx, path, per_byte) {
NodeWalkResult::Matched { end } => {
cur = end;
idx += 1;
}
NodeWalkResult::NoMatch { position, expected } => {
if idx == 0 {
// Seq didn't even start.
return NodeWalkResult::NoMatch { position, expected };
}
// Mid-shape: did we run out of input or hit a
// wrong token?
let post_ws = skip_whitespace(source, position);
let kind = if post_ws >= source.len() {
return NodeWalkResult::Incomplete { position: post_ws, expected };
} else {
FailureKind::Mismatch { expected }
};
return NodeWalkResult::Failed { position: post_ws, kind };
}
NodeWalkResult::Incomplete { position, expected } => {
return NodeWalkResult::Incomplete { position, expected };
}
NodeWalkResult::Failed { position, kind } => {
return NodeWalkResult::Failed { position, kind };
}
}
}
NodeWalkResult::Matched { end: cur }
}
fn walk_optional(
source: &str,
position: usize,
child: &Node,
ctx: &mut WalkContext,
path: &mut MatchedPath,
per_byte: &mut Vec<ByteClass>,
) -> NodeWalkResult {
let saved_path_len = path.items.len();
let saved_byte_len = per_byte.len();
match walk_node(source, position, child, ctx, path, per_byte) {
NodeWalkResult::Matched { end } => NodeWalkResult::Matched { end },
NodeWalkResult::NoMatch { .. } => {
path.items.truncate(saved_path_len);
per_byte.truncate(saved_byte_len);
NodeWalkResult::Matched { end: position }
}
other => other,
}
}
fn merge_expected(dst: &mut Vec<Expectation>, src: Vec<Expectation>) {
for e in src {
if !dst.contains(&e) {
dst.push(e);
}
}
}