oli/rdbms-playground
1
0
Fork 0
Code Issues 4 Pull Requests Actions Packages Projects Releases 1 Wiki Activity

ADR-0024 Phase A: walker framework + app-lifecycle commands

Browse Source 
Stand up the unified-grammar tree walker alongside the existing
chumsky parser and migrate the eleven app-lifecycle commands
(quit, help, rebuild, save / save as, new, load, export, import,
mode, messages) end-to-end. The router in parse_tokens consults
the walker first; non-migrated commands still fall through to
chumsky.

Scope:
- src/dsl/grammar/{mod,app}.rs: Node enum (13 kinds), Word /
  IdentSource / HintMode / HighlightClass / ValidationError /
  CommandNode types, REGISTRY of the eleven app commands.
- src/dsl/walker/{mod,driver,context,outcome,lex_helpers}.rs:
  scannerless byte-level walker, per-node-kind dispatch with
  Choice/Seq/Optional backtracking, WalkContext (Phase B-D
  schema fields stubbed), WalkOutcome with Match/Incomplete/
  Mismatch/ValidationFailed.
- src/dsl/parser.rs: try_walker_route() runs first in
  parse_tokens; bridge converts WalkOutcome to ParseError
  preserving catalog wording (mode.unknown / messages.unknown
  surface verbatim via friendly::translate). Legacy
  try_parse_app_path_command deleted; chumsky's bare-keyword
  app branches remain unreachable until Phase F sweep.

Walker design choices worth noting:
- mode <value> / messages <value> use Choice(Word, Word, Ident)
  so known keywords appear in the expected-set; the trailing
  Ident catch-all funnels unknown values into the friendly
  validator that always errors with the catalog wording.
- save / save as is one CommandNode (Optional(Word("as"))) -
  closes the round-5 "save Tab can't offer as" limitation
  structurally.
- Path-bearing UX shipped per ADR-0024: BarePath terminates at
  whitespace; paths with spaces use the (not-yet-wired) quoted
  form. Existing tests pass on the new shape.

Tests:
- 28 new walker-specific tests in dsl::walker::tests covering
  every app-lifecycle command, friendly-error wording for
  mode/messages unknown values, trailing-garbage detection,
  whitespace tolerance, and routing fall-through.
- Total: 805 passed, 0 failed, 1 ignored (was 777 / 1).
- cargo clippy --all-targets -- -D warnings clean.
This commit is contained in:
claude@clouddev1
2026-05-15 06:39:29 +00:00
parent 3e1ff83f26
commit 50b3542050
9 changed files with 1696 additions and 60 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/dsl/walker/mod.rs
+432
View File
@@ -0,0 +1,432 @@
//! Walker entry point (ADR-0024 §architecture).
//!
//! The walker is the single source of truth for the migrated
//! commands. Phase A wires the parse consumer; completion +
//! highlighting still flow through the chumsky path until
//! Phase D / F.
//!
//! Routing rule (ADR-0024 §migration): the input's first
//! identifier-shape token decides whether the walker owns this
//! command. If it matches a registered entry word, the walker
//! takes over end-to-end (success or failure). Otherwise, the
//! router falls through to the chumsky parser, which still
//! carries every non-migrated command's grammar through Phase F.
pub mod context;
pub mod driver;
pub mod lex_helpers;
pub mod outcome;
use crate::dsl::command::Command;
use crate::dsl::grammar;
use crate::dsl::walker::context::WalkContext;
use crate::dsl::walker::driver::{FailureKind, NodeWalkResult, walk_node};
use crate::dsl::walker::lex_helpers::{consume_ident, skip_whitespace};
use crate::dsl::walker::outcome::{
Expectation, MatchedPath, WalkBound, WalkOutcome, WalkResult,
};
pub use context::ColumnInfo;
/// Public walk entry. `bound` is `EndOfInput` for parse;
/// `Position(cursor)` for completion / hint (Phase A: not yet
/// wired).
///
/// Returns:
/// - `(Some(WalkResult), Some(Command))` on full match — the
/// AST builder produced a typed Command.
/// - `(Some(WalkResult), None)` on failure where the walker
/// committed (matched the entry word). Caller surfaces the
/// walker's error.
/// - `(None, None)` when the entry word doesn't match any
/// registered command — the router falls through to chumsky.
pub fn walk(
source: &str,
bound: WalkBound,
ctx: &mut WalkContext,
) -> (Option<WalkResult>, Option<Command>) {
// Phase A only consumes EndOfInput; Position would slice
// the source, which is the same operation.
let effective_source: &str = match bound {
WalkBound::EndOfInput => source,
WalkBound::Position(end) => &source[..end.min(source.len())],
};
let start = skip_whitespace(effective_source, 0);
if start >= effective_source.len() {
return (None, None);
}
// Identify the command by its entry word. If the first
// identifier-shape token isn't a registered entry, the
// walker yields to chumsky.
let Some((kw_start, kw_end)) = consume_ident(effective_source, start) else {
return (None, None);
};
let entry_text = &effective_source[kw_start..kw_end];
let Some((command_idx, command_node)) = grammar::command_for_entry_word(entry_text)
else {
return (None, None);
};
let mut path = MatchedPath::new();
let mut per_byte = Vec::new();
// Record the entry-word match.
path.push(crate::dsl::walker::outcome::MatchedItem {
kind: crate::dsl::walker::outcome::MatchedKind::Word(command_node.entry.primary),
text: entry_text.to_string(),
span: (kw_start, kw_end),
});
per_byte.push(crate::dsl::walker::outcome::ByteClass {
start: kw_start,
end: kw_end,
class: grammar::HighlightClass::Keyword,
});
let outcome = match walk_node(
effective_source,
kw_end,
&command_node.shape,
ctx,
&mut path,
&mut per_byte,
) {
NodeWalkResult::Matched { end } => {
let trailing = skip_whitespace(effective_source, end);
if trailing < effective_source.len() {
WalkOutcome::Mismatch {
position: trailing,
expected: vec![Expectation::EndOfInput],
}
} else {
WalkOutcome::Match { command_idx }
}
}
NodeWalkResult::NoMatch { position, expected } => {
// The shape required content the user hasn't typed.
// (Optional/empty-Seq shapes always return Matched
// even when skipped, so reaching NoMatch here means
// the command really wanted something more.)
let post = skip_whitespace(effective_source, position);
if post >= effective_source.len() {
WalkOutcome::Incomplete { position: post, expected }
} else {
WalkOutcome::Mismatch { position: post, expected }
}
}
NodeWalkResult::Incomplete { position, expected } => {
WalkOutcome::Incomplete { position, expected }
}
NodeWalkResult::Failed { position, kind } => match kind {
FailureKind::Mismatch { expected } => {
WalkOutcome::Mismatch { position, expected }
}
FailureKind::Validation(error) => {
WalkOutcome::ValidationFailed { position, error }
}
},
};
let cmd = if matches!(outcome, WalkOutcome::Match { .. }) {
(command_node.ast_builder)(&path).ok()
} else {
None
};
let result = WalkResult {
outcome,
matched_path: path,
per_byte_class: per_byte,
};
(Some(result), cmd)
}
#[cfg(test)]
mod tests {
//! Walker behaviour tests — Phase A (ADR-0024 §migration).
//!
//! These cover every app-lifecycle command the walker now
//! owns. Each input is paired with its expected `Command`
//! output (the differential-against-chumsky check
//! materialised as hand-curated expectations — same role
//! the differential test scaffolding plays per ADR-0024
//! §test-discipline).
//!
//! The handoff document lists these tests as "walker-
//! specific tests for trie-only features" — they pin down
//! the walker's contract for the migrated commands so
//! Phase B-F migrations can refactor without regression.
use crate::dsl::command::{AppCommand, Command, MessagesValue, ModeValue};
use crate::dsl::parser::parse_command;
fn parse(input: &str) -> Result<Command, crate::dsl::ParseError> {
parse_command(input)
}
// ---- Bare no-arg commands ---------------------------------
#[test]
fn walker_parses_quit() {
assert_eq!(parse("quit").unwrap(), Command::App(AppCommand::Quit));
}
#[test]
fn walker_parses_help() {
assert_eq!(parse("help").unwrap(), Command::App(AppCommand::Help));
}
#[test]
fn walker_parses_rebuild() {
assert_eq!(parse("rebuild").unwrap(), Command::App(AppCommand::Rebuild));
}
#[test]
fn walker_parses_new() {
assert_eq!(parse("new").unwrap(), Command::App(AppCommand::New));
}
#[test]
fn walker_parses_load() {
assert_eq!(parse("load").unwrap(), Command::App(AppCommand::Load));
}
// ---- Save / save as ---------------------------------------
#[test]
fn walker_parses_save() {
assert_eq!(parse("save").unwrap(), Command::App(AppCommand::Save));
}
#[test]
fn walker_parses_save_as() {
assert_eq!(parse("save as").unwrap(), Command::App(AppCommand::SaveAs));
}
#[test]
fn walker_save_keywords_case_insensitive() {
assert_eq!(parse("SAVE").unwrap(), Command::App(AppCommand::Save));
assert_eq!(parse("Save AS").unwrap(), Command::App(AppCommand::SaveAs));
}
// ---- Mode -------------------------------------------------
#[test]
fn walker_parses_mode_simple() {
assert_eq!(
parse("mode simple").unwrap(),
Command::App(AppCommand::Mode {
value: ModeValue::Simple,
})
);
}
#[test]
fn walker_parses_mode_advanced() {
assert_eq!(
parse("mode advanced").unwrap(),
Command::App(AppCommand::Mode {
value: ModeValue::Advanced,
})
);
}
#[test]
fn walker_mode_unknown_value_emits_friendly_error() {
let err = parse("mode foo").unwrap_err();
match err {
crate::dsl::ParseError::Invalid { message, .. } => {
// The catalog wording for `mode.unknown` carries
// the user's value verbatim.
assert!(message.contains("foo"), "got: {message}");
}
other => panic!("expected Invalid, got {other:?}"),
}
}
// ---- Messages ---------------------------------------------
#[test]
fn walker_parses_messages_bare() {
assert_eq!(
parse("messages").unwrap(),
Command::App(AppCommand::Messages { value: None })
);
}
#[test]
fn walker_parses_messages_short() {
assert_eq!(
parse("messages short").unwrap(),
Command::App(AppCommand::Messages {
value: Some(MessagesValue::Short),
})
);
}
#[test]
fn walker_parses_messages_verbose() {
assert_eq!(
parse("messages verbose").unwrap(),
Command::App(AppCommand::Messages {
value: Some(MessagesValue::Verbose),
})
);
}
#[test]
fn walker_messages_unknown_value_emits_friendly_error() {
let err = parse("messages bogus").unwrap_err();
match err {
crate::dsl::ParseError::Invalid { message, .. } => {
assert!(message.contains("bogus"), "got: {message}");
}
other => panic!("expected Invalid, got {other:?}"),
}
}
// ---- Export -----------------------------------------------
#[test]
fn walker_parses_export_bare() {
assert_eq!(
parse("export").unwrap(),
Command::App(AppCommand::Export { path: None })
);
}
#[test]
fn walker_parses_export_with_path() {
assert_eq!(
parse("export backups/MyExport.zip").unwrap(),
Command::App(AppCommand::Export {
path: Some("backups/MyExport.zip".to_string()),
})
);
}
#[test]
fn walker_export_trims_trailing_whitespace() {
// Pre-migration the source-slice helper trimmed; the
// walker treats " " after `export` as zero BarePath
// matches and produces the bare form.
assert_eq!(
parse("export ").unwrap(),
Command::App(AppCommand::Export { path: None })
);
}
// ---- Import -----------------------------------------------
#[test]
fn walker_parses_import_bare() {
assert_eq!(
parse("import").unwrap(),
Command::App(AppCommand::Import {
path: String::new(),
target: None,
})
);
}
#[test]
fn walker_parses_import_with_path() {
assert_eq!(
parse("import some/file.zip").unwrap(),
Command::App(AppCommand::Import {
path: "some/file.zip".to_string(),
target: None,
})
);
}
#[test]
fn walker_parses_import_with_path_and_target() {
assert_eq!(
parse("import some/file.zip as MyImported").unwrap(),
Command::App(AppCommand::Import {
path: "some/file.zip".to_string(),
target: Some("MyImported".to_string()),
})
);
}
#[test]
fn walker_import_keeps_as_inside_path() {
// The lexer-free walker terminates `BarePath` at the
// first whitespace byte. `path/asfile.zip` is one
// token; the `as` *inside* it stays part of the path.
assert_eq!(
parse("import path/asfile.zip").unwrap(),
Command::App(AppCommand::Import {
path: "path/asfile.zip".to_string(),
target: None,
})
);
}
#[test]
fn walker_import_trailing_as_without_target_errors() {
let err = parse("import foo.zip as ").unwrap_err();
match err {
crate::dsl::ParseError::Invalid { message, expected, .. } => {
// Phase A: the friendly `project.import_empty_target`
// wording moves out of the parser; the walker's
// structural error names the `target` slot.
assert!(
message.contains("target") || expected.iter().any(|e| e == "target"),
"expected mention of target slot; got message={message:?}, expected={expected:?}"
);
}
other => panic!("expected Invalid, got {other:?}"),
}
}
// ---- Routing fall-through ---------------------------------
#[test]
fn walker_does_not_engage_for_non_app_keywords() {
// The router falls through to the chumsky path. The
// existing chumsky parser produces this Command.
assert!(matches!(
parse("drop table Customers").unwrap(),
Command::DropTable { .. }
));
}
#[test]
fn walker_does_not_engage_for_unknown_first_token() {
// Not an entry word — chumsky yields its usual
// unknown-command error.
assert!(parse("frobulate").is_err());
}
// ---- Trailing-garbage detection ---------------------------
#[test]
fn walker_quit_with_trailing_garbage_errors() {
assert!(parse("quit nonsense").is_err());
}
#[test]
fn walker_save_with_trailing_garbage_errors() {
assert!(parse("save Customers").is_err());
}
// ---- Whitespace tolerance ---------------------------------
#[test]
fn walker_tolerates_leading_and_internal_whitespace() {
assert_eq!(parse(" quit ").unwrap(), Command::App(AppCommand::Quit));
assert_eq!(
parse("save as").unwrap(),
Command::App(AppCommand::SaveAs)
);
assert_eq!(
parse("mode\tadvanced").unwrap(),
Command::App(AppCommand::Mode {
value: ModeValue::Advanced,
})
);
}
}