oli/rdbms-playground
1
0
Fork 0
Code Issues 4 Pull Requests Actions Packages Projects Releases 1 Wiki Activity

walker: 3a — category-grouped mode-aware dispatch (ADR-0033 Amendment 1)

Browse Source 
Replaces ADR-0033 §2's original Node::Guard + Choice(SQL,DSL) mechanism,
which was found during 3a to be unworkable: any guard-in-Choice approach
forces a walk_choice change (walk_choice falls through only on NoMatch, so
simple-mode valid-DSL would wrongly surface "this is SQL"), and walk_seq
treats a NoMatch past idx 0 as a hard Failed, breaking advanced-mode DSL
fall-through.

Mechanism (Amendment 1): each REGISTRY entry is tagged
CommandCategory::{Simple, Advanced}, generalising the whole-command
is_advanced_only gate. walk() becomes a thin dispatcher over decide()
(mode-aware candidate selection: simple commits the DSL node or emits the
"this is SQL" hint; advanced tries SQL first, DSL as a full-line fallback)
and an extracted walk_one_command(); speculative match-testing runs on a
scratch WalkContext so the caller's context is only touched by the
committed walk. No Node::Guard, no walk_choice/walk_seq change.

6 dispatch smoke tests on a shared-entry-word smoke registry; 1446 baseline
green; clippy clean.
This commit is contained in:
claude@clouddev1
2026-05-21 18:18:50 +00:00
parent a37a0b7d40
commit 4e16d97fe0
5 changed files with 720 additions and 92 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/dsl/walker/mod.rs
+463 -40
View File
@@ -280,11 +280,11 @@ pub fn completion_probe_in_mode(
let mode_filtered_entries = || -> Vec<outcome::Expectation> {
REGISTRY
.iter()
.filter(|c| {
.filter(|(c, _)| {
mode == crate::mode::Mode::Advanced
|| !is_advanced_only(c.entry.primary)
})
.map(|c| outcome::Expectation::Word(c.entry.primary))
.map(|(c, _)| outcome::Expectation::Word(c.entry.primary))
.collect()
};
@@ -1529,11 +1529,11 @@ pub fn expected_at_input_in_mode(
let mode_filtered = || -> Vec<outcome::Expectation> {
REGISTRY
.iter()
.filter(|c| {
.filter(|(c, _)| {
mode == crate::mode::Mode::Advanced
|| !is_advanced_only(c.entry.primary)
})
.map(|c| outcome::Expectation::Word(c.entry.primary))
.map(|(c, _)| outcome::Expectation::Word(c.entry.primary))
.collect()
};
@@ -1614,7 +1614,7 @@ fn expected_for_hint_snapshot(
let entry_words = || -> Vec<outcome::Expectation> {
REGISTRY
.iter()
.map(|c| outcome::Expectation::Word(c.entry.primary))
.map(|(c, _)| outcome::Expectation::Word(c.entry.primary))
.collect()
};
@@ -1690,11 +1690,267 @@ pub fn walk<'a>(
return (None, None);
};
let entry_text = &effective_source[kw_start..kw_end];
let Some((command_idx, command_node)) = grammar::command_for_entry_word(entry_text)
else {
let candidates = grammar::commands_for_entry_word(entry_text);
if candidates.is_empty() {
// First token isn't a registered entry word — yield to
// the chumsky path.
return (None, None);
};
}
// ADR-0033 Amendment 1 — category-grouped, mode-aware
// dispatch. `decide` chooses which registered candidate to
// commit (or emits the "this is SQL" hint), running any
// speculative match-testing on scratch contexts so the
// caller's `ctx` is only ever touched by the committed walk.
match decide(
effective_source,
kw_start,
kw_end,
&candidates,
ctx.mode,
ctx.schema,
) {
Decision::Commit { idx, node } => {
let (result, cmd) =
walk_one_command(effective_source, source, kw_start, kw_end, idx, node, ctx);
(Some(result), cmd)
}
Decision::ThisIsSql { primary } => (
Some(this_is_sql_result(entry_text, primary, kw_start, kw_end)),
None,
),
}
}
/// The dispatcher's choice for a given input (ADR-0033
/// Amendment 1): commit a specific registered candidate, or emit
/// the simple-mode "this is SQL" hint.
enum Decision {
/// Walk this candidate into the caller's `WalkContext`.
Commit {
idx: usize,
node: &'static crate::dsl::grammar::CommandNode,
},
/// Simple mode with SQL-shaped input: emit
/// `advanced_mode.sql_in_simple`, carrying this entry literal.
ThisIsSql { primary: &'static str },
}
/// Category-grouped, mode-aware dispatch decision (ADR-0033
/// Amendment 1).
///
/// Pure with respect to the caller's context: any speculative
/// match-testing runs on a fresh scratch `WalkContext` (see
/// `scratch_outcome`), so `decide` never mutates the caller's
/// accumulators.
///
/// - **Simple mode** commits the DSL (`Simple`) candidate. With
/// no DSL candidate (a SQL-only entry word) it emits the
/// "this is SQL" hint. For a shared entry word whose DSL shape
/// does not match but whose SQL shape does, it also emits the
/// hint — so `delete … returning *` in simple mode points the
/// user at advanced mode rather than at a bare DSL parse error.
/// - **Advanced mode** tries `Advanced` candidates first, then
/// the `Simple` candidate as a fallback; the first full match
/// wins. When none fully match it commits the candidate that
/// progressed furthest (advanced-first on ties) so the surfaced
/// error is the most informative.
fn decide(
effective_source: &str,
kw_start: usize,
kw_end: usize,
candidates: &[(
usize,
&'static crate::dsl::grammar::CommandNode,
crate::dsl::grammar::CommandCategory,
)],
mode: crate::mode::Mode,
schema: Option<&crate::completion::SchemaCache>,
) -> Decision {
use crate::dsl::grammar::CommandCategory;
let advanced: Vec<(usize, &'static crate::dsl::grammar::CommandNode)> = candidates
.iter()
.filter(|(_, _, cat)| *cat == CommandCategory::Advanced)
.map(|(i, n, _)| (*i, *n))
.collect();
let simple: Vec<(usize, &'static crate::dsl::grammar::CommandNode)> = candidates
.iter()
.filter(|(_, _, cat)| *cat == CommandCategory::Simple)
.map(|(i, n, _)| (*i, *n))
.collect();
match mode {
crate::mode::Mode::Simple => {
let Some(&(sidx, snode)) = simple.first() else {
// No DSL candidate — the entry word is SQL-only.
let primary = candidates.first().map_or("", |(_, n, _)| n.entry.primary);
return Decision::ThisIsSql { primary };
};
if advanced.is_empty() {
return Decision::Commit { idx: sidx, node: snode };
}
// Shared entry word: prefer the DSL node; only point
// at advanced mode when the DSL shape does not match
// but the SQL shape does.
if scratch_full_match(effective_source, kw_start, kw_end, snode, mode, schema) {
return Decision::Commit { idx: sidx, node: snode };
}
let (_, anode) = advanced[0];
if scratch_full_match(effective_source, kw_start, kw_end, anode, mode, schema) {
return Decision::ThisIsSql {
primary: anode.entry.primary,
};
}
Decision::Commit { idx: sidx, node: snode }
}
crate::mode::Mode::Advanced => {
// Advanced candidates first, DSL as the fallback.
let ordered: Vec<(usize, &'static crate::dsl::grammar::CommandNode)> =
advanced.iter().chain(simple.iter()).copied().collect();
// `candidates` is non-empty (the caller checked), so
// `ordered` is non-empty too.
if ordered.len() == 1 {
let (idx, node) = ordered[0];
return Decision::Commit { idx, node };
}
for &(idx, node) in &ordered {
if scratch_full_match(effective_source, kw_start, kw_end, node, mode, schema) {
return Decision::Commit { idx, node };
}
}
// None fully matched — commit the furthest-progress
// candidate, keeping the first (advanced) on ties.
let mut best = ordered[0];
let mut best_progress =
scratch_progress(effective_source, kw_start, kw_end, best.1, mode, schema);
for &(idx, node) in &ordered[1..] {
let progress =
scratch_progress(effective_source, kw_start, kw_end, node, mode, schema);
if progress > best_progress {
best = (idx, node);
best_progress = progress;
}
}
Decision::Commit {
idx: best.0,
node: best.1,
}
}
}
}
/// Build the `advanced_mode.sql_in_simple` result for a SQL entry
/// word typed in simple mode (ADR-0030 §2, ADR-0033 Amendment 1).
/// The entry word stays highlighted as a keyword; the input
/// carries an ERROR verdict (it will not run here).
fn this_is_sql_result(
entry_text: &str,
primary: &'static str,
kw_start: usize,
kw_end: usize,
) -> WalkResult {
let mut path = MatchedPath::new();
let mut per_byte = Vec::new();
path.push(crate::dsl::walker::outcome::MatchedItem {
kind: crate::dsl::walker::outcome::MatchedKind::Word(primary),
text: entry_text.to_string(),
span: (kw_start, kw_end),
});
per_byte.push(crate::dsl::walker::outcome::ByteClass {
start: kw_start,
end: kw_end,
class: grammar::HighlightClass::Keyword,
});
WalkResult {
outcome: WalkOutcome::ValidationFailed {
position: kw_start,
error: crate::dsl::grammar::ValidationError {
message_key: "advanced_mode.sql_in_simple",
args: vec![("command", primary.to_string())],
},
},
matched_path: path,
per_byte_class: per_byte,
diagnostics: Vec::new(),
tail_expected: Vec::new(),
}
}
/// Run `walk_one_command` on a fresh scratch `WalkContext` so the
/// dispatcher can test a candidate without disturbing the
/// caller's accumulators (ADR-0033 Amendment 1).
fn scratch_outcome(
effective_source: &str,
kw_start: usize,
kw_end: usize,
node: &'static crate::dsl::grammar::CommandNode,
mode: crate::mode::Mode,
schema: Option<&crate::completion::SchemaCache>,
) -> WalkOutcome {
let mut sctx = schema.map_or_else(context::WalkContext::new, context::WalkContext::with_schema);
sctx.mode = mode;
let (result, _cmd) =
walk_one_command(effective_source, effective_source, kw_start, kw_end, 0, node, &mut sctx);
result.outcome
}
/// Whether a candidate fully matches the input (a clean
/// `WalkOutcome::Match`), tested on a scratch context.
fn scratch_full_match(
effective_source: &str,
kw_start: usize,
kw_end: usize,
node: &'static crate::dsl::grammar::CommandNode,
mode: crate::mode::Mode,
schema: Option<&crate::completion::SchemaCache>,
) -> bool {
matches!(
scratch_outcome(effective_source, kw_start, kw_end, node, mode, schema),
WalkOutcome::Match { .. }
)
}
/// How far (byte position) a candidate's walk progressed. A full
/// match scores the whole input; a failure scores its failure
/// position. Used only to tie-break when no candidate fully
/// matches.
fn scratch_progress(
effective_source: &str,
kw_start: usize,
kw_end: usize,
node: &'static crate::dsl::grammar::CommandNode,
mode: crate::mode::Mode,
schema: Option<&crate::completion::SchemaCache>,
) -> usize {
match scratch_outcome(effective_source, kw_start, kw_end, node, mode, schema) {
WalkOutcome::Match { .. } => effective_source.len(),
WalkOutcome::Incomplete { position, .. }
| WalkOutcome::Mismatch { position, .. }
| WalkOutcome::ValidationFailed { position, .. } => position,
}
}
/// Walk a *single* committed command's shape and produce its
/// `WalkResult` + optional `Command` (ADR-0033 Amendment 1).
///
/// Factored out of `walk` so the dispatcher's speculative
/// match-testing (`scratch_outcome`) reuses the exact same walk +
/// outcome-mapping + AST-builder + diagnostic path on a scratch
/// context, while the committed walk runs into the caller's
/// context. `source` is the full (unbounded) input the AST
/// builder reads for SQL command text; `effective_source` is the
/// bound-trimmed slice the walker matches against.
fn walk_one_command<'a>(
effective_source: &str,
source: &str,
kw_start: usize,
kw_end: usize,
command_idx: usize,
command_node: &'static crate::dsl::grammar::CommandNode,
ctx: &mut WalkContext<'a>,
) -> (WalkResult, Option<Command>) {
let entry_text = &effective_source[kw_start..kw_end];
let mut path = MatchedPath::new();
let mut per_byte = Vec::new();
@@ -1710,37 +1966,6 @@ pub fn walk<'a>(
class: grammar::HighlightClass::Keyword,
});
// Mode gate (ADR-0030 §2): an advanced-only command (a SQL
// form) typed in simple mode is *recognised as SQL* and
// yields a precise hint — "this is SQL; switch with `mode
// advanced`, or prefix the line with `:`" — rather than
// being walked normally or rejected as an unknown command.
// The entry word stays highlighted as a keyword (it is one);
// the input carries an ERROR verdict (it will not run here).
if ctx.mode == crate::mode::Mode::Simple
&& grammar::is_advanced_only(command_node.entry.primary)
{
return (
Some(WalkResult {
outcome: WalkOutcome::ValidationFailed {
position: kw_start,
error: crate::dsl::grammar::ValidationError {
message_key: "advanced_mode.sql_in_simple",
args: vec![(
"command",
command_node.entry.primary.to_string(),
)],
},
},
matched_path: path,
per_byte_class: per_byte,
diagnostics: Vec::new(),
tail_expected: Vec::new(),
}),
None,
);
}
let mut tail_expected: Vec<Expectation> = Vec::new();
let outcome = match walk_node(
effective_source,
@@ -1879,7 +2104,7 @@ pub fn walk<'a>(
tail_expected,
diagnostics,
};
(Some(result), cmd)
(result, cmd)
}
#[cfg(test)]
@@ -4361,3 +4586,201 @@ mod projection_before_from_tests {
);
}
}
/// Sub-phase 3a — category-grouped, mode-aware dispatch
/// (ADR-0033 Amendment 1).
///
/// These exercise the dispatch mechanism end-to-end on a *smoke*
/// registry: a single shared entry word (`smk`) carrying a
/// `Simple` DSL node and an `Advanced` SQL node with
/// distinguishable tails (`dsltail` / `sqltail`). The dispatch
/// functions (`decide`, `walk_one_command`, `this_is_sql_result`)
/// are module-private; this child module reaches them via
/// `super::*`. The smoke nodes never enter the real `REGISTRY`,
/// so production dispatch is unaffected.
#[cfg(test)]
mod dispatch_3a_tests {
use super::*;
use crate::dsl::command::{AppCommand, Command};
use crate::dsl::grammar::{
CommandCategory, CommandNode, Node, ValidationError, Word,
};
use crate::dsl::walker::lex_helpers::{consume_ident, skip_whitespace};
use crate::dsl::walker::outcome::MatchedPath;
use crate::mode::Mode;
// Distinct dummy commands so a test can tell which node a walk
// committed to (the outcome alone doesn't distinguish them).
fn dsl_builder(_: &MatchedPath, _: &str) -> Result<Command, ValidationError> {
Ok(Command::App(AppCommand::Help))
}
fn sql_builder(_: &MatchedPath, _: &str) -> Result<Command, ValidationError> {
Ok(Command::App(AppCommand::Quit))
}
static SMOKE_DSL: CommandNode = CommandNode {
entry: Word::keyword("smk"),
shape: Node::Word(Word::keyword("dsltail")),
ast_builder: dsl_builder,
help_id: None,
usage_ids: &[],
};
static SMOKE_SQL: CommandNode = CommandNode {
entry: Word::keyword("smk"),
shape: Node::Word(Word::keyword("sqltail")),
ast_builder: sql_builder,
help_id: None,
usage_ids: &[],
};
type Candidates = Vec<(usize, &'static CommandNode, CommandCategory)>;
/// A shared entry word: both a DSL and a SQL node under `smk`.
/// Listed SQL-first to prove `decide` partitions by category
/// rather than relying on registry order.
fn shared() -> Candidates {
vec![
(0, &SMOKE_SQL, CommandCategory::Advanced),
(1, &SMOKE_DSL, CommandCategory::Simple),
]
}
/// A SQL-only entry word (no DSL fallback) — models `select`.
fn sql_only() -> Candidates {
vec![(0, &SMOKE_SQL, CommandCategory::Advanced)]
}
fn kw(input: &str) -> (usize, usize) {
let start = skip_whitespace(input, 0);
consume_ident(input, start).expect("entry word")
}
fn run_decide(input: &str, mode: Mode, cands: &Candidates) -> Decision {
let (ks, ke) = kw(input);
decide(input, ks, ke, cands, mode, None)
}
/// Mirror `walk`'s dispatch: decide, then either walk the
/// committed node or build the "this is SQL" result. Returns
/// the resulting outcome plus the committed command (if any).
fn dispatch(input: &str, mode: Mode, cands: &Candidates) -> (WalkOutcome, Option<Command>) {
let (ks, ke) = kw(input);
let entry_text = &input[ks..ke];
match decide(input, ks, ke, cands, mode, None) {
Decision::Commit { idx, node } => {
let mut ctx = context::WalkContext::new();
ctx.mode = mode;
let (res, cmd) = walk_one_command(input, input, ks, ke, idx, node, &mut ctx);
(res.outcome, cmd)
}
Decision::ThisIsSql { primary } => {
(this_is_sql_result(entry_text, primary, ks, ke).outcome, None)
}
}
}
fn committed_node(input: &str, mode: Mode, cands: &Candidates) -> &'static CommandNode {
match run_decide(input, mode, cands) {
Decision::Commit { node, .. } => node,
Decision::ThisIsSql { .. } => panic!("expected Commit, got ThisIsSql for {input:?}"),
}
}
// ---- Exit-gate case 1: Simple + DSL input → DSL match ------
#[test]
fn simple_mode_dsl_input_matches_dsl() {
let cands = shared();
assert!(
std::ptr::eq(committed_node("smk dsltail", Mode::Simple, &cands), &SMOKE_DSL),
"simple mode must commit the DSL node for DSL input",
);
let (outcome, cmd) = dispatch("smk dsltail", Mode::Simple, &cands);
assert!(matches!(outcome, WalkOutcome::Match { .. }), "got {outcome:?}");
assert_eq!(cmd, Some(Command::App(AppCommand::Help)));
}
// ---- Exit-gate case 2: Advanced + SQL input → SQL match ----
#[test]
fn advanced_mode_sql_input_matches_sql() {
let cands = shared();
assert!(
std::ptr::eq(committed_node("smk sqltail", Mode::Advanced, &cands), &SMOKE_SQL),
"advanced mode must commit the SQL node for SQL input",
);
let (outcome, cmd) = dispatch("smk sqltail", Mode::Advanced, &cands);
assert!(matches!(outcome, WalkOutcome::Match { .. }), "got {outcome:?}");
assert_eq!(cmd, Some(Command::App(AppCommand::Quit)));
}
// ---- Exit-gate case 3: Simple + SQL-only input →
// ValidationFailed advanced_mode.sql_in_simple ----------
#[test]
fn simple_mode_sql_only_input_is_this_is_sql() {
// Shared word, but the input matches only the SQL tail.
let cands = shared();
match run_decide("smk sqltail", Mode::Simple, &cands) {
Decision::ThisIsSql { primary } => assert_eq!(primary, "smk"),
Decision::Commit { idx, .. } => {
panic!("expected ThisIsSql, got Commit {{ idx: {idx} }}")
}
}
let (outcome, cmd) = dispatch("smk sqltail", Mode::Simple, &cands);
match outcome {
WalkOutcome::ValidationFailed { error, .. } => {
assert_eq!(error.message_key, "advanced_mode.sql_in_simple");
}
other => panic!("expected ValidationFailed, got {other:?}"),
}
assert_eq!(cmd, None);
}
/// A pure SQL-only entry word (no DSL node, like `select`) in
/// simple mode also yields the "this is SQL" hint — the
/// behaviour the old whole-command `is_advanced_only` gate
/// produced, now via `decide`.
#[test]
fn simple_mode_sql_only_entry_word_is_this_is_sql() {
let cands = sql_only();
let (outcome, _) = dispatch("smk sqltail", Mode::Simple, &cands);
match outcome {
WalkOutcome::ValidationFailed { error, .. } => {
assert_eq!(error.message_key, "advanced_mode.sql_in_simple");
}
other => panic!("expected ValidationFailed, got {other:?}"),
}
}
// ---- Exit-gate case 4 / 5: Advanced + DSL-only input →
// DSL match via fallback (the R1-equivalent invariant) --
#[test]
fn advanced_mode_dsl_input_falls_back_to_dsl() {
// `dsltail` matches the DSL node but NOT the SQL node.
// Advanced mode tries SQL first; it must fall back to the
// DSL node rather than surfacing the SQL node's failure.
let cands = shared();
assert!(
std::ptr::eq(committed_node("smk dsltail", Mode::Advanced, &cands), &SMOKE_DSL),
"advanced mode must fall back to DSL when SQL doesn't match",
);
let (outcome, cmd) = dispatch("smk dsltail", Mode::Advanced, &cands);
assert!(matches!(outcome, WalkOutcome::Match { .. }), "got {outcome:?}");
assert_eq!(cmd, Some(Command::App(AppCommand::Help)));
}
/// In advanced mode a non-shared DSL entry word (no Advanced
/// candidate) still commits the single DSL node.
#[test]
fn advanced_mode_dsl_only_entry_word_commits_dsl() {
let cands: Candidates = vec![(0, &SMOKE_DSL, CommandCategory::Simple)];
assert!(std::ptr::eq(
committed_node("smk dsltail", Mode::Advanced, &cands),
&SMOKE_DSL,
));
let (outcome, _) = dispatch("smk dsltail", Mode::Advanced, &cands);
assert!(matches!(outcome, WalkOutcome::Match { .. }), "got {outcome:?}");
}
}