oli/rdbms-playground
1
0
Fork 0
Code Issues 4 Pull Requests Actions Packages Projects Releases 1 Wiki Activity

ADR-0024 Phase B: DDL commands without value literals

Browse Source 
Migrate the five DDL commands at four entry words: drop (drop
table / drop column / drop relationship), add (add column /
add 1:n relationship), rename (rename column), change (change
column). The walker route now owns these end-to-end; chumsky
declarations remain unreachable for these inputs but stay
until Phase F.

Walker extensions:
- New node kinds: NumberLit (with optional content validator)
  and Literal(&str) (verbatim byte sequence with word-boundary
  lookahead — used for the `1` in `add 1:n …` so it surfaces
  as `\`1\`` in the expected-set, matching the existing
  parse_error_pedagogy contract).
- Flag (--name) terminal — Phase A stubbed; now wired to the
  walker driver with consume_flag() in lex_helpers.
- Repeated combinator with optional separator and `min` floor.
  Used by referential clauses (0..2 `on <delete|update>` runs)
  and change-column flags (0..N --force-conversion /
  --dont-convert; AST builder enforces mutual exclusion).
- Optional now propagates its inner's expectations as a
  `skipped` field on the Matched result. Seq accumulates these
  across children so the next failure's expected-set surfaces
  the full union — closes the keyword-completion regression
  (`add column ` must offer `to`, `table`, plus the table-name
  identifier slot).
- Expectation::Ident gained a `source: IdentSource` field; the
  parser-side bridge maps Tables/Columns/Relationships/Types
  to the IdentSlot::expected_label strings ("table name",
  "column name", …) so the existing completion engine's
  schema-cache lookup still resolves.
- Walker error wording now includes "after `<consumed>`,
  expected …" framing — matches the chumsky-side test
  contract for structural errors mid-shape.
- AST-builder validation errors now propagate as
  WalkOutcome::ValidationFailed (not the generic "AST builder
  failed" fallback), so `change column … --force-conversion
  --dont-convert` and repeated `on delete` clauses surface
  their friendly catalog wording verbatim.

Grammar additions:
- src/dsl/grammar/shared.rs: type-name validator (TYPE_VALIDATOR
  uses Type::from_str via parse.custom.unknown_type catalog),
  qualified_column sub-grammar, referential action keyword
  (`cascade`/`restrict`/`set null`/`no action`), repeated
  on-clauses.
- src/dsl/grammar/ddl.rs: drop/add/rename/change CommandNodes
  with inline shapes (per-use-site `role` annotations let the
  AST builder discriminate parent vs child columns, etc.).
  The four entry words each have one CommandNode whose `shape`
  is a Choice across sub-forms.

Tests:
- 14 new walker-specific tests covering all DDL forms (bare
  drop table, drop column with optional connectives, drop
  relationship by name and by endpoints, add column with type
  validator, rename column, change column with each flag form
  + mutual-exclusion check, add 1:n relationship minimal /
  full, repeated-clause-twice rejection).
- Total: 819 passed, 0 failed, 1 ignored (was 805 / 1).
- cargo clippy --all-targets -- -D warnings clean.
This commit is contained in:
claude@clouddev1
2026-05-15 06:59:27 +00:00
parent 50b3542050
commit 7e79ca865a
8 changed files with 1400 additions and 62 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/dsl/walker/driver.rs
+290 -37
View File
@@ -23,7 +23,9 @@
use crate::dsl::grammar::{HighlightClass, Node, ValidationError};
use crate::dsl::walker::context::WalkContext;
use crate::dsl::walker::lex_helpers::{consume_bare_path, consume_ident, skip_whitespace};
use crate::dsl::walker::lex_helpers::{
consume_bare_path, consume_flag, consume_ident, consume_number_literal, skip_whitespace,
};
use crate::dsl::walker::outcome::{
ByteClass, Expectation, MatchedItem, MatchedKind, MatchedPath,
};
@@ -32,6 +34,13 @@ use crate::dsl::walker::outcome::{
pub enum NodeWalkResult {
Matched {
end: usize,
/// Expectations contributed by Optional children that
/// skipped (matched zero terminals). Walker callers
/// merge these into the next failure's expected set so
/// completion sees the full "what could have appeared
/// here" union, not just the strictly-required next
/// terminal.
skipped: Vec<Expectation>,
},
/// Did not engage at this position. Caller decides whether
/// this is benign (Optional, Choice fallthrough) or a hard
@@ -52,6 +61,13 @@ pub enum NodeWalkResult {
},
}
const fn matched(end: usize) -> NodeWalkResult {
NodeWalkResult::Matched {
end,
skipped: Vec::new(),
}
}
#[derive(Debug, Clone)]
pub enum FailureKind {
Mismatch { expected: Vec<Expectation> },
@@ -76,22 +92,25 @@ pub fn walk_node(
validator,
highlight_override: _,
} => walk_ident(source, pos, *src, role, *validator, path, per_byte),
Node::NumberLit
| Node::StringLit
| Node::BlobLit
| Node::Flag(_)
| Node::Repeated { .. }
| Node::DynamicSubgrammar(_) => {
// Phase A: not exercised by app-lifecycle commands.
// Reaching this branch means a Phase B+ grammar got
// declared without the walker support landing yet —
// surface as a hard failure so the test suite catches
// it loudly instead of silently mis-parsing.
Node::NumberLit { validator } => walk_number_lit(source, pos, *validator, path, per_byte),
Node::Literal(literal) => walk_literal(source, pos, literal, path, per_byte),
Node::StringLit | Node::BlobLit | Node::DynamicSubgrammar(_) => {
// Phase A-B: not exercised yet. Reaching this branch
// means a Phase D+ grammar got declared without the
// walker support landing — surface as a hard failure
// so tests catch it loudly rather than silently
// mis-parsing.
NodeWalkResult::Failed {
position: pos,
kind: FailureKind::Mismatch { expected: vec![] },
}
}
Node::Flag(name) => walk_flag(source, pos, name, path, per_byte),
Node::Repeated {
inner,
separator,
min,
} => walk_repeated(source, pos, inner, *separator, *min, ctx, path, per_byte),
Node::BarePath => walk_bare_path(source, pos, path, per_byte),
Node::Choice(children) => walk_choice(source, pos, children, ctx, path, per_byte),
Node::Seq(children) => walk_seq(source, pos, children, ctx, path, per_byte),
@@ -127,7 +146,7 @@ fn walk_word(
end,
class: HighlightClass::Keyword,
});
NodeWalkResult::Matched { end }
NodeWalkResult::Matched { end, skipped: Vec::new() }
} else {
NodeWalkResult::NoMatch {
position,
@@ -155,9 +174,7 @@ fn walk_punct(
end: position + 1,
class: HighlightClass::Punct,
});
NodeWalkResult::Matched {
end: position + 1,
}
matched(position + 1)
} else {
NodeWalkResult::NoMatch {
position,
@@ -169,7 +186,7 @@ fn walk_punct(
fn walk_ident(
source: &str,
position: usize,
_src: crate::dsl::grammar::IdentSource,
src: crate::dsl::grammar::IdentSource,
role: &'static str,
validator: Option<crate::dsl::grammar::IdentValidator>,
path: &mut MatchedPath,
@@ -178,7 +195,7 @@ fn walk_ident(
let Some((start, end)) = consume_ident(source, position) else {
return NodeWalkResult::NoMatch {
position,
expected: vec![Expectation::Ident { role }],
expected: vec![Expectation::Ident { role, source: src }],
};
};
let text = source[start..end].to_string();
@@ -200,7 +217,197 @@ fn walk_ident(
end,
class: HighlightClass::Identifier,
});
NodeWalkResult::Matched { end }
NodeWalkResult::Matched { end, skipped: Vec::new() }
}
fn walk_literal(
source: &str,
position: usize,
literal: &'static str,
path: &mut MatchedPath,
per_byte: &mut Vec<ByteClass>,
) -> NodeWalkResult {
let bytes = source.as_bytes();
let lit_bytes = literal.as_bytes();
if position + lit_bytes.len() > bytes.len() {
return NodeWalkResult::NoMatch {
position,
expected: vec![Expectation::Literal(literal)],
};
}
if &bytes[position..position + lit_bytes.len()] != lit_bytes {
return NodeWalkResult::NoMatch {
position,
expected: vec![Expectation::Literal(literal)],
};
}
// Lookahead: if the literal is a single digit / alphabetic
// run, the next byte must not extend it (so `1` doesn't
// half-match `12`).
let end = position + lit_bytes.len();
let last = lit_bytes[lit_bytes.len() - 1];
let last_is_word = last.is_ascii_alphanumeric() || last == b'_';
if last_is_word && end < bytes.len() {
let next = bytes[end];
if next.is_ascii_alphanumeric() || next == b'_' {
return NodeWalkResult::NoMatch {
position,
expected: vec![Expectation::Literal(literal)],
};
}
}
// Highlight class follows the literal's shape: digits get
// Number; letters get Keyword; mixed defaults to Keyword.
let class = if lit_bytes.iter().all(|b| b.is_ascii_digit()) {
HighlightClass::Number
} else {
HighlightClass::Keyword
};
path.push(MatchedItem {
kind: MatchedKind::Word(literal),
text: literal.to_string(),
span: (position, end),
});
per_byte.push(ByteClass {
start: position,
end,
class,
});
NodeWalkResult::Matched { end, skipped: Vec::new() }
}
fn walk_number_lit(
source: &str,
position: usize,
validator: Option<crate::dsl::grammar::NumberValidator>,
path: &mut MatchedPath,
per_byte: &mut Vec<ByteClass>,
) -> NodeWalkResult {
let Some((start, end)) = consume_number_literal(source, position) else {
return NodeWalkResult::NoMatch {
position,
expected: vec![Expectation::NumberLit],
};
};
let text = source[start..end].to_string();
if let Some(v) = validator
&& let Err(err) = v(&text)
{
return NodeWalkResult::Failed {
position: start,
kind: FailureKind::Validation(err),
};
}
path.push(MatchedItem {
kind: MatchedKind::NumberLit,
text,
span: (start, end),
});
per_byte.push(ByteClass {
start,
end,
class: HighlightClass::Number,
});
NodeWalkResult::Matched { end, skipped: Vec::new() }
}
fn walk_flag(
source: &str,
position: usize,
name: &'static str,
path: &mut MatchedPath,
per_byte: &mut Vec<ByteClass>,
) -> NodeWalkResult {
let Some((start, end)) = consume_flag(source, position) else {
return NodeWalkResult::NoMatch {
position,
expected: vec![Expectation::Flag(name)],
};
};
// `consume_flag` guarantees `start..end` covers `--<body>`.
let body = &source[start + 2..end];
if body != name {
return NodeWalkResult::NoMatch {
position,
expected: vec![Expectation::Flag(name)],
};
}
path.push(MatchedItem {
kind: MatchedKind::Flag(name),
text: source[start..end].to_string(),
span: (start, end),
});
per_byte.push(ByteClass {
start,
end,
class: HighlightClass::Flag,
});
NodeWalkResult::Matched { end, skipped: Vec::new() }
}
#[allow(clippy::too_many_arguments)]
fn walk_repeated(
source: &str,
position: usize,
inner: &Node,
separator: Option<&Node>,
min: usize,
ctx: &mut WalkContext,
path: &mut MatchedPath,
per_byte: &mut Vec<ByteClass>,
) -> NodeWalkResult {
let mut cur = position;
let mut count = 0_usize;
let mut last_expected: Option<Vec<Expectation>> = None;
loop {
let saved_path_len = path.items.len();
let saved_byte_len = per_byte.len();
let result = if count == 0 {
walk_node(source, cur, inner, ctx, path, per_byte)
} else if let Some(sep) = separator {
let sep_saved_path = path.items.len();
let sep_saved_byte = per_byte.len();
match walk_node(source, cur, sep, ctx, path, per_byte) {
NodeWalkResult::Matched { end, .. } => {
walk_node(source, end, inner, ctx, path, per_byte)
}
NodeWalkResult::NoMatch { .. } => {
path.items.truncate(sep_saved_path);
per_byte.truncate(sep_saved_byte);
break;
}
other => return other,
}
} else {
walk_node(source, cur, inner, ctx, path, per_byte)
};
match result {
NodeWalkResult::Matched { end, .. } => {
cur = end;
count += 1;
}
NodeWalkResult::NoMatch { expected, .. } => {
path.items.truncate(saved_path_len);
per_byte.truncate(saved_byte_len);
last_expected = Some(expected);
break;
}
other => return other,
}
}
if count < min {
return NodeWalkResult::NoMatch {
position: cur,
expected: last_expected.unwrap_or_default(),
};
}
// The "could continue with another inner" expectations
// become this Repeated's `skipped` set so the caller's
// expected-set surfaces them at completion time.
NodeWalkResult::Matched {
end: cur,
skipped: last_expected.unwrap_or_default(),
}
}
fn walk_bare_path(
@@ -226,7 +433,7 @@ fn walk_bare_path(
end,
class: HighlightClass::String,
});
NodeWalkResult::Matched { end }
NodeWalkResult::Matched { end, skipped: Vec::new() }
}
fn walk_choice(
@@ -242,13 +449,12 @@ fn walk_choice(
let saved_path_len = path.items.len();
let saved_byte_len = per_byte.len();
match walk_node(source, position, child, ctx, path, per_byte) {
NodeWalkResult::Matched { end } => return NodeWalkResult::Matched { end },
m @ NodeWalkResult::Matched { .. } => return m,
NodeWalkResult::NoMatch { expected, .. } => {
path.items.truncate(saved_path_len);
per_byte.truncate(saved_byte_len);
merge_expected(&mut all_expected, expected);
}
// Once a choice branch commits, propagate its outcome.
other => return other,
}
}
@@ -268,28 +474,67 @@ fn walk_seq(
) -> NodeWalkResult {
let mut cur = position;
let mut idx = 0;
// Carries expectations from skipped-Optional children so
// that a NoMatch on a later child reports the union of "you
// could have typed any of these" — making the completion
// engine see optional connectives that haven't been typed.
let mut pending_skipped: Vec<Expectation> = Vec::new();
for child in children {
match walk_node(source, cur, child, ctx, path, per_byte) {
NodeWalkResult::Matched { end } => {
NodeWalkResult::Matched { end, skipped } => {
if end == cur {
// Child matched zero terminals (Optional skipped,
// empty Repeated, empty Seq). Accumulate its
// would-be expectations into pending.
for e in skipped {
if !pending_skipped.contains(&e) {
pending_skipped.push(e);
}
}
} else {
// Child consumed terminals — the "missing optional"
// window closed; reset the pending list.
pending_skipped.clear();
pending_skipped.extend(skipped);
}
cur = end;
idx += 1;
}
NodeWalkResult::NoMatch { position, expected } => {
NodeWalkResult::NoMatch {
position,
mut expected,
} => {
// Merge pending skipped-optional expectations with this
// child's expected set.
for e in std::mem::take(&mut pending_skipped) {
if !expected.contains(&e) {
expected.push(e);
}
}
if idx == 0 {
// Seq didn't even start.
return NodeWalkResult::NoMatch { position, expected };
}
// Mid-shape: did we run out of input or hit a
// wrong token?
let post_ws = skip_whitespace(source, position);
let kind = if post_ws >= source.len() {
return NodeWalkResult::Incomplete { position: post_ws, expected };
} else {
FailureKind::Mismatch { expected }
if post_ws >= source.len() {
return NodeWalkResult::Incomplete {
position: post_ws,
expected,
};
}
return NodeWalkResult::Failed {
position: post_ws,
kind: FailureKind::Mismatch { expected },
};
return NodeWalkResult::Failed { position: post_ws, kind };
}
NodeWalkResult::Incomplete { position, expected } => {
NodeWalkResult::Incomplete {
position,
mut expected,
} => {
for e in std::mem::take(&mut pending_skipped) {
if !expected.contains(&e) {
expected.push(e);
}
}
return NodeWalkResult::Incomplete { position, expected };
}
NodeWalkResult::Failed { position, kind } => {
@@ -297,7 +542,10 @@ fn walk_seq(
}
}
}
NodeWalkResult::Matched { end: cur }
NodeWalkResult::Matched {
end: cur,
skipped: pending_skipped,
}
}
fn walk_optional(
@@ -311,11 +559,16 @@ fn walk_optional(
let saved_path_len = path.items.len();
let saved_byte_len = per_byte.len();
match walk_node(source, position, child, ctx, path, per_byte) {
NodeWalkResult::Matched { end } => NodeWalkResult::Matched { end },
NodeWalkResult::NoMatch { .. } => {
m @ NodeWalkResult::Matched { .. } => m,
NodeWalkResult::NoMatch { expected, .. } => {
// Skip the optional but carry the inner's expectations
// so the caller's expected-set sees them.
path.items.truncate(saved_path_len);
per_byte.truncate(saved_byte_len);
NodeWalkResult::Matched { end: position }
NodeWalkResult::Matched {
end: position,
skipped: expected,
}
}
other => other,
}