Three fixes from the user's testing run, plus an
investigation note on a fourth.
#4 Sticky hint during cycling. The previous code recomputed
candidates_at_cursor at the post-Tab cursor position, which
made the panel whiplash through "what comes next at the new
cursor" between cycles. ambient_hint now short-circuits to
the memo's stored candidate list while the memo is alive —
so Tab Tab Tab keeps showing the same list with the
selection moving, then snaps to the post-Tab ambient state
once any non-Tab key clears the memo.
#2 Candidate ordering and kind-coloured rendering. New
`Candidate { text, kind: Keyword|Identifier }` carries the
classification through completion, last-completion memo,
and ambient-hint payload. candidates_at_cursor now sorts
keywords first (alphabetical), identifiers second
(alphabetical), and the hint-panel renderer colours keywords
in `tok_keyword` and identifiers in `tok_identifier`.
Keyword-vs-identifier name collisions resolve in favour of
the keyword (rare; the user can still address their table
via different syntax).
#3 tok_identifier no longer matches theme.fg. Identifiers
in the input pane now render in a distinct cool grey-blue
(dark) / dark steel-blue (light), so they stand out from
prose-like default text without competing with keyword
purple. Same colour drives the identifier candidates in
the hint panel for visual consistency input ↔ hint.
Limitation worth knowing: "keywords first, alphabetical"
is not the same as grammatical order. For "add column "
the hint shows `table to` not `to table` — chumsky's
expected-set doesn't preserve combinator-source order, and
encoding it in the registry adds maintenance overhead the
fix doesn't cleanly justify. Marked for future revisit if
it bites.
#1 (Tab does nothing on "add column ") — not reproduced
through App::update. The internal logic works correctly:
"add column " + Tab inserts "Customers ", second Tab
cycles to "Orders ", third to "Thing ". The most likely
explanation is a stale binary or a terminal-level event
intercept (tmux focus, kitty-keyboard protocol differences,
etc.) — needs user verification with a fresh build.
Tests: 747 passing, 0 failing, 1 ignored (744 baseline →
+3: 2 new completion-ordering cases including the
keyword-wins-on-name-collision edge, plus 1 hint-mid-cycle
sticky test). Clippy clean.
Per the user's #5: "if our candidate selection works
correctly, then entering a character that removes all matches
is the same as entering an invalid token." Closes the loop
between schema cache (8c/8d) and live error feedback (4).
New `completion::invalid_ident_at_cursor(input, cursor, cache)`
returns `Some(InvalidIdent { range, found, slot })` when:
- the cursor is on a partial identifier-shaped token;
- the parser's expected-set at the start of that token
contains a known-set IdentSlot (TableName / Column /
RelationshipName);
- no schema entry across those slots prefix-matches the
typed text.
`render_input_runs` extended to take a `&SchemaCache` and
overlay the invalid-identifier range with `tok_error` —
same visual treatment as the parse-error overlay (4),
unified red signal regardless of which detector fires.
`ambient_hint` extended to surface `hint.ambient_invalid_ident`
when invalid_ident_at_cursor returns Some — wording
"no such {kind}: `{found}`" mirrors ADR-0019's engine-error
voice for consistency. Catalog + KEYS_AND_PLACEHOLDERS
declaration added; validator passes.
Render priority: candidates win over invalid-ident
(if any schema match exists for the partial prefix, the
state is "in-progress completion" not "invalid"). Falls
through to the existing parse-error/incomplete/Valid
framings otherwise.
NewName slots are filtered out at the source — typing
into a "user invents this name" position is never invalid
(per `IdentSlot::completes_from_schema`).
Tests: 744 passing, 0 failing, 1 ignored (738 baseline →
+6: 5 invalid_ident_at_cursor cases covering
unknown-prefix-fires, prefix-match-doesn't-fire,
NewName-immune, no-cursor-token, keyword-slot-immune;
plus 1 ambient_hint integration test). Clippy clean.
This closes ADR-0022. Stages 1-8e together deliver the
ambient-typing-assistance feature: token highlighting,
error overlay, hint panel ambient, hint panel multi-
candidate display with scroll markers, Tab/Shift-Tab
cycling with one-keystroke Esc/Backspace undo, schema-aware
identifier completion, and invalid-identifier live
feedback. Total stage-8 footprint: 5 commits, ~1600 lines.
`IdentSlot` gains `expected_label()` and the round-trip
`from_expected_label()`. The four slot kinds map to the
user-facing labels "identifier" (NewName), "table name",
"column name", "relationship name".
`ident_ctx(slot)` now actually applies `slot.expected_label()`
as the chumsky label (was documentation-only after stage 6).
Parser errors and the hint panel's "expected: …" prose now
read with the slot-specific name: "expected table name"
instead of the generic "expected identifier". One parser
test updated accordingly; the four catalog `parse.token.*`
keys are unaffected (the slot labels are a parallel surface).
New `completion::SchemaCache { tables, columns,
relationships }` struct + `for_slot(slot) -> &[String]`
accessor. Empty by default; runtime wiring lands in a
follow-on substage. NewName slots return `&[]`
unconditionally.
`candidates_at_cursor` extended to accept `&SchemaCache`:
when the parser's expected-set includes a slot label,
schema candidates from the cache are added alongside the
keyword candidates. Both sources are then prefix-filtered,
combined, sorted, deduplicated. App::schema_cache field
threaded into both the App-side completion paths and the
ambient_hint computation in ui.
Tests: 738 passing, 0 failing, 1 ignored (730 baseline →
+8: 2 IdentSlot label round-trip tests, 6 completion-with-cache
cases covering table/column/relationship slots, prefix
filtering, empty cache, and NewName-no-candidates).
Clippy clean.
User-visible: identifier completion infrastructure is in
place but the cache is always empty — runtime wiring (the
next substage) will populate it on project load and after
successful DDL, at which point Tab on identifier slots
starts offering schema names.
Refactor `ambient_hint` to return a richer enum:
- `Prose(String)` — the existing single-line hint (Valid /
incomplete-with-no-keywords / definite-error states);
- `Candidates { items, selected }` — multi-candidate (or
single-candidate) keyword completion at the cursor.
When `candidates_at_cursor` returns Some, the new
`Candidates` variant wins over the prose framing — the
candidate list is more actionable than "expected: `data` or
`table`". `selected` tracks the live `LastCompletion` memo's
selection_idx for the renderer to highlight.
`render_candidate_line` (new helper in ui.rs):
- All items fit → render space-separated; selected item
rendered bold + theme.fg, others theme.muted.
- Overflow → window centred on the selected item (or
item 0 with no selection); `< ` / ` >` markers at the
edges (per the user's #2). Window expands right-first
then left-first to use available width.
- Returns `Line<'static>` (items cloned into spans) so the
caller doesn't fight lifetimes between the
AmbientHint::Candidates payload and the rendered Line.
Updated callers in ui.rs and input_render tests for the new
signature. Added `ambient_hint_with_memo_carries_selected_index`
test asserting the renderer-side `selected` plumbing.
Tests: 730 passing, 0 failing, 1 ignored (728 baseline →
+2 net: -3 reworked + 5 new candidate-related cases).
Clippy clean.
Stage 8c will plumb identifier completion (schema cache +
candidate fetch from worker on demand or pre-cache) and add
the invalid-identifier hint variant.
ParseError::Invalid gains an `expected: Vec<String>` field —
the human-rendered names of the patterns chumsky was looking
for at the failure point (`\`create\``, `identifier`, etc.).
Empty for custom errors, which have no expected-set framing.
Populated by a new `describe_expected()` helper in parser.rs
that humanise() also delegates to (eliminates duplication).
`input_render::ambient_hint(input) -> Option<String>` returns
the hint-panel content per ADR-0022 §6:
- empty input → None (caller falls back to panel.hint_empty);
- Valid → t!("hint.ambient_complete") ("submit with Enter");
- IncompleteAtEof → t!("hint.ambient_expected", expected = …)
listing the parser's expected next tokens, oxford-joined;
- DefiniteErrorAt → t!("hint.ambient_error_with_usage", …)
composing the parse-error message with the matching
parse.usage.* template if a known entry keyword was
consumed, else the bare message.
Catalog gains the three hint.ambient_* keys + validator
declarations.
ui::render_hint_panel resolution order:
1. explicit app.hint (modal contexts) wins;
2. simple-mode + non-empty input → ambient_hint;
3. fallback to panel.hint_empty.
Advanced mode (persistent + one-shot `:`) bypasses ambient
hinting per ADR-0022 §12.
Snapshot: highlighted_input_all_token_classes rebaselined
because the hint panel now displays an ambient hint instead
of the empty placeholder when input is non-empty.
Tests: 698 passing, 0 failing, 1 ignored (693 baseline →
+5 ambient_hint cases). Clippy clean.
Stage 6 introduces the IdentSlot taxonomy + parser audit so
identifier-typed slots can yield schema-aware completion
candidates in stage 8.
Add `classify_input(&str) -> InputState` that returns one of
{Empty, Valid, IncompleteAtEof, DefiniteErrorAt(byte)}.
The renderer uses this to overlay tok_error on the failing
token of mid-typed input that can never be valid.
ParseError::Invalid gains an `at_eof: bool` field populated
by `into_parse_error`:
- structural failures: at_eof = found.is_none()
(chumsky's own "ran out of input" discriminator);
- custom errors from try_map: at_eof = true,
conservatively.
The conservative custom-error classification is a deliberate
under-highlighting bias. It means three classes of error
currently DO NOT get a live red overlay (only on submit):
- "tables need at least one column" (correct: this is
genuinely an incomplete state — adding `with pk ...` fixes it);
- "unknown type 'varchar'" (sub-optimal: should overlay);
- "--force-conversion and --dont-convert are mutually
exclusive" (sub-optimal: should overlay).
The trade-off is documented inline on the at_eof field. A
future refinement could carry an explicit definite/incomplete
tag through Custom errors (would change RichReason::Custom's
payload from String to a typed value).
render_input_runs now applies the overlay on the failing
token's run before injecting the cursor. Tokens after the
error keep their lex-class colour — fixes one thing at a
time per ADR-0022 §4. Lex errors continue to render in
tok_error from stage 2.
Pattern-matches on ParseError::Invalid throughout the
codebase use `..` and are unaffected; only the two
constructions in parser.rs needed updating.
Tests: 693 passing, 0 failing, 1 ignored (683 baseline →
+10: 7 classify + overlay tests, +1 adapted full-command
test, +2 valid-vs-incomplete coverage). Clippy clean.
Stage 5 lights up the hint panel as the verbose-feedback
surface — needs the InputState classifier from this stage.
Lift `dsl::ECHO_PREFIX = "running: "` as a public const,
with a unit test asserting `t!("dsl.running", input = "")`
matches it. The catalog template is now contracted to equal
`format!("{ECHO_PREFIX}{input}")` — a translator changing
the prefix breaks the test.
Add `input_render::lex_to_runs(input, theme)` — a
cursor-less variant of `render_input_runs` for use cases
(echo lines, future hint panel) that need token-class
colouring without an inverted cursor.
ui::render_output_line: when the line is an Echo submitted
in Simple mode, peel the prefix and re-tokenise the rest
through lex_to_runs, rendering each token at its class
colour. Advanced-mode echoes and any echo whose body
unexpectedly lacks the prefix fall through to the plain
rendering.
Tests: 683 passing, 0 failing, 1 ignored (682 baseline →
+1 echo_prefix_matches_catalog_template). Clippy clean
(uses let-chain to keep the if condition flat).
Stage 4 adds render-time parse + error overlay so the
failing token in mid-typed input lights up in the error
colour.
New `input_render` module with `render_input_runs(input,
cursor_byte, theme) -> Vec<StyledRun>`. Lexes the input,
assigns each token its `theme.token_color`, preserves
whitespace gaps as `theme.fg` runs, and injects the cursor
by splitting the run that contains it into before/under/after
sub-spans (under marked Modifier::REVERSED). End-of-input
cursor is an empty-range sentinel rendered as an inverted
space.
ui::render_input_panel switches over EffectiveMode: simple
mode goes through render_input_runs + a small runs_to_spans
helper that borrows from the input string; advanced modes
(persistent + one-shot `:`) keep the previous plain
before/under/after rendering since the DSL lexer doesn't
speak SQL (ADR-0022 §12).
Multi-byte UTF-8 in string literals is handled by walking
to the next char boundary when splitting the cursor run,
mirroring the previous renderer.
Tests: 682 passing, 0 failing, 1 ignored (672 baseline →
+10: 9 input_render unit tests covering each token class,
cursor placements, multi-byte, full-command shape; +1 new
"all token classes" UI snapshot). Clippy clean.
Caveat (noted inline in the new snapshot test): the
TestBackend/render_to_string path records text symbols
only, not ratatui style. The new snapshot is therefore a
text-layout regression net; the unit tests in
input_render::tests are the authoritative regression net
for colour mappings.
Stage 3 wires the same colouring into simple-mode echo
lines in the output panel.
claude@clouddev1