rdbms-playground/docs/adr/0022-ambient-typing-assistance.md

# ADR-0022: Ambient typing assistance — colour, hint panel, completion (I3 + I4)

## Status

Accepted.

Subsumes the originally-planned I4 (syntax highlighting) and
I3 (tab completion) into a single coherent feature: ambient
typing assistance. Builds on ADR-0020 (the lexer + parser-over-tokens
that this ADR consumes) and ADR-0021 (per-command usage
templates, which this ADR promotes from on-submit-only to live
ambient feedback). Cross-references ADR-0001 (theme conventions),
ADR-0010 (worker-thread access for schema queries), ADR-0019
(catalog conventions for any new strings), and ADR-0009 (the
closed grammar surface that defines what each token class means).

## Context

ADR-0019, 0020, and 0021 closed the on-submit feedback gap: a
parse error now renders with a caret, a structural / custom
message, and a per-command usage template. Once the user
hits Enter, they get a thorough explanation.

What's missing is *during* typing. Today the input field
renders as plain foreground colour with one inverted character
at the cursor. There is no signal for any of:

- "is `crete` a keyword or a typo?";
- "what comes after `create table Customers`?";
- "is `Customers` actually a table that exists?";
- "did I open this string and forget to close it?";
- "does Tab do anything here?".

Three separate mechanisms are commonly built to fix this:
**syntax highlighting** (colour), **tab completion** (Tab
key), and a **hint / status surface** (in our case the
already-present hint panel, which has been empty most of the
time). Each is correct on its own but they answer the same
underlying question — *what does the user need to know mid-typing?*
— and planning them separately produces three loose pieces
that drift apart in voice, mechanism, and what they cover.

The framing this ADR adopts:

- Colour is the **silent always-on** layer.
- The hint panel is the **verbose always-visible** layer.
- Tab is the **accept-this-suggestion action**.

These layer cleanly, with each one taking the load the other
two cannot bear:

- Colour catches lexer-level garbage and signals "this
  token is in error" instantly, but cannot explain why.
- The hint panel always says what comes next in prose, but
  cannot tell the user "the third character you typed is
  wrong" — colour does that.
- Tab does nothing on its own; the hint panel tells the
  user when Tab would help.

The hint panel is the central design move: it converts the
"Tab opens a popup over your text" anti-pattern into "the
already-visible panel becomes the chooser when Tab activates
it", which keeps screen real estate stable and removes any
modal floating UI.

## Decision

### 1. Three mid-typing states

Every keystroke produces one of three classifications, derived
from `lex(input)` followed by `parse_tokens(...)`:

1. **Valid-so-far.** Lex produces zero `Error` tokens AND
   parse succeeds OR parse fails at end-of-input (more input
   would make it valid).
2. **Definite error.** Lex produces an `Error` token, OR
   parse fails at a position before end-of-input (a token
   appears where no production accepts it).
3. **Incomplete-but-plausible.** A subset of valid-so-far:
   parse fails at end-of-input. The user's input is on
   track but not done.

The three states drive what colour, hint, and tab all do.

### 2. Layered feedback channels

Four channels operate independently, layered:

| Channel              | Layer        | Always on | Driver              |
|----------------------|--------------|-----------|---------------------|
| Token-class colour   | Silent       | Yes       | Lexer               |
| Error overlay        | Silent       | When applies | Parse position    |
| Hint panel ambient   | Verbose      | Yes       | Parse expected-set + state |
| Hint panel completion| Verbose      | Tab-triggered | User action       |

Each channel knows its layer's job and stays in it.

### 3. Token-class colour (the original I4 scope)

`Theme` gains seven token-class colour fields:

```rust
pub struct Theme {
    // ... existing fields ...
    pub tok_keyword: Color,
    pub tok_identifier: Color,
    pub tok_number: Color,
    pub tok_string: Color,
    pub tok_punct: Color,
    pub tok_flag: Color,
    pub tok_error: Color,
}
```

Both `Theme::dark()` and `Theme::light()` populate all seven
with WCAG-AA-contrasting values. Where two classes can
plausibly share `theme.fg`, the field still exists so a
future palette refresh can distinguish them without code
churn.

`ui::render_input_panel` runs `lex(&app.input)` per render.
Each token contributes a styled `Span` using its class
colour; whitespace gaps are preserved as `theme.fg` spans.
The cursor reverse-styling is injected by splitting the span
that contains the cursor's byte position into
(before, under, after) sub-spans, marking `under` REVERSED
while preserving its colour.

Per-render lex is microsecond-cheap. No caching.

### 4. Error overlay (parse-error highlighting on the failing token)

Render-time parse, in addition to lex. Then:

- If parse succeeds, no overlay.
- If parse fails at end-of-input (incomplete-but-plausible),
  no overlay.
- If parse fails at a token position before end-of-input,
  the token at that position renders with `theme.tok_error`
  overlay (foreground only, no underline / reverse — see
  reasoning in §6 of the previous draft).
- Lex `Error` tokens always render with `theme.tok_error`
  regardless of parse outcome.

Subsequent tokens after the error position render in their
normal lex-class colours. Rationale: the user fixes one
thing at a time; cascading red across the rest of the input
is visually overwhelming and rarely informative.

### 5. Echo lines: same colour treatment for simple-mode echoes

`render_output_line` for `OutputKind::Echo + Mode::Simple`
peels the catalog-bound `running: ` prefix, lexes the rest,
and renders the tokens identically to the input panel.
Advanced-mode echoes stay plain. Lift the
`dsl::ECHO_PREFIX = "running: "` constant + a unit test
asserting `t!("dsl.running", input = "")` matches it (so a
translator changing the prefix breaks the test, not the
render).

### 6. Hint panel ambient mode (always visible)

The hint panel currently shows `panel.hint_empty` when
`app.hint` is `None`. This ADR repurposes it as the verbose
typing-assistance surface. Three sub-states, one per
mid-typing state:

- **Valid-so-far + complete** (parse succeeds, all tokens
  consumed): "submit with Enter" — short and unobtrusive.
- **Valid-so-far + incomplete-but-plausible** (parse fails at
  EOF): "expected: <oxford-joined expected set>". Pulls
  from chumsky's expected-set at the failure point. For
  multi-entry families ("`add` family expects `column` or
  `1`") this is one line of natural prose.
- **Definite error**: "<short description of error> — usage:
  <template>". The usage template is the same `parse.usage.*`
  catalog content rendered on submit (ADR-0021), now also
  surfaced live. Multi-entry families render the matching
  family member only, picked by the same `usage::matched_entry`
  logic. Keeps the panel one-or-two-lines-tall.

Empty input keeps the existing `panel.hint_empty` content —
the hint panel only takes over when the user starts typing.

The catalog gains a small set of templates for these states:

```yaml
hint:
  ambient_complete: "submit with Enter"
  ambient_expected: "expected: {expected}"
  ambient_error_with_usage: "{message} — usage: {usage}"
```

### 7. Hint panel completion mode (Tab-triggered)

When the user presses Tab on a slot with multiple candidates,
the hint panel switches into completion mode. Visually the
panel shows the candidate list with one highlighted; the
input field is unchanged.

Mechanics:

- **Single candidate** → Tab inserts it immediately (with a
  trailing space if the next token would expect one). No
  mode change.
- **Multiple candidates** → Tab opens completion mode. The
  hint panel shows the candidates, one per line (or
  comma-joined if they fit), with the first highlighted.
- **Zero candidates** → Tab is a no-op.

While in completion mode, the input event flow shifts:

| Key                 | Behaviour                                  |
|---------------------|--------------------------------------------|
| Tab / Down          | Move highlight to next candidate           |
| Shift+Tab / Up      | Move highlight to previous candidate       |
| Enter               | Accept highlighted candidate               |
| Esc                 | Close completion mode without accepting   |
| Letter / digit / `_`| Append to input AND narrow candidate list  |
| Backspace           | Delete from input AND widen candidate list (or close mode if empty) |
| Other (cursor moves, paste, etc.) | Close completion mode, then process key normally |

Completion mode is a sticky state on `App` (`completion: Option<CompletionState>`).
It survives until accepted, cancelled, or the input changes
in a way that no candidate matches anymore (close + return
to ambient).

### 8. Identifier slot taxonomy

Identifier completion needs to know *what kind of identifier*
fits at the cursor. The parser is the only thing that knows
this — every `ident()` call has a semantic role. We make
that role explicit by replacing `ident()` with a tagged
variant at each call site:

```rust
pub enum IdentSlot {
    /// A name the user is inventing — no completion candidates.
    /// Examples: new table name, new column name, new relationship
    /// alias.
    NewName,
    /// An existing table.
    TableName,
    /// An existing column in a specific table. The table is bound
    /// by an earlier-consumed token in the same command.
    ColumnIn(TableRef),
    /// An existing relationship.
    RelationshipName,
}

pub enum TableRef {
    /// The table identifier is the Nth `ident()` call in this
    /// command's parser combinator chain.
    Earlier(usize),
}
```

`ident_ctx(slot)` is a small wrapper around the existing
`ident()` combinator that records the slot type alongside
the parsed identifier in the parser's `extra` data. The AST
itself is unchanged — slot tags are render-time concerns,
not AST concerns.

Concretely, every `ident()` call in `dsl/parser.rs` is
audited and becomes one of:

```rust
ident_ctx(IdentSlot::NewName)            // create table <Name>
ident_ctx(IdentSlot::TableName)          // drop table <T>
ident_ctx(IdentSlot::ColumnIn(Earlier(0)))  // drop column <T>: <Col>
// etc.
```

A unit test asserts every parser combinator references
`ident_ctx`, not bare `ident()`, so future commands can't
forget to tag.

The taxonomy is intentionally minimal for v1. Future
extensions (e.g. relationship endpoints needing both
parent-table and child-table awareness) are amendments to
this enum.

### 9. Schema query plumbing

The completion engine needs current schema data to enumerate
candidates for `TableName`, `ColumnIn`, and `RelationshipName`
slots. Per ADR-0010, all database access goes through the
worker thread.

Decision: a single new request:

```rust
Request::ListNamesFor(IdentSlot) → NameList
pub struct NameList { pub names: Vec<String> }
```

The worker computes from cached metadata; reply is small
and cheap. The completion engine fires this on Tab when
the slot is identifier-typed; until the reply arrives the
hint panel shows "(loading…)". For a teaching tool with a
small schema, the reply is effectively instantaneous; we
do not pre-cache or invalidate-on-write.

If schema mutates between Tab and Enter (the user pressed
Tab, then a background event changed the schema, then
Enter), the worst case is the user accepts a stale name —
caught by the next parse on submit. Acceptable.

### 10. Tab on keyword slots

Keyword candidates come from the parser's expected-token-set
at the cursor. The expected-set is the same data that
already drives the structural error wording (ADR-0020 §9
hook). For a slot with one keyword candidate, Tab inserts
it directly; for multiple, Tab opens completion mode with
the keyword list.

Punctuation that the parser expects (`:`, `(`, `,`) is
*not* offered by Tab — punctuation is too short to benefit
from completion, and inserting it without a following
identifier is an awkward halfway state. Tab on a slot whose
expected-set is purely punctuation is a no-op; the hint
panel's ambient prose still names what's expected.

### 11. Cursor position interpretation

The completion engine needs to know what slot the cursor is
at. Two cases:

- **Cursor at end of input**: the slot is whatever comes
  next after the last consumed token. Easiest case.
- **Cursor inside the input**: split the input at the
  cursor, lex the prefix, parse the prefix, and treat the
  failure-at-EOF state as the cursor slot. The suffix (the
  text after the cursor) is ignored for completion purposes.

This means tab completion in the middle of a typed-out
command works the same way as at the end — the completion
engine pretends the input ends at the cursor.

### 12. Mode interaction (simple vs. advanced; persistent vs. one-shot)

Ambient typing assistance is a **simple-mode** feature.
Advanced mode (persistent or `:`-one-shot) renders input
plain, has no hint panel ambient content beyond
`panel.hint_empty`, and Tab is a no-op. Justification: the
DSL lexer + parser don't speak SQL; using them for SQL
input would mark almost everything as Identifier/Error and
mislead. Future SQL-subset ADR (Q4) decides whether to
extend this.

The mode-banner colour (red for advanced, blue for simple)
already gives the user a strong "you are in a different
mode" signal; the absence of typing assistance reinforces
it.

### 13. Performance posture

Per render: lex + parse on the current input. For typical
input sizes (~80 chars, ~10 tokens), this is well under
100 µs total — orders of magnitude under ratatui's frame
budget. Schema queries on Tab incur one worker round-trip
(~1 ms in practice); the hint panel shows `(loading…)`
between Tab and reply. No caching, no debouncing in v1.

If profiling later shows render-loop pressure, options are
(a) cache the parse result keyed on input-string identity,
(b) lex+parse only on input change, not every render. Both
are local optimisations; not part of this ADR.

## Amendment 1 — Advanced-mode ambient assistance re-enabled (2026-05-21)

This amendment **supersedes §12's carve-out** that ambient
typing assistance is a simple-mode-only feature.

### The obsolete premise

§12 disabled ambient assistance in advanced mode because, at the
time, "the DSL lexer + parser don't speak SQL; using them for SQL
input would mark almost everything as Identifier/Error and
mislead." That premise no longer holds. ADR-0030 / ADR-0031 /
ADR-0032 moved the SQL surface into the **same unified, mode-aware
walker grammar** (ADR-0023 / ADR-0024). The walker now speaks SQL:
in `Mode::Advanced` it parses `SELECT` (and, from ADR-0033, DML),
highlights SQL keywords, resolves slot hints, and produces
completion candidates — exactly the inputs ambient assistance
needs.

### The bug this fixes

Despite the walker being mode-aware, the **UI never surfaced**
advanced-mode ambient assistance:

- `render_hint_panel` hard-returned `None` for advanced mode (the
  stale §12 gate), so the hint panel showed only `panel.hint_empty`
  — no prose hints and no candidate preview for SQL.
- The hint resolver (`hint_resolution_at_input` →
  `expected_for_hint_snapshot`) and `ambient_hint` never threaded
  the mode, so even the engine-level calls defaulted to
  `Mode::Simple` and gated a SQL statement as "this is SQL".

The result: in advanced mode, hinting and completion-preview for
SQL were completely dead, even for a bare `SELECT`.

This gap survived Phase 2 because ADR-0032's cross-cut matrix rows
for "Tab completion works for SQL keywords" / "Hint-panel prose
appears at every SQL slot" were validated by **engine-level** tests
(`completion_probe_in_mode(…, Advanced)`, `hint_mode_*` called
directly) — which prove the walker *can* produce SQL
hints/candidates but never exercise the UI that suppressed them.
This is the "free-for-free claim shipping without a real-app test"
failure mode the project's process pins call out.

### What changed

- **`ambient_hint_in_mode(input, cursor, memo, cache, mode)`** —
  the mode-aware ambient entry point. `ambient_hint` is now a thin
  wrapper that forwards `Mode::Simple`. Its sub-calls
  (`input_diagnostics_in_mode`, `hint_resolution_at_input_in_mode`,
  `candidates_at_cursor_in_mode`, the fallback `parse_command_in_mode`)
  all run in the supplied `mode`.
- **`hint_resolution_at_input_in_mode` + `expected_for_hint_snapshot`**
  now set `ctx.mode`, so the hint walk respects the active mode.
- **`render_hint_panel`** calls `ambient_hint_in_mode` with the
  effective mode for *all* modes (no more advanced-mode `None`).
- **One-shot `:` handling.** In one-shot advanced mode the raw
  input carries the `:` sigil, which is not part of the grammar.
  The panel strips it (mirroring `App::submit`) before the ambient
  walk, so `: sel` hints `select` rather than the sigil.

### What still holds

- **Simple mode is unchanged.** The simple-mode entry point keeps
  gating SQL as "this is SQL"; advanced assistance is opt-in via
  mode, never leaked into simple mode (regression-locked).
- **Syntax highlighting** already ran with `Mode::Advanced` and is
  unaffected.
- **The validity indicator** was already mode-aware (ADR-0032
  §10.6); this amendment aligns the ambient hint panel with it.
- **§13 performance posture** is unchanged — one walk per render,
  now in the active mode.

### Coverage

App-level regression at the layer Phase 2 missed:
`src/ui.rs::advanced_mode_hint_panel_surfaces_sql_candidates`
(renders the panel in advanced mode and asserts the FROM-slot
table candidate appears). Ambient-layer locks:
`src/input_render.rs::advanced_mode_ambient_offers_sql_from_slot_candidate`
and `simple_mode_ambient_does_not_surface_sql_candidates`.

### 14. Catalog additions

```yaml
hint:
  ambient_complete: "submit with Enter"
  ambient_expected: "expected: {expected}"
  ambient_error_with_usage: "{message} — usage: {usage}"
  completion_loading: "(loading suggestions…)"
  completion_none: "no completions for this position"
panel:
  hint_completion_title: "Completions ({count})"
  # The existing `panel.hint_empty` keeps its current text.
```

### 15. Snapshot test churn

Existing UI snapshots in `src/snapshots/` cover the input
and output panels at fg-only colour. The render changes
re-baseline those snapshots. New snapshots cover:

- Input field with one token of each lex class.
- Input field with a definite-error overlay.
- Input field with cursor mid-token.
- Hint panel in each ambient sub-state.
- Hint panel in completion mode with multiple candidates.
- Hint panel after Tab on a single candidate (visual confirm
  of inserted text + closed completion mode).

The snapshots are the regression net for "did we change the
visual output unexpectedly".

## Amendment 2 — Candidate ordering: schema identifiers before keywords (2026-05-21)

This amendment **reverses the candidate-ordering call made in the
handoff-14 ranker discussion** (keywords before schema
identifiers). That call was never recorded in an ADR — it lived
only in `tests/typing_surface/candidate_ordering.rs` — so this
amendment also gives the ordering a decision record.

### The obsolete premise

Handoff-14 ordered command-part keywords before schema
identifiers on the rationale that "grammar parts are read before
the content that fills them," so `add column to table T` reads in
order. That held while candidate lists were short. The SQL surface
(ADR-0030/0031/0032/0033) made lists long — an expression position
such as `where Name ` or `order by ` legitimately offers the
column names *plus* the full expression-continuation keyword run
(`is not like between in and or`, plus `asc`/`desc` in ORDER BY).

The hint panel's candidate line is **single-row and
window-scrolled** (`render_candidate_line`): when it overflows it
centres on the selected item, or on item 0 when nothing is
selected (the ambient, just-typed state). With keywords first, the
schema identifiers sat at the tail and scrolled off behind the `>`
marker — invisible until the user Tab-cycled to them.

### The decision

Schema identifiers (table / column / relationship names) now sort
**before** keywords in the candidate list. A name the user would
otherwise have to look up is the highest-value completion —
valuable even to experts, who come to know the keywords over time —
so it must stay visible by default. Within each section the prior
rules are unchanged: identifiers alphabetised; keywords in
grammar-declaration order (`to` before `table`); then type names,
composite literals, branching punct, flags. The existing colour
split (`tok_identifier` teal vs `tok_keyword` purple, §colour)
makes the section boundary legible once both are on screen.

### Related fix — Repeated trailing optionals

This amendment shipped alongside a `walk_repeated` fix: a
comma-separated list (`Repeated`) was discarding the last matched
item's trailing-optional expectations at a clean item boundary, so
`order by Name ` offered no `asc`/`desc`, `select Name ` no `as`,
and `create table … Code(text) ` no `not`/`unique`/`default`/
`check`. Those now surface (the separator `,` itself is
deliberately not surfaced). This is what made identifier
visibility pressing — the lists these positions produce are now
both correct and long.

### Deferred — two-line hint box

As hint lists grow, a **two-line candidate box** (more candidates
visible without scrolling) is worth considering. Deferred for now
on screen-space grounds; recorded so it is not lost.

### Coverage

`tests/typing_surface/candidate_ordering.rs` rewritten to assert
identifiers precede keywords (header invariant #2 inverted; the
`to`-before-`table` keyword-order invariant #1 retained).
`completion::tests::identifiers_come_before_keywords_in_grammar_order`
and `identifiers_precede_keywords_at_expression_position` lock the
ordering; `order_by_after_sort_item_offers_direction`,
`projection_after_item_offers_alias_keyword`, and
`create_table_after_column_spec_offers_constraints` lock the
trailing-optional fix. ~20 typing-surface snapshots re-baselined.

## Out of scope

Deliberately deferred to keep this ADR shippable as a single
feature:

1. **"Did you mean?" near-keyword detection** (typed
   `crete`, suggest `create`). Useful but a separate
   feature with its own UX (highlight the typo? offer
   replace?). Future ADR.
2. **Fuzzy matching in completion mode** (typed `Cmrs`,
   match `Customers`). Today completion mode narrows by
   prefix only.
3. **Inline ghost text** (the rest of the unique completion
   shown ahead of the cursor in dim colour). The hint panel
   already covers the same role; ghost text would be
   redundant and risks visual clutter.
4. **User-customisable keybindings.** Tab / arrows / Enter
   / Esc are hard-coded.
5. **Schema completion across projects.** Completion is
   scoped to the currently loaded project's schema.
6. **Live-error highlighting *between* tokens** (e.g. a
   wavy underline spanning two tokens that together form
   an invalid sequence). Single-token error overlay only.
7. **Multi-line input.** Out of this ADR; tracked
   separately as I1.
8. **Highlighting in past parse-error renderings** in
   output history. Echo lines are highlighted; the
   error-block content is not retroactively re-styled.
9. **SQL highlighting / completion in advanced mode.**
   Waits on Q4.
10. **Identifier completion that crosses table boundaries**
    (e.g. completing column names across all tables). v1
    requires a `ColumnIn(table)` binding from earlier in
    the command.

## Consequences

### Positive

- **Single coherent feature** for typing assistance
  instead of three loose pieces. Voice, mechanism, and
  scope agree across colour / hint / completion.
- **The hint panel is finally useful.** It earns its
  screen real estate with always-visible content.
- **Tab does the right thing in every position.** Single
  candidate: insert. Multiple: open chooser. Zero: no-op
  (and the hint panel says why).
- **No floating UI.** Completion lives in the existing
  hint panel; no popup that overlaps text.
- **Schema-aware from day one.** `IdentSlot` taxonomy is
  in the parser; the completion engine knows what to ask.
- **Live error feedback.** Definite errors light up the
  failing token mid-typing; the hint panel surfaces the
  usage template on the spot.
- **Error overlay does not require a Tab.** Passive
  feedback you cannot miss.

### Costs

- **Largest single ADR-driven change so far**: theme + ui
  + app event flow + parser combinator audit + worker
  request type + new App state for completion mode +
  catalog additions + snapshot rebaselining. Estimated
  1500-2500 lines across changes plus tests.
- **Render-time parse** is new work. Cheap absolutely but
  it is per-keystroke. Risk of perf regression on very
  large input strings (none today; the DSL is one-line
  commands).
- **Identifier-slot taxonomy** is a new concept the parser
  combinators must thread through every `ident()` call.
  One-time refactor.
- **Schema-query worker request** adds one new entry to
  the worker's request enum.
- **App state grows** for completion mode (Option<CompletionState>),
  with input-event routing changes when the state is
  active.
- **Snapshot tests churn.** ~6 new snapshots, several
  existing ones rebaselined.
- **Catalog grows** by a handful of `hint.*` entries.

### Neutral

- **Public parser API**: gains the `IdentSlot` enum and
  the per-slot tagging via `ident_ctx`. The
  `parse_command` signature is unchanged.
- **Theme dependency direction unchanged.**
- **Tab key was previously unused** in the input panel; no
  conflict.

## Implementation notes

### Order of operations

Implementation lands as a sequence of green-after-each commits:

1. **Theme colours (small, ~150 lines).** Add seven
   `tok_*` fields with light + dark values. Add
   `Theme::token_color(&TokenKind) -> Color` helper.
2. **`ui::lex_to_spans` + input panel rewrite (medium,
   ~200 lines + snapshot rebaseline).** Live token
   colouring on the input field; cursor splitting logic.
3. **Echo-line rewrite (small, ~80 lines).**
   Simple-mode echo lines highlighted; `dsl::ECHO_PREFIX`
   constant + test.
4. **Render-time parse + error overlay (medium, ~150
   lines).** Classify input into one of three states;
   overlay error styling on the failing token.
5. **Hint panel ambient (medium, ~250 lines).** Three
   sub-states; catalog additions; rendering. The hint
   panel begins to earn its space.
6. **Identifier-slot taxonomy + parser tagging (medium,
   ~300 lines + parser audit).** `IdentSlot` enum,
   `ident_ctx` wrapper, audit every `ident()` call,
   per-command unit test asserting tagging coverage.
7. **Schema query plumbing (medium, ~200 lines).** New
   worker request, App-side dispatch, async handling
   model.
8. **Completion mode + Tab/arrow/Enter/Esc bindings
   (medium-large, ~400 lines).** App state, event
   routing, hint-panel render variant, completion-list
   filtering as user types.
9. **Snapshot test pass + manual smoke (small).**

Each stage is a checkpoint commit. The commit messages name
the stage and the cumulative state.

### Things that interact subtly

- **Cursor splitting at multi-byte UTF-8 boundaries.**
  The current input renderer handles this; the new
  span-based renderer must keep it. Walk to next char
  boundary when splitting a token's span at the cursor.
- **Empty input.** Lex returns `vec![]`, parse returns
  Empty, hint panel falls back to `panel.hint_empty`.
- **Input mid-token at cursor.** The completion engine
  treats the token-prefix-up-to-cursor as the typed
  prefix; candidate filtering matches against that
  prefix.
- **Mode transitions during completion mode.** If the
  user toggles persistent advanced mode (or types `:`)
  while completion mode is active, completion mode closes
  immediately.
- **Project switch / load while completion mode is
  active.** Same: close completion mode before any project
  state change is applied.
- **Schema cache after a DDL command.** The first
  ListNamesFor request after a schema-mutating command
  fetches fresh data; no explicit cache-invalidation
  mechanism in v1 because the worker re-reads metadata on
  each call.
- **Completion of a partial token.** Typing `Cust`, then
  Tab, where one candidate matches: the engine replaces
  `Cust` with `Customers` (not appends). The replace span
  is the partial-identifier token's span.
- **Tab at end of complete-and-valid input.** The
  expected-set may be empty or contain only `end of
  input`; Tab is a no-op (the hint panel says "submit
  with Enter").
- **`messages` setting.** Verbosity governs engine-error
  rendering (ADR-0019); ambient hint content is unaffected
  by it. Hint always shows the same level of detail.
- **Render performance.** Lex + parse + classify per render
  for typical input is microseconds; profile if bigger
  inputs land later (Query DSL, multi-line).