walker+completion: surface list trailing-optionals + identifiers-first ordering (ADR-0022 Amendment 2)

walk_repeated discarded the last matched item's trailing-optional
expectations at a clean item boundary, so a comma-separated list
offered no continuation after a complete item: `order by Name `
gave no asc/desc, `select Name ` no `as`, `create table …
Code(text) ` no not/unique/default/check. Capture the last item's
skipped set and surface it when the list ends at an item boundary
(the separator `,` itself is deliberately not surfaced).

That fix made expression-position candidate lists long, which
exposed a visibility problem: the hint panel's candidate line is
single-row and window-scrolls on overflow, centring on item 0 when
nothing is selected — so with keywords-first, schema identifiers
scrolled off behind the `>` marker. Reverse the ordering: schema
identifiers (table/column/relationship names) now sort before
keywords, since a name the user would have to look up is the
highest-value completion and must stay visible (keywords are
learned over time; the tok_identifier/tok_keyword colour split
marks the boundary). This reverses the handoff-14 keywords-first
call, now recorded in ADR-0022 Amendment 2.

Tests: walker expected-set + completion-layer regressions for the
trailing-optionals and the ordering; candidate_ordering.rs header
invariant inverted; ~20 typing-surface snapshots re-baselined; a
two-line hint box recorded as a deferred follow-up.
This commit is contained in:
claude@clouddev1
2026-05-21 21:52:49 +00:00
parent 43c49f4d1b
commit 7f68a53f86
28 changed files with 716 additions and 329 deletions
@@ -480,6 +480,75 @@ re-baseline those snapshots. New snapshots cover:
The snapshots are the regression net for "did we change the
visual output unexpectedly".
## Amendment 2 — Candidate ordering: schema identifiers before keywords (2026-05-21)
This amendment **reverses the candidate-ordering call made in the
handoff-14 ranker discussion** (keywords before schema
identifiers). That call was never recorded in an ADR — it lived
only in `tests/typing_surface/candidate_ordering.rs` — so this
amendment also gives the ordering a decision record.
### The obsolete premise
Handoff-14 ordered command-part keywords before schema
identifiers on the rationale that "grammar parts are read before
the content that fills them," so `add column to table T` reads in
order. That held while candidate lists were short. The SQL surface
(ADR-0030/0031/0032/0033) made lists long — an expression position
such as `where Name ` or `order by ` legitimately offers the
column names *plus* the full expression-continuation keyword run
(`is not like between in and or`, plus `asc`/`desc` in ORDER BY).
The hint panel's candidate line is **single-row and
window-scrolled** (`render_candidate_line`): when it overflows it
centres on the selected item, or on item 0 when nothing is
selected (the ambient, just-typed state). With keywords first, the
schema identifiers sat at the tail and scrolled off behind the `>`
marker — invisible until the user Tab-cycled to them.
### The decision
Schema identifiers (table / column / relationship names) now sort
**before** keywords in the candidate list. A name the user would
otherwise have to look up is the highest-value completion —
valuable even to experts, who come to know the keywords over time —
so it must stay visible by default. Within each section the prior
rules are unchanged: identifiers alphabetised; keywords in
grammar-declaration order (`to` before `table`); then type names,
composite literals, branching punct, flags. The existing colour
split (`tok_identifier` teal vs `tok_keyword` purple, §colour)
makes the section boundary legible once both are on screen.
### Related fix — Repeated trailing optionals
This amendment shipped alongside a `walk_repeated` fix: a
comma-separated list (`Repeated`) was discarding the last matched
item's trailing-optional expectations at a clean item boundary, so
`order by Name ` offered no `asc`/`desc`, `select Name ` no `as`,
and `create table … Code(text) ` no `not`/`unique`/`default`/
`check`. Those now surface (the separator `,` itself is
deliberately not surfaced). This is what made identifier
visibility pressing — the lists these positions produce are now
both correct and long.
### Deferred — two-line hint box
As hint lists grow, a **two-line candidate box** (more candidates
visible without scrolling) is worth considering. Deferred for now
on screen-space grounds; recorded so it is not lost.
### Coverage
`tests/typing_surface/candidate_ordering.rs` rewritten to assert
identifiers precede keywords (header invariant #2 inverted; the
`to`-before-`table` keyword-order invariant #1 retained).
`completion::tests::identifiers_come_before_keywords_in_grammar_order`
and `identifiers_precede_keywords_at_expression_position` lock the
ordering; `order_by_after_sort_item_offers_direction`,
`projection_after_item_offers_alias_keyword`, and
`create_table_after_column_spec_offers_constraints` lock the
trailing-optional fix. ~20 typing-surface snapshots re-baselined.
## Out of scope
Deliberately deferred to keep this ADR shippable as a single