claude@clouddev1
443 lines
19 KiB
Markdown
443 lines
19 KiB
Markdown
# Session handoff — 2026-05-15 (11)
|
|
|
|
Eleventh handover. Continues from handoff-10. This session
|
|
completed every deferred item from handoffs 9 + 10: **ADR-0024
|
|
Phase F (full) step 5 (walker-driven completion), the Ranker
|
|
hook scaffolding, the HintMode dispatch layer, Phase D (full)
|
|
end-to-end schema-aware value typing, and the round-5
|
|
optional-suffix completion gap.** Six commits total (five
|
|
feature commits + this handoff).
|
|
|
|
The codebase now reaches the steady-state ADR-0024 envisioned:
|
|
one declaration per command, walker as single source of truth,
|
|
typed value slots rejecting mis-shaped input at parse time.
|
|
|
|
## State at handoff
|
|
|
|
**Branch:** `main`. Working tree clean. **Local HEAD is
|
|
`8188fa5`**, ahead of `origin/main` by ten commits across two
|
|
sessions (user pushes asynchronously).
|
|
|
|
Commits since handoff-10's baseline (`044173b`):
|
|
|
|
```
|
|
bbe1252 ADR-0024 Phase F (full) step 5: walker-driven completion
|
|
7ae1a0f ADR-0024 ranker hook scaffolding
|
|
8581779 ADR-0024 HintMode dispatch via walker_hint_mode_at_input
|
|
abebd79 ADR-0024 Phase D (full): schema-aware value typing
|
|
8188fa5 ADR-0024 round-5 follow-up: surface tail-Optional expectations
|
|
```
|
|
|
|
**Tests:** **830 passing, 0 failing, 1 ignored** (up from 806
|
|
at handoff-10's baseline). Net +24 — these all under
|
|
`dsl::walker::tests::phase_d_*`, `dsl::walker::tests::hint_mode_*`,
|
|
`completion::tests::ranker_*`, and the round-5 optional-suffix
|
|
tests. The ignored test is still the same `\`\`\`ignore`
|
|
doc-test in `src/friendly/mod.rs`.
|
|
|
|
**Clippy:** clean with `nursery` lints + `-D warnings`.
|
|
|
|
## What shipped this session
|
|
|
|
### Step 5 — Walker-driven completion (commit bbe1252)
|
|
|
|
The completion engine no longer round-trips through the
|
|
`ParseError::Invalid::expected: Vec<String>` bridge. New helper
|
|
`walker::expected_at_input(source) -> Vec<Expectation>` returns
|
|
structured expectations with full `IdentSource + role` info
|
|
attached. The completion engine pattern-matches `Expectation`
|
|
variants directly:
|
|
|
|
- `Word(w)` / `Literal(s)` → keyword candidates (alphabetic
|
|
filter).
|
|
- `Ident { source: IdentSource::Types }` → type names.
|
|
- `Flag(body)` → `--{body}` candidates.
|
|
- `Literal("1")` → composite-literal opener (`1:n`).
|
|
- `Ident { source, .. }` where `source.completes_from_schema()` →
|
|
schema identifier candidates.
|
|
|
|
`is_value_literal_signature` now matches on `Expectation::Word`
|
|
literals + `NumberLit` + `StringLit` variants instead of
|
|
backticked strings. `invalid_ident_at_cursor` and
|
|
`typing_name_at_cursor` adopt the same structured path.
|
|
|
|
Zero observable-behaviour change: every existing completion
|
|
test passes unchanged. The win is clean code and lossless
|
|
information flow — the chumsky-era string round-trip is gone.
|
|
|
|
### Ranker hook (commit 7ae1a0f)
|
|
|
|
ADR-0024 §ranker-layer scaffolding:
|
|
|
|
- `pub type Ranker = fn(Vec<Candidate>) -> Vec<Candidate>;`
|
|
- `pub const fn identity_ranker(c) -> Vec<Candidate>` returns
|
|
its input unchanged.
|
|
- `candidates_at_cursor_with(input, cursor, cache, ranker)`
|
|
applies a custom ranker; the default
|
|
`candidates_at_cursor(input, cursor, cache)` delegates with
|
|
`identity_ranker`.
|
|
|
|
Three tests cover identity preservation, custom reordering, and
|
|
the empty-list-collapses-to-None edge.
|
|
|
|
No production caller passes a non-identity ranker yet — hooks
|
|
for frequency-based ranking, content-aware priors, and recency
|
|
plug in here without touching grammar declarations.
|
|
|
|
### HintMode dispatch (commit 8581779)
|
|
|
|
ADR-0024 §HintMode-per-node specified a dispatch layer between
|
|
the walker's expected-set and the hint-panel renderer. This
|
|
session lands the dispatch shape:
|
|
|
|
- `walker::hint_mode_at_input(source) -> Option<HintMode>`
|
|
pattern-matches the walker's expected-set for the
|
|
value-literal-slot signature (returns
|
|
`ProseOnly("hint.value_literal_slot")`) and for
|
|
`Ident { source: NewName }` (returns
|
|
`ForceProse("hint.ambient_typing_name")`).
|
|
- `input_render::ambient_hint` consults the hint mode before
|
|
the candidate / prose ladder. `ProseOnly` emits the catalog
|
|
prose at empty prefix; `ForceProse` consults
|
|
`typing_name_at_cursor` for the post-name probe and emits
|
|
the typing-name prose.
|
|
- A separate `expected_for_hint(source)` returns empty on
|
|
`WalkOutcome::Match` so the hint resolver doesn't surface
|
|
prose at the end of a valid command (where the completion
|
|
engine offers optional-suffix candidates instead).
|
|
|
|
This is the structural shape the ADR specified; the actual
|
|
detection inside `hint_mode_at_input` is signature-matching
|
|
against the expected-set (same logic the previous post-hoc
|
|
detectors used, relocated to one place). The detection becomes
|
|
node-attached in Phase D (already wired below).
|
|
|
|
`HintMode` gains `PartialEq + Eq` for tests; docstring
|
|
rewritten to describe live semantics.
|
|
|
|
8 new walker tests pin the hint resolver across
|
|
value-literal-slot positions, NewName-slot positions, and the
|
|
non-fire cases.
|
|
|
|
### Phase D (full) — schema-aware value typing (commit abebd79)
|
|
|
|
The central design claim of ADR-0024 §Phase D. Insert / update
|
|
/ delete value slots dispatch on the user-facing column type at
|
|
parse time, rejecting mis-shaped input with localised wording
|
|
instead of waiting for the bind-time error.
|
|
|
|
End-to-end pipeline:
|
|
|
|
1. **SchemaCache extension** (`src/completion.rs`): new
|
|
`TableColumn { name, user_type }` per-table type metadata
|
|
stored in `SchemaCache.table_columns: HashMap<String,
|
|
Vec<TableColumn>>`. `columns_for_table(name)` is the
|
|
case-insensitive lookup helper.
|
|
2. **WalkContext schema plumbing** (`src/dsl/walker/context.rs`):
|
|
`WalkContext<'a>` gains a lifetime and a
|
|
`schema: Option<&'a SchemaCache>`. `WalkContext::new()` is
|
|
the schemaless default; `with_schema(s)` is the schema-aware
|
|
constructor.
|
|
3. **Schema-aware parse entry point**
|
|
(`src/dsl/parser.rs`): `parse_command_with_schema(input,
|
|
schema)` is the new public variant. `parse_command(input)`
|
|
delegates with `None` for back-compat.
|
|
4. **`Node::Ident` writes_table/writes_column** flags
|
|
(`src/dsl/grammar/mod.rs`): when set, the walker writes the
|
|
matched identifier's resolution into `WalkContext`.
|
|
5. **Walker driver `DynamicSubgrammar` dispatch**
|
|
(`src/dsl/walker/driver.rs`): the factory resolves the inner
|
|
Node at walk time; the result is `Box::leak`ed so its
|
|
static-slice fields (Choice/Seq) have the lifetime the
|
|
walker expects (per ADR-0024 §sub-grammars).
|
|
6. **Typed value slot factories**
|
|
(`src/dsl/grammar/shared.rs`): one per `Type` —
|
|
`int_slot` (integer-only NumberLit + null), `real_slot`,
|
|
`decimal_slot`, `bool_slot`, `text_slot`, `date_slot`,
|
|
`datetime_slot`, `blob_slot`. `slot_for_type(ty)` is the
|
|
dispatcher. `current_column_value(ctx)` reads
|
|
`current_column` and dispatches. `column_value_list(ctx)`
|
|
reads `current_table_columns` and emits a Seq of typed
|
|
slots separated by commas.
|
|
7. **Data-command grammar uses the typed slots**
|
|
(`src/dsl/grammar/data.rs`): `INSERT_VALUES_LIST` becomes
|
|
`DynamicSubgrammar(column_value_list)`. `UPDATE_ASSIGNMENT`
|
|
and `WHERE_CLAUSE` use
|
|
`PER_COLUMN_VALUE = DynamicSubgrammar(current_column_value)`.
|
|
Insert / update / delete table-name slots set
|
|
`writes_table: true`; update / delete column-name slots set
|
|
`writes_column: true`.
|
|
8. **Runtime populates schema-with-types** (`src/runtime.rs`):
|
|
`refresh_schema_cache` calls `describe_table` for each
|
|
table and populates `SchemaCache::table_columns`. Live
|
|
typing benefits immediately on project load and after
|
|
every successful DDL.
|
|
9. **App dispatches with schema** (`src/app.rs`):
|
|
`dispatch_dsl` routes through `parse_command_with_schema`.
|
|
|
|
Catalog: new `parse.custom.bind_type_mismatch` entry with
|
|
`{found}` / `{expected}` placeholders, surfaced by the
|
|
int_slot and decimal_slot validators.
|
|
|
|
Fallback semantics: when the schema can't resolve a table (or
|
|
the walker is schemaless), `DynamicSubgrammar` factories fall
|
|
back to the schemaless `VALUE_LITERAL` choice — the
|
|
pre-Phase-D behaviour, so every existing test passes
|
|
unchanged.
|
|
|
|
11 new walker tests cover schema-aware insert / update /
|
|
delete: typed acceptance per column, decimal rejection at int
|
|
columns, null acceptance at every slot, multi-assignment
|
|
per-column dispatch, schemaless fallback.
|
|
|
|
### Round-5 follow-up — tail_expected (commit 8188fa5)
|
|
|
|
Closes the long-standing "save Tab offers `as`" gap. Pre-this
|
|
session, `save ` parsed as a complete `save` command, the
|
|
walker returned `Match { command_idx }`, and the completion
|
|
engine had nothing to mine.
|
|
|
|
`WalkResult` gains a `tail_expected: Vec<Expectation>` field.
|
|
The walker's top-level `Matched` branch copies the outer
|
|
shape's skipped-Optional expectations into it.
|
|
`expected_at_input` returns `tail_expected` on `Match`.
|
|
Completion now surfaces `as` (for `save `), `short` /
|
|
`verbose` (for `messages `), and any other Optional suffix
|
|
expectation as Tab candidates.
|
|
|
|
`hint_mode_at_input` deliberately does NOT consume
|
|
`tail_expected` (a separate `expected_for_hint` returns empty
|
|
on Match). Reasoning: completion offering optional suffixes is
|
|
helpful ("you could continue with `as`"); the hint panel
|
|
showing prose like "Type a name" at the end of a valid command
|
|
would be misleading ("the command is complete, but you're
|
|
forcing me to continue").
|
|
|
|
Two new tests: `save_space_offers_as_via_tail_expected`,
|
|
`messages_space_offers_short_and_verbose_via_tail_expected`.
|
|
|
|
## What's still pending — for future work
|
|
|
|
### 1. `CommandNode.help_id` not consumed
|
|
|
|
Every CommandNode declares `help_id: Option<&'static str>` (e.g.
|
|
`app.quit`, `data.insert`) pointing into a catalog namespace
|
|
that doesn't exist yet. The current help system reads
|
|
`help.in_app_body` — one big hand-curated catalog entry.
|
|
|
|
Wiring `help_id` would mean:
|
|
- Add 20 per-command help catalog entries.
|
|
- Replace the `help.in_app_body` lookup with iteration over
|
|
REGISTRY → translate each `help_id`.
|
|
- The in_app_body composition logic lives in
|
|
`app::note_help`.
|
|
|
|
Benefit: new commands auto-include in help without YAML edits.
|
|
Cost: medium refactor; user-facing strings need careful
|
|
rewording.
|
|
|
|
Not blocking; not user-visible regression.
|
|
|
|
### 2. `Box::leak` memory growth per dynamic walk
|
|
|
|
`Node::DynamicSubgrammar` dispatch uses `Box::leak` in the
|
|
walker driver. Per-walk leak is bounded by command-shape
|
|
complexity (~6 Node allocations for a typical
|
|
`insert into T values (...)` with 5 columns). For
|
|
per-keystroke completion polling, the leak grows over time.
|
|
|
|
ADR-0024 §sub-grammars sketched a per-walk arena as the
|
|
alternative. Implementation would replace the Box::leak with
|
|
a `WalkContext::arena: Vec<Box<Node>>` that lives for the
|
|
walk's duration. Worth doing if memory growth becomes
|
|
measurable (it isn't today).
|
|
|
|
### 3. Replay path is schemaless
|
|
|
|
`runtime::run_replay` calls `parse_command(line)` without a
|
|
schema, so typed-slot rejections don't fire during replay —
|
|
bind-time errors still catch the same cases, so this is a UX
|
|
nicety not a correctness gap. The schema mutates during replay
|
|
(create / add columns), so the simplest "thread schema through"
|
|
approach would need a per-line re-fetch.
|
|
|
|
Worth doing once a user actually trips this. Today it just
|
|
means a `replay` that contains `insert into T values (3.14)` at
|
|
an int column fails at bind time instead of at parse time.
|
|
|
|
### 4. HintMode is detection-based, not node-attached
|
|
|
|
The ADR design says HintMode annotations should live on
|
|
grammar nodes (so `date_slot` could carry
|
|
`ProseOnly("hint.date_format")` and the resolver dispatches on
|
|
each expected node's HintMode). Today the resolver
|
|
pattern-matches the walker's expected-set signature for the
|
|
value-literal slot and the NewName ident slot.
|
|
|
|
Functionally equivalent — but the structural form would let
|
|
typed value slots carry per-type prose hints (a date slot
|
|
shows "Type a date as 'YYYY-MM-DD'" instead of the generic
|
|
value-literal hint). Phase D's typed slots are ready to receive
|
|
the annotations; just plumb HintMode through Expectation when
|
|
emitting from `Choice` / `Ident` / `NumberLit` / `StringLit`
|
|
nodes.
|
|
|
|
### 5. Ranker is scaffolding-only
|
|
|
|
No production caller passes a non-identity ranker. Frequency
|
|
ranking, content-aware priors, recency hooks — all future
|
|
work that plugs in via `candidates_at_cursor_with`.
|
|
|
|
### 6. Dead `parse.token.*` catalog entries
|
|
|
|
The five structural-class entries
|
|
(`parse.token.identifier/number/string_literal/flag/end_of_input`)
|
|
and three lex-error entries
|
|
(`parse.token.error.{bad_flag,unknown_char,unterminated_string}`)
|
|
remain in the catalog. Today's walker doesn't surface them
|
|
(structural classification feeds highlighting, not catalog
|
|
lookup). Carried from handoff-10 §8 — conservative call: leave
|
|
in. They cost nothing and reintroducing them would be cheap if
|
|
a future need arises.
|
|
|
|
## Sharp edges and invariants
|
|
|
|
These are the load-bearing decisions worth remembering:
|
|
|
|
- **Phase D fallback is behaviour-preserving.** A
|
|
`DynamicSubgrammar` factory called with no schema in
|
|
WalkContext, or with a table the schema doesn't know about,
|
|
returns the schemaless `VALUE_LITERAL` choice. Every test
|
|
that called `parse_command(input)` without a schema continues
|
|
to pass with the existing semantics.
|
|
- **`Box::leak` per walk.** See pending §2 above.
|
|
- **`tail_expected` is for completion, not hint.** Surfacing
|
|
optional-suffix expectations through the hint resolver would
|
|
display prose like "Type a name" at the end of a complete
|
|
command. The split via `expected_for_hint` enforces this
|
|
invariant.
|
|
- **`writes_table` resolves columns at the same site.** The
|
|
walker writes `current_table = Some(matched)` AND resolves
|
|
`current_table_columns` from `schema.columns_for_table`. If
|
|
the schema lookup misses, `current_table_columns` is None and
|
|
downstream DynamicSubgrammar fallbacks engage. The column
|
|
list is a *snapshot* at the moment of the table ident match —
|
|
if the schema cache refreshes mid-walk (it doesn't today),
|
|
the walk would use the old snapshot. This is a feature, not a
|
|
bug — schema-cache consistency belongs to the runtime, not
|
|
the walker.
|
|
- **Schema-aware parse cost.** Every `dispatch_dsl` call now
|
|
walks the schema-aware path with a borrowed SchemaCache. The
|
|
cost is one `HashMap::get(table)` per Tables ident write and
|
|
one `Vec::clone` for the column list. Negligible.
|
|
- **`Repeated(UPDATE_ASSIGNMENT, ',', 1)` preserves
|
|
current_column write semantics.** Each repetition writes the
|
|
matched column name, overwriting the previous iteration's
|
|
value. The `=` and the typed value slot dispatch against the
|
|
most recent write. Correct.
|
|
|
|
## ADR index (delta vs. handoff-10)
|
|
|
|
```
|
|
0014 Data operations, value literals, and auto-show
|
|
— Phase D now rejects mis-shaped values at parse time
|
|
instead of at bind time. Same final behaviour; better UX.
|
|
0024 Unified grammar tree: execution plan (ACCEPTED)
|
|
— A through F (full all 5 steps) + D (full) landed.
|
|
HintMode dispatch landed (detection-based; node-attached
|
|
form is non-blocking future work).
|
|
Ranker hook landed (scaffolding).
|
|
```
|
|
|
|
## Repository layout (delta vs. handoff-10)
|
|
|
|
Files significantly modified:
|
|
|
|
```
|
|
src/dsl/grammar/mod.rs — Node::Ident writes_table/
|
|
writes_column flags
|
|
src/dsl/grammar/data.rs — insert / update / delete use
|
|
DynamicSubgrammar value slots
|
|
src/dsl/grammar/shared.rs — typed value slots
|
|
(int_slot, decimal_slot, …),
|
|
slot_for_type,
|
|
current_column_value,
|
|
column_value_list
|
|
src/dsl/walker/context.rs — WalkContext gains lifetime
|
|
and schema reference;
|
|
current_column changes from
|
|
ColumnInfo to TableColumn
|
|
src/dsl/walker/driver.rs — DynamicSubgrammar dispatch
|
|
via Box::leak; writes_table/
|
|
writes_column semantics in
|
|
walk_ident
|
|
src/dsl/walker/mod.rs — expected_at_input,
|
|
hint_mode_at_input,
|
|
expected_for_hint;
|
|
WalkResult::tail_expected
|
|
surfacing on Match
|
|
src/dsl/walker/outcome.rs — WalkResult::tail_expected
|
|
src/dsl/parser.rs — parse_command_with_schema
|
|
src/completion.rs — SchemaCache::table_columns,
|
|
columns_for_table, TableColumn;
|
|
structured Expectation
|
|
consumption throughout;
|
|
Ranker / identity_ranker;
|
|
candidates_at_cursor_with
|
|
src/input_render.rs — HintMode dispatch via
|
|
walker_hint_mode_at_input;
|
|
hint_leading_slice helper
|
|
src/app.rs — parse_command_with_schema in
|
|
dispatch_dsl
|
|
src/runtime.rs — describe_table loop in
|
|
refresh_schema_cache
|
|
src/friendly/strings/en-US.yaml — parse.custom.bind_type_mismatch
|
|
src/friendly/keys.rs — keys.rs declaration
|
|
```
|
|
|
|
Files added: none (only new tests inside existing modules).
|
|
|
|
## How to take over
|
|
|
|
1. **Read this file.**
|
|
2. **Read handoff-10** (`20260515-handoff-10.md`) for the prior
|
|
session's Phase F (full) work that landed steps 1-4 and
|
|
set the stage.
|
|
3. **Read `CLAUDE.md`** for the working-style rules.
|
|
4. **Read ADR-0024** for the design intent now fully realised.
|
|
5. **Skim `src/dsl/grammar/mod.rs`** — Node enum, IdentSource,
|
|
HintMode, CommandNode contract.
|
|
6. **Skim `src/dsl/walker/mod.rs`** — walk(), expected_at_input,
|
|
hint_mode_at_input, tail_expected propagation.
|
|
7. **Skim `src/dsl/grammar/shared.rs`** — typed value slots
|
|
and the two dynamic sub-grammars.
|
|
8. **Run `cargo test`** — should report 830 passing, 0
|
|
failing, 1 ignored.
|
|
9. **Run `cargo clippy --all-targets -- -D warnings`** —
|
|
clean.
|
|
10. **If picking up pending items**: see §1-§6 above for the
|
|
open work. §1 (help_id wiring) and §4 (node-attached
|
|
HintMode) are the most tractable; the rest are background
|
|
notes more than action items.
|
|
|
|
### End-to-end smoke (post-this-session)
|
|
|
|
User-visible improvements vs. handoff-10:
|
|
|
|
- **Typed value rejection at parse time.** With a populated
|
|
schema, `insert into T values (3.14)` at an int column fires
|
|
a parse-time error ("value '3.14' is not a valid integer")
|
|
instead of waiting for the bind-time `bind_type_mismatch`
|
|
error. Same wording, earlier surface.
|
|
- **Save Tab offers `as`.** Type `save ` and press Tab → the
|
|
completion menu offers `as` (closing the round-5 gap).
|
|
- **Messages Tab offers `short` / `verbose`.** Same shape.
|
|
- **Walker-direct completion.** Same user-visible candidates
|
|
as before; the round-trip through formatted strings is gone.
|
|
Future hint improvements (per-column-type prose,
|
|
frequency-based ranking) plug in without touching grammar
|
|
declarations.
|
|
|
|
Everything else is internal — code health, clearer
|
|
abstractions, fewer surprises for the next maintainer.
|