claude@clouddev1
753 lines
32 KiB
Markdown
753 lines
32 KiB
Markdown
# Session handoff — 2026-05-12 (7)
|
|
|
|
Seventh handover. The previous session (handoff-6) shipped
|
|
ADR-0019 (friendly error layer + i18n catalog). This session
|
|
designed and **fully implemented ADR-0020 (tokenization layer)
|
|
and ADR-0021 (per-command usage templates / H1a)**, then
|
|
designed and **shipped all eight stages of ADR-0022 (ambient
|
|
typing assistance)** — colour highlighting, error overlay,
|
|
hint panel, non-modal Tab completion with cycling, schema
|
|
cache, identifier completion, invalid-identifier detection.
|
|
Four rounds of real-app testing produced follow-up fixes.
|
|
|
|
The next agent picks up a clean baseline with no design
|
|
deferrals or known bugs — only a partial test pass from the
|
|
user that may turn up further findings.
|
|
|
|
## State at handoff
|
|
|
|
**Branch:** `main`. Working tree clean. **Up to date with
|
|
`origin/main` at `c247f55`** — the user pushed along the way,
|
|
so there is no in-flight commit queue to publish. Verified via
|
|
`git status` at handoff time (the per-message "X commits
|
|
ahead" lines earlier in the session were stale snapshots — do
|
|
re-check `git status` rather than trust running totals).
|
|
|
|
Commits since handoff-6 (in chronological order, 26 total):
|
|
|
|
```
|
|
857ee75 ADR-0020 + ADR-0021: tokenization layer and parse-error pedagogy (H1a)
|
|
fdaf7e3 ADR-0020 implementation: lexer + parser refactor over &[Token]
|
|
11071ae ADR-0021 implementation: per-command usage templates in parse errors
|
|
f0632af ADR-0022: ambient typing assistance (unifies I3 + I4)
|
|
00c9dea ADR-0022 stage 1/8: theme token-class colour fields
|
|
cafc455 ADR-0022 stage 2/8: input panel — token-class highlighting
|
|
39da399 ADR-0022 stage 3/8: simple-mode echo lines highlighted
|
|
313d4f8 ADR-0022 stage 4/8: render-time parse + error overlay
|
|
9c4857e ADR-0022 stage 5/8: hint panel ambient typing assistance
|
|
6845df1 ADR-0022 stage 6/8: IdentSlot taxonomy + parser audit
|
|
aea3224 ADR-0022 stage 7/8: schema query plumbing
|
|
06e8d1e ADR-0022 stage 8a: non-modal keyword completion + Esc/Backspace undo
|
|
faebeed ADR-0022 stage 8b: hint panel candidate list with scroll markers
|
|
51a8d9a ADR-0022 stage 8c: IdentSlot propagation + SchemaCache API
|
|
7a32c13 ADR-0022 stage 8d: schema cache refresh wiring
|
|
8214e41 ADR-0022 stage 8e: invalid-identifier detection + hint variant
|
|
bd1cce6 ADR-0022 stage 8 follow-up: fixes from real-app testing
|
|
f94a999 ADR-0022 stage 8 follow-up r2: completion UX fixes from real testing
|
|
22119d6 ADR-0022 follow-up r3: identifier colour, NewName hint, "Next:" wording, "type" label
|
|
c247f55 ADR-0022 follow-up r4: column-type completion
|
|
```
|
|
|
|
**Tests:** **760 passing, 0 failing, 1 ignored** (up from 610
|
|
at handoff-6's baseline; +150 over this session). The ignored
|
|
test is the same `\`\`\`ignore` doc-test in `src/friendly/mod.rs`
|
|
that was already ignored at handoff-6 — not new debt.
|
|
|
|
Per-area test deltas:
|
|
|
|
- ADR-0020 (lexer + parser refactor): +40 (29 lexer + 11
|
|
keyword/punct)
|
|
- ADR-0021 (per-command usage): +18 (8 usage + 1 token-vocab
|
|
completeness + 9 parse-error-pedagogy integration tests)
|
|
- ADR-0022 stages 1-8e: +37 (4 theme + 9 input_render core
|
|
+ 5 input_render ambient + 5 db schema-cache + ~14
|
|
completion + others)
|
|
- ADR-0022 round-1 follow-up: +3
|
|
- ADR-0022 round-2 follow-up: +3
|
|
- ADR-0022 round-3 follow-up: +5
|
|
- ADR-0022 round-4 follow-up: +5
|
|
|
|
**Clippy:** clean with `nursery` lints enabled.
|
|
|
|
**Release build:** not measured this session; the existing
|
|
~7.8 MB binary will grow modestly from the new modules
|
|
(completion, input_render, keyword, lexer, usage,
|
|
ident_slot) but they're all hand-rolled, no new
|
|
dependencies.
|
|
|
|
## What's implemented (delta vs. handoff-6)
|
|
|
|
### ADR-0020 — tokenization layer for the DSL parser
|
|
|
|
Amends ADR-0001. Adds a lexer between the source string and
|
|
the chumsky grammar; chumsky now parses `&[Token]` instead
|
|
of `&str`.
|
|
|
|
What it provides:
|
|
|
|
- **`src/dsl/lexer.rs`** — tokenizer producing
|
|
`Vec<Token>` with byte-offset spans. Always succeeds:
|
|
lex-shape errors (unterminated string, unrecognised
|
|
character, malformed flag) embed as `TokenKind::Error(_)`
|
|
tokens in the stream rather than as a `Result` variant.
|
|
Rationale: I4 (syntax highlighting) needs to render
|
|
partial / invalid input uniformly; the parser sees
|
|
`Error` tokens and raises a structural error at that
|
|
point.
|
|
- **`src/dsl/keyword.rs`** — `Keyword` and `Punct` enums
|
|
declared via `define_keywords!` / `define_punct!`
|
|
`macro_rules!` invocations. Single source of truth: the
|
|
enum, the lex-side string→variant mapping, the
|
|
variant→literal rendering, and the
|
|
`parse.token.keyword.*` / `parse.token.punct.*`
|
|
catalog-key derivation all come from one declaration
|
|
block. Adding a keyword is one line of Rust + one line
|
|
of YAML.
|
|
- **Parser refactor.** All combinators in
|
|
`src/dsl/parser.rs` rewritten to operate on `&[Token]`.
|
|
Keyword aggregation across `choice` now works natively
|
|
("expected `data` or `table`" instead of just "expected
|
|
`table`"). Custom `try_map` content errors (unknown
|
|
type, mutually-exclusive flags, "with pk needs at least
|
|
one column", "specified twice") survive unchanged.
|
|
- **`replay` bare-path UX preserved** via a one-place
|
|
source-slice special case (ADR-0020 §6): after matching
|
|
`Keyword(Replay)`, the parser reads the remainder of the
|
|
source string directly rather than reassembling tokens.
|
|
- **I3 / I4 hooks committed at the parser level**: lexer
|
|
always produces a stream; parser's expected-token-set is
|
|
queryable.
|
|
- **Honest history note** in the ADR: the no-lexer shape
|
|
in `dsl/parser.rs` arose incrementally without ADR-level
|
|
deliberation against the known I3/I4/H1a requirements.
|
|
ADR-0020 corrects that.
|
|
|
|
### ADR-0021 — parser-as-source-of-truth for H1a
|
|
|
|
Builds on ADR-0020. Pulls forward H1a from
|
|
`requirements.md`.
|
|
|
|
What it provides:
|
|
|
|
- **`src/dsl/usage.rs`** — per-command `UsageEntry`
|
|
registry keyed off entry-keyword. Multi-entry families
|
|
(`add`, `drop`, `show`) return multiple keys;
|
|
unique-entry keywords return one. `matched_entry(tokens,
|
|
failure_position)` resolves the entry keyword from the
|
|
consumed prefix.
|
|
- **Catalog sections** under `parse.usage.<command>`
|
|
(templates) and `parse.token.{keyword,punct,
|
|
identifier,...}` (single-token vocabulary).
|
|
- **Renderer (in `app.rs::dispatch_dsl`)** composes three
|
|
blocks: caret + structural/custom error + usage template
|
|
(or "available commands:" fallback when no entry
|
|
keyword was consumed, e.g. `frobulate Customers`).
|
|
- **`tests/parse_error_pedagogy.rs`** — 9 integration
|
|
tests covering create/add/drop/show/frobulate/update/
|
|
insert pedagogy cases.
|
|
- **Anchor-phrase compliance preserved** (ADR-0019 §10):
|
|
existing tests assert on substrings like "no such
|
|
table", "already exists"; unchanged.
|
|
|
|
### ADR-0022 — ambient typing assistance (the big one)
|
|
|
|
Unified design that subsumed the originally-planned
|
|
separate ADRs for syntax highlighting (I4) and tab
|
|
completion (I3). The framing: colour, hint panel, and Tab
|
|
are three answers to the same question — what does the
|
|
user need to know mid-typing? — and planning them
|
|
separately produces three loose pieces that drift apart.
|
|
|
|
What it provides (eight implementation stages + four
|
|
testing-round follow-ups):
|
|
|
|
- **Token-class colouring** in the input pane and
|
|
simple-mode echo history (stages 1-3). Theme gains seven
|
|
`tok_*` fields. Identifier colour was tuned twice from
|
|
user feedback; final values are `#56B6C2` (dark) /
|
|
`#0F6B76` (light) — vivid teal to make identifiers
|
|
stand out.
|
|
- **Render-time parse + error overlay** on the failing
|
|
token (stage 4). `ParseError::Invalid` gained an
|
|
`at_eof: bool` flag; custom errors map to
|
|
`at_eof = true` conservatively to avoid over-highlighting.
|
|
- **Hint panel ambient mode** (stage 5) — sub-states
|
|
Valid/IncompleteAtEof/DefiniteError/InvalidIdent/
|
|
TypingName. Three-block composition (caret + error +
|
|
usage) when relevant.
|
|
- **Identifier-slot taxonomy** (stage 6): `IdentSlot`
|
|
enum (`NewName`, `TableName`, `Column`,
|
|
`RelationshipName`) with `expected_label()` /
|
|
`from_expected_label()` round-trip. `ident_ctx(slot)`
|
|
wrapper labels each call site; every `ident()` in the
|
|
parser was audited and tagged.
|
|
- **Schema query plumbing** (stage 7) — `Database::list_names_for(slot)`
|
|
worker request.
|
|
- **Schema cache refresh wiring** (stage 8d) —
|
|
`AppEvent::SchemaCacheRefreshed`; runtime posts at every
|
|
TablesRefreshed site (project load, post-DDL, rebuild,
|
|
replay).
|
|
- **Non-modal Tab completion** (stage 8a):
|
|
- **Single candidate** → insert with trailing space, no
|
|
memo. Chains naturally: `a` Tab → `add `, Tab → `add
|
|
column `.
|
|
- **Multi candidate** → insert WITHOUT trailing space,
|
|
create memo. Tab cycles forward (Shift-Tab backward,
|
|
starting from last); space (or any non-Tab key) clears
|
|
the memo and inserts itself naturally, producing the
|
|
final `<chosen> ` form.
|
|
- **Esc/Backspace while memo alive** → restore original
|
|
text + cursor in one keystroke. The user's preference
|
|
for symmetric insert-and-undo is baked in.
|
|
- **Multi-candidate hint panel** (stage 8b) with `<` / `>`
|
|
scroll markers when items overflow; selected item bold.
|
|
Kind-coloured (keyword purple, identifier teal).
|
|
- **Sticky hint during cycling** — while the memo is
|
|
alive, the hint shows the memo's candidate list with
|
|
the new selection, NOT what comes next at the post-Tab
|
|
cursor.
|
|
- **Grammar-order ordering** — keywords come first in
|
|
chumsky's source-order traversal (so `to` before
|
|
`table` for `add column [to] [table] …`), then schema
|
|
identifiers alphabetised. Achieved by dropping a sort
|
|
in `describe_expected` rather than alphabetising
|
|
blindly.
|
|
- **Invalid-identifier detection** (stage 8e) — when the
|
|
user types text that doesn't prefix-match any schema
|
|
entry at a known-set slot, render red overlay + "no
|
|
such {kind}: `{found}`" hint.
|
|
- **NewName slot probe** (round 3) — at user-invents-the-
|
|
name positions, the hint reads "Type a name, then `(`"
|
|
(or whatever follows) instead of the technical "next:
|
|
…" that would surface once the partial identifier got
|
|
consumed. The "what follows" is computed by re-parsing
|
|
with a single-letter placeholder identifier
|
|
substituted at the cursor.
|
|
- **Type-name completion** (round 4) — `Type::all()`
|
|
surfaces as Tab candidates at `(` slots, filtered by
|
|
partial prefix, in declaration order
|
|
(text/int/real/decimal/bool/date/datetime/blob/serial/
|
|
shortid). Type names live outside the Keyword enum
|
|
(ADR-0020 §2 keeps them as identifiers validated by
|
|
`Type::from_str`) so they needed their own completion
|
|
path via the `TYPE_SLOT_LABEL` constant.
|
|
- **"Next:" wording** instead of "expected:" — friendlier
|
|
framing. Hint sentences are now capitalised
|
|
(Submit/Next/Type/No such).
|
|
|
|
## Open testing work — picks up next session
|
|
|
|
The user has noted that every testing round has turned up
|
|
issues, and the next session begins with another testing
|
|
pass. Issues found and **fixed** so far (rounds 1-4):
|
|
|
|
1. **r1**: hint ordering (keywords alphabetised vs. grammar
|
|
order); hint-panel kind colouring; `tok_identifier`
|
|
equal to `theme.fg` (no contrast).
|
|
2. **r2**: stuck-on-unique completions (memo created for
|
|
single candidate caused Tab to no-op visibly); grammar
|
|
order regression (alphabetical sort was burying chumsky's
|
|
source order).
|
|
3. **r3**: identifier colour still too subtle; "expected: …"
|
|
reading as a leaked diagnostic; "Next: something else"
|
|
after `(` (unlabelled `type_keyword`); typing into a
|
|
NewName slot showed the post-consume "expected: `(`"
|
|
prose.
|
|
4. **r4**: column-type completion entirely missing — type
|
|
names aren't in the Keyword enum so the completion
|
|
engine never saw them.
|
|
|
|
The fixes from each round are described in the matching
|
|
commit messages. **Nothing is currently outstanding from
|
|
the user's reports** — but the user has explicitly noted
|
|
that they will run another testing pass and expect to find
|
|
more, so the next session should:
|
|
|
|
1. Do nothing structural until the user has tested. Wait.
|
|
2. Expect findings to be small surface-level UX issues
|
|
like the previous rounds (wording, ordering, colour
|
|
adjustments, edge cases in completion / hint panel /
|
|
render).
|
|
3. Approach each finding by tracing the symptom to the
|
|
minimum-viable layer (completion engine,
|
|
`ambient_hint`, render-time classifier, parser
|
|
labelling, catalog wording), then applying the fix
|
|
there.
|
|
|
|
### Known surface-level wart (not user-reported but
|
|
visible)
|
|
|
|
The `create table Customers ` case still shows the
|
|
lowercase custom-error wording "tables need at least one
|
|
column. Add `with pk`..." in the hint panel — because
|
|
that message is hand-written in `parser.rs` source code
|
|
rather than coming through the catalog. Capitalising it
|
|
needs the user's sign-off (rewriting a parser error
|
|
string). Flagged in commit `22119d6`'s message.
|
|
|
|
## Recommended next move (after testing)
|
|
|
|
### Push to test the still-quiet areas
|
|
|
|
What hasn't been tested yet in user-facing terms:
|
|
|
|
- **Shift-Tab cycling** (only forward-Tab has been
|
|
exercised in the user's reports).
|
|
- **Esc / Backspace undo** of a multi-candidate Tab
|
|
(memo path).
|
|
- **Cursor-in-the-middle-of-input completion** (most
|
|
tests have been cursor-at-end). The completion engine
|
|
handles mid-input but it hasn't been driven through
|
|
the TUI by the user.
|
|
- **Long candidate lists** that trigger the `<` `>`
|
|
scroll markers — schema with many tables.
|
|
- **Multi-byte UTF-8** in string literals + identifiers
|
|
(the lexer handles this but it hasn't been driven from
|
|
the keyboard).
|
|
- **Advanced mode** behaviour — should be plain-text
|
|
render with no hint panel content, no Tab completion.
|
|
Confirmed by code but not user-tested in this session.
|
|
- **Replay** with bare paths and quoted paths,
|
|
particularly the path special-case (ADR-0020 §6) under
|
|
the new lexer.
|
|
- **Interaction with modals** — should completion be
|
|
suppressed when a modal is open? Code currently
|
|
short-circuits via `if self.modal.is_some() return
|
|
handle_modal_key(key)` BEFORE the completion match
|
|
arms, so completion is correctly suppressed.
|
|
|
|
### After testing — three sizeable pieces in priority order
|
|
|
|
1. **A1 (CI workflow)**. Has been the "easy quick win"
|
|
for several sessions now and hasn't shipped. Standard
|
|
GitHub Actions YAML at `.github/workflows/ci.yml`;
|
|
cross-platform Linux / macOS / Windows; `cargo test`
|
|
+ `cargo clippy --all-targets -- -D warnings`. Locks
|
|
in the 760-test green baseline. 1-2 hours.
|
|
|
|
2. **Query DSL ADR + implementation**. Biggest
|
|
remaining design piece. Discussed in handoff-6:
|
|
extend `show data` into a SELECT-style command with
|
|
WHERE / projection / order; expose generated SQL as
|
|
a pedagogical hook; bundle C5a's complex WHERE
|
|
into one coherent feature. Then QA1 (EXPLAIN QUERY
|
|
PLAN) becomes meaningful. Probably 600-1000 lines of
|
|
ADR + 800-1500 of implementation.
|
|
|
|
3. **Constraint management surface (C3)**. UNIQUE /
|
|
CHECK / NOT NULL DDL operations. The friendly-error
|
|
layer (ADR-0019) has CHECK wording ready; the
|
|
missing piece is the DDL surface itself.
|
|
|
|
### V-series UX projects
|
|
|
|
Still pending from handoff-6:
|
|
|
|
- V4 — session log + Markdown export.
|
|
- V1/V2 — relationship rendering (the "two structures
|
|
+ arrow" view).
|
|
- V3 — ER diagram export.
|
|
|
|
### Smaller items still on the table
|
|
|
|
- I1 — multi-line input (Enter inserts newline,
|
|
Ctrl-Enter submits).
|
|
- I1b — readline shortcuts (Ctrl-A/E, Ctrl-W/K/U).
|
|
- C4 — m:n convenience (auto-junction-table).
|
|
|
|
### Tracked but bounded to other ADRs
|
|
|
|
- Q1 (SQL handling in advanced mode) — waits on Q4 (SQL
|
|
subset ADR), which itself waits on the lexer model
|
|
for advanced-mode tokenization. Now that ADR-0020 has
|
|
established the DSL-side lexer, the Q4 ADR can decide
|
|
whether to share / wrap `sqlparser-rs`'s tokens or
|
|
add a parallel SQL lexer.
|
|
- U-series undo/snapshot.
|
|
- Settings persistence — feeds the deferred
|
|
`messages` persistence and (when it arrives) any
|
|
user-configurable theme picker.
|
|
|
|
## Sharp edges and subtleties (delta vs. handoff-6)
|
|
|
|
Carried-over edges still apply. New ones this session:
|
|
|
|
- **The parser combinators are now `Parser<'a, &'a [Token],
|
|
…>` instead of `Parser<'a, &'a str, …>`.** Helper
|
|
combinators in `dsl/parser.rs`: `kw(Keyword)`,
|
|
`punct(Punct)`, `ident_inner()` /
|
|
`ident_ctx(IdentSlot)`, `number_literal()`,
|
|
`string_literal()`, `string_payload()`, `flag(name)`.
|
|
All defined inline at the top of the parser combinator
|
|
section.
|
|
|
|
- **`ident_inner` is the only call that produces a bare
|
|
identifier match.** Every command parser combinator
|
|
must use `ident_ctx(slot)`. There is no compile-time
|
|
enforcement (`ident_inner` is `fn` not a sealed type),
|
|
only convention + code review.
|
|
|
|
- **`ParseError::Invalid` gained two fields**:
|
|
`at_eof: bool` (stage 4) and `expected: Vec<String>`
|
|
(stage 5). Pattern matches with `..` are unaffected;
|
|
the two constructions in `dsl/parser.rs` populate
|
|
both correctly. `at_eof` for custom errors is
|
|
conservatively `true` — known limitation, noted on
|
|
the field's docstring.
|
|
|
|
- **Type names are NOT keywords.** They lex as
|
|
`Identifier` tokens; the parser's `type_keyword()`
|
|
helper uses `Type::from_str` to classify, which emits
|
|
the existing "unknown type 'X' (expected one of: …)"
|
|
custom error on miss. The completion engine
|
|
recognises the `"type"` label (from
|
|
`ident_inner().labelled("type")` inside
|
|
`type_keyword`) via the `TYPE_SLOT_LABEL` constant in
|
|
`completion.rs`. Adding a new type means: add the
|
|
variant + keyword to `Type::all()` (ADR-0005), and
|
|
the completion / error paths pick it up automatically.
|
|
|
|
- **The lexer eats whitespace.** `padded()` combinators
|
|
are gone from the parser entirely. Whitespace
|
|
positioning between tokens is recoverable from token
|
|
spans if anyone needs it.
|
|
|
|
- **`replay` parses via a special-case** in
|
|
`try_parse_replay_with_bare_path` (before the chumsky
|
|
parser runs). Quoted paths go through chumsky;
|
|
everything else is source-sliced from the byte after
|
|
`replay`. Documented in ADR-0020 §6.
|
|
|
|
- **The hint panel render now branches** on the
|
|
`AmbientHint` enum: `Prose(String)` or
|
|
`Candidates { items, selected }`. The renderer in
|
|
`ui.rs::render_candidate_line` builds spans with
|
|
per-candidate kind colouring and `<` / `>` scroll
|
|
markers when overflowing.
|
|
|
|
- **The schema cache refresh fires from the runtime**,
|
|
not from the App. App receives `AppEvent::SchemaCacheRefreshed(cache)`
|
|
and stores it. Refresh sites: project load, after DDL,
|
|
after rebuild, after replay. Best-effort: a failed
|
|
`list_names_for` for one slot kind leaves that field
|
|
empty in the cache rather than dropping the whole
|
|
refresh.
|
|
|
|
- **Tab/Shift-Tab key handling is at the TOP of
|
|
`handle_key`** — before the modal short-circuit's
|
|
match arm and before the regular key matcher. The
|
|
ordering is: `if modal { handle_modal_key }; match
|
|
Tab/BackTab/Esc-with-memo/Backspace-with-memo { … };
|
|
match (the existing key matcher) { … }`. Esc /
|
|
Backspace match arms have `if self.last_completion.is_some()`
|
|
guards so they fall through to normal behaviour when
|
|
no memo is alive.
|
|
|
|
- **`completion::candidates_at_cursor` is sync.** It
|
|
consults the in-memory `SchemaCache` rather than the
|
|
worker thread. The cache may be slightly stale
|
|
between a DDL command and its
|
|
`SchemaCacheRefreshed` event; acceptable per
|
|
ADR-0022 §9.
|
|
|
|
- **`NewName` slots return `&[]` from `SchemaCache::for_slot`**
|
|
— the user invents these names. The `typing_name_at_cursor`
|
|
function handles them with a friendlier hint
|
|
("Type a name, then …") via a placeholder-substitution
|
|
re-parse.
|
|
|
|
- **The `TYPE_SLOT_LABEL = "type"` const** in
|
|
`completion.rs` must equal what
|
|
`dsl::parser::type_keyword` labels its `select_ref!`
|
|
with. The strings are physically separate; no
|
|
compile-time link. If a future maintainer changes one,
|
|
the other must change too.
|
|
|
|
- **The macro-generated `Keyword::ALL` and `Punct::ALL`
|
|
arrays** are the canonical iteration source for
|
|
catalog-validity tests. A new keyword or punct that
|
|
forgets a YAML entry under `parse.token.*` fails the
|
|
`keys_validate_against_catalog` test loudly.
|
|
|
|
## ADR index (read these before touching the related areas)
|
|
|
|
```
|
|
0000 Record architecture decisions (process)
|
|
0001 Language and TUI framework (Rust + Ratatui)
|
|
— amended by ADR-0020 (adds tokenization layer)
|
|
0002 Database engine (User-facing posture)
|
|
0003 Input modes and command dispatch
|
|
0004 Project file format (amended by 0015)
|
|
0005 Column type vocabulary
|
|
0006 Undo snapshots and replay log
|
|
0007 Sharing and export (amended by 0015 amendment 1)
|
|
0008 Testing approach
|
|
0009 DSL command syntax conventions
|
|
0010 Database access via worker thread
|
|
0011 FK column type compatibility
|
|
0012 Internal metadata for user-facing column types
|
|
0013 Relationships, naming, and rebuild-table strategy
|
|
0014 Data operations, value literals, and auto-show
|
|
0015 Project storage runtime
|
|
0016 Pretty table rendering for data and structure views
|
|
0017 Column type-change compatibility
|
|
0018 Auto-fill contracts for serial and shortid columns
|
|
0019 Friendly error layer (H1) and i18n message catalog
|
|
0020 Tokenization layer for the DSL parser
|
|
— IMPLEMENTED (this session). Amends ADR-0001.
|
|
0021 Parser-as-source-of-truth for H1a
|
|
— IMPLEMENTED (this session). Builds on ADR-0020.
|
|
0022 Ambient typing assistance (I3 + I4 unified)
|
|
— IMPLEMENTED through stage 8e + 4 testing rounds.
|
|
```
|
|
|
|
## Repository layout (delta vs. handoff-6)
|
|
|
|
```
|
|
src/
|
|
completion.rs — NEW. SchemaCache, Candidate,
|
|
CandidateKind, Completion,
|
|
LastCompletion structs.
|
|
candidates_at_cursor,
|
|
invalid_ident_at_cursor,
|
|
typing_name_at_cursor.
|
|
input_render.rs — NEW. StyledRun, AmbientHint,
|
|
InputState. render_input_runs
|
|
(token-class colour + parse-
|
|
error overlay + invalid-ident
|
|
overlay + cursor injection).
|
|
ambient_hint (the resolution
|
|
ladder: memo → candidates →
|
|
invalid_ident → typing_name →
|
|
parse-prose fallback).
|
|
classify_input, lex_to_runs.
|
|
dsl/
|
|
ident_slot.rs — NEW. IdentSlot enum +
|
|
expected_label /
|
|
from_expected_label round-trip.
|
|
keyword.rs — NEW. define_keywords! /
|
|
define_punct! macros + ALL
|
|
static tables.
|
|
lexer.rs — NEW. Token, TokenKind,
|
|
LexError, Span. lex() always
|
|
succeeds; embeds Error tokens
|
|
in the stream.
|
|
parser.rs — Heavily refactored. Parser
|
|
now operates on &[Token].
|
|
ident_ctx(slot) wrapper.
|
|
ParseError gained at_eof +
|
|
expected fields.
|
|
usage.rs — NEW. UsageEntry registry +
|
|
matched_entry +
|
|
entry_keywords_alphabetised.
|
|
app.rs — Tab/BackTab/Esc-with-memo/
|
|
Backspace-with-memo handlers.
|
|
completion_tab_forward /
|
|
_backward,
|
|
start_or_complete_at / _last,
|
|
commit_unique / commit_multi,
|
|
replace_inserted,
|
|
undo_last_completion.
|
|
AppEvent::SchemaCacheRefreshed
|
|
handler.
|
|
last_completion: Option<…>,
|
|
schema_cache: SchemaCache
|
|
fields on App.
|
|
render path in dispatch_dsl
|
|
composes the three-block
|
|
parse error per ADR-0021 §2.
|
|
ui.rs — render_input_panel uses
|
|
input_render::render_input_runs
|
|
for simple mode; advanced
|
|
modes fall back to plain
|
|
before/under/after.
|
|
render_output_line peels the
|
|
dsl::ECHO_PREFIX from simple-
|
|
mode echo lines and re-
|
|
tokenises (lex_to_runs).
|
|
render_hint_panel dispatches
|
|
on AmbientHint variant;
|
|
render_candidate_line composes
|
|
spans with kind colouring +
|
|
`<` `>` scroll markers.
|
|
theme.rs — Seven new tok_* Color fields
|
|
on Theme. Theme::token_color
|
|
helper. WCAG-AA contrast
|
|
values for dark + light.
|
|
event.rs — AppEvent::SchemaCacheRefreshed
|
|
variant.
|
|
runtime.rs — refresh_schema_cache helper +
|
|
call sites at every
|
|
TablesRefreshed point.
|
|
friendly/
|
|
keys.rs — Massive expansion. hint.*,
|
|
parse.usage.*, parse.token.*,
|
|
ambient_*, invalid_ident,
|
|
typing_name, typing_name_then
|
|
keys all validated.
|
|
strings/en-US.yaml — Same expansion on the YAML
|
|
side. Hint sentences are
|
|
capitalised (Submit / Next /
|
|
Type / No such).
|
|
docs/
|
|
adr/
|
|
0020-tokenization-layer-for-the-dsl-parser.md
|
|
0021-parser-as-source-of-truth-for-h1a.md
|
|
0022-ambient-typing-assistance.md
|
|
README.md — indexed
|
|
handoff/
|
|
20260512-handoff-7.md — this file
|
|
tests/
|
|
parse_error_pedagogy.rs — NEW (ADR-0021). 9 integration
|
|
tests for caret + structural
|
|
error + usage template
|
|
composition.
|
|
```
|
|
|
|
## How to take over
|
|
|
|
1. Read this file.
|
|
2. Read `CLAUDE.md` for the working-style rules.
|
|
3. Read `docs/requirements.md` for the granular progress
|
|
table (note: probably wants an update reflecting the
|
|
stage-8 completion of I3 and I4).
|
|
4. **Expect the user to begin with another testing pass.**
|
|
Resist the urge to ship anything new until they have.
|
|
Past rounds: r1-r4 each turned up real findings that
|
|
needed surgical fixes.
|
|
5. When findings arrive, **trace each one to the minimum
|
|
relevant layer** before patching:
|
|
- Completion behaviour → `src/completion.rs`
|
|
(`candidates_at_cursor`,
|
|
`invalid_ident_at_cursor`, `typing_name_at_cursor`).
|
|
- Hint panel content → `src/input_render.rs::ambient_hint`.
|
|
- Hint panel render → `src/ui.rs::render_candidate_line` /
|
|
`render_hint_panel`.
|
|
- Input pane colours → `src/theme.rs` (tok_* fields).
|
|
- Parser shape / labels → `src/dsl/parser.rs`.
|
|
- Catalog wording → `src/friendly/strings/en-US.yaml`
|
|
(and `keys.rs` for the placeholder declarations).
|
|
6. After testing settles, the next bigger move is **A1
|
|
(CI workflow)** — quick win that locks in the now-
|
|
substantial 760-test baseline before more design work.
|
|
7. Then **Query DSL ADR + implementation** is the
|
|
recommended bigger piece.
|
|
8. Run `cargo test` to confirm the 760-test green
|
|
baseline.
|
|
9. Run `cargo clippy --all-targets` to confirm
|
|
clippy-clean.
|
|
|
|
### End-to-end smoke test (current state, post-r4)
|
|
|
|
Demonstrates ambient typing assistance. Replaces handoff-6's
|
|
smoke test, which is now stale (it predates ADR-0022).
|
|
|
|
```
|
|
$ rm -rf /tmp/handoff7-smoke
|
|
$ rdbms-playground --data-dir /tmp/handoff7-smoke
|
|
|
|
# Inside the app — empty input. Hint panel shows the existing
|
|
# panel.hint_empty content (no ambient interference until you
|
|
# type).
|
|
|
|
# Type a single character. Hint immediately offers candidates.
|
|
a -- hint: "add"
|
|
(single Keyword
|
|
candidate, purple)
|
|
|
|
# Tab inserts with trailing space (single candidate, no memo).
|
|
<Tab> -- input becomes "add "
|
|
hint: "column"
|
|
(next required kw)
|
|
|
|
# Tab again chains through unique completions.
|
|
<Tab> -- input becomes "add column "
|
|
hint: "to table"
|
|
(multi: `to`, `table`,
|
|
in source order)
|
|
|
|
# Multi-candidate Tab: inserts WITHOUT trailing space.
|
|
<Tab> -- input "add column to"
|
|
(memo alive,
|
|
`to` highlighted)
|
|
|
|
# Tab cycles within the memo.
|
|
<Tab> -- input "add column table"
|
|
(memo still alive,
|
|
`table` highlighted)
|
|
|
|
# Pressing space commits the choice. Memo clears.
|
|
<space> -- input "add column table "
|
|
|
|
# Continue. Tab on the table-name slot offers schema entries
|
|
# in cyan-teal (identifier colour).
|
|
<Tab> -- if schema has Customers,
|
|
Orders, etc., they cycle.
|
|
|
|
# Type a fresh column name yourself.
|
|
Customers: Email -- builds "add column to
|
|
table Customers: Email"
|
|
|
|
# Now hint reads "Type a name, then `(`" — wait, that's at the
|
|
# NewName slot. After typing Email the parser has consumed it.
|
|
# Hint reads "Next: `(`".
|
|
|
|
# Type the type slot.
|
|
(de -- hint: "decimal"
|
|
(single match,
|
|
keyword colour)
|
|
|
|
<Tab> -- input completes to
|
|
"...Email (decimal "
|
|
with trailing space
|
|
|
|
# Type close paren and submit.
|
|
)<Enter> -- command runs.
|
|
|
|
# Invalid identifier feedback:
|
|
show data Custp -- if no table starts
|
|
with Custp, `Custp`
|
|
renders red and hint
|
|
says "No such table:
|
|
`Custp`"
|
|
|
|
# Unknown command fallback:
|
|
frobulate widgets -- on submit, hint
|
|
shows usage block;
|
|
live hint shows
|
|
"available commands"
|
|
fallback.
|
|
|
|
quit
|
|
```
|
|
|
|
### Manual spot-checks worth running
|
|
|
|
- Single-Tab chaining: `a` Tab Tab Tab Tab — should
|
|
build `add column to ` (the unique-chain runs out at
|
|
position 4 where `to` and `table` are both possible).
|
|
- Multi-Tab cycling: `show ` Tab Tab Tab — should cycle
|
|
data → table → data (wrap).
|
|
- Esc undo: `show ` Tab Esc — should restore to `show `.
|
|
- Backspace undo (symmetric): `show ` Tab Backspace —
|
|
should also restore to `show `.
|
|
- Sticky hint: `show ` Tab Tab — hint stays as the
|
|
two-item candidate list with the selection moving.
|
|
- Invalid identifier: type `show data X` where no table
|
|
starts with X — `X` should render red + "No such table"
|
|
hint.
|
|
- Type completion: `add column to T: Name (` Tab Tab Tab
|
|
— cycles through types.
|
|
- Long candidate list: with many tables (say 12+),
|
|
position at `show data ` to see the `<` / `>` scroll
|
|
markers in the hint panel.
|
|
- Advanced mode: `:show data` (one-shot) or `mode advanced`
|
|
— input renders plain, hint shows existing
|
|
`panel.hint_empty` content, Tab is a no-op.
|
|
- Replay with bare path: `replay history.log` — should
|
|
work as before via the source-slice special case.
|
|
- Replay with quoted path: `replay 'my project/data.log'` —
|
|
chumsky path.
|
|
- Multi-byte UTF-8 inside a string literal: `insert into T
|
|
values ('café')` — lexer + render must not panic.
|