handoff
This commit is contained in:
@@ -0,0 +1,752 @@
|
||||
# Session handoff — 2026-05-12 (7)
|
||||
|
||||
Seventh handover. The previous session (handoff-6) shipped
|
||||
ADR-0019 (friendly error layer + i18n catalog). This session
|
||||
designed and **fully implemented ADR-0020 (tokenization layer)
|
||||
and ADR-0021 (per-command usage templates / H1a)**, then
|
||||
designed and **shipped all eight stages of ADR-0022 (ambient
|
||||
typing assistance)** — colour highlighting, error overlay,
|
||||
hint panel, non-modal Tab completion with cycling, schema
|
||||
cache, identifier completion, invalid-identifier detection.
|
||||
Four rounds of real-app testing produced follow-up fixes.
|
||||
|
||||
The next agent picks up a clean baseline with no design
|
||||
deferrals or known bugs — only a partial test pass from the
|
||||
user that may turn up further findings.
|
||||
|
||||
## State at handoff
|
||||
|
||||
**Branch:** `main`. Working tree clean. **Up to date with
|
||||
`origin/main` at `c247f55`** — the user pushed along the way,
|
||||
so there is no in-flight commit queue to publish. Verified via
|
||||
`git status` at handoff time (the per-message "X commits
|
||||
ahead" lines earlier in the session were stale snapshots — do
|
||||
re-check `git status` rather than trust running totals).
|
||||
|
||||
Commits since handoff-6 (in chronological order, 26 total):
|
||||
|
||||
```
|
||||
857ee75 ADR-0020 + ADR-0021: tokenization layer and parse-error pedagogy (H1a)
|
||||
fdaf7e3 ADR-0020 implementation: lexer + parser refactor over &[Token]
|
||||
11071ae ADR-0021 implementation: per-command usage templates in parse errors
|
||||
f0632af ADR-0022: ambient typing assistance (unifies I3 + I4)
|
||||
00c9dea ADR-0022 stage 1/8: theme token-class colour fields
|
||||
cafc455 ADR-0022 stage 2/8: input panel — token-class highlighting
|
||||
39da399 ADR-0022 stage 3/8: simple-mode echo lines highlighted
|
||||
313d4f8 ADR-0022 stage 4/8: render-time parse + error overlay
|
||||
9c4857e ADR-0022 stage 5/8: hint panel ambient typing assistance
|
||||
6845df1 ADR-0022 stage 6/8: IdentSlot taxonomy + parser audit
|
||||
aea3224 ADR-0022 stage 7/8: schema query plumbing
|
||||
06e8d1e ADR-0022 stage 8a: non-modal keyword completion + Esc/Backspace undo
|
||||
faebeed ADR-0022 stage 8b: hint panel candidate list with scroll markers
|
||||
51a8d9a ADR-0022 stage 8c: IdentSlot propagation + SchemaCache API
|
||||
7a32c13 ADR-0022 stage 8d: schema cache refresh wiring
|
||||
8214e41 ADR-0022 stage 8e: invalid-identifier detection + hint variant
|
||||
bd1cce6 ADR-0022 stage 8 follow-up: fixes from real-app testing
|
||||
f94a999 ADR-0022 stage 8 follow-up r2: completion UX fixes from real testing
|
||||
22119d6 ADR-0022 follow-up r3: identifier colour, NewName hint, "Next:" wording, "type" label
|
||||
c247f55 ADR-0022 follow-up r4: column-type completion
|
||||
```
|
||||
|
||||
**Tests:** **760 passing, 0 failing, 1 ignored** (up from 610
|
||||
at handoff-6's baseline; +150 over this session). The ignored
|
||||
test is the same `\`\`\`ignore` doc-test in `src/friendly/mod.rs`
|
||||
that was already ignored at handoff-6 — not new debt.
|
||||
|
||||
Per-area test deltas:
|
||||
|
||||
- ADR-0020 (lexer + parser refactor): +40 (29 lexer + 11
|
||||
keyword/punct)
|
||||
- ADR-0021 (per-command usage): +18 (8 usage + 1 token-vocab
|
||||
completeness + 9 parse-error-pedagogy integration tests)
|
||||
- ADR-0022 stages 1-8e: +37 (4 theme + 9 input_render core
|
||||
+ 5 input_render ambient + 5 db schema-cache + ~14
|
||||
completion + others)
|
||||
- ADR-0022 round-1 follow-up: +3
|
||||
- ADR-0022 round-2 follow-up: +3
|
||||
- ADR-0022 round-3 follow-up: +5
|
||||
- ADR-0022 round-4 follow-up: +5
|
||||
|
||||
**Clippy:** clean with `nursery` lints enabled.
|
||||
|
||||
**Release build:** not measured this session; the existing
|
||||
~7.8 MB binary will grow modestly from the new modules
|
||||
(completion, input_render, keyword, lexer, usage,
|
||||
ident_slot) but they're all hand-rolled, no new
|
||||
dependencies.
|
||||
|
||||
## What's implemented (delta vs. handoff-6)
|
||||
|
||||
### ADR-0020 — tokenization layer for the DSL parser
|
||||
|
||||
Amends ADR-0001. Adds a lexer between the source string and
|
||||
the chumsky grammar; chumsky now parses `&[Token]` instead
|
||||
of `&str`.
|
||||
|
||||
What it provides:
|
||||
|
||||
- **`src/dsl/lexer.rs`** — tokenizer producing
|
||||
`Vec<Token>` with byte-offset spans. Always succeeds:
|
||||
lex-shape errors (unterminated string, unrecognised
|
||||
character, malformed flag) embed as `TokenKind::Error(_)`
|
||||
tokens in the stream rather than as a `Result` variant.
|
||||
Rationale: I4 (syntax highlighting) needs to render
|
||||
partial / invalid input uniformly; the parser sees
|
||||
`Error` tokens and raises a structural error at that
|
||||
point.
|
||||
- **`src/dsl/keyword.rs`** — `Keyword` and `Punct` enums
|
||||
declared via `define_keywords!` / `define_punct!`
|
||||
`macro_rules!` invocations. Single source of truth: the
|
||||
enum, the lex-side string→variant mapping, the
|
||||
variant→literal rendering, and the
|
||||
`parse.token.keyword.*` / `parse.token.punct.*`
|
||||
catalog-key derivation all come from one declaration
|
||||
block. Adding a keyword is one line of Rust + one line
|
||||
of YAML.
|
||||
- **Parser refactor.** All combinators in
|
||||
`src/dsl/parser.rs` rewritten to operate on `&[Token]`.
|
||||
Keyword aggregation across `choice` now works natively
|
||||
("expected `data` or `table`" instead of just "expected
|
||||
`table`"). Custom `try_map` content errors (unknown
|
||||
type, mutually-exclusive flags, "with pk needs at least
|
||||
one column", "specified twice") survive unchanged.
|
||||
- **`replay` bare-path UX preserved** via a one-place
|
||||
source-slice special case (ADR-0020 §6): after matching
|
||||
`Keyword(Replay)`, the parser reads the remainder of the
|
||||
source string directly rather than reassembling tokens.
|
||||
- **I3 / I4 hooks committed at the parser level**: lexer
|
||||
always produces a stream; parser's expected-token-set is
|
||||
queryable.
|
||||
- **Honest history note** in the ADR: the no-lexer shape
|
||||
in `dsl/parser.rs` arose incrementally without ADR-level
|
||||
deliberation against the known I3/I4/H1a requirements.
|
||||
ADR-0020 corrects that.
|
||||
|
||||
### ADR-0021 — parser-as-source-of-truth for H1a
|
||||
|
||||
Builds on ADR-0020. Pulls forward H1a from
|
||||
`requirements.md`.
|
||||
|
||||
What it provides:
|
||||
|
||||
- **`src/dsl/usage.rs`** — per-command `UsageEntry`
|
||||
registry keyed off entry-keyword. Multi-entry families
|
||||
(`add`, `drop`, `show`) return multiple keys;
|
||||
unique-entry keywords return one. `matched_entry(tokens,
|
||||
failure_position)` resolves the entry keyword from the
|
||||
consumed prefix.
|
||||
- **Catalog sections** under `parse.usage.<command>`
|
||||
(templates) and `parse.token.{keyword,punct,
|
||||
identifier,...}` (single-token vocabulary).
|
||||
- **Renderer (in `app.rs::dispatch_dsl`)** composes three
|
||||
blocks: caret + structural/custom error + usage template
|
||||
(or "available commands:" fallback when no entry
|
||||
keyword was consumed, e.g. `frobulate Customers`).
|
||||
- **`tests/parse_error_pedagogy.rs`** — 9 integration
|
||||
tests covering create/add/drop/show/frobulate/update/
|
||||
insert pedagogy cases.
|
||||
- **Anchor-phrase compliance preserved** (ADR-0019 §10):
|
||||
existing tests assert on substrings like "no such
|
||||
table", "already exists"; unchanged.
|
||||
|
||||
### ADR-0022 — ambient typing assistance (the big one)
|
||||
|
||||
Unified design that subsumed the originally-planned
|
||||
separate ADRs for syntax highlighting (I4) and tab
|
||||
completion (I3). The framing: colour, hint panel, and Tab
|
||||
are three answers to the same question — what does the
|
||||
user need to know mid-typing? — and planning them
|
||||
separately produces three loose pieces that drift apart.
|
||||
|
||||
What it provides (eight implementation stages + four
|
||||
testing-round follow-ups):
|
||||
|
||||
- **Token-class colouring** in the input pane and
|
||||
simple-mode echo history (stages 1-3). Theme gains seven
|
||||
`tok_*` fields. Identifier colour was tuned twice from
|
||||
user feedback; final values are `#56B6C2` (dark) /
|
||||
`#0F6B76` (light) — vivid teal to make identifiers
|
||||
stand out.
|
||||
- **Render-time parse + error overlay** on the failing
|
||||
token (stage 4). `ParseError::Invalid` gained an
|
||||
`at_eof: bool` flag; custom errors map to
|
||||
`at_eof = true` conservatively to avoid over-highlighting.
|
||||
- **Hint panel ambient mode** (stage 5) — sub-states
|
||||
Valid/IncompleteAtEof/DefiniteError/InvalidIdent/
|
||||
TypingName. Three-block composition (caret + error +
|
||||
usage) when relevant.
|
||||
- **Identifier-slot taxonomy** (stage 6): `IdentSlot`
|
||||
enum (`NewName`, `TableName`, `Column`,
|
||||
`RelationshipName`) with `expected_label()` /
|
||||
`from_expected_label()` round-trip. `ident_ctx(slot)`
|
||||
wrapper labels each call site; every `ident()` in the
|
||||
parser was audited and tagged.
|
||||
- **Schema query plumbing** (stage 7) — `Database::list_names_for(slot)`
|
||||
worker request.
|
||||
- **Schema cache refresh wiring** (stage 8d) —
|
||||
`AppEvent::SchemaCacheRefreshed`; runtime posts at every
|
||||
TablesRefreshed site (project load, post-DDL, rebuild,
|
||||
replay).
|
||||
- **Non-modal Tab completion** (stage 8a):
|
||||
- **Single candidate** → insert with trailing space, no
|
||||
memo. Chains naturally: `a` Tab → `add `, Tab → `add
|
||||
column `.
|
||||
- **Multi candidate** → insert WITHOUT trailing space,
|
||||
create memo. Tab cycles forward (Shift-Tab backward,
|
||||
starting from last); space (or any non-Tab key) clears
|
||||
the memo and inserts itself naturally, producing the
|
||||
final `<chosen> ` form.
|
||||
- **Esc/Backspace while memo alive** → restore original
|
||||
text + cursor in one keystroke. The user's preference
|
||||
for symmetric insert-and-undo is baked in.
|
||||
- **Multi-candidate hint panel** (stage 8b) with `<` / `>`
|
||||
scroll markers when items overflow; selected item bold.
|
||||
Kind-coloured (keyword purple, identifier teal).
|
||||
- **Sticky hint during cycling** — while the memo is
|
||||
alive, the hint shows the memo's candidate list with
|
||||
the new selection, NOT what comes next at the post-Tab
|
||||
cursor.
|
||||
- **Grammar-order ordering** — keywords come first in
|
||||
chumsky's source-order traversal (so `to` before
|
||||
`table` for `add column [to] [table] …`), then schema
|
||||
identifiers alphabetised. Achieved by dropping a sort
|
||||
in `describe_expected` rather than alphabetising
|
||||
blindly.
|
||||
- **Invalid-identifier detection** (stage 8e) — when the
|
||||
user types text that doesn't prefix-match any schema
|
||||
entry at a known-set slot, render red overlay + "no
|
||||
such {kind}: `{found}`" hint.
|
||||
- **NewName slot probe** (round 3) — at user-invents-the-
|
||||
name positions, the hint reads "Type a name, then `(`"
|
||||
(or whatever follows) instead of the technical "next:
|
||||
…" that would surface once the partial identifier got
|
||||
consumed. The "what follows" is computed by re-parsing
|
||||
with a single-letter placeholder identifier
|
||||
substituted at the cursor.
|
||||
- **Type-name completion** (round 4) — `Type::all()`
|
||||
surfaces as Tab candidates at `(` slots, filtered by
|
||||
partial prefix, in declaration order
|
||||
(text/int/real/decimal/bool/date/datetime/blob/serial/
|
||||
shortid). Type names live outside the Keyword enum
|
||||
(ADR-0020 §2 keeps them as identifiers validated by
|
||||
`Type::from_str`) so they needed their own completion
|
||||
path via the `TYPE_SLOT_LABEL` constant.
|
||||
- **"Next:" wording** instead of "expected:" — friendlier
|
||||
framing. Hint sentences are now capitalised
|
||||
(Submit/Next/Type/No such).
|
||||
|
||||
## Open testing work — picks up next session
|
||||
|
||||
The user has noted that every testing round has turned up
|
||||
issues, and the next session begins with another testing
|
||||
pass. Issues found and **fixed** so far (rounds 1-4):
|
||||
|
||||
1. **r1**: hint ordering (keywords alphabetised vs. grammar
|
||||
order); hint-panel kind colouring; `tok_identifier`
|
||||
equal to `theme.fg` (no contrast).
|
||||
2. **r2**: stuck-on-unique completions (memo created for
|
||||
single candidate caused Tab to no-op visibly); grammar
|
||||
order regression (alphabetical sort was burying chumsky's
|
||||
source order).
|
||||
3. **r3**: identifier colour still too subtle; "expected: …"
|
||||
reading as a leaked diagnostic; "Next: something else"
|
||||
after `(` (unlabelled `type_keyword`); typing into a
|
||||
NewName slot showed the post-consume "expected: `(`"
|
||||
prose.
|
||||
4. **r4**: column-type completion entirely missing — type
|
||||
names aren't in the Keyword enum so the completion
|
||||
engine never saw them.
|
||||
|
||||
The fixes from each round are described in the matching
|
||||
commit messages. **Nothing is currently outstanding from
|
||||
the user's reports** — but the user has explicitly noted
|
||||
that they will run another testing pass and expect to find
|
||||
more, so the next session should:
|
||||
|
||||
1. Do nothing structural until the user has tested. Wait.
|
||||
2. Expect findings to be small surface-level UX issues
|
||||
like the previous rounds (wording, ordering, colour
|
||||
adjustments, edge cases in completion / hint panel /
|
||||
render).
|
||||
3. Approach each finding by tracing the symptom to the
|
||||
minimum-viable layer (completion engine,
|
||||
`ambient_hint`, render-time classifier, parser
|
||||
labelling, catalog wording), then applying the fix
|
||||
there.
|
||||
|
||||
### Known surface-level wart (not user-reported but
|
||||
visible)
|
||||
|
||||
The `create table Customers ` case still shows the
|
||||
lowercase custom-error wording "tables need at least one
|
||||
column. Add `with pk`..." in the hint panel — because
|
||||
that message is hand-written in `parser.rs` source code
|
||||
rather than coming through the catalog. Capitalising it
|
||||
needs the user's sign-off (rewriting a parser error
|
||||
string). Flagged in commit `22119d6`'s message.
|
||||
|
||||
## Recommended next move (after testing)
|
||||
|
||||
### Push to test the still-quiet areas
|
||||
|
||||
What hasn't been tested yet in user-facing terms:
|
||||
|
||||
- **Shift-Tab cycling** (only forward-Tab has been
|
||||
exercised in the user's reports).
|
||||
- **Esc / Backspace undo** of a multi-candidate Tab
|
||||
(memo path).
|
||||
- **Cursor-in-the-middle-of-input completion** (most
|
||||
tests have been cursor-at-end). The completion engine
|
||||
handles mid-input but it hasn't been driven through
|
||||
the TUI by the user.
|
||||
- **Long candidate lists** that trigger the `<` `>`
|
||||
scroll markers — schema with many tables.
|
||||
- **Multi-byte UTF-8** in string literals + identifiers
|
||||
(the lexer handles this but it hasn't been driven from
|
||||
the keyboard).
|
||||
- **Advanced mode** behaviour — should be plain-text
|
||||
render with no hint panel content, no Tab completion.
|
||||
Confirmed by code but not user-tested in this session.
|
||||
- **Replay** with bare paths and quoted paths,
|
||||
particularly the path special-case (ADR-0020 §6) under
|
||||
the new lexer.
|
||||
- **Interaction with modals** — should completion be
|
||||
suppressed when a modal is open? Code currently
|
||||
short-circuits via `if self.modal.is_some() return
|
||||
handle_modal_key(key)` BEFORE the completion match
|
||||
arms, so completion is correctly suppressed.
|
||||
|
||||
### After testing — three sizeable pieces in priority order
|
||||
|
||||
1. **A1 (CI workflow)**. Has been the "easy quick win"
|
||||
for several sessions now and hasn't shipped. Standard
|
||||
GitHub Actions YAML at `.github/workflows/ci.yml`;
|
||||
cross-platform Linux / macOS / Windows; `cargo test`
|
||||
+ `cargo clippy --all-targets -- -D warnings`. Locks
|
||||
in the 760-test green baseline. 1-2 hours.
|
||||
|
||||
2. **Query DSL ADR + implementation**. Biggest
|
||||
remaining design piece. Discussed in handoff-6:
|
||||
extend `show data` into a SELECT-style command with
|
||||
WHERE / projection / order; expose generated SQL as
|
||||
a pedagogical hook; bundle C5a's complex WHERE
|
||||
into one coherent feature. Then QA1 (EXPLAIN QUERY
|
||||
PLAN) becomes meaningful. Probably 600-1000 lines of
|
||||
ADR + 800-1500 of implementation.
|
||||
|
||||
3. **Constraint management surface (C3)**. UNIQUE /
|
||||
CHECK / NOT NULL DDL operations. The friendly-error
|
||||
layer (ADR-0019) has CHECK wording ready; the
|
||||
missing piece is the DDL surface itself.
|
||||
|
||||
### V-series UX projects
|
||||
|
||||
Still pending from handoff-6:
|
||||
|
||||
- V4 — session log + Markdown export.
|
||||
- V1/V2 — relationship rendering (the "two structures
|
||||
+ arrow" view).
|
||||
- V3 — ER diagram export.
|
||||
|
||||
### Smaller items still on the table
|
||||
|
||||
- I1 — multi-line input (Enter inserts newline,
|
||||
Ctrl-Enter submits).
|
||||
- I1b — readline shortcuts (Ctrl-A/E, Ctrl-W/K/U).
|
||||
- C4 — m:n convenience (auto-junction-table).
|
||||
|
||||
### Tracked but bounded to other ADRs
|
||||
|
||||
- Q1 (SQL handling in advanced mode) — waits on Q4 (SQL
|
||||
subset ADR), which itself waits on the lexer model
|
||||
for advanced-mode tokenization. Now that ADR-0020 has
|
||||
established the DSL-side lexer, the Q4 ADR can decide
|
||||
whether to share / wrap `sqlparser-rs`'s tokens or
|
||||
add a parallel SQL lexer.
|
||||
- U-series undo/snapshot.
|
||||
- Settings persistence — feeds the deferred
|
||||
`messages` persistence and (when it arrives) any
|
||||
user-configurable theme picker.
|
||||
|
||||
## Sharp edges and subtleties (delta vs. handoff-6)
|
||||
|
||||
Carried-over edges still apply. New ones this session:
|
||||
|
||||
- **The parser combinators are now `Parser<'a, &'a [Token],
|
||||
…>` instead of `Parser<'a, &'a str, …>`.** Helper
|
||||
combinators in `dsl/parser.rs`: `kw(Keyword)`,
|
||||
`punct(Punct)`, `ident_inner()` /
|
||||
`ident_ctx(IdentSlot)`, `number_literal()`,
|
||||
`string_literal()`, `string_payload()`, `flag(name)`.
|
||||
All defined inline at the top of the parser combinator
|
||||
section.
|
||||
|
||||
- **`ident_inner` is the only call that produces a bare
|
||||
identifier match.** Every command parser combinator
|
||||
must use `ident_ctx(slot)`. There is no compile-time
|
||||
enforcement (`ident_inner` is `fn` not a sealed type),
|
||||
only convention + code review.
|
||||
|
||||
- **`ParseError::Invalid` gained two fields**:
|
||||
`at_eof: bool` (stage 4) and `expected: Vec<String>`
|
||||
(stage 5). Pattern matches with `..` are unaffected;
|
||||
the two constructions in `dsl/parser.rs` populate
|
||||
both correctly. `at_eof` for custom errors is
|
||||
conservatively `true` — known limitation, noted on
|
||||
the field's docstring.
|
||||
|
||||
- **Type names are NOT keywords.** They lex as
|
||||
`Identifier` tokens; the parser's `type_keyword()`
|
||||
helper uses `Type::from_str` to classify, which emits
|
||||
the existing "unknown type 'X' (expected one of: …)"
|
||||
custom error on miss. The completion engine
|
||||
recognises the `"type"` label (from
|
||||
`ident_inner().labelled("type")` inside
|
||||
`type_keyword`) via the `TYPE_SLOT_LABEL` constant in
|
||||
`completion.rs`. Adding a new type means: add the
|
||||
variant + keyword to `Type::all()` (ADR-0005), and
|
||||
the completion / error paths pick it up automatically.
|
||||
|
||||
- **The lexer eats whitespace.** `padded()` combinators
|
||||
are gone from the parser entirely. Whitespace
|
||||
positioning between tokens is recoverable from token
|
||||
spans if anyone needs it.
|
||||
|
||||
- **`replay` parses via a special-case** in
|
||||
`try_parse_replay_with_bare_path` (before the chumsky
|
||||
parser runs). Quoted paths go through chumsky;
|
||||
everything else is source-sliced from the byte after
|
||||
`replay`. Documented in ADR-0020 §6.
|
||||
|
||||
- **The hint panel render now branches** on the
|
||||
`AmbientHint` enum: `Prose(String)` or
|
||||
`Candidates { items, selected }`. The renderer in
|
||||
`ui.rs::render_candidate_line` builds spans with
|
||||
per-candidate kind colouring and `<` / `>` scroll
|
||||
markers when overflowing.
|
||||
|
||||
- **The schema cache refresh fires from the runtime**,
|
||||
not from the App. App receives `AppEvent::SchemaCacheRefreshed(cache)`
|
||||
and stores it. Refresh sites: project load, after DDL,
|
||||
after rebuild, after replay. Best-effort: a failed
|
||||
`list_names_for` for one slot kind leaves that field
|
||||
empty in the cache rather than dropping the whole
|
||||
refresh.
|
||||
|
||||
- **Tab/Shift-Tab key handling is at the TOP of
|
||||
`handle_key`** — before the modal short-circuit's
|
||||
match arm and before the regular key matcher. The
|
||||
ordering is: `if modal { handle_modal_key }; match
|
||||
Tab/BackTab/Esc-with-memo/Backspace-with-memo { … };
|
||||
match (the existing key matcher) { … }`. Esc /
|
||||
Backspace match arms have `if self.last_completion.is_some()`
|
||||
guards so they fall through to normal behaviour when
|
||||
no memo is alive.
|
||||
|
||||
- **`completion::candidates_at_cursor` is sync.** It
|
||||
consults the in-memory `SchemaCache` rather than the
|
||||
worker thread. The cache may be slightly stale
|
||||
between a DDL command and its
|
||||
`SchemaCacheRefreshed` event; acceptable per
|
||||
ADR-0022 §9.
|
||||
|
||||
- **`NewName` slots return `&[]` from `SchemaCache::for_slot`**
|
||||
— the user invents these names. The `typing_name_at_cursor`
|
||||
function handles them with a friendlier hint
|
||||
("Type a name, then …") via a placeholder-substitution
|
||||
re-parse.
|
||||
|
||||
- **The `TYPE_SLOT_LABEL = "type"` const** in
|
||||
`completion.rs` must equal what
|
||||
`dsl::parser::type_keyword` labels its `select_ref!`
|
||||
with. The strings are physically separate; no
|
||||
compile-time link. If a future maintainer changes one,
|
||||
the other must change too.
|
||||
|
||||
- **The macro-generated `Keyword::ALL` and `Punct::ALL`
|
||||
arrays** are the canonical iteration source for
|
||||
catalog-validity tests. A new keyword or punct that
|
||||
forgets a YAML entry under `parse.token.*` fails the
|
||||
`keys_validate_against_catalog` test loudly.
|
||||
|
||||
## ADR index (read these before touching the related areas)
|
||||
|
||||
```
|
||||
0000 Record architecture decisions (process)
|
||||
0001 Language and TUI framework (Rust + Ratatui)
|
||||
— amended by ADR-0020 (adds tokenization layer)
|
||||
0002 Database engine (User-facing posture)
|
||||
0003 Input modes and command dispatch
|
||||
0004 Project file format (amended by 0015)
|
||||
0005 Column type vocabulary
|
||||
0006 Undo snapshots and replay log
|
||||
0007 Sharing and export (amended by 0015 amendment 1)
|
||||
0008 Testing approach
|
||||
0009 DSL command syntax conventions
|
||||
0010 Database access via worker thread
|
||||
0011 FK column type compatibility
|
||||
0012 Internal metadata for user-facing column types
|
||||
0013 Relationships, naming, and rebuild-table strategy
|
||||
0014 Data operations, value literals, and auto-show
|
||||
0015 Project storage runtime
|
||||
0016 Pretty table rendering for data and structure views
|
||||
0017 Column type-change compatibility
|
||||
0018 Auto-fill contracts for serial and shortid columns
|
||||
0019 Friendly error layer (H1) and i18n message catalog
|
||||
0020 Tokenization layer for the DSL parser
|
||||
— IMPLEMENTED (this session). Amends ADR-0001.
|
||||
0021 Parser-as-source-of-truth for H1a
|
||||
— IMPLEMENTED (this session). Builds on ADR-0020.
|
||||
0022 Ambient typing assistance (I3 + I4 unified)
|
||||
— IMPLEMENTED through stage 8e + 4 testing rounds.
|
||||
```
|
||||
|
||||
## Repository layout (delta vs. handoff-6)
|
||||
|
||||
```
|
||||
src/
|
||||
completion.rs — NEW. SchemaCache, Candidate,
|
||||
CandidateKind, Completion,
|
||||
LastCompletion structs.
|
||||
candidates_at_cursor,
|
||||
invalid_ident_at_cursor,
|
||||
typing_name_at_cursor.
|
||||
input_render.rs — NEW. StyledRun, AmbientHint,
|
||||
InputState. render_input_runs
|
||||
(token-class colour + parse-
|
||||
error overlay + invalid-ident
|
||||
overlay + cursor injection).
|
||||
ambient_hint (the resolution
|
||||
ladder: memo → candidates →
|
||||
invalid_ident → typing_name →
|
||||
parse-prose fallback).
|
||||
classify_input, lex_to_runs.
|
||||
dsl/
|
||||
ident_slot.rs — NEW. IdentSlot enum +
|
||||
expected_label /
|
||||
from_expected_label round-trip.
|
||||
keyword.rs — NEW. define_keywords! /
|
||||
define_punct! macros + ALL
|
||||
static tables.
|
||||
lexer.rs — NEW. Token, TokenKind,
|
||||
LexError, Span. lex() always
|
||||
succeeds; embeds Error tokens
|
||||
in the stream.
|
||||
parser.rs — Heavily refactored. Parser
|
||||
now operates on &[Token].
|
||||
ident_ctx(slot) wrapper.
|
||||
ParseError gained at_eof +
|
||||
expected fields.
|
||||
usage.rs — NEW. UsageEntry registry +
|
||||
matched_entry +
|
||||
entry_keywords_alphabetised.
|
||||
app.rs — Tab/BackTab/Esc-with-memo/
|
||||
Backspace-with-memo handlers.
|
||||
completion_tab_forward /
|
||||
_backward,
|
||||
start_or_complete_at / _last,
|
||||
commit_unique / commit_multi,
|
||||
replace_inserted,
|
||||
undo_last_completion.
|
||||
AppEvent::SchemaCacheRefreshed
|
||||
handler.
|
||||
last_completion: Option<…>,
|
||||
schema_cache: SchemaCache
|
||||
fields on App.
|
||||
render path in dispatch_dsl
|
||||
composes the three-block
|
||||
parse error per ADR-0021 §2.
|
||||
ui.rs — render_input_panel uses
|
||||
input_render::render_input_runs
|
||||
for simple mode; advanced
|
||||
modes fall back to plain
|
||||
before/under/after.
|
||||
render_output_line peels the
|
||||
dsl::ECHO_PREFIX from simple-
|
||||
mode echo lines and re-
|
||||
tokenises (lex_to_runs).
|
||||
render_hint_panel dispatches
|
||||
on AmbientHint variant;
|
||||
render_candidate_line composes
|
||||
spans with kind colouring +
|
||||
`<` `>` scroll markers.
|
||||
theme.rs — Seven new tok_* Color fields
|
||||
on Theme. Theme::token_color
|
||||
helper. WCAG-AA contrast
|
||||
values for dark + light.
|
||||
event.rs — AppEvent::SchemaCacheRefreshed
|
||||
variant.
|
||||
runtime.rs — refresh_schema_cache helper +
|
||||
call sites at every
|
||||
TablesRefreshed point.
|
||||
friendly/
|
||||
keys.rs — Massive expansion. hint.*,
|
||||
parse.usage.*, parse.token.*,
|
||||
ambient_*, invalid_ident,
|
||||
typing_name, typing_name_then
|
||||
keys all validated.
|
||||
strings/en-US.yaml — Same expansion on the YAML
|
||||
side. Hint sentences are
|
||||
capitalised (Submit / Next /
|
||||
Type / No such).
|
||||
docs/
|
||||
adr/
|
||||
0020-tokenization-layer-for-the-dsl-parser.md
|
||||
0021-parser-as-source-of-truth-for-h1a.md
|
||||
0022-ambient-typing-assistance.md
|
||||
README.md — indexed
|
||||
handoff/
|
||||
20260512-handoff-7.md — this file
|
||||
tests/
|
||||
parse_error_pedagogy.rs — NEW (ADR-0021). 9 integration
|
||||
tests for caret + structural
|
||||
error + usage template
|
||||
composition.
|
||||
```
|
||||
|
||||
## How to take over
|
||||
|
||||
1. Read this file.
|
||||
2. Read `CLAUDE.md` for the working-style rules.
|
||||
3. Read `docs/requirements.md` for the granular progress
|
||||
table (note: probably wants an update reflecting the
|
||||
stage-8 completion of I3 and I4).
|
||||
4. **Expect the user to begin with another testing pass.**
|
||||
Resist the urge to ship anything new until they have.
|
||||
Past rounds: r1-r4 each turned up real findings that
|
||||
needed surgical fixes.
|
||||
5. When findings arrive, **trace each one to the minimum
|
||||
relevant layer** before patching:
|
||||
- Completion behaviour → `src/completion.rs`
|
||||
(`candidates_at_cursor`,
|
||||
`invalid_ident_at_cursor`, `typing_name_at_cursor`).
|
||||
- Hint panel content → `src/input_render.rs::ambient_hint`.
|
||||
- Hint panel render → `src/ui.rs::render_candidate_line` /
|
||||
`render_hint_panel`.
|
||||
- Input pane colours → `src/theme.rs` (tok_* fields).
|
||||
- Parser shape / labels → `src/dsl/parser.rs`.
|
||||
- Catalog wording → `src/friendly/strings/en-US.yaml`
|
||||
(and `keys.rs` for the placeholder declarations).
|
||||
6. After testing settles, the next bigger move is **A1
|
||||
(CI workflow)** — quick win that locks in the now-
|
||||
substantial 760-test baseline before more design work.
|
||||
7. Then **Query DSL ADR + implementation** is the
|
||||
recommended bigger piece.
|
||||
8. Run `cargo test` to confirm the 760-test green
|
||||
baseline.
|
||||
9. Run `cargo clippy --all-targets` to confirm
|
||||
clippy-clean.
|
||||
|
||||
### End-to-end smoke test (current state, post-r4)
|
||||
|
||||
Demonstrates ambient typing assistance. Replaces handoff-6's
|
||||
smoke test, which is now stale (it predates ADR-0022).
|
||||
|
||||
```
|
||||
$ rm -rf /tmp/handoff7-smoke
|
||||
$ rdbms-playground --data-dir /tmp/handoff7-smoke
|
||||
|
||||
# Inside the app — empty input. Hint panel shows the existing
|
||||
# panel.hint_empty content (no ambient interference until you
|
||||
# type).
|
||||
|
||||
# Type a single character. Hint immediately offers candidates.
|
||||
a -- hint: "add"
|
||||
(single Keyword
|
||||
candidate, purple)
|
||||
|
||||
# Tab inserts with trailing space (single candidate, no memo).
|
||||
<Tab> -- input becomes "add "
|
||||
hint: "column"
|
||||
(next required kw)
|
||||
|
||||
# Tab again chains through unique completions.
|
||||
<Tab> -- input becomes "add column "
|
||||
hint: "to table"
|
||||
(multi: `to`, `table`,
|
||||
in source order)
|
||||
|
||||
# Multi-candidate Tab: inserts WITHOUT trailing space.
|
||||
<Tab> -- input "add column to"
|
||||
(memo alive,
|
||||
`to` highlighted)
|
||||
|
||||
# Tab cycles within the memo.
|
||||
<Tab> -- input "add column table"
|
||||
(memo still alive,
|
||||
`table` highlighted)
|
||||
|
||||
# Pressing space commits the choice. Memo clears.
|
||||
<space> -- input "add column table "
|
||||
|
||||
# Continue. Tab on the table-name slot offers schema entries
|
||||
# in cyan-teal (identifier colour).
|
||||
<Tab> -- if schema has Customers,
|
||||
Orders, etc., they cycle.
|
||||
|
||||
# Type a fresh column name yourself.
|
||||
Customers: Email -- builds "add column to
|
||||
table Customers: Email"
|
||||
|
||||
# Now hint reads "Type a name, then `(`" — wait, that's at the
|
||||
# NewName slot. After typing Email the parser has consumed it.
|
||||
# Hint reads "Next: `(`".
|
||||
|
||||
# Type the type slot.
|
||||
(de -- hint: "decimal"
|
||||
(single match,
|
||||
keyword colour)
|
||||
|
||||
<Tab> -- input completes to
|
||||
"...Email (decimal "
|
||||
with trailing space
|
||||
|
||||
# Type close paren and submit.
|
||||
)<Enter> -- command runs.
|
||||
|
||||
# Invalid identifier feedback:
|
||||
show data Custp -- if no table starts
|
||||
with Custp, `Custp`
|
||||
renders red and hint
|
||||
says "No such table:
|
||||
`Custp`"
|
||||
|
||||
# Unknown command fallback:
|
||||
frobulate widgets -- on submit, hint
|
||||
shows usage block;
|
||||
live hint shows
|
||||
"available commands"
|
||||
fallback.
|
||||
|
||||
quit
|
||||
```
|
||||
|
||||
### Manual spot-checks worth running
|
||||
|
||||
- Single-Tab chaining: `a` Tab Tab Tab Tab — should
|
||||
build `add column to ` (the unique-chain runs out at
|
||||
position 4 where `to` and `table` are both possible).
|
||||
- Multi-Tab cycling: `show ` Tab Tab Tab — should cycle
|
||||
data → table → data (wrap).
|
||||
- Esc undo: `show ` Tab Esc — should restore to `show `.
|
||||
- Backspace undo (symmetric): `show ` Tab Backspace —
|
||||
should also restore to `show `.
|
||||
- Sticky hint: `show ` Tab Tab — hint stays as the
|
||||
two-item candidate list with the selection moving.
|
||||
- Invalid identifier: type `show data X` where no table
|
||||
starts with X — `X` should render red + "No such table"
|
||||
hint.
|
||||
- Type completion: `add column to T: Name (` Tab Tab Tab
|
||||
— cycles through types.
|
||||
- Long candidate list: with many tables (say 12+),
|
||||
position at `show data ` to see the `<` / `>` scroll
|
||||
markers in the hint panel.
|
||||
- Advanced mode: `:show data` (one-shot) or `mode advanced`
|
||||
— input renders plain, hint shows existing
|
||||
`panel.hint_empty` content, Tab is a no-op.
|
||||
- Replay with bare path: `replay history.log` — should
|
||||
work as before via the source-slice special case.
|
||||
- Replay with quoted path: `replay 'my project/data.log'` —
|
||||
chumsky path.
|
||||
- Multi-byte UTF-8 inside a string literal: `insert into T
|
||||
values ('café')` — lexer + render must not panic.
|
||||
Reference in New Issue
Block a user