rdbms-playground/docs/adr/0058-clause-concept-hints.md

# ADR-0058: Clause-concept hints — a `hint.concept.*` layer surfaced by cursor position (issue #37)

## Status

Accepted — **implemented 2026-06-23** on branch `feat/clause-concept-hints`
(pending merge). Number **0058**, reserved up front via the reserve-first
flow (ADR-0059) rather than the older placeholder-until-merge default —
this ADR is the first to carry a number stable from creation. Extends
**ADR-0053** (contextual `hint`, D3/D4) additively; does not supersede it.
Closes the deferred extension tracked as Gitea issue **#37**.

Implementation: `Node::Concept { topic, inner }` (`dsl/grammar/mod.rs`) +
the span-recording driver arm (`dsl/walker/driver.rs`) +
`WalkContext::concept_spans` / `ConceptSpan` (`dsl/walker/context.rs`);
the `concept_topic_at_cursor` resolver (`dsl/walker/mod.rs`); 10
`Node::Concept` wrappers across `shared.rs` / `ddl.rs` /
`sql_create_table.rs` (one shared `REFERENTIAL_CLAUSES`, two extracted
cardinality sub-`Seq`s); the mode-keyed `emit_tier3_block` +
`note_hint_for_input` layering (`app.rs`); the seven `hint.concept.*`
blocks + `hint.block.concept_heading` (`strings/en-US.yaml`) with
`keys.rs` declarations; and the test set (14 resolver unit tests, 3
comprehensiveness gates incl. the recursive `Node::Concept` visitor, 3 F1
integration tests, the `hint_block_with_concept` snapshot). Suite green
across all tiers; `fmt`/`clippy` clean.

Revised after a `/runda` review (2026-06-22) that grounded the mechanism
against the code and caught two design gaps: the clause mapping (D4)
covered only the advanced-SQL constraint grammar and missed the
simple-mode constraint suffix (ADR-0029); and the single-`example`
content model collided with ADR-0053 D6's mode-correct-example rule for
topics reachable in both modes. Both are resolved here — D4 now maps both
grammars, and D5 adopts **mode-keyed examples** (user decision,
2026-06-22).

References ADR-0053 (the three-tier `hint` model + the `hint.cmd.*` /
`hint.err.*` corpus), ADR-0024 (the unified grammar/walker, `HintMode`-
per-node, `MatchedPath` spans), ADR-0022 (ambient tier-2 typing
assistance), ADR-0013/0045 (the relationship + `m:n` grammar), ADR-0035
(advanced-mode `CREATE TABLE` constraint grammar).

## Context

ADR-0053 delivers tier-3 teaching hints at **per-form** granularity: one
`hint.cmd.<form>` block (`what` / `example` / `concept`) per command form,
surfaced by F1 on the live input. Position-awareness *within* a form is
owned by tier-2 (the ambient candidate list / slot prose, ADR-0022).

During ADR-0053 Phase B we identified a third teaching point the per-form
model does not serve: **concepts that live inside a clause**, where a
learner may want teaching deeper than tier-2's candidate list but narrower
than the whole-form block. The canonical examples are the most
load-bearing relational concepts in the tool:

- the **referential actions** clause — `… on delete ⟨cascade | set null |
  restrict | no action⟩` (and `on update`);
- the **cardinality** marker — `1:n` (one parent, many children) and
  `m:n` (many-to-many, auto-junction per ADR-0045);
- the **primary-key** declaration — `with pk(…)` in simple mode and the
  `primary key` constraint in `create table (…)`;
- the **column-constraint slots** in `create table (…)` — `unique`,
  `check (…)`, `foreign key` / `references`.

A student parked in one of these clauses is, by definition, looking at the
exact relational concept that clause embodies. ADR-0053 explicitly
deferred this ("clause-concept hints", issue #37) and recorded that the
per-form keying does **not** lock it out: a later `hint.concept.<topic>`
namespace can be surfaced when the cursor sits in a recognized clause,
layered on top of the per-form block.

This ADR is that follow-up. It is **pedagogy-led, not metrics-led** — we
do not expect to accumulate usage statistics about which clauses learners
pause at, so the clause set is chosen by relational-teaching value, not
data.

### Two user decisions taken up front (2026-06-22)

The issue left two scope questions open. Both were decided by the user
before design:

1. **Detection precision: "anywhere inside the clause."** The hint fires
   whenever the cursor sits *within* a recognized clause's span —
   including parked inside already-typed text (e.g. `add 1:n
   relation|ship …`) — not only at the slot boundary where the next token
   is about to be typed.
2. **Clause scope: all four families** — referential actions, the
   create-table constraint slots, `with pk`, and `1:n` / `m:n`
   cardinality.

### Why the existing `HintMode`/`pending_hint_mode` mechanism is not enough

Tier-2 slot prose is driven by `Node::Hinted { mode, inner }`: entering
the node sets `ctx.pending_hint_mode`, and the resolver reads it at the
end of the cursor-trimmed walk (`hint_resolution_at_input_in_mode`,
`src/dsl/walker/mod.rs:107`). Crucially, the driver **clears
`pending_hint_mode` on every successful match** (`src/dsl/walker/
driver.rs:154`): it records only "the Hinted slot the cursor is *about to
fill*", and the moment the slot's content matches, the signal is gone.

That is exactly the *boundary* semantics tier-2 wants — and exactly the
opposite of "anywhere inside the clause". The instant the learner finishes
typing `cascade`, `pending_hint_mode` clears, so a cursor parked on the
finished `on delete cascade` would see nothing. Decision (1) above
therefore needs a **span-based** mechanism that survives the clause being
fully matched, not the single-slot `pending_hint_mode`.

## Decision

### D1 — A span-recording grammar wrapper, `Node::Concept`

Add a new grammar combinator, sibling to `Node::Hinted`:

```rust
Node::Concept { topic: &'static str, inner: &'static Node }
```

It is a pass-through for matching (it walks `inner` and returns `inner`'s
result verbatim — it never changes what parses), but as a side effect the
driver records the **byte span the node covered** into a new context
field:

```rust
// WalkContext
pub concept_spans: Vec<ConceptSpan>,   // { topic: &'static str, start: usize, end: usize }
```

- `start` = the first non-whitespace byte the node began matching at
  (post `skip_whitespace`, matching the driver's existing convention).
- `end` = the byte position the node's walk reached — the end of the
  matched span on a full match, or the stop position on an incomplete /
  failed inner. This makes the span the real extent of the clause as typed
  so far.

Unlike `pending_hint_mode`, `concept_spans` is **append-only across the
whole walk** and is never cleared on match — so a fully-typed clause keeps
its span, satisfying decision (1).

**Precise push rule** (grounded in `NodeWalkResult`,
`src/dsl/walker/driver.rs:97`): walk `inner`, then push `{ topic, start,
end }` iff the inner result is one of `Matched { end }` (end = `end`),
`Incomplete { position }`, or `Failed { position }` (end = `position`) —
i.e. the node **committed** (consumed at least its first token, or ran out
mid-clause). On `NoMatch { .. }` (the node never engaged — e.g. a `Choice`
tried this branch and it didn't apply) push **nothing**. This is the
`NodeWalkResult`-level analogue of the driver's existing
`pending_hint_mode` clear-on-match leak-avoidance, and it means a failed
`Choice` branch leaves no stale span.

`Node::Concept` is **orthogonal to `Node::Hinted`** — a clause that wants
both a tier-2 slot prose *and* a tier-3 concept span nests them (`Concept{
… inner: Hinted{ … } }`). Neither changes the parse.

### D2 — Cursor → topic resolution: innermost containing span

A new resolver:

```rust
pub fn concept_topic_at_cursor(
    input: &str, cursor: usize,
    schema: Option<&SchemaCache>, mode: Mode,
) -> Option<&'static str>
```

walks the **full** input buffer with `WalkBound::EndOfInput` (the dormant
`WalkBound::Position` path is *not* used — see D3 rationale), collects
`ctx.concept_spans`, and returns the `topic` of the **innermost span
containing `cursor`**. "Innermost" = the containing span with the latest
`start` (equivalently, the narrowest), so when a `references` clause
(constraint slot) wraps a referential-actions clause, a cursor on `on
delete` resolves to the more specific `referential_actions`, while a
cursor on `references Parent` resolves to `foreign_key`.

Containment is inclusive of `start` and `end` (`start <= cursor <= end`),
so the boundary case — cursor *at* the point of entering the clause —
also resolves, meaning this subsumes the simpler boundary detection for
free.

The `cursor` passed in is the one **already computed by `feedback_view()`**
(`src/app.rs:3204` returns `(view, cursor, _off)`), which strips the `:`
one-shot sigil and adjusts the cursor offset to index into the stripped
`view`. The concept resolver walks that same `view`, so the `:`-strip is
handled with no extra offset arithmetic.

### D3 — Why full-buffer span-containment, not `WalkBound::Position`

`WalkBound::Position(cursor)` (defined, `#[allow(dead_code)]`,
`src/dsl/walker/outcome.rs:26`) slices the source at the cursor and walks
the prefix. Two reasons it is the wrong tool here:

- **It discards text after the cursor.** A cursor parked early in a
  complete command (`add 1:n| relationship from … on delete cascade`)
  would truncate to `add 1:` and lose the parsed structure that tells us
  we are inside a well-formed relationship command. Full-buffer walking
  keeps the whole parse; the cursor is used *only* for span containment.
- **It still rides `pending_hint_mode`'s clear-on-match**, which D1
  already establishes is unsuitable for inside-clause detection.

Span-recording over a full `EndOfInput` walk needs no new walk-bound
behaviour and leaves the dormant `Position` path untouched.

### D4 — The clause set and their topics (both grammars)

**Seven** `hint.concept.<topic>` blocks, placed via `Node::Concept`
wrappers across **both** the simple-mode (`grammar/ddl.rs`, ADR-0029) and
advanced-mode (`grammar/sql_create_table.rs`, ADR-0035) grammars. The
codebase has **two distinct constraint grammars** — simple-mode `create
table` column constraints (`COLUMN_CONSTRAINT_SUFFIX`, `ddl.rs:1082`:
`not null` / `unique` / `default` / `check`; PK is `with pk`, FK is a
relationship) and the advanced-SQL constraint set (`COL_CONSTRAINT_CHOICES`,
`sql_create_table.rs`: adds `primary key` / `references`). A topic
reachable in both modes is wrapped in **both** grammars (or once, if the
node is physically shared — e.g. `REFERENTIAL_CLAUSES` in `shared.rs` is
reused by the relationship command *and* the advanced `references` clause).

| Topic | Modes | Grammar node(s) to wrap |
|---|---|---|
| `referential_actions` | both | `REFERENTIAL_CLAUSES` (`shared.rs` — one wrapper, reused in both) |
| `cardinality_one_to_many` | simple only | a new `CARDINALITY_1N` sub-`Seq` extracted from `ADD_RELATIONSHIP_NODES` (`ddl.rs:479`) |
| `cardinality_many_to_many` | simple only | a new `CARDINALITY_MN` sub-`Seq` extracted from `CREATE_M2N` head (`ddl.rs`) |
| `primary_key` | both | `WITH_PK` (`ddl.rs`, simple) + the `primary key` constraint node (`sql_create_table.rs`, advanced) |
| `unique` | both | the `unique` constraint node in `ddl.rs` (`UNIQUE_CONSTRAINT`, `:1051`) + in `sql_create_table.rs` |
| `check` | both | the `check (…)` constraint node in `ddl.rs` (`:1064`) + in `sql_create_table.rs` |
| `foreign_key` | advanced only | `REFERENCES_CLAUSE` (`sql_create_table.rs:206`) |

If a `unique` / `check` node turns out to be physically shared between the
two grammars (as `REFERENTIAL_CLAUSES` is), one wrapper suffices — the
mode-keyed example (D5) is selected by the **live mode** at F1 time, not by
which grammar node matched, so wrapper-sharing never affects correctness.
The exact wrapper count is pinned during implementation; the *topic* set is
the seven above.

**Nesting is intentional.** `foreign_key`'s `REFERENCES_CLAUSE` contains
`referential_actions`' `REFERENTIAL_CLAUSES`, so a cursor on `references
Parent` resolves to `foreign_key` and a cursor on the inner `on delete`
resolves to `referential_actions` — D2's innermost-wins handles it.

**Deliberate omissions** (faithful to issue #37's explicit list of
"primary key | unique | check | foreign key"): the **`not null`** and
**`default`** column constraints get **no** concept block in v1. They are
mechanical rather than the headline relational concepts #37 targets;
adding them later is purely additive (a wrapper + a block + a coverage
entry).

Cardinality is **split** (`_one_to_many` vs `_many_to_many`) rather than a
single "cardinality" block: the two are pedagogically distinct — `m:n`
auto-creates a junction table (ADR-0045) — the cursor is always
unambiguously in one or the other, and both are **simple-mode-only**
(`1:n`/`m:n` are app syntax, not SQL), so each carries a single plain
example with no mode conflict.

`referential_actions` is one block covering both `on delete` and `on
update` and all four actions (the action set is the concept; the block
explains the choice rather than each action in isolation).

### D5 — The content model: `what` / `example` / `concept`, with mode-keyed examples

Each `hint.concept.<topic>` block keeps the ADR-0053 D3 three-part shape —
`what` / `example` / `concept`. The one extension is the **example**: to
honour ADR-0053 D6's "mode-correct example, no cross-mode sharing" rule
for topics reachable in both modes (the `/runda` finding), a both-mode
topic carries **two** examples keyed by mode; a single-mode topic carries
one plain example.

```yaml
# both-mode topic — mode-keyed example
hint.concept.referential_actions:
  what: "Decide what happens to child rows when the parent they point at is deleted, or its key changes."
  example:
    simple: "add 1:n relationship from Customers.id to Orders.customer_id on delete cascade"
    advanced: "create table Orders (id int primary key, customer_id int references Customers(id) on delete cascade)"
  concept: "A foreign key forbids orphans by default (restrict). `cascade`
    deletes the children with the parent; `set null` keeps them but clears
    the link. The action encodes a real rule about your data."

# single-mode topic — one plain example
hint.concept.cardinality_one_to_many:
  what: "Link two tables so one parent row can own many child rows."
  example: "add 1:n relationship from Customers.id to Orders.customer_id"
  concept: "‘1:n’ is one parent, many children. The child table holds the
    foreign key column pointing back at the parent — that single shared
    value is what groups a parent's children together."
```

`what`/`example` are always present; `concept` is the teaching payload
(it is what makes this tier-3). The both-mode topics are
`referential_actions`, `primary_key`, `unique`, `check`; the single-mode
(simple-only) topics are `cardinality_one_to_many`,
`cardinality_many_to_many`; `foreign_key` is single-mode (advanced-only).

The blocks live under the existing `hint:` namespace in
`src/friendly/strings/en-US.yaml`, alongside `hint.cmd.*` / `hint.err.*`,
and every leaf key is declared in
`src/friendly/keys.rs::KEYS_AND_PLACEHOLDERS` (the validator requires
exhaustive declaration — no prefix exemption for the `hint.*` families).
For both-mode topics that means `hint.concept.<topic>.example.simple` +
`.example.advanced`; for single-mode topics, `hint.concept.<topic>.example`.

**Authoring constraints** (project copy rules — see `CLAUDE.md`): concept
text is user-facing, so it must **never name the database engine** and
must **not say "DSL"** — use "simple mode" / "advanced mode". The `concept`
prose teaches the relational idea in plain language; examples are
copyable, mode-correct lines.

### D6 — Rendering: the concept block layers under the form block

This is an **additive** F1 surface. When F1 fires on live input
(`note_hint_for_input`, `src/app.rs:3200`):

1. Resolve and emit the per-form `hint.cmd.<form>` block exactly as today
   (unchanged).
2. **Then** call `concept_topic_at_cursor(buffer, cursor, …)`. If it
   yields a topic and `hint.concept.<topic>.what` exists, emit a second
   tier-3 block beneath the form block.

The two blocks are visually distinguished by their **heading**: the form
block keeps the existing `hint.block.heading` ("Hint"); the concept block
carries a distinct sub-heading from a new `hint.block.concept_heading`
(e.g. "About this clause") so the learner sees the form-level help *and*
the clause-level concept without confusing the two.

`emit_tier3_block` gains **two** small parameters: the heading key, and an
optional mode used to resolve the example. Its example lookup becomes:
prefer `<stem>.example.<mode>` (where `<mode>` is `simple`/`advanced` from
the live `effective_mode`), else fall back to a plain `<stem>.example`.
This keeps a single renderer for all three tier-3 families — `hint.cmd.*`
and `hint.err.*` keep a plain `.example` (lookup falls straight through),
while `hint.concept.*` both-mode topics resolve the mode-keyed variant.
The mode-correct-example contract of ADR-0053 D6 is thereby honoured by
the *content keying* + this lookup, not by duplicating the renderer.

The concept block is **never shown alone** and **never replaces** the form
block — consistent with the issue ("layered on top of … does not
replace"). If the per-form resolution fails (rare; falls back to tier-2),
the concept layer is skipped too — F1 keeps a single coherent behaviour.

The submitted `hint` command (last-error route, D2 of ADR-0053) is
**unchanged** — it has no cursor, so concept hints do not apply there.

The rendered shape of a form-block-plus-concept-block is locked by a new
`insta` snapshot (`hint_block_with_concept`).

### D7 — Comprehensiveness, validation, fallback

- **Validation (build/test gate):** every `hint.concept.<topic>` leaf key
  is declared in `keys.rs`; the existing `keys_validate_against_catalog`
  test fails on a typo or an undeclared key, in either direction.
- **Comprehensiveness coverage test:** extend the ADR-0053
  comprehensiveness test with a `CONCEPT_TOPICS` table — each entry a
  `(topic, ModeCoverage)` where `ModeCoverage` is `Both` or
  `Single` — asserting: (a) every topic resolves to `hint.concept.<topic>`
  with a `.what` (and, per its `ModeCoverage`, either `.example.simple`
  +`.example.advanced` for `Both`, or a plain `.example` for `Single`);
  and (b) every `Node::Concept` wrapper reachable from the REGISTRY
  carries a topic in the table. (b) needs a **recursive `Node`-tree
  visitor** — `Node` is a recursive enum (`Seq(&[Node])` / `Choice` /
  `Optional` / `Repeated` / `Hinted` / `Concept`), so the test walks each
  REGISTRY shape collecting `Node::Concept` topics. A wrapper added
  without content (or a topic with a wrapper missing) fails the test —
  this is what makes the clause set enforceable rather than aspirational,
  since `keys.rs` only checks that *referenced* keys resolve.
- **Graceful degrade:** `emit_tier3_block` already returns `false` when a
  `.what` key is absent, so a missing concept block degrades to "form
  block only" — never a blank or an error. The validation above means
  that path is a safety net, not the plan.

## Forks

- **Detection precision = anywhere inside the clause** (user, 2026-06-22)
  rather than slot-boundary-only. Drives the span-recording mechanism
  (D1) over the simpler `pending_hint_mode` reuse.
- **Clause scope = all four families** (user, 2026-06-22) rather than the
  issue's "start with referential actions + constraint slots" subset.
- **Mechanism = full-buffer span-containment, not `WalkBound::Position`**
  (D3) — keeps the whole parse and leaves the dormant cursor-bound path
  untouched. *Engineering choice; rationale in D3.*
- **Mode-keyed examples for both-mode topics** (user, 2026-06-22,
  resolving the `/runda` finding) rather than splitting into
  `_simple`/`_advanced` blocks — one concept block per topic, two examples
  selected by live mode (D5/D6). Honours ADR-0053 D6's mode-correct-example
  rule without duplicating concept prose.
- **Cardinality split into two blocks** (D4) rather than one — the `1:n`
  and `m:n` concepts diverge (junction table). *Pedagogy choice.*
- **`primary_key` shared between `with pk` and the constraint slot** (D4)
  — one concept, two syntaxes (a mode-keyed example per D5). *Content
  choice.*
- **Distinct sub-heading for the concept block** (D6) rather than merging
  into one block or an undifferentiated second "Hint" — keeps form-level
  and clause-level guidance legible. *UX choice.*

## Consequences

- **A fourth contextual surface exists** within tier-3: clause-level
  teaching, deeper than tier-2's candidate list, narrower than the
  per-form block, surfaced by cursor position on F1.
- **One new grammar node** (`Node::Concept`), **one new `WalkContext`
  field** (`concept_spans`), **one new resolver**
  (`concept_topic_at_cursor`), **two new renderer parameters** (heading
  key + example mode), and **~10 `Node::Concept` wrappers** across both
  constraint grammars (exact count pinned at implementation per D4's
  shared-node note), plus **two extracted cardinality sub-`Seq` nodes**.
  No change to what parses; no change to tier-2; no change to the
  submitted `hint` path.
- **A new content sub-namespace** — `hint.concept.*` (seven topics; the
  four both-mode topics carry two examples each) — enters the catalogue
  under `hint:`, validated by `keys.rs` and the comprehensiveness test.
  Ongoing surface: a future clause worth teaching ships its
  `Node::Concept` wrapper + block + coverage entry together (a checklist
  item for feature ADRs that add clauses).
- **CHANGELOG:** a new `[Unreleased] → Added` bullet (user-facing F1
  behaviour), phrased under the copy rules (no engine name; "simple
  mode"/"advanced mode", not "DSL").
- **Testing:** Tier-1 unit tests for `concept_topic_at_cursor` across each
  clause (cursor at boundary, cursor inside typed text, cursor outside →
  `None`, nested `references`→`on delete` innermost-wins, both modes);
  the F1 layering logic (form block always; concept block appended iff a
  topic resolves; concept skipped when the form block falls back); the
  `:` one-shot strip carried through (it must still resolve advanced
  forms); Tier-2 `insta` snapshot `hint_block_with_concept`; Tier-3
  integration tests (type a partial relationship command, move the cursor
  into `on delete`, F1 → form block + referential-actions concept block,
  **buffer / cursor / completion memo untouched**); the comprehensiveness
  + `keys.rs` validation tests (D7).

## Out of scope

- **Concept hints on the submitted `hint` command** — OOS: no cursor; the
  last-error route is unchanged (D6).
- **A concept block for every clause in the grammar** — OOS for v1: the
  eight wrappers cover the highest-value relational concepts (D4). More
  can be added additively later, gated by the comprehensiveness test.
- **Surfacing concept text in the always-on tier-2 ambient panel** — OOS
  (rejected): tier-2 stays terse by design (ADR-0022); the concept layer
  is on-demand via F1, consistent with all of tier-3.
- **A `hint.concept.<topic>` reachable by an explicit argument** (e.g.
  `hint cascade`) — OOS: ADR-0053 D1 keeps `hint` argument-free; `help
  <topic>` owns explicit reference lookup.
- **Localisation beyond `en-US`** — OOS (deferred): the catalogue is i18n-
  structured (ADR-0019) but English-only for v1 (requirements X2).

## Content inventory (implementation tracking)

Seven `hint.concept.<topic>` blocks, authored to the ADR-0053 D7 voice
(`what` / `concept` always; example(s) per mode coverage):

- [x] `referential_actions` *(both — 2 examples)* — on delete/update;
      cascade / set null / restrict / no action.
- [x] `cardinality_one_to_many` *(simple — 1)* — one parent owns many
      children; the FK lives on the child.
- [x] `cardinality_many_to_many` *(simple — 1)* — junction table; ADR-0045
      auto-creation.
- [x] `primary_key` *(both — 2)* — uniquely identifies a row; shared by
      `with pk` (simple) and the `create table` constraint (advanced).
- [x] `unique` *(both — 2)* — no two rows share the value; distinct from a
      PK.
- [x] `check` *(both — 2)* — a per-row rule enforced on write.
- [x] `foreign_key` *(advanced — 1)* — a child column promising to point
      at a real parent.

Implementation artifacts: the `Node::Concept` wrappers (D4 table, ~10
across both grammars + the two extracted cardinality sub-`Seq`s); the
`CONCEPT_TOPICS` coverage table (topic + `ModeCoverage`); the `keys.rs`
declarations (mode-keyed example leaves for the four both-mode topics);
the `hint_block_with_concept` snapshot; the CHANGELOG bullet.