rdbms-playground/docs/adr/draft-clause-concept-hints.md

ADR-XXXX: Clause-concept hints — a hint.concept.* layer surfaced by cursor position (issue #37)

Status

Draft — implemented 2026-06-23 on branch feat/clause-concept-hints (pending merge). Number assigned at merge-to-main per ADR-0000 "Numbering discipline" (drafted under draft-clause-concept-hints.md / ADR-XXXX; README index entry added at merge). Extends ADR-0053 (contextual hint, D3/D4) additively; does not supersede it. Closes the deferred extension tracked as Gitea issue #37.

Implementation: Node::Concept { topic, inner } (dsl/grammar/mod.rs) + the span-recording driver arm (dsl/walker/driver.rs) + WalkContext::concept_spans / ConceptSpan (dsl/walker/context.rs); the concept_topic_at_cursor resolver (dsl/walker/mod.rs); 10 Node::Concept wrappers across shared.rs / ddl.rs / sql_create_table.rs (one shared REFERENTIAL_CLAUSES, two extracted cardinality sub-Seqs); the mode-keyed emit_tier3_block + note_hint_for_input layering (app.rs); the seven hint.concept.* blocks + hint.block.concept_heading (strings/en-US.yaml) with keys.rs declarations; and the test set (14 resolver unit tests, 3 comprehensiveness gates incl. the recursive Node::Concept visitor, 3 F1 integration tests, the hint_block_with_concept snapshot). Suite green across all tiers; fmt/clippy clean.

Revised after a /runda review (2026-06-22) that grounded the mechanism against the code and caught two design gaps: the clause mapping (D4) covered only the advanced-SQL constraint grammar and missed the simple-mode constraint suffix (ADR-0029); and the single-example content model collided with ADR-0053 D6's mode-correct-example rule for topics reachable in both modes. Both are resolved here — D4 now maps both grammars, and D5 adopts mode-keyed examples (user decision, 2026-06-22).

References ADR-0053 (the three-tier hint model + the hint.cmd.* / hint.err.* corpus), ADR-0024 (the unified grammar/walker, HintMode- per-node, MatchedPath spans), ADR-0022 (ambient tier-2 typing assistance), ADR-0013/0045 (the relationship + m:n grammar), ADR-0035 (advanced-mode CREATE TABLE constraint grammar).

Context

ADR-0053 delivers tier-3 teaching hints at per-form granularity: one hint.cmd.<form> block (what / example / concept) per command form, surfaced by F1 on the live input. Position-awareness within a form is owned by tier-2 (the ambient candidate list / slot prose, ADR-0022).

During ADR-0053 Phase B we identified a third teaching point the per-form model does not serve: concepts that live inside a clause, where a learner may want teaching deeper than tier-2's candidate list but narrower than the whole-form block. The canonical examples are the most load-bearing relational concepts in the tool:

• the referential actions clause — … on delete ⟨cascade | set null | restrict | no action⟩ (and on update);
• the cardinality marker — 1:n (one parent, many children) and m:n (many-to-many, auto-junction per ADR-0045);
• the primary-key declaration — with pk(…) in simple mode and the primary key constraint in create table (…);
• the column-constraint slots in create table (…) — unique, check (…), foreign key / references.

A student parked in one of these clauses is, by definition, looking at the exact relational concept that clause embodies. ADR-0053 explicitly deferred this ("clause-concept hints", issue #37) and recorded that the per-form keying does not lock it out: a later hint.concept.<topic> namespace can be surfaced when the cursor sits in a recognized clause, layered on top of the per-form block.

This ADR is that follow-up. It is pedagogy-led, not metrics-led — we do not expect to accumulate usage statistics about which clauses learners pause at, so the clause set is chosen by relational-teaching value, not data.

Two user decisions taken up front (2026-06-22)

The issue left two scope questions open. Both were decided by the user before design:

1. Detection precision: "anywhere inside the clause." The hint fires whenever the cursor sits within a recognized clause's span — including parked inside already-typed text (e.g. add 1:n relation|ship …) — not only at the slot boundary where the next token is about to be typed.
2. Clause scope: all four families — referential actions, the create-table constraint slots, with pk, and 1:n / m:n cardinality.

Why the existing HintMode/pending_hint_mode mechanism is not enough

Tier-2 slot prose is driven by Node::Hinted { mode, inner }: entering the node sets ctx.pending_hint_mode, and the resolver reads it at the end of the cursor-trimmed walk (hint_resolution_at_input_in_mode, src/dsl/walker/mod.rs:107). Crucially, the driver clears pending_hint_mode on every successful match (src/dsl/walker/ driver.rs:154): it records only "the Hinted slot the cursor is about to fill", and the moment the slot's content matches, the signal is gone.

That is exactly the boundary semantics tier-2 wants — and exactly the opposite of "anywhere inside the clause". The instant the learner finishes typing cascade, pending_hint_mode clears, so a cursor parked on the finished on delete cascade would see nothing. Decision (1) above therefore needs a span-based mechanism that survives the clause being fully matched, not the single-slot pending_hint_mode.

Decision

D1 — A span-recording grammar wrapper, Node::Concept

Add a new grammar combinator, sibling to Node::Hinted:

Node::Concept { topic: &'static str, inner: &'static Node }

It is a pass-through for matching (it walks inner and returns inner's result verbatim — it never changes what parses), but as a side effect the driver records the byte span the node covered into a new context field:

// WalkContext
pub concept_spans: Vec<ConceptSpan>,   // { topic: &'static str, start: usize, end: usize }

• start = the first non-whitespace byte the node began matching at (post skip_whitespace, matching the driver's existing convention).
• end = the byte position the node's walk reached — the end of the matched span on a full match, or the stop position on an incomplete / failed inner. This makes the span the real extent of the clause as typed so far.

Unlike pending_hint_mode, concept_spans is append-only across the whole walk and is never cleared on match — so a fully-typed clause keeps its span, satisfying decision (1).

Precise push rule (grounded in NodeWalkResult, src/dsl/walker/driver.rs:97): walk inner, then push { topic, start, end } iff the inner result is one of Matched { end } (end = end), Incomplete { position }, or Failed { position } (end = position) — i.e. the node committed (consumed at least its first token, or ran out mid-clause). On NoMatch { .. } (the node never engaged — e.g. a Choice tried this branch and it didn't apply) push nothing. This is the NodeWalkResult-level analogue of the driver's existing pending_hint_mode clear-on-match leak-avoidance, and it means a failed Choice branch leaves no stale span.

Node::Concept is orthogonal to Node::Hinted — a clause that wants both a tier-2 slot prose and a tier-3 concept span nests them (Concept{ … inner: Hinted{ … } }). Neither changes the parse.

D2 — Cursor → topic resolution: innermost containing span

A new resolver:

pub fn concept_topic_at_cursor(
    input: &str, cursor: usize,
    schema: Option<&SchemaCache>, mode: Mode,
) -> Option<&'static str>

walks the full input buffer with WalkBound::EndOfInput (the dormant WalkBound::Position path is not used — see D3 rationale), collects ctx.concept_spans, and returns the topic of the innermost span containing cursor. "Innermost" = the containing span with the latest start (equivalently, the narrowest), so when a references clause (constraint slot) wraps a referential-actions clause, a cursor on on delete resolves to the more specific referential_actions, while a cursor on references Parent resolves to foreign_key.

Containment is inclusive of start and end (start <= cursor <= end), so the boundary case — cursor at the point of entering the clause — also resolves, meaning this subsumes the simpler boundary detection for free.

The cursor passed in is the one already computed by feedback_view() (src/app.rs:3204 returns (view, cursor, _off)), which strips the : one-shot sigil and adjusts the cursor offset to index into the stripped view. The concept resolver walks that same view, so the :-strip is handled with no extra offset arithmetic.

D3 — Why full-buffer span-containment, not WalkBound::Position

WalkBound::Position(cursor) (defined, #[allow(dead_code)], src/dsl/walker/outcome.rs:26) slices the source at the cursor and walks the prefix. Two reasons it is the wrong tool here:

• It discards text after the cursor. A cursor parked early in a complete command (add 1:n| relationship from … on delete cascade) would truncate to add 1: and lose the parsed structure that tells us we are inside a well-formed relationship command. Full-buffer walking keeps the whole parse; the cursor is used only for span containment.
• It still rides pending_hint_mode's clear-on-match, which D1 already establishes is unsuitable for inside-clause detection.

Span-recording over a full EndOfInput walk needs no new walk-bound behaviour and leaves the dormant Position path untouched.

D4 — The clause set and their topics (both grammars)

Seven hint.concept.<topic> blocks, placed via Node::Concept wrappers across both the simple-mode (grammar/ddl.rs, ADR-0029) and advanced-mode (grammar/sql_create_table.rs, ADR-0035) grammars. The codebase has two distinct constraint grammars — simple-mode create table column constraints (COLUMN_CONSTRAINT_SUFFIX, ddl.rs:1082: not null / unique / default / check; PK is with pk, FK is a relationship) and the advanced-SQL constraint set (COL_CONSTRAINT_CHOICES, sql_create_table.rs: adds primary key / references). A topic reachable in both modes is wrapped in both grammars (or once, if the node is physically shared — e.g. REFERENTIAL_CLAUSES in shared.rs is reused by the relationship command and the advanced references clause).

Topic	Modes	Grammar node(s) to wrap
referential_actions	both	REFERENTIAL_CLAUSES (shared.rs — one wrapper, reused in both)
cardinality_one_to_many	simple only	a new CARDINALITY_1N sub-Seq extracted from ADD_RELATIONSHIP_NODES (ddl.rs:479)
cardinality_many_to_many	simple only	a new CARDINALITY_MN sub-Seq extracted from CREATE_M2N head (ddl.rs)
primary_key	both	WITH_PK (ddl.rs, simple) + the primary key constraint node (sql_create_table.rs, advanced)
unique	both	the unique constraint node in ddl.rs (UNIQUE_CONSTRAINT, :1051) + in sql_create_table.rs
check	both	the check (…) constraint node in ddl.rs (:1064) + in sql_create_table.rs
foreign_key	advanced only	REFERENCES_CLAUSE (sql_create_table.rs:206)

If a unique / check node turns out to be physically shared between the two grammars (as REFERENTIAL_CLAUSES is), one wrapper suffices — the mode-keyed example (D5) is selected by the live mode at F1 time, not by which grammar node matched, so wrapper-sharing never affects correctness. The exact wrapper count is pinned during implementation; the topic set is the seven above.

Nesting is intentional. foreign_key's REFERENCES_CLAUSE contains referential_actions' REFERENTIAL_CLAUSES, so a cursor on references Parent resolves to foreign_key and a cursor on the inner on delete resolves to referential_actions — D2's innermost-wins handles it.

Deliberate omissions (faithful to issue #37's explicit list of "primary key | unique | check | foreign key"): the not null and default column constraints get no concept block in v1. They are mechanical rather than the headline relational concepts #37 targets; adding them later is purely additive (a wrapper + a block + a coverage entry).

Cardinality is split (_one_to_many vs _many_to_many) rather than a single "cardinality" block: the two are pedagogically distinct — m:n auto-creates a junction table (ADR-0045) — the cursor is always unambiguously in one or the other, and both are simple-mode-only (1:n/m:n are app syntax, not SQL), so each carries a single plain example with no mode conflict.

referential_actions is one block covering both on delete and on update and all four actions (the action set is the concept; the block explains the choice rather than each action in isolation).

D5 — The content model: what / example / concept, with mode-keyed examples

Each hint.concept.<topic> block keeps the ADR-0053 D3 three-part shape — what / example / concept. The one extension is the example: to honour ADR-0053 D6's "mode-correct example, no cross-mode sharing" rule for topics reachable in both modes (the /runda finding), a both-mode topic carries two examples keyed by mode; a single-mode topic carries one plain example.

# both-mode topic — mode-keyed example
hint.concept.referential_actions:
  what: "Decide what happens to child rows when the parent they point at is deleted, or its key changes."
  example:
    simple: "add 1:n relationship from Customers.id to Orders.customer_id on delete cascade"
    advanced: "create table Orders (id int primary key, customer_id int references Customers(id) on delete cascade)"
  concept: "A foreign key forbids orphans by default (restrict). `cascade`
    deletes the children with the parent; `set null` keeps them but clears
    the link. The action encodes a real rule about your data."

# single-mode topic — one plain example
hint.concept.cardinality_one_to_many:
  what: "Link two tables so one parent row can own many child rows."
  example: "add 1:n relationship from Customers.id to Orders.customer_id"
  concept: "‘1:n’ is one parent, many children. The child table holds the
    foreign key column pointing back at the parent — that single shared
    value is what groups a parent's children together."

what/example are always present; concept is the teaching payload (it is what makes this tier-3). The both-mode topics are referential_actions, primary_key, unique, check; the single-mode (simple-only) topics are cardinality_one_to_many, cardinality_many_to_many; foreign_key is single-mode (advanced-only).

The blocks live under the existing hint: namespace in src/friendly/strings/en-US.yaml, alongside hint.cmd.* / hint.err.*, and every leaf key is declared in src/friendly/keys.rs::KEYS_AND_PLACEHOLDERS (the validator requires exhaustive declaration — no prefix exemption for the hint.* families). For both-mode topics that means hint.concept.<topic>.example.simple + .example.advanced; for single-mode topics, hint.concept.<topic>.example.

Authoring constraints (project copy rules — see CLAUDE.md): concept text is user-facing, so it must never name the database engine and must not say "DSL" — use "simple mode" / "advanced mode". The concept prose teaches the relational idea in plain language; examples are copyable, mode-correct lines.

D6 — Rendering: the concept block layers under the form block

This is an additive F1 surface. When F1 fires on live input (note_hint_for_input, src/app.rs:3200):

1. Resolve and emit the per-form hint.cmd.<form> block exactly as today (unchanged).
2. Then call concept_topic_at_cursor(buffer, cursor, …). If it yields a topic and hint.concept.<topic>.what exists, emit a second tier-3 block beneath the form block.

The two blocks are visually distinguished by their heading: the form block keeps the existing hint.block.heading ("Hint"); the concept block carries a distinct sub-heading from a new hint.block.concept_heading (e.g. "About this clause") so the learner sees the form-level help and the clause-level concept without confusing the two.

emit_tier3_block gains two small parameters: the heading key, and an optional mode used to resolve the example. Its example lookup becomes: prefer <stem>.example.<mode> (where <mode> is simple/advanced from the live effective_mode), else fall back to a plain <stem>.example. This keeps a single renderer for all three tier-3 families — hint.cmd.* and hint.err.* keep a plain .example (lookup falls straight through), while hint.concept.* both-mode topics resolve the mode-keyed variant. The mode-correct-example contract of ADR-0053 D6 is thereby honoured by the content keying + this lookup, not by duplicating the renderer.

The concept block is never shown alone and never replaces the form block — consistent with the issue ("layered on top of … does not replace"). If the per-form resolution fails (rare; falls back to tier-2), the concept layer is skipped too — F1 keeps a single coherent behaviour.

The submitted hint command (last-error route, D2 of ADR-0053) is unchanged — it has no cursor, so concept hints do not apply there.

The rendered shape of a form-block-plus-concept-block is locked by a new insta snapshot (hint_block_with_concept).

D7 — Comprehensiveness, validation, fallback

• Validation (build/test gate): every hint.concept.<topic> leaf key is declared in keys.rs; the existing keys_validate_against_catalog test fails on a typo or an undeclared key, in either direction.
• Comprehensiveness coverage test: extend the ADR-0053 comprehensiveness test with a CONCEPT_TOPICS table — each entry a (topic, ModeCoverage) where ModeCoverage is Both or Single — asserting: (a) every topic resolves to hint.concept.<topic> with a .what (and, per its ModeCoverage, either .example.simple +.example.advanced for Both, or a plain .example for Single); and (b) every Node::Concept wrapper reachable from the REGISTRY carries a topic in the table. (b) needs a recursive Node-tree visitor — Node is a recursive enum (Seq(&[Node]) / Choice / Optional / Repeated / Hinted / Concept), so the test walks each REGISTRY shape collecting Node::Concept topics. A wrapper added without content (or a topic with a wrapper missing) fails the test — this is what makes the clause set enforceable rather than aspirational, since keys.rs only checks that referenced keys resolve.
• Graceful degrade: emit_tier3_block already returns false when a .what key is absent, so a missing concept block degrades to "form block only" — never a blank or an error. The validation above means that path is a safety net, not the plan.

Forks

• Detection precision = anywhere inside the clause (user, 2026-06-22) rather than slot-boundary-only. Drives the span-recording mechanism (D1) over the simpler pending_hint_mode reuse.
• Clause scope = all four families (user, 2026-06-22) rather than the issue's "start with referential actions + constraint slots" subset.
• Mechanism = full-buffer span-containment, not WalkBound::Position (D3) — keeps the whole parse and leaves the dormant cursor-bound path untouched. Engineering choice; rationale in D3.
• Mode-keyed examples for both-mode topics (user, 2026-06-22, resolving the /runda finding) rather than splitting into _simple/_advanced blocks — one concept block per topic, two examples selected by live mode (D5/D6). Honours ADR-0053 D6's mode-correct-example rule without duplicating concept prose.
• Cardinality split into two blocks (D4) rather than one — the 1:n and m:n concepts diverge (junction table). Pedagogy choice.
• primary_key shared between with pk and the constraint slot (D4) — one concept, two syntaxes (a mode-keyed example per D5). Content choice.
• Distinct sub-heading for the concept block (D6) rather than merging into one block or an undifferentiated second "Hint" — keeps form-level and clause-level guidance legible. UX choice.

Consequences

• A fourth contextual surface exists within tier-3: clause-level teaching, deeper than tier-2's candidate list, narrower than the per-form block, surfaced by cursor position on F1.
• One new grammar node (Node::Concept), one new WalkContext field (concept_spans), one new resolver (concept_topic_at_cursor), two new renderer parameters (heading key + example mode), and ~10 Node::Concept wrappers across both constraint grammars (exact count pinned at implementation per D4's shared-node note), plus two extracted cardinality sub-Seq nodes. No change to what parses; no change to tier-2; no change to the submitted hint path.
• A new content sub-namespace — hint.concept.* (seven topics; the four both-mode topics carry two examples each) — enters the catalogue under hint:, validated by keys.rs and the comprehensiveness test. Ongoing surface: a future clause worth teaching ships its Node::Concept wrapper + block + coverage entry together (a checklist item for feature ADRs that add clauses).
• CHANGELOG: a new [Unreleased] → Added bullet (user-facing F1 behaviour), phrased under the copy rules (no engine name; "simple mode"/"advanced mode", not "DSL").
• Testing: Tier-1 unit tests for concept_topic_at_cursor across each clause (cursor at boundary, cursor inside typed text, cursor outside → None, nested references→on delete innermost-wins, both modes); the F1 layering logic (form block always; concept block appended iff a topic resolves; concept skipped when the form block falls back); the : one-shot strip carried through (it must still resolve advanced forms); Tier-2 insta snapshot hint_block_with_concept; Tier-3 integration tests (type a partial relationship command, move the cursor into on delete, F1 → form block + referential-actions concept block, buffer / cursor / completion memo untouched); the comprehensiveness
  • keys.rs validation tests (D7).

Out of scope

• Concept hints on the submitted hint command — OOS: no cursor; the last-error route is unchanged (D6).
• A concept block for every clause in the grammar — OOS for v1: the eight wrappers cover the highest-value relational concepts (D4). More can be added additively later, gated by the comprehensiveness test.
• Surfacing concept text in the always-on tier-2 ambient panel — OOS (rejected): tier-2 stays terse by design (ADR-0022); the concept layer is on-demand via F1, consistent with all of tier-3.
• A hint.concept.<topic> reachable by an explicit argument (e.g. hint cascade) — OOS: ADR-0053 D1 keeps hint argument-free; help <topic> owns explicit reference lookup.
• Localisation beyond en-US — OOS (deferred): the catalogue is i18n- structured (ADR-0019) but English-only for v1 (requirements X2).

Content inventory (implementation tracking)

Seven hint.concept.<topic> blocks, authored to the ADR-0053 D7 voice (what / concept always; example(s) per mode coverage):

• referential_actions (both — 2 examples) — on delete/update; cascade / set null / restrict / no action.
• cardinality_one_to_many (simple — 1) — one parent owns many children; the FK lives on the child.
• cardinality_many_to_many (simple — 1) — junction table; ADR-0045 auto-creation.
• primary_key (both — 2) — uniquely identifies a row; shared by with pk (simple) and the create table constraint (advanced).
• unique (both — 2) — no two rows share the value; distinct from a PK.
• check (both — 2) — a per-row rule enforced on write.
• foreign_key (advanced — 1) — a child column promising to point at a real parent.

Implementation artifacts: the Node::Concept wrappers (D4 table, ~10 across both grammars + the two extracted cardinality sub-Seqs); the CONCEPT_TOPICS coverage table (topic + ModeCoverage); the keys.rs declarations (mode-keyed example leaves for the four both-mode topics); the hint_block_with_concept snapshot; the CHANGELOG bullet.