From 9aa7e2ede0a6b4fbf5779df7cded8cfff46ddeb1 Mon Sep 17 00:00:00 2001 From: "claude@clouddev1" Date: Mon, 18 May 2026 21:27:52 +0000 Subject: [PATCH] =?UTF-8?q?docs:=20add=20ADR-0028=20=E2=80=94=20query=20pl?= =?UTF-8?q?ans=20(EXPLAIN=20QUERY=20PLAN)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The QA1/QA2 design: an `explain` prefix command over `show data` / `update` / `delete` that runs EXPLAIN QUERY PLAN (without executing the statement) and renders the result as an annotated tree. Plan steps keep the engine's own wording; an annotation taxonomy marks full scans, index use, and the automatic-index "you should add an index here" case. Introduces a general styled-output-line mechanism — an OutputLine may carry per-span styling — realising the per-span theming ADR-0016 deferred; the plan renderer is its first consumer. The explained SQL is shown above the tree as standard, copy-pasteable SQL. - docs/adr/0028-query-plans.md — the ADR. - docs/adr/README.md — index entry. - docs/requirements.md — QA2 [~] -> [ ]; QA1 note reconciled (designed in ADR-0028). --- docs/adr/0028-query-plans.md | 305 +++++++++++++++++++++++++++++++++++ docs/adr/README.md | 1 + docs/requirements.md | 14 +- 3 files changed, 315 insertions(+), 5 deletions(-) create mode 100644 docs/adr/0028-query-plans.md diff --git a/docs/adr/0028-query-plans.md b/docs/adr/0028-query-plans.md new file mode 100644 index 0000000..b6e6fc2 --- /dev/null +++ b/docs/adr/0028-query-plans.md @@ -0,0 +1,305 @@ +# ADR-0028: Query plans (`EXPLAIN QUERY PLAN`) + +## Status + +Accepted + +## Context + +`QA1` commits to running `EXPLAIN QUERY PLAN` on demand and +rendering the result as an annotated tree that highlights +full table scans, index use, and join order. `QA2` — the +rendering specifics (tree layout, annotation taxonomy, +colour scheme) — was deferred pending this ADR. This ADR +covers both: the `explain` command and how its output is +rendered. + +Two earlier decisions make this the right moment: + +- **ADR-0025** gave the playground real, persistent + indexes. +- **ADR-0026** designed `show data … where ` — a + *filtered* query. + +Both matter, because a query plan is only pedagogically +interesting for a filtered query: an unfiltered +`SELECT * FROM t` is always a full scan, and an index can +never appear. The teaching payoff — a plan that visibly +flips from a scan to an index search once an index exists +— needs a `WHERE`. So a genuinely interesting `explain` +depends on ADR-0026 being implemented; this ADR, like +ADR-0026 and ADR-0027, is design-ahead. + +`explain` also continues the arc the WHERE-expression work +began: it is one more step from the simple DSL toward real +SQL, introducing the actual `EXPLAIN QUERY PLAN` concept +and the engine's own plan vocabulary (`SCAN`, `SEARCH`, +`USING INDEX`). + +One rendering obstacle shapes the decision. The output +panel today stores each line as plain text with a single +`OutputKind`, and `render_output_line` colours the whole +line by that kind. A useful plan wants finer colour — +marking the *parts* of a step that signal a scan versus an +index. ADR-0016 deferred exactly this ("per-cell theming … +V4 territory") but explicitly anticipated it (its OOS-3: +"sets up the architecture, defers the colours until the +query DSL ships"). This ADR realizes it. + +## Decision + +### 1. Invocation — the `explain` prefix + +A plan is requested by prefixing a query with `explain`: + +``` +explain show data [where ] [limit ] +explain update set … (where | --all-rows) +explain delete from (where | --all-rows) +``` + +- `explain` mirrors SQL's own `EXPLAIN QUERY PLAN `; + the prefix teaches the real concept directly. +- It applies to the three commands that issue a query with + a row-finding step: `show data`, `update`, `delete`. +- **`EXPLAIN QUERY PLAN` does not execute the statement.** + So `explain update …` and `explain delete …` show how + the engine *would* locate the rows and change nothing — + a safe way to inspect a destructive operation's plan. + This is stated explicitly because it is pedagogically + useful and not self-evident. +- The wrapped command must be well-formed: a complete + `update` / `delete` still needs its `where` or + `--all-rows`. `explain` of an incomplete command is the + same parse error the command alone would be. + +**Grammar.** `explain` is a new top-level command. After +the `explain` keyword its shape is a `Choice` over the +three explainable commands' shapes, reached through the +`Subgrammar` node (ADR-0026) — the `show` / `update` / +`delete` grammars are *referenced*, not duplicated, so the +explained command is parsed, completed, hinted, and +highlighted exactly as it is on its own. + +**AST.** `Command::Explain { query: Box }`. The +inner `Command` is the ordinary parsed command; the +runtime recognizes the `Explain` wrapper and routes it to +the plan path instead of normal execution. + +### 2. Capturing the plan + +The database worker prepares `EXPLAIN QUERY PLAN `, +where `` is exactly the SQL the inner command would +otherwise have run, and reads back the result rows. Each +row is `(id, parent, notused, detail)` — the same +read-only, multi-row shape as the existing `PRAGMA`-backed +reads (`read_table_indexes`). + +- The inner command's SQL is produced by the *same* + construction logic that builds it for real execution, so + the plan is the plan of the actual query, not an + approximation. This means the SQL-building step is + separated from execution for the explainable commands. +- `EXPLAIN QUERY PLAN` determines the plan from the + statement's structure and the schema, not from parameter + *values*. The statement is still prepared with the inner + command's parameters bound (so a parameterised `WHERE` + prepares cleanly); the bound values do not affect the + plan. +- A new `Request::ExplainPlan` / `do_explain_plan`, and a + `QueryPlan` result type carrying the tree, flow back + through `CommandOutcome` / `AppEvent` like other command + results. + +### 3. The plan tree + +The `id` / `parent` columns form a tree. It is rendered +indented, with box-drawing connectors (`├─`, `└─`, `│`), +the way a file tree is drawn. Each node's text is the +engine's `detail` string **verbatim** — `SCAN Customers`, +`SEARCH Customers USING INDEX Customers_email_idx +(Email=?)`, and so on. Nothing is reworded. + +Verbatim text is a deliberate pedagogical choice: it +teaches the real vocabulary a learner meets in every +database tool. The `detail` strings name no engine +*product*, so the ADR-0002 "no engine name in user-facing +strings" rule is satisfied as-is. + +The block is preceded by the usual command echo, and the +SQL being explained is shown above the tree — seeing the +generated SQL beside its plan is itself part of the +simple → advanced bridge. + +The displayed SQL is **standard SQL**, rendered to read as +a complete, copy-pasteable query: identifiers are +double-quoted (the ISO delimited-identifier form); `WHERE` +literals are shown *inline* (`WHERE "Email" = +'alice@example.com'`) rather than as the `?` placeholders +the statement is actually prepared with (§2 — execution +and plan capture keep the parameters); and inequality is +written `<>` even when the user typed `!=`. The one clause +the display carries that the user did not type is the +implicit `ORDER BY ` that `limit` adds (ADR-0026 §5) +— itself a worthwhile lesson. + +### 4. Annotation taxonomy + +Each node's `detail` string is classified, by matching it +against a small table of substring patterns, into one of: + +| Category | Recognised by | Reading | +|---|---|---| +| Full scan | `SCAN `, no index | every row read — expensive | +| Index search | `SEARCH … USING INDEX …` | indexed lookup — efficient | +| Covering index | `USING COVERING INDEX` | indexed, no table fetch | +| Primary-key lookup | `USING INTEGER PRIMARY KEY` | direct row lookup | +| Automatic index | `USING AUTOMATIC … INDEX` | the engine built a *temporary* index because none existed — the strongest "add an index here" signal | +| Temp B-tree | `USE TEMP B-TREE FOR …` | sorting / grouping with no index to lean on | +| Neutral | anything else / structural | — | + +A `detail` that matches no pattern renders neutral rather +than failing — the engine's plan vocabulary may grow. + +The **automatic index** category is the most important +teaching moment: it is the case where the learner *should* +have added an index and the engine quietly compensated. It +is called out distinctly, not folded in with plain scans. + +### 5. Styled output lines (the rendering mechanism) + +Span-level colour needs the output panel to colour *parts* +of a line. Today an `OutputLine` is +`{ text: String, kind: OutputKind }` and +`render_output_line` colours the whole line by `kind` — +except simple-mode echo lines, which already render +multi-span from re-lexed token runs +(`input_render::lex_to_runs` → runs of +`{ byte_range, style }`). + +This ADR adds, to `OutputLine`, an optional **styled-runs** +payload — a list of `{ byte_range, style-class }` over the +line's text, the same shape `lex_to_runs` already +produces. `render_output_line` gains one branch: when the +payload is present, render the text as spans per the runs; +otherwise fall back to the existing whole-line `kind` +styling. The echo path is unchanged. + +The runs carry a *semantic style class*, resolved to a +concrete colour at render time from the active theme — not +a baked-in colour — so the styling stays correct +regardless of theme. + +This is a **general** capability: any output line may now +carry rich styling. The plan renderer is its first and, +for now, only consumer; the existing renderers +(`render_structure`, `render_data_table`) keep producing +plain lines. The mechanism is the per-span styling +ADR-0016 anticipated, and V4's session-log work will reuse +it — the same "general mechanism, single current consumer" +shape as ADR-0027's diagnostics model. + +Scroll math is unaffected: one display row per plan node; +styled spans do not change the line count. + +### 6. Colour scheme + +Beyond neutral text the plan needs an "efficient" colour +and an "expensive" colour: + +- **Efficient** — index search, covering index, primary-key + lookup — green. +- **Expensive** — full scan, temp B-tree — amber, reusing + the `warning` colour introduced by ADR-0027. +- **Automatic index** — amber as well, but with a distinct + marker (an icon or short tag) so it reads as "you should + add an index", not merely "this is slow". +- Connector glyphs and table / index *names* stay neutral; + only the category-bearing keywords of the `detail` string + carry the category colour. + +`theme.system`'s green is the existing "normal output" +colour; a plan-specific efficient colour distinct from it +avoids "green means two things". The exact theme fields are +an implementation detail; the requirement is that the +scheme is legible on both light and dark backgrounds +(NFR-5, NFR-7). + +### 7. Out of scope + +- **Explaining raw advanced-mode SQL.** There is no SQL + parser yet (`Q1`); `explain` covers the simple-mode DSL + queries. When SQL parsing lands, `explain` extends to it. +- **`EXPLAIN`** (the bytecode form) — only + `EXPLAIN QUERY PLAN`. The bytecode dump is not a teaching + surface. +- **Cost estimates / row-count predictions** — + `EXPLAIN QUERY PLAN` does not provide them and this ADR + does not invent them. +- **Re-styling existing output.** The styled-line mechanism + (§5) is available to all output, but this ADR only wires + the plan renderer to it; `render_structure` / + `render_data_table` are untouched. +- **A plan history, or multiple plan tabs.** + +## Consequences + +- A new `explain` command — `Command::Explain`, + `Request::ExplainPlan` + `do_explain_plan`, a `QueryPlan` + result type, and `CommandOutcome` / `AppEvent` variants. +- `explain` covers `show data` / `update` / `delete`; + because `EXPLAIN QUERY PLAN` never executes, explaining a + destructive command is safe. +- The SQL-construction step for the explainable commands is + separated from execution, so the same SQL feeds both real + execution and `EXPLAIN QUERY PLAN`. +- `OutputLine` gains an optional styled-runs payload, and + `render_output_line` a branch to honour it — a general + per-span output-styling capability (ADR-0016's OOS-3 + realized), with the plan renderer as its first consumer. +- A new `render_explain_plan` in `output_render.rs` + producing the styled tree; a small substring-pattern + table for the annotation taxonomy. +- Theme gains plan colours (an efficient colour distinct + from `system`; `warning` reused for expensive). +- Depends on ADR-0026: a plan that flips between a scan and + an index search needs `show data … where`. The feature + works against whatever queries exist; it is fully + realised once C5a is implemented. +- Builds toward `Q1`: when advanced-mode SQL lands, + `explain` extends to cover it. + +## Implementation notes + +A sensible order, each step test-guarded: + +1. The styled-output-line mechanism — the `OutputLine` + styled-runs payload and the `render_output_line` branch. + No user-visible change on its own. +2. The `explain` grammar (the prefix plus the + `Subgrammar`-referenced query shapes) and + `Command::Explain` with its AST builder. +3. Separating SQL construction from execution for + `show data` / `update` / `delete`; `Request::ExplainPlan` + / `do_explain_plan`; the `QueryPlan` result and its + `CommandOutcome` / `AppEvent` wiring. +4. `render_explain_plan` — the tree layout, the annotation + taxonomy, and the styled runs; the theme colours. +5. Typing-surface matrix cells for `explain`. + +## See also + +- ADR-0002 — database engine; the "no engine name in + user-facing strings" rule (plan `detail` strings name no + product, so verbatim text complies). +- ADR-0016 — pretty table rendering; its OOS-3 anticipated + the per-span output styling §5 realizes. +- ADR-0024 — the unified grammar tree the `explain` command + plugs into. +- ADR-0025 — indexes; what makes a query plan + pedagogically interesting. +- ADR-0026 — complex WHERE expressions; the filtered query + worth explaining, and the `Subgrammar` node `explain` + reuses to reference the query grammars. +- ADR-0027 — the diagnostics model; the same "general + mechanism, single current consumer" shape as §5. diff --git a/docs/adr/README.md b/docs/adr/README.md index 9717f0e..0b592ed 100644 --- a/docs/adr/README.md +++ b/docs/adr/README.md @@ -33,3 +33,4 @@ This directory contains the project's ADRs, recorded per - [ADR-0025 — Indexes](0025-indexes.md) — **Accepted**, `add index` / `drop index`, persistence, rebuild-table preservation, and items-list display (`C3` index portion + `S2`) - [ADR-0026 — Complex WHERE expressions](0026-complex-where-expressions.md) — **Accepted**, stratified recursive expression grammar (`AND`/`OR`/`NOT`, comparisons, `LIKE`, `IS NULL`, `IN`, `BETWEEN`) for `update` / `delete` / `show data` filters; `show data` gains `where` + `limit`; adds the `Subgrammar` node and a recursive `Expr` AST (`C5a`) - [ADR-0027 — Input-field validity indicator](0027-input-validity-indicator.md) — **Accepted**, a debounced `[ERR]` / `[WRN]` marker at the input row's right edge, backed by a walker diagnostics-severity model (parse-outcome + schema-existence); advisory, never blocks submission (`S6`) +- [ADR-0028 — Query plans (`EXPLAIN QUERY PLAN`)](0028-query-plans.md) — **Accepted**, an `explain` prefix command over `show data` / `update` / `delete`; an annotated, span-styled plan tree; introduces the `OutputLine` styled-runs mechanism (ADR-0016's deferred per-span styling) (`QA1` / `QA2`) diff --git a/docs/requirements.md b/docs/requirements.md index 88ed1df..dbff64c 100644 --- a/docs/requirements.md +++ b/docs/requirements.md @@ -350,11 +350,15 @@ group enabled. (Earlier reference points: 1006 after ADR-0024 - [ ] **QA1** `EXPLAIN QUERY PLAN` is run on demand for queries; output is rendered as an annotated tree highlighting full scans, index use, and join order. - *(Unblocked by ADR-0025: indexes now exist, so a plan for - `show data where =` visibly changes with an - index. Still needs the QA2 rendering ADR.)* -- [~] **QA2** Plan rendering specifics (tree layout, annotation - taxonomy, colour scheme) — design and ADR pending. + *(Designed in ADR-0028: the `explain` prefix over + `show data` / `update` / `delete`, with a span-styled plan + tree. The pedagogical payoff — a plan that flips between a + full scan and an index search — needs a filtered query + (`show data … where`, designed in ADR-0026) plus indexes + (ADR-0025, done). Implementation pending.)* +- [ ] **QA2** Plan rendering specifics — tree layout, annotation + taxonomy, colour scheme. Designed in ADR-0028 (§3–§6); + implementation pending. ## Hints, help, errors