# ADR-0028: Query plans (`EXPLAIN QUERY PLAN`) ## Status Accepted ## Context `QA1` commits to running `EXPLAIN QUERY PLAN` on demand and rendering the result as an annotated tree that highlights full table scans, index use, and join order. `QA2` — the rendering specifics (tree layout, annotation taxonomy, colour scheme) — was deferred pending this ADR. This ADR covers both: the `explain` command and how its output is rendered. Two earlier decisions make this the right moment: - **ADR-0025** gave the playground real, persistent indexes. - **ADR-0026** designed `show data … where ` — a *filtered* query. Both matter, because a query plan is only pedagogically interesting for a filtered query: an unfiltered `SELECT * FROM t` is always a full scan, and an index can never appear. The teaching payoff — a plan that visibly flips from a scan to an index search once an index exists — needs a `WHERE`. So a genuinely interesting `explain` depends on ADR-0026 being implemented; this ADR, like ADR-0026 and ADR-0027, is design-ahead. `explain` also continues the arc the WHERE-expression work began: it is one more step from the simple DSL toward real SQL, introducing the actual `EXPLAIN QUERY PLAN` concept and the engine's own plan vocabulary (`SCAN`, `SEARCH`, `USING INDEX`). One rendering obstacle shapes the decision. The output panel today stores each line as plain text with a single `OutputKind`, and `render_output_line` colours the whole line by that kind. A useful plan wants finer colour — marking the *parts* of a step that signal a scan versus an index. ADR-0016 deferred exactly this ("per-cell theming … V4 territory") but explicitly anticipated it (its OOS-3: "sets up the architecture, defers the colours until the query DSL ships"). This ADR realizes it. ## Decision ### 1. Invocation — the `explain` prefix A plan is requested by prefixing a query with `explain`: ``` explain show data [where ] [limit ] explain update set … (where | --all-rows) explain delete from (where | --all-rows) ``` - `explain` mirrors SQL's own `EXPLAIN QUERY PLAN `; the prefix teaches the real concept directly. - It applies to the three commands that issue a query with a row-finding step: `show data`, `update`, `delete`. - **`EXPLAIN QUERY PLAN` does not execute the statement.** So `explain update …` and `explain delete …` show how the engine *would* locate the rows and change nothing — a safe way to inspect a destructive operation's plan. This is stated explicitly because it is pedagogically useful and not self-evident. - The wrapped command must be well-formed: a complete `update` / `delete` still needs its `where` or `--all-rows`. `explain` of an incomplete command is the same parse error the command alone would be. **Grammar.** `explain` is a new top-level command. After the `explain` keyword its shape is a `Choice` over the three explainable commands' shapes, reached through the `Subgrammar` node (ADR-0026) — the `show` / `update` / `delete` grammars are *referenced*, not duplicated, so the explained command is parsed, completed, hinted, and highlighted exactly as it is on its own. **AST.** `Command::Explain { query: Box }`. The inner `Command` is the ordinary parsed command; the runtime recognizes the `Explain` wrapper and routes it to the plan path instead of normal execution. ### 2. Capturing the plan The database worker prepares `EXPLAIN QUERY PLAN `, where `` is exactly the SQL the inner command would otherwise have run, and reads back the result rows. Each row is `(id, parent, notused, detail)` — the same read-only, multi-row shape as the existing `PRAGMA`-backed reads (`read_table_indexes`). - The inner command's SQL is produced by the *same* construction logic that builds it for real execution, so the plan is the plan of the actual query, not an approximation. This means the SQL-building step is separated from execution for the explainable commands. - `EXPLAIN QUERY PLAN` determines the plan from the statement's structure and the schema, not from parameter *values*. The statement is still prepared with the inner command's parameters bound (so a parameterised `WHERE` prepares cleanly); the bound values do not affect the plan. - A new `Request::ExplainPlan` / `do_explain_plan`, and a `QueryPlan` result type carrying the tree, flow back through `CommandOutcome` / `AppEvent` like other command results. ### 3. The plan tree The `id` / `parent` columns form a tree. It is rendered indented, with box-drawing connectors (`├─`, `└─`, `│`), the way a file tree is drawn. Each node's text is the engine's `detail` string **verbatim** — `SCAN Customers`, `SEARCH Customers USING INDEX Customers_email_idx (Email=?)`, and so on. Nothing is reworded. Verbatim text is a deliberate pedagogical choice: it teaches the real vocabulary a learner meets in every database tool. The `detail` strings name no engine *product*, so the ADR-0002 "no engine name in user-facing strings" rule is satisfied as-is. The block is preceded by the usual command echo, and the SQL being explained is shown above the tree — seeing the generated SQL beside its plan is itself part of the simple → advanced bridge. The displayed SQL is **standard SQL**, rendered to read as a complete, copy-pasteable query: identifiers are double-quoted (the ISO delimited-identifier form); `WHERE` literals are shown *inline* (`WHERE "Email" = 'alice@example.com'`) rather than as the `?` placeholders the statement is actually prepared with (§2 — execution and plan capture keep the parameters); and inequality is written `<>` even when the user typed `!=`. The one clause the display carries that the user did not type is the implicit `ORDER BY ` that `limit` adds (ADR-0026 §5) — itself a worthwhile lesson. ### 4. Annotation taxonomy Each node's `detail` string is classified, by matching it against a small table of substring patterns, into one of: | Category | Recognised by | Reading | |---|---|---| | Full scan | `SCAN `, no index | every row read — expensive | | Index search | `SEARCH … USING INDEX …` | indexed lookup — efficient | | Covering index | `USING COVERING INDEX` | indexed, no table fetch | | Primary-key lookup | `USING INTEGER PRIMARY KEY` | direct row lookup | | Automatic index | `USING AUTOMATIC … INDEX` | the engine built a *temporary* index because none existed — the strongest "add an index here" signal | | Temp B-tree | `USE TEMP B-TREE FOR …` | sorting / grouping with no index to lean on | | Neutral | anything else / structural | — | A `detail` that matches no pattern renders neutral rather than failing — the engine's plan vocabulary may grow. The **automatic index** category is the most important teaching moment: it is the case where the learner *should* have added an index and the engine quietly compensated. It is called out distinctly, not folded in with plain scans. ### 5. Styled output lines (the rendering mechanism) Span-level colour needs the output panel to colour *parts* of a line. Today an `OutputLine` is `{ text: String, kind: OutputKind }` and `render_output_line` colours the whole line by `kind` — except simple-mode echo lines, which already render multi-span from re-lexed token runs (`input_render::lex_to_runs` → runs of `{ byte_range, style }`). This ADR adds, to `OutputLine`, an optional **styled-runs** payload — a list of `{ byte_range, style-class }` over the line's text, the same shape `lex_to_runs` already produces. `render_output_line` gains one branch: when the payload is present, render the text as spans per the runs; otherwise fall back to the existing whole-line `kind` styling. The echo path is unchanged. The runs carry a *semantic style class*, resolved to a concrete colour at render time from the active theme — not a baked-in colour — so the styling stays correct regardless of theme. This is a **general** capability: any output line may now carry rich styling. The plan renderer is its first and, for now, only consumer; the existing renderers (`render_structure`, `render_data_table`) keep producing plain lines. The mechanism is the per-span styling ADR-0016 anticipated, and V4's session-log work will reuse it — the same "general mechanism, single current consumer" shape as ADR-0027's diagnostics model. Scroll math is unaffected: one display row per plan node; styled spans do not change the line count. ### 6. Colour scheme Beyond neutral text the plan needs an "efficient" colour and an "expensive" colour: - **Efficient** — index search, covering index, primary-key lookup — green. - **Expensive** — full scan, temp B-tree — amber, reusing the `warning` colour introduced by ADR-0027. - **Automatic index** — amber as well, but with a distinct marker (an icon or short tag) so it reads as "you should add an index", not merely "this is slow". - Connector glyphs and table / index *names* stay neutral; only the category-bearing keywords of the `detail` string carry the category colour. `theme.system`'s green is the existing "normal output" colour; a plan-specific efficient colour distinct from it avoids "green means two things". The exact theme fields are an implementation detail; the requirement is that the scheme is legible on both light and dark backgrounds (NFR-5, NFR-7). ### 7. Out of scope - **Explaining raw advanced-mode SQL.** There is no SQL parser yet (`Q1`); `explain` covers the simple-mode DSL queries. When SQL parsing lands, `explain` extends to it. - **`EXPLAIN`** (the bytecode form) — only `EXPLAIN QUERY PLAN`. The bytecode dump is not a teaching surface. - **Cost estimates / row-count predictions** — `EXPLAIN QUERY PLAN` does not provide them and this ADR does not invent them. - **Re-styling existing output.** The styled-line mechanism (§5) is available to all output, but this ADR only wires the plan renderer to it; `render_structure` / `render_data_table` are untouched. - **A plan history, or multiple plan tabs.** ## Consequences - A new `explain` command — `Command::Explain`, `Request::ExplainPlan` + `do_explain_plan`, a `QueryPlan` result type, and `CommandOutcome` / `AppEvent` variants. - `explain` covers `show data` / `update` / `delete`; because `EXPLAIN QUERY PLAN` never executes, explaining a destructive command is safe. - The SQL-construction step for the explainable commands is separated from execution, so the same SQL feeds both real execution and `EXPLAIN QUERY PLAN`. - `OutputLine` gains an optional styled-runs payload, and `render_output_line` a branch to honour it — a general per-span output-styling capability (ADR-0016's OOS-3 realized), with the plan renderer as its first consumer. - A new `render_explain_plan` in `output_render.rs` producing the styled tree; a small substring-pattern table for the annotation taxonomy. - Theme gains plan colours (an efficient colour distinct from `system`; `warning` reused for expensive). - Depends on ADR-0026: a plan that flips between a scan and an index search needs `show data … where`. The feature works against whatever queries exist; it is fully realised once C5a is implemented. - Builds toward `Q1`: when advanced-mode SQL lands, `explain` extends to cover it. ## Implementation notes A sensible order, each step test-guarded: 1. The styled-output-line mechanism — the `OutputLine` styled-runs payload and the `render_output_line` branch. No user-visible change on its own. 2. The `explain` grammar (the prefix plus the `Subgrammar`-referenced query shapes) and `Command::Explain` with its AST builder. 3. Separating SQL construction from execution for `show data` / `update` / `delete`; `Request::ExplainPlan` / `do_explain_plan`; the `QueryPlan` result and its `CommandOutcome` / `AppEvent` wiring. 4. `render_explain_plan` — the tree layout, the annotation taxonomy, and the styled runs; the theme colours. 5. Typing-surface matrix cells for `explain`. ## See also - ADR-0002 — database engine; the "no engine name in user-facing strings" rule (plan `detail` strings name no product, so verbatim text complies). - ADR-0016 — pretty table rendering; its OOS-3 anticipated the per-span output styling §5 realizes. - ADR-0024 — the unified grammar tree the `explain` command plugs into. - ADR-0025 — indexes; what makes a query plan pedagogically interesting. - ADR-0026 — complex WHERE expressions; the filtered query worth explaining, and the `Subgrammar` node `explain` reuses to reference the query grammars. - ADR-0027 — the diagnostics model; the same "general mechanism, single current consumer" shape as §5.