rdbms-playground/docs/adr/0025-indexes.md

ADR-0025: Indexes

Status

Accepted

Context

The requirements checklist (C3) commits to indexes as part of the schema-constraint surface, and S2 commits to the items list showing "tables and per-table indexes". Neither is implemented yet.

Indexes are the natural next teaching topic after relationships: they are the structure that makes EXPLAIN QUERY PLAN (QA1) pedagogically interesting — the plan for a filtered query visibly changes from a full scan to an index search once an index exists. QA1 itself is a deliberate follow-up (it needs its own rendering ADR and a query worth explaining); this ADR sets it up by giving the playground real indexes.

Three design problems shape the decision:

1. SQLite owns the index namespace. Unlike foreign keys — which have no name slot, the problem ADR-0013 solved with an internal metadata table — an index in SQLite is a named object. sqlite_master and PRAGMA index_list / index_info carry the name, table, column list, and uniqueness natively. There is nothing app-specific to store.
2. DROP TABLE silently drops a table's indexes. The rebuild-table primitive (ADR-0013) — used by change-column- type and every relationship operation — drops and recreates the table. Once indexes exist, every such operation would erase them unless the primitive is taught to preserve them.
3. playground.db is a derived artifact (ADR-0004 / ADR-0015). Indexes must round-trip through project.yaml or they vanish on rebuild, export, and import.

Decision

Grammar

Indexes are declared and removed via DSL commands following ADR-0009 (required clauses keyword-based; optional names introduced by as per the ADR-0013 convention; -- flags for opt-ins):

add index [as <name>] on <Table> (<col>[, <col>...])

drop index <name>
drop index on <Table> (<col>[, <col>...])

• add index is a third branch of the existing add command, alongside add column and add 1:n relationship; drop index is a new branch of the existing drop command.
• as <name> is optional. The as keyword introduces the name, matching add 1:n relationship [as <name>] (ADR-0013 established as as the convention for optional names).
• on <Table> uses the keyword on — the SQL-natural word for CREATE INDEX ... ON table, and pedagogically aligned.
• The column list is parenthesised and comma-separated, the same shape as create table and insert. One or more columns; multiple columns produce a composite index in the given order. An empty list () is a parse error.
• Column-list completion resolves against the named table, reusing the dynamic-subgrammar mechanism that already drives insert into T (...) column candidates.

add unique index is not part of this ADR — see Out of scope.

Auto-name format

When as <name> is omitted, the executor generates <Table>_<col1>[_<col2>...]_idx, mirroring the descriptive, subject-first style of ADR-0013's relationship auto-names.

Examples:

• add index on Customers (email) → Customers_email_idx
• add index on Orders (CustId, Date) → Orders_CustId_Date_idx

If the generated name is already taken — which happens exactly when the same columns of the same table are already indexed — the command is refused with a friendly error naming the existing index (a second index on an identical column set is redundant). A duplicate explicit name is likewise a friendly error.

Drop forms

drop index accepts two forms, mirroring drop relationship:

• drop index <name> — for users who named the index or know the generated name.
• drop index on <Table> (<col>...) — the positional form, resolved by matching the table and exact column set against the table's indexes. No match is a friendly error; more than one match is an ambiguity error listing the candidates and advising the user to drop by name.

Storage — no metadata table

Indexes do not get a __rdbms_playground_indexes table. SQLite stores everything the application needs natively:

• PRAGMA index_list(<table>) — index name, uniqueness, and origin (c = CREATE INDEX, u = UNIQUE constraint, pk = primary key).
• PRAGMA index_info(<index>) — the ordered column list.

The application reads indexes through these pragmas. Only origin = 'c' indexes are treated as user indexes; the automatic indexes SQLite creates to back primary keys and UNIQUE constraints are not surfaced as user indexes.

This is a deliberate divergence from the ADR-0013 relationship precedent. Relationships needed a metadata table because SQL foreign keys have no name slot; indexes have one, so the divergence is justified — adding a metadata table would duplicate state SQLite already owns and create a consistency hazard.

The in-memory representation is a small structural value (name, table, ordered columns) carried by db.rs, persistence, and the renderer.

project.yaml persistence

A top-level indexes: list is added to project.yaml, mirroring relationships:. Each entry records the index name, its table, and its ordered column list:

indexes:
  - name: Customers_email_idx
    table: Customers
    columns: [email]

• version: stays 1. The field is additive and optional: the serde_yml reader marks it #[serde(default)], so project files written before this change parse unchanged. No migrator is required (the ADR-0015 §F3 framework stays empty).
• The hand-rolled writer emits indexes: [] when there are none, consistent with how tables/relationships render.
• SchemaSnapshot gains an indexes vector alongside tables and relationships.
• rebuild_from_text recreates each index (via CREATE INDEX) after the tables are built. export / import carry indexes because they operate on the text artifacts.

Rebuild-table interaction

The rebuild_table primitive (ADR-0013) is extended so it no longer loses indexes:

1. Before the DROP TABLE, capture the table's user indexes structurally (name + ordered columns) via PRAGMA index_list / index_info, filtered to origin = 'c'.
2. After the ALTER TABLE ... RENAME, recreate them with CREATE INDEX.

Recreation is parameterised by an optional column-rename map and a set of dropped columns, so the same primitive serves every caller:

• add / drop relationship, change column type — the column set is unchanged; indexes are recreated verbatim.
• rename column — an index referencing the old column name is regenerated with the new name; the index keeps its own name. No error.
• drop column — see below.

Because indexes are captured structurally (not as raw SQL text), regeneration after a rename is a clean substitution rather than SQL string-munging.

Drop / rename column interaction

• rename column and change column type preserve any covering index transparently, per the rebuild rules above.
• drop column is refused by default when an index covers the dropped column. The error names the offending index(es) and advises dropping them first. This matches the existing conservative posture of drop column, which already refuses primary-key and FK-involved columns.
• A new --cascade flag on drop column opts in to the cascading behaviour: covering indexes are dropped automatically and each is reported in the result note.

drop column <col> from table <Table> [--cascade]

Command::DropColumn gains a cascade: bool. --cascade is the first destructive cascade flag in the DSL; future cascading drops should follow the same opt-in -- pattern. Indexes that do not cover the dropped column are recreated normally regardless of the flag.

Display — structure view

render_structure (the table-structure view in the output panel) gains an Indexes: section, rendered after the relationship sections and only when the table has at least one user index:

Customers
  Id    [serial PK]
  Email [text]
  Indexes:
    Customers_email_idx (Email)
    cust_lookup (Email, Name)

add index and drop index return the affected table's description, so the auto-show pattern (ADR-0014) displays the updated structure — including this section — after the command, the same as add column and add relationship.

Display — items list (S2)

The items list (left panel) becomes a nested list: each table, with its indexes indented beneath it.

Tables
  Customers
    Customers_email_idx
    cust_lookup
  Orders
    Orders_date_idx

This satisfies S2 ("the items list shows tables and per-table indexes; designed to extend to additional element kinds … without restructuring") — the nested model is that extensible structure; future kinds (relationships, views) slot in as further child rows.

The panel's data model changes from a flat Vec<String> of table names to a structured list (table name plus its index names), populated by a schema refresh that now also reads indexes. Index rows are display-only: the current-table highlight behaviour is unchanged, and selecting an index row carries no new action in this ADR.

Errors and edge cases

All user-facing strings obey the ADR-0002 rule — "the database" / "the engine", never the engine product name.

• add index on a non-existent table → friendly error.
• add index naming a column the table does not have → friendly error naming the column.
• Duplicate explicit index name, or an auto-name collision (same table + column set) → friendly error naming the existing index.
• drop index <name> for an unknown name → friendly error.
• drop index on T(cols) with no match → friendly error; with multiple matches → ambiguity error listing candidates.
• Internal __rdbms_* tables are not user tables, so the table identifier never resolves to one.
• add index / drop index are DSL DDL commands, available in simple mode, appended to history.log, and replayable — consistent with add column / add relationship.

Out of scope

Explicitly excluded from this ADR:

• UNIQUE indexes (add unique index). A unique index is also a constraint; UNIQUE is tracked as its own C3 sub-item and is a distinct teaching concern.
• Partial indexes (CREATE INDEX ... WHERE), expression / computed indexes, and per-column DESC / collation modifiers — advanced features beyond the playground's pedagogical aim. Plain column-list indexes only.
• EXPLAIN QUERY PLAN / QA1 — the deliberate follow-up. It needs its own rendering ADR (QA2) and builds on the indexes this ADR delivers.

Consequences

• The playground gains real, persistent indexes, advancing the index portion of C3 and satisfying S2.
• The rebuild-table primitive now preserves indexes. This also closes a latent bug: once indexes exist, column rename / type-change would otherwise silently drop them — there are no indexes today, so the bug is latent rather than live, but the fix ships with the feature that would trigger it.
• A new structural index representation threads through db.rs, persistence, and output_render.
• No new internal table — a deliberate divergence from the ADR-0013 relationship precedent, justified by SQLite owning the index namespace natively.
• The items panel is no longer a flat list; the nested model is the S2-mandated extension point for future element kinds.
• drop column --cascade establishes the opt-in -- flag pattern for destructive cascades.
• EXPLAIN QUERY PLAN (QA1) becomes worthwhile: once it lands, show data <T> where <col> = <val> is a query whose plan visibly changes when an index on <col> exists.

Implementation notes

Two details settled differently from the sketch above, recorded here so the decision text and the code agree:

• rename column / drop column do not use the rebuild primitive. Both run native ALTER TABLE (the playground targets SQLite 3.25+/3.35+). ALTER TABLE … RENAME COLUMN already rewrites index definitions that reference the renamed column, so rename needs no index code at all. drop column detects covering indexes directly and either refuses or, with --cascade, issues DROP INDEX before the column drop. Only change column (and the relationship operations) go through the rebuild primitive, and there the column set is unchanged, so the captured indexes are recreated verbatim — no column-rename map or dropped-column set is needed.

• The items list keeps a flat app.tables plus a cache map. Rather than restructuring app.tables and the TablesRefreshed event payload, per-table index names ride in SchemaCache::table_indexes, populated by the existing schema-cache refresh. The panel renders the ordered table list with each table's indexes indented beneath — the S2 nested view — reading the two together.

See also

• ADR-0004 / ADR-0015 (project file format and storage runtime)
• ADR-0009 (DSL command syntax conventions)
• ADR-0012 (internal column metadata — and why indexes diverge from that precedent)
• ADR-0013 (relationships, the rebuild-table primitive, and the as <name> convention)
• ADR-0014 (auto-show after writes)
• ADR-0023 / ADR-0024 (the unified grammar tree the new commands plug into)