rdbms-playground/docs/adr/0013-relationships-and-rebuild-table.md

ADR-0013: Relationships, naming, and the rebuild-table strategy

Status

Accepted

Context

This iteration introduces foreign-key relationships between tables (C3 partial), which exposes two coupled design problems:

1. SQLite's ALTER TABLE cannot add or drop a FOREIGN KEY constraint on an existing table. The accepted SQLite recipe for this is the rebuild-table dance (create a new table with the desired shape, copy the data, drop the old, rename). This same machinery is what B2 (column drops/renames/type changes) needs in future iterations, so the cost is shared.
2. SQL foreign-key constraints have no user-name slot. A relationship is identified internally by its (child, column, parent, column) quadruple. Pedagogically, the user's mental model is more natural with named relationships ("the cust_orders relationship") that survive renaming and feature in drop / future modify commands. A name field has to come from somewhere we own.

Additionally, the SQLite schema model treats a foreign key as a property of the child table only. A learner viewing the parent table normally has no clue that other tables refer to it. This asymmetry runs counter to the relational mental model.

Decision

Grammar

Relationships are declared and removed via DSL commands that follow ADR-0009 (required clauses keyword-based; -- flags for opt-ins):

add 1:n relationship [as <name>] from <Parent>.<col> to <Child>.<col>
                                    [on delete <action>]
                                    [on update <action>]
                                    [--create-fk]

drop relationship <name>
drop relationship from <Parent>.<col> to <Child>.<col>

• as <name> is optional. The keyword as is required to introduce the name so the parser is unambiguous in the presence of from (an unkeyed identifier could otherwise be confused with the parent table identifier).
• on delete / on update clauses are independently optional and can appear in either order; default action is no action.
• --create-fk auto-creates the child column with the appropriate type (Type::fk_target_type() of the parent column, per ADR-0011) when it does not yet exist. Without the flag, missing child column is a friendly error advising the user to add the column first or use the flag.
• Drop accepts both forms — the named form for users who picked a name; the positional form for users who let it auto-generate.

Auto-name format

When as <name> is omitted, the executor generates <Parent>_<parent_col>_to_<Child>_<child_col>, matching the direction of the user's from <Parent>.<col> to <Child>.<col> syntax. Example: a relationship from Customers.id to Orders.CustId is named Customers_id_to_Orders_CustId. Long but unambiguous, and reads in the same direction as the declaration.

Internal metadata

A new internal table tracks relationship metadata:

CREATE TABLE __rdbms_playground_relationships (
    name           TEXT NOT NULL UNIQUE,
    parent_table   TEXT NOT NULL,
    parent_column  TEXT NOT NULL,
    child_table    TEXT NOT NULL,
    child_column   TEXT NOT NULL,
    on_delete      TEXT NOT NULL,
    on_update      TEXT NOT NULL,
    PRIMARY KEY (child_table, child_column)
) STRICT;

The PK on (child_table, child_column) enforces our convention "at most one FK per child column" — true in typical SQL practice even though SQLite would technically allow more. The unique name ensures no two relationships collide in identification.

Created at connection open alongside __rdbms_playground_columns (ADR-0012); follows the __rdbms_* internal-table naming convention. list_tables filters this out of user-visible listings.

Symmetric description

describe_table populates two collections on TableDescription:

• outbound_relationships — relationships where this table is the child (it has the FK column).
• inbound_relationships — relationships where this table is the parent (other tables reference one of its PK columns).

Both are populated by querying the relationships metadata table (child-side for outbound; parent-side for inbound), which keeps the symmetric view consistent with the underlying constraints that SQLite enforces.

The structure view in the output panel renders both sections when present:

Customers
  Id [serial PK]
  Email [text]
  Referenced by:
    Orders.CustId → Id  (Customers_id_to_Orders_CustId, on delete cascade, on update no action)

Orders
  Id [serial PK]
  CustId [int]
  References:
    CustId → Customers.Id  (Customers_id_to_Orders_CustId, on delete cascade, on update no action)

Type compatibility check

Per ADR-0011, the FK column type must match the parent column's fk_target_type(). The executor checks this:

• For --create-fk: the column is created with that type, no user-side decision required.
• For an existing column: type is compared. Mismatch yields a friendly error naming both types and the expected one. The user learns through the error rather than via a silent promotion.

The non-PK-target case (FK referencing a non-PK column) is rejected with a clear error. UNIQUE-target FKs land when UNIQUE constraints (C3 partial) do.

Rebuild-table dance

The rebuild_table primitive performs the SQLite-recommended ALTER-via-rebuild sequence:

PRAGMA foreign_keys = OFF;        -- must be outside any tx
BEGIN;
DROP TABLE IF EXISTS __rdbms_rebuild_<name>;
CREATE TABLE __rdbms_rebuild_<name> (...) STRICT;
INSERT INTO __rdbms_rebuild_<name> (cols-on-both)
   SELECT (cols-on-both) FROM <name>;
DROP TABLE <name>;
ALTER TABLE __rdbms_rebuild_<name> RENAME TO <name>;
<metadata updates>
PRAGMA foreign_key_check;          -- abort on any returned rows
COMMIT;
PRAGMA foreign_keys = ON;

Key properties:

• foreign_keys is toggled at the connection level outside the transaction, because the pragma is a no-op inside one.
• The IF EXISTS clause defends against a leftover rebuild table from a previous failed attempt.
• Data is copied column-by-name — only columns present on both old and new schemas are transferred. Auto-created columns (e.g. via --create-fk) start as NULL in existing rows.
• foreign_key_check runs before commit. Any returned rows mean existing data violates the new constraint, which we surface as a clear error and roll back.
• The metadata-updates closure runs inside the same transaction so the schema and the metadata stay consistent.

This primitive is reused by add_relationship and drop_relationship. Future B2 work (drop column, rename column, change column type) plugs into the same primitive without re-implementing the dance.

Drop-table interaction

A drop table request is rejected when the table has inbound relationships — dropping it would leave dangling FK constraints in the children. The error names the offending relationships; the user drops those first, then drops the table.

Outbound relationships are removed naturally with the table; the metadata rows are cleaned up alongside the drop in the same transaction.

Modify deferred

modify relationship <name> [on delete <action>] [on update <action>] would be a one-step way to change actions without typing both the drop and the add. The mechanics reuse the rebuild dance. Deferred to a follow-up iteration; users can achieve the same via drop + add today.

Consequences

• Pedagogically honest: the user creates tables, adds columns, declares relationships in the natural order. Type mismatches and missing columns are caught at declaration time with actionable errors.
• The rebuild-table primitive is the load-bearing piece for B2 too; future column drops, renames, and type changes layer on top.
• The relationship metadata table is a new authority alongside the column metadata table (ADR-0012). Both are owned by the app and round-trip through the project file format (track 2) when that lands.
• Without I/U/D (C5) the FK constraints are observable but not yet demonstrable (you can't try inserting a violating row). C5 is the obvious follow-up.
• The as <name> keyword is established. Future commands that accept optional names should follow the same convention to stay parser-unambiguous.
• drop table becomes more conservative — it now refuses when inbound relationships exist. This is a learnability win (failure has a clear cause) and matches relational expectations.

See also

• ADR-0009 (DSL command syntax conventions)
• ADR-0010 (DB worker thread)
• ADR-0011 (FK column type compatibility)
• ADR-0012 (Internal column metadata)