rdbms-playground/docs/plans/20260525-adr-0035-sql-ddl-4b.md

# Plan: ADR-0035 Phase 4, sub-phase 4b — foreign keys in `CREATE TABLE`

Add foreign keys to advanced-mode SQL `CREATE TABLE`: inline
`<col> <type> REFERENCES <parent>[(<pcol>)] [ON DELETE …] [ON UPDATE …]`
and table-level `[CONSTRAINT <name>] FOREIGN KEY (<ccol>) REFERENCES
<parent>[(<pcol>)] [ON DELETE …] [ON UPDATE …]`. Each FK is the SQL
spelling of an ADR-0013 **named relationship**: one `CREATE TABLE`
creates the table *and* its relationship metadata in one transaction =
**one undo step** (ADR-0035 §5). Builds on 4a/4a.2/4a.3.

## 1. Baseline

- Tests: **1769 passing, 0 failing, 1 ignored**; clippy clean. Branch
  `main`, last commit `60111f6` (4a.3). 4b starts here.

## 2. Decisions locked with the user (do not re-litigate)

1. **Self-referencing FK supported** (2026-05-25): `CREATE TABLE emp
   (id int primary key, mgr int REFERENCES emp(id))`. When the FK's
   parent table equals the table being created, the referenced column is
   validated against the columns/PK **being defined** (not `read_schema`,
   which would fail — the table doesn't exist yet). Metadata row carries
   `parent_table = child_table`.
2. **Bare `REFERENCES <parent>` supported** (2026-05-25, standard SQL +
   the §7 "trust like SQL" posture): resolves to the parent's
   **single-column PK**; a parent with a composite PK is a clear error
   ("name the referenced column"). The explicit `REFERENCES parent(col)`
   form is the primary one (ADR §4).
3. **Inline `REFERENCES` is always auto-named; only the table-level
   `FOREIGN KEY` form takes `CONSTRAINT <name>`** (ADR §4/§5). Auto-name:
   `<Parent>_<pcol>_to_<Child>_<ccol>` (ADR-0013).
4. **PK-target only** (ADR-0013/0011): the referenced column must be a
   primary key; UNIQUE-target FKs stay deferred (as for `add
   relationship`). Type compatibility via `Type::fk_target_type`
   (ADR-0011).
5. **One statement = one command = one transaction = one undo step**
   (ADR-0035 §5/§10), including a multi-FK `CREATE TABLE`.

## 3. Phase 1 — Requirements checklist (4b)

### Functional

- [ ] Inline column FK `REFERENCES <parent>[(<pcol>)] [ON DELETE <a>]
  [ON UPDATE <a>]` parses (advanced mode); child column = the column it
  is attached to.
- [ ] Table-level `[CONSTRAINT <name>] FOREIGN KEY (<ccol>) REFERENCES
  <parent>[(<pcol>)] [ON DELETE <a>] [ON UPDATE <a>]` parses; `<ccol>`
  is the child column.
- [ ] `ON DELETE` / `ON UPDATE` each optional, either order, default
  `no action`; actions `cascade` / `restrict` / `set null` / `no action`
  (reuse `shared::REFERENTIAL_CLAUSES`).
- [ ] Multiple FKs in one statement → multiple relationships, all in one
  transaction / one undo step.
- [ ] Self-referencing FK (parent = table being created) validated
  against the columns/PK being defined.
- [ ] Bare `REFERENCES <parent>` resolves to the parent's single-column
  PK; composite-PK parent → friendly "name the column" error.
- [ ] Validation (reused with `do_add_relationship`): parent column is a
  PK; child column type == `parent_type.fk_target_type()`; relationship
  name unique. Engine-neutral errors.
- [ ] Structural execution: `do_create_table` emits the `FOREIGN KEY`
  clause(s) **identically to `schema_to_ddl`** (the §6.1 rule) and writes
  `__rdbms_playground_relationships` rows in its transaction.
- [ ] FK still-rejected test from 4a.3 (`foreign_key_still_rejected`)
  flipped to accepted.

### Cross-cutting / round-trip

- [ ] A table created with FK(s) round-trips: relationships appear in
  `project.yaml`, survive `rebuild_from_text`, and the FK is enforced
  (a violating insert fails) before and after rebuild.
- [ ] `describe` shows the relationship (outbound on the child, inbound
  on the parent) — FKs are first-class (unlike table CHECK, FK metadata
  *is* surfaced by `do_describe_table`).
- [ ] One undo step removes the table **and** its relationship rows;
  redo recreates both.
- [ ] `history.log` / replay: the literal SQL line replays as a write.

### Testing (ADR-0008 four tiers)

- [ ] **Tier 1** (builder): inline + table-level FK captured (child/
  parent/columns/actions); `CONSTRAINT <name>`; bare `REFERENCES`
  (parent_column = None); multiple FKs; ON DELETE/UPDATE in both orders;
  self-ref shape; FK alongside CHECK/PK/UNIQUE (depth/element coexistence).
- [ ] **Tier 3** (`tests/sql_create_table.rs`): FK enforced (violating
  insert fails); cascade behaviour; auto-name + explicit name in
  `REL_TABLE`; type-mismatch + non-PK-target + composite-bare-ref errors;
  self-ref enforced; **survives rebuild**; `describe` shows the relation.
- [ ] **Undo** Tier-3: one step; undo drops table + relationship rows.

## 4. Architecture & design

### 4.1 Grammar (`src/dsl/grammar/sql_create_table.rs`)

- **Action clauses**: reuse `shared::REFERENTIAL_CLAUSES` (the
  `on <delete|update> <action>` repeat, 0..2) — same keywords as SQL.
- **Inline FK** — new `COL_CONSTRAINT_CHOICES` branch
  `REFERENCES <parent_ident> [ '(' <pcol_ident> ')' ] REFERENTIAL_CLAUSES`.
  Idents: `fk_parent_table`, `fk_parent_column` roles
  (`IdentSource::Schema` for the parent so it completes from the cache;
  parent-column also schema-ish — confirm the right source; the table
  being created is `NewName`).
- **Table-level FK** — new `ELEMENT_CHOICES` branch
  `[CONSTRAINT <name_ident>] FOREIGN KEY '(' <ccol_ident> ')' REFERENCES
  <parent_ident> [ '(' <pcol_ident> ')' ] REFERENTIAL_CLAUSES`. Idents:
  `fk_name` (NewName), `fk_child_column`, `fk_parent_table`,
  `fk_parent_column`.
- The bare `REFERENCES parent` form: make the `( pcol )` group
  `Optional`.

### 4.2 Builder (`build_sql_create_table`, `ddl.rs`)

Greedy-consume each FK clause when its keyword is hit (parens consumed
**inside** the handler, so they don't perturb the §4a.3 `depth` tracker):

- On `Word("references")` (inline; `column_open == true`): child column
  = `columns.last()`; read the following `fk_parent_table` ident,
  optional `( fk_parent_column )`, then drain referential clauses
  (`while peek == on { … }`, mapping `delete`/`update` → actions via the
  existing `parse_action` logic). Push an `FkSpec { name: None, … }`.
- On `Word("constraint")`: stash `pending_fk_name = next fk_name ident`.
- On `Word("foreign")` (table-level; `column_open == false`): consume
  `key ( fk_child_column ) references fk_parent_table [ ( fk_parent_column ) ]`
  + referential clauses; push `FkSpec { name: pending_fk_name.take(), … }`.

A small `FkSpec { name: Option<String>, child_column: String,
parent_table: String, parent_column: Option<String>, on_delete,
on_update }` accumulates into `Command::SqlCreateTable.foreign_keys`.

### 4.3 Command AST (`src/dsl/command.rs`)

`SqlCreateTable` gains `foreign_keys: Vec<SqlForeignKey>` where
`SqlForeignKey { name: Option<String>, child_column: String,
parent_table: String, parent_column: Option<String>, on_delete:
ReferentialAction, on_update: ReferentialAction }`. (`parent_column`
optional = the bare-ref form, resolved at execution.)

### 4.4 Worker — `do_create_table` (the only executor)

Threaded a `foreign_keys: &[SqlForeignKey]` param (and through
`Request`/method/dispatch). Inside, **before** building the DDL:

For each FK, resolve + validate (reusing helpers, §4.5):
1. **Resolve `parent_column`** (bare form): self-ref → the statement's
   own single-column `primary_key` (composite → error); else →
   `read_schema(parent).primary_key` single column (composite → error).
2. **Parent is a PK**: self-ref → `parent_column ∈ primary_key` being
   defined; else → `read_schema(parent)` column with `primary_key`.
   Obtain `parent_user_type`.
3. **Child type compat**: child column must be among the `columns` being
   defined; its type must equal `parent_user_type.fk_target_type()`
   (shared helper; friendly mismatch error).
4. **Name**: `resolve_relationship_name(name, parent, pcol, child, ccol)`
   then uniqueness check (shared helpers).

Then emit `FOREIGN KEY (<ccol>) REFERENCES <parent>(<resolved_pcol>) ON
DELETE <od> ON UPDATE <ou>` **after the CHECK clauses**, exactly as
`schema_to_ddl` does (always explicit pcol + both actions, so create-DDL
== rebuild-DDL — no drift). Write each `REL_TABLE` row in the existing
transaction (before `finalize_persistence`, so the snapshot/yaml sees it).

`do_describe_table(name)` already surfaces the new relationships.

### 4.5 Shared helpers (refactor `do_add_relationship`, don't duplicate)

Factor out of `do_add_relationship`, used by both it and
`do_create_table`:

- `fk_type_mismatch_error(...)` / a `check_fk_type_compat(parent_type,
  child_type, …) -> Result<()>`.
- `resolve_relationship_name(name, parent, pcol, child, ccol) -> String`.
- `ensure_relationship_name_unique(conn, name) -> Result<()>`.
- `insert_relationship_metadata(tx, name, parent, pcol, child, ccol,
  on_delete, on_update) -> Result<()>`.

`do_add_relationship` keeps its `read_schema`-based validation + rebuild;
`do_create_table` validates against the in-statement specs. Only the
naming/uniqueness/metadata-write + type-compat message are shared.

### 4.6 No new persistence/round-trip structures

FKs already round-trip: `read_schema` reads them via
`pragma_foreign_key_list`; `RelationshipSchema` → `project.yaml`;
`build_read_schema` re-emits them inline on rebuild via `schema_to_ddl`.
4b adds **no** new `TableSchema`/YAML field — it reuses the existing
relationship plumbing entirely. (The litmus test: an FK is a structure
the model already persists.)

### 4.7 Friendly catalog / keys

New diagnostics may be needed (non-PK target, type mismatch,
composite-bare-ref, unknown parent table) — but most reuse
`do_add_relationship`'s existing engine-neutral errors. Any new
`help_id`/diagnostic key gets a `keys.rs` entry **and** an `en-US.yaml`
body (lockstep test) with engine-neutral wording. Help/usage skeleton
refresh stays batched into **4i** (consistent with 4a.2/4a.3).

## 5. Out of 4b scope

- `DROP TABLE` (4c), indexes (4d), `ALTER TABLE` (4e–4h).
- UNIQUE-target FKs (deferred with `add relationship`).
- `MATCH` / `DEFERRABLE` and other engine-specific FK extras (ADR §12 —
  parse error).

## 6. Open items / implementer calls

1. **`IdentSource` for the parent table/column slots** — RESOLVED:
   `IdentSource::Tables`/`Columns` (matching the `add relationship`
   endpoints; completion + existence hint). **Known wrinkle deferred to
   4i (user-confirmed 2026-05-25):** a self-referencing FK
   (`references <self>`) parses + executes correctly, but the pre-submit
   schema-existence indicator falsely flags the not-yet-created self
   table as unknown. Tracked in **three places** so it is not lost:
   ADR-0035 §13 4i bullet, a `NOTE (4i)` code comment at `FK_PARENT_TABLE`
   (`src/dsl/grammar/sql_create_table.rs`), and here. The 4i fix teaches
   the diagnostic about the `CREATE TABLE` target.
2. **Two-DDL-generators drift** — assert by a create→rebuild test that
   the FK survives and is enforced after rebuild (the §6.1 net).
3. **Self-ref column-order / forward ref** — `REFERENCES emp(id)` where
   `id` is defined in the same statement; SQLite accepts it. Confirm by
   test, incl. the bare self-ref `REFERENCES emp`.

## 7. Devil's Advocate review of this plan

- **Reuse vs fork?** `do_create_table` stays the single create executor;
  FK validation/naming/metadata-write are shared helpers with
  `do_add_relationship` — no second relationship path. ✓
- **DDL drift?** `do_create_table` emits the FK clause byte-identical to
  `schema_to_ddl` (explicit resolved pcol + both actions), guarded by a
  survives-rebuild test — the 4a `serial` lesson applied. ✓
- **Bare-ref ambiguity?** Resolved to the single-column PK with an
  explicit composite-PK error — not silently guessed. ✓
- **Self-ref soundness?** Validated against the in-statement specs;
  SQLite accepts the self-FK DDL; proven by an enforced + rebuild test. ✓
- **One undo step?** All FK rows + the table land in the one
  `snapshot_then`-wrapped create transaction. A dedicated undo test. ✓
- **Silent scope drop?** UNIQUE-target + FK extras explicitly out (parse
  error / deferred), consistent with `add relationship`. ✓

## 8. Implementation sequence (test-first)

1. **Command + grammar (inline + table FK) + builder** — Tier-1 builder
   tests (inline, table-level, CONSTRAINT name, bare ref, multi-FK,
   action orders, self-ref shape, FK+CHECK coexistence) → red → add
   `SqlForeignKey`, the grammar branches (reusing
   `shared::REFERENTIAL_CLAUSES`), the builder handlers, thread the field
   through `Request`/method/dispatch/runtime → green. Flip
   `foreign_key_still_rejected` → accepted.
2. **Worker execution + shared helpers** — Tier-3 (FK enforced, cascade,
   auto/explicit name in `REL_TABLE`, errors: non-PK, type mismatch,
   composite-bare-ref; self-ref enforced) → red → factor the shared
   helpers out of `do_add_relationship`, add FK resolve/validate + DDL
   emission + metadata writes to `do_create_table` → green.
3. **Round-trip + describe + undo** — Tier-3 (survives rebuild + still
   enforced; `describe` shows the relation; one undo step drops table +
   rels) → green.
4. **Catalog/keys** — any new diagnostic keys in lockstep + vocab audit.
5. **Full sweep** — `cargo test` (no regression from 1769) + clippy.
6. **Docs** — ADR-0035 Status/§13 4b note; README; `requirements.md` Q1.
   Propose commit; wait for approval.

## 9. Exit gate

- All §3 items satisfied; four tiers green, zero skips; no regression
  from the 1769 baseline; written DA pass on the delivered slice; clippy
  clean.