ADR-0018 implementation: auto-fill contracts for serial and shortid
Generalises serial and shortid beyond their previous restricted forms: - `serial` is no longer restricted to single-column PK. Non-PK serial columns get an emitted UNIQUE constraint and use application-side MAX(col)+1 at INSERT time (rowid alias still drives the PK case for free; per ADR-0010 worker-thread serialisation, the read-then-insert sequence is safe). - `shortid` columns auto-fill existing null cells when the column is materialised — `add column T: x (shortid)` on a non-empty table no longer leaves rows in a not-really-valid NULL state. - `int -> serial` joins the type-change matrix as always-clean identity (closes the asymmetry vs `text -> shortid`); other sources are refused with a route-via-int hint. - `change column T: x (serial|shortid)` fills null source cells with sequence / generated values in the same rebuild transaction. Internal infrastructure: - ReadColumn gains `unique: bool`; read_schema detects single- column UNIQUE indexes via pragma_index_list / pragma_index_info; schema_to_ddl emits inline UNIQUE for non-PK columns. - ColumnSchema (persistence) gains `unique: bool` so the flag survives YAML round-trip and rebuild-from-text reconstructs it faithfully — preserves the "serial -> int leaves UNIQUE in place" promise across save/load cycles. - ChangeColumnTypeResult.client_side now carries `auto_filled` + `auto_fill_kind` alongside `transformed` + `lossy`; the app handler renders separate note lines when both apply. - AddColumnResult is a new return type carrying pre-rendered [client-side] note lines for the auto-fill paths. Tests: 519 -> 534 (+15). Clippy clean.
This commit is contained in:
@@ -0,0 +1,385 @@
|
|||||||
|
# ADR-0018: Auto-fill contracts for `serial` and `shortid` columns
|
||||||
|
|
||||||
|
## Status
|
||||||
|
|
||||||
|
Accepted.
|
||||||
|
|
||||||
|
Amends ADR-0005 (column type vocabulary), ADR-0014 (data
|
||||||
|
operations and value model), and ADR-0017 (column type-change
|
||||||
|
compatibility). Pulls part of C3's UNIQUE-constraint emission
|
||||||
|
forward as internal infrastructure.
|
||||||
|
|
||||||
|
## Context
|
||||||
|
|
||||||
|
`serial` and `shortid` are the two auto-generated types in
|
||||||
|
ADR-0005's vocabulary. Today they have asymmetric and
|
||||||
|
under-specified semantics:
|
||||||
|
|
||||||
|
1. **`serial` only on PK.** `Type::Serial.sqlite_strict_extra()`
|
||||||
|
returns `" PRIMARY KEY"`, and `do_add_column` explicitly
|
||||||
|
refuses serial. The implicit user-facing model is "serial =
|
||||||
|
auto-incrementing PK". This is an artefact of SQLite's only
|
||||||
|
free auto-increment mechanism (the rowid alias on `INTEGER
|
||||||
|
PRIMARY KEY`); other RDBMS — PostgreSQL's `SEQUENCE`, MySQL's
|
||||||
|
`AUTO_INCREMENT` — let auto-incrementing columns exist
|
||||||
|
anywhere. Our pedagogical intent is the broader model; the
|
||||||
|
restriction is incidental to our backend choice and leaks
|
||||||
|
that choice into the user-facing surface (against ADR-0002).
|
||||||
|
|
||||||
|
2. **`int → serial` is statically refused** in ADR-0017's
|
||||||
|
transformer matrix, while `text → shortid` is per-cell-
|
||||||
|
classified. Yet both target types are equally
|
||||||
|
"auto-generated with a uniqueness contract" — the asymmetry
|
||||||
|
isn't principled.
|
||||||
|
|
||||||
|
3. **`add column T: x (shortid)` on a non-empty table leaves
|
||||||
|
existing rows NULL.** Per the design contract, shortids are
|
||||||
|
unique non-null identifiers — so the column ends up in a
|
||||||
|
not-really-valid state until the user issues UPDATEs. The
|
||||||
|
auto-fill logic that runs at INSERT time for omitted shortid
|
||||||
|
values doesn't run at column-materialisation time.
|
||||||
|
|
||||||
|
4. **No UNIQUE constraint emission today.** A non-PK serial
|
||||||
|
column would need a UNIQUE constraint to enforce its contract
|
||||||
|
(the rowid trick isn't available off the PK). The same
|
||||||
|
applies to non-PK shortid: today it relies on a probabilistic
|
||||||
|
"won't collide" argument, not a database-enforced
|
||||||
|
guarantee. `schema_to_ddl` only emits NOT NULL inline plus PK
|
||||||
|
inline / table-level. No path emits UNIQUE.
|
||||||
|
|
||||||
|
This ADR resolves all four gaps with a single unifying
|
||||||
|
principle.
|
||||||
|
|
||||||
|
## Decision
|
||||||
|
|
||||||
|
### 1. The unifying principle
|
||||||
|
|
||||||
|
> Auto-generated column types honour their generation contract
|
||||||
|
> on every path that creates or transitions the column.
|
||||||
|
|
||||||
|
Concretely: a column declared (or converted to be) `serial` or
|
||||||
|
`shortid` always satisfies its contract — non-null,
|
||||||
|
auto-generated, unique — by the time the operation completes.
|
||||||
|
The user does not have to issue a follow-up UPDATE. The
|
||||||
|
mechanism is hidden; the user-facing model is "values appear
|
||||||
|
automatically".
|
||||||
|
|
||||||
|
### 2. `serial`: dual-implementation, single semantic
|
||||||
|
|
||||||
|
`serial` is generalised from "auto-incrementing PK" to
|
||||||
|
"auto-incrementing integer column". The column may be the table
|
||||||
|
PK or any non-PK column; the user-facing semantic is identical.
|
||||||
|
|
||||||
|
The implementation switches transparently:
|
||||||
|
|
||||||
|
- **PK case** (single-column PK on this column): rowid alias.
|
||||||
|
`INTEGER PRIMARY KEY` in DDL; SQLite's free auto-increment
|
||||||
|
applies. Unchanged from today.
|
||||||
|
- **Non-PK case**: app-level `MAX(col) + 1` lookup at INSERT
|
||||||
|
time, plus an emitted UNIQUE constraint on the column. The
|
||||||
|
worker-thread serialisation (ADR-0010) makes the read-then-
|
||||||
|
insert sequence safe without explicit locking — only one
|
||||||
|
INSERT runs at a time on the connection.
|
||||||
|
|
||||||
|
User-visible help, error messages, and `[client-side]` notes
|
||||||
|
refer to `serial` columns as "auto-incrementing" or
|
||||||
|
"auto-generated". The PK / non-PK distinction is an internal
|
||||||
|
implementation detail (ADR-0002 user-facing posture).
|
||||||
|
|
||||||
|
### 3. `shortid`: tighten the contract at column materialisation
|
||||||
|
|
||||||
|
Today: shortid generation runs only when an INSERT omits the
|
||||||
|
value. Rows existing at the moment a shortid column is created
|
||||||
|
remain NULL until the user issues an UPDATE.
|
||||||
|
|
||||||
|
Going forward: any null cell in a shortid column gets a freshly-
|
||||||
|
generated value at the operation that creates that condition:
|
||||||
|
|
||||||
|
- `add column T: x (shortid)` on a non-empty table fills every
|
||||||
|
existing row's `x` with a generated shortid before the
|
||||||
|
operation completes.
|
||||||
|
- `change column T: x (shortid)` from `text` (or any other
|
||||||
|
matrix-permitted source) fills any null cells with generated
|
||||||
|
shortids in the same rebuild transaction.
|
||||||
|
|
||||||
|
Generator collisions (vanishingly rare given the 10⁷–10⁸
|
||||||
|
namespace; see ADR-0014 §"shortid auto-generation") trigger up
|
||||||
|
to 5 retries per cell. Exhausting retries fails the operation
|
||||||
|
with a friendly diagnostic; in practice this indicates either a
|
||||||
|
generator-state bug or a pathological RNG and is not user-
|
||||||
|
recoverable.
|
||||||
|
|
||||||
|
### 4. UNIQUE story
|
||||||
|
|
||||||
|
Auto-generated non-PK columns gain an emitted UNIQUE constraint
|
||||||
|
to enforce their contract:
|
||||||
|
|
||||||
|
- Non-PK `serial`: gains UNIQUE on creation / conversion-to-
|
||||||
|
serial. Required for the contract; the rowid trick isn't
|
||||||
|
available off the PK.
|
||||||
|
- Non-PK `shortid`: gains UNIQUE on creation / conversion-to-
|
||||||
|
shortid. Strengthens today's probabilistic guarantee into a
|
||||||
|
database-enforced one.
|
||||||
|
- PK case for either type: PK already implies UNIQUE+NOT NULL.
|
||||||
|
No additional constraint needed.
|
||||||
|
|
||||||
|
The reverse direction (`serial → int`, `shortid → text`) leaves
|
||||||
|
the UNIQUE constraint **in place**. The user has not signalled
|
||||||
|
intent to drop the uniqueness guarantee; only the auto-
|
||||||
|
generation contract was dropped. When constraint-management
|
||||||
|
lands as a user-facing feature (C3-track), the user can
|
||||||
|
explicitly drop the UNIQUE if desired.
|
||||||
|
|
||||||
|
This ADR pulls forward the **internal infrastructure** to emit
|
||||||
|
and read UNIQUE constraints — `schema_to_ddl` gains UNIQUE-
|
||||||
|
column-clause emission; `read_schema` gains UNIQUE detection
|
||||||
|
via `pragma_index_list` + `pragma_index_info`; `ReadColumn`
|
||||||
|
gains a `unique: bool` field. The **user-facing constraint
|
||||||
|
surface** (declaring UNIQUE in `with pk … unique …` or via
|
||||||
|
`add unique`, dropping UNIQUE, naming UNIQUE constraints) is
|
||||||
|
not in scope here and remains C3-track work.
|
||||||
|
|
||||||
|
### 5. INSERT-path changes
|
||||||
|
|
||||||
|
For non-PK `serial` columns, when the column is omitted from
|
||||||
|
an INSERT (the existing skip-list at db.rs:3111 already covers
|
||||||
|
serial and shortid identically), the executor:
|
||||||
|
|
||||||
|
1. Queries `SELECT COALESCE(MAX(col), 0) + 1 FROM T` inside the
|
||||||
|
same transaction.
|
||||||
|
2. Binds the result as the column's value.
|
||||||
|
|
||||||
|
The MAX-based seeding mirrors SQLite's rowid behaviour: gaps
|
||||||
|
left by user-supplied explicit values are jumped over (the next
|
||||||
|
auto-fill is `MAX + 1`, not "the smallest available integer").
|
||||||
|
|
||||||
|
Worker-thread serialisation (ADR-0010) prevents the classic
|
||||||
|
read-modify-write race; the pattern is safe for our single-
|
||||||
|
writer model.
|
||||||
|
|
||||||
|
### 6. `add_column` changes
|
||||||
|
|
||||||
|
`do_add_column` lifts its blanket serial refusal (db.rs:1374).
|
||||||
|
The new behaviour is determined by the source table's state:
|
||||||
|
|
||||||
|
- **`add column T: x (serial)` on an empty table**: emit
|
||||||
|
`ALTER TABLE T ADD COLUMN x INTEGER UNIQUE`. Every table has
|
||||||
|
a PK by construction (the parser refuses `create table`
|
||||||
|
without `with pk`), so the "no PK" branch doesn't arise —
|
||||||
|
the new column joins as a non-PK serial.
|
||||||
|
- **`add column T: x (serial)` on a non-empty table**: route
|
||||||
|
through the rebuild-table primitive (ADR-0013). Create new
|
||||||
|
table with `x INTEGER UNIQUE`. Copy rows, filling `x` with
|
||||||
|
values 1..N in declaration order. Emit a `[client-side]` note
|
||||||
|
(§7).
|
||||||
|
- **`add column T: x (shortid)` on a non-empty table**: route
|
||||||
|
through the rebuild-table primitive. Create new table with
|
||||||
|
`x TEXT UNIQUE`. Copy rows, generating a fresh shortid for
|
||||||
|
each (collision-retried per §3). Emit a `[client-side]` note.
|
||||||
|
|
||||||
|
The empty-table path can stay on `ALTER TABLE ADD COLUMN` for
|
||||||
|
efficiency; the non-empty path needs the rebuild because we
|
||||||
|
need to populate the new column atomically with table
|
||||||
|
creation.
|
||||||
|
|
||||||
|
### 7. `change column` to `serial` / `shortid`
|
||||||
|
|
||||||
|
`change column T: x (serial)` from any matrix-permitted source
|
||||||
|
type (today: `int`; future expansions follow the same rule):
|
||||||
|
|
||||||
|
1. Run the per-cell dry-run (ADR-0017 §2). For non-null cells,
|
||||||
|
classify via the transformer matrix: source must produce an
|
||||||
|
integer (the existing serial pre-condition).
|
||||||
|
2. Refuse if existing non-null values have duplicates
|
||||||
|
(uniqueness collision, ADR-0017 §4.3).
|
||||||
|
3. Auto-fill any null cells with sequential values continuing
|
||||||
|
from `MAX(non-null values) + 1` (or starting at 1 if none).
|
||||||
|
4. Refuse if the auto-fill would itself produce a collision —
|
||||||
|
in practice, this can only happen if the user supplied
|
||||||
|
non-null values that already overlap the would-be sequence
|
||||||
|
(e.g., existing values [1, 2, 5] with two nulls — fill would
|
||||||
|
be 6 and 7, no collision; existing values [1, 2, 6] with
|
||||||
|
nulls — fill would be 3 and 4, no collision; the sequence
|
||||||
|
uses MAX+1, not gap-filling, so this case doesn't actually
|
||||||
|
arise — but state the rule defensively).
|
||||||
|
5. Rebuild the table with the new column type plus UNIQUE (per
|
||||||
|
§4) plus the transformed + auto-filled values.
|
||||||
|
6. Emit `[client-side]` notes (§7).
|
||||||
|
|
||||||
|
`change column T: x (shortid)` from `text`:
|
||||||
|
|
||||||
|
1. Run the per-cell dry-run. Non-null cells classify via the
|
||||||
|
text → shortid transformer (ADR-0017 §3) — must match the
|
||||||
|
shortid grammar.
|
||||||
|
2. Refuse if existing non-null shortid-valid values have
|
||||||
|
duplicates.
|
||||||
|
3. Auto-fill null cells with generated shortids (collision-
|
||||||
|
retried per §3, including against the existing values).
|
||||||
|
4. Rebuild with TEXT + UNIQUE + the validated + auto-filled
|
||||||
|
values.
|
||||||
|
5. Emit `[client-side]` notes.
|
||||||
|
|
||||||
|
### 8. Conversion matrix amendments to ADR-0017
|
||||||
|
|
||||||
|
ADR-0017 §3 "Statically refused" is amended:
|
||||||
|
|
||||||
|
- `int → serial` is **removed** from the static refusal list and
|
||||||
|
added as a **per-cell-classified** matrix entry: clean for
|
||||||
|
non-null integers (with the post-transformation uniqueness
|
||||||
|
check from §4.3), with null-cell auto-fill per §7 above.
|
||||||
|
- The general "Anything → `serial`" refusal is replaced with a
|
||||||
|
more specific list: `text → serial`, `real → serial`, etc.
|
||||||
|
remain refused for v1 (route via int first); `bool → serial`
|
||||||
|
remains refused (cross-domain).
|
||||||
|
- `text → shortid` is unchanged from ADR-0017 (still per-cell-
|
||||||
|
classified). The contract enforcement at column-materialisation
|
||||||
|
is new.
|
||||||
|
|
||||||
|
ADR-0017 §4.3 (uniqueness check) is amended to apply to
|
||||||
|
"PK columns and shortid columns and any column that gains a
|
||||||
|
UNIQUE constraint as part of the operation" — i.e., non-PK
|
||||||
|
serial / shortid targets are uniqueness-checked.
|
||||||
|
|
||||||
|
### 9. Client-side notes
|
||||||
|
|
||||||
|
ADR-0017 §6 introduced the `[client-side]` pattern: when the
|
||||||
|
playground rewrote any cell value, the success summary tells
|
||||||
|
the learner "the tool did this for you; raw SQL would need a
|
||||||
|
`CAST` or application-level code." This ADR extends the pattern
|
||||||
|
to auto-fill operations:
|
||||||
|
|
||||||
|
- **`add column T: x (serial)` on non-empty table**:
|
||||||
|
> [client-side] N row(s) given auto-generated serial values
|
||||||
|
> 1..N. In raw SQL this would need an explicit UPDATE to
|
||||||
|
> populate.
|
||||||
|
|
||||||
|
- **`add column T: x (shortid)` on non-empty table**:
|
||||||
|
> [client-side] N row(s) given auto-generated shortid values.
|
||||||
|
> In raw SQL this would need an explicit UPDATE to populate.
|
||||||
|
|
||||||
|
- **`change column T: x (serial)` with M null cells**:
|
||||||
|
> [client-side] M null cell(s) given auto-generated serial
|
||||||
|
> values. In raw SQL this would need an explicit UPDATE to
|
||||||
|
> populate.
|
||||||
|
|
||||||
|
- **`change column T: x (shortid)` with M null cells**:
|
||||||
|
> [client-side] M null cell(s) given auto-generated shortid
|
||||||
|
> values. In raw SQL this would need an explicit UPDATE to
|
||||||
|
> populate.
|
||||||
|
|
||||||
|
When both an ADR-0017 transformation note AND an ADR-0018
|
||||||
|
auto-fill note apply to the same operation (e.g., `change
|
||||||
|
column T: x (shortid)` from text where some cells need
|
||||||
|
validation and others need auto-fill), both notes are emitted
|
||||||
|
on separate lines. The success path emits them after the `[ok]`
|
||||||
|
summary and before the structure-render block.
|
||||||
|
|
||||||
|
### 10. Engine-vocabulary cleanup
|
||||||
|
|
||||||
|
While here, fix the existing user-facing string in
|
||||||
|
`do_add_column`'s serial refusal (db.rs:1374): the message
|
||||||
|
names "SQLite's ALTER TABLE" — an ADR-0002 user-facing posture
|
||||||
|
violation. This message is being replaced anyway as part of
|
||||||
|
lifting the refusal; the replacement uses abstract "the
|
||||||
|
database" / "the engine" phrasing.
|
||||||
|
|
||||||
|
## Resolutions
|
||||||
|
|
||||||
|
Three points called out as "open" during drafting, resolved
|
||||||
|
before acceptance:
|
||||||
|
|
||||||
|
1. **No-PK empty-table case**: not reachable. Every table has
|
||||||
|
a PK by construction — the `create table` parser refuses
|
||||||
|
input that produces an empty PK list. `add column T: x
|
||||||
|
(serial)` on an empty table therefore always lands on a
|
||||||
|
table that already has a PK, and the new `x` column is a
|
||||||
|
non-PK serial (gains UNIQUE per §4).
|
||||||
|
|
||||||
|
2. **Serial sequencing under explicit user inserts**: MAX+1.
|
||||||
|
If the user explicitly inserts `id = 100`, the next auto-
|
||||||
|
fill yields 101. Gappy sequences are accepted (e.g., if
|
||||||
|
the user later inserts `id = 200`, the next auto-fill is
|
||||||
|
201; the gap 102..199 is not back-filled). MAX+1 matches
|
||||||
|
SQLite's rowid behaviour for the PK case, so both
|
||||||
|
implementation paths feel uniform to the user, and gap-
|
||||||
|
detection is more expensive than its pedagogical value.
|
||||||
|
|
||||||
|
3. **UNIQUE emission style**: inline column constraint
|
||||||
|
(`x INTEGER UNIQUE`). Cleaner DDL while we don't have a
|
||||||
|
user-facing constraint surface that would benefit from
|
||||||
|
named, separately-managed indexes. Revisitable when C3
|
||||||
|
lands the user-facing constraint feature; the
|
||||||
|
`read_schema` detection via `pragma_index_list` works for
|
||||||
|
either form.
|
||||||
|
|
||||||
|
## Out of scope
|
||||||
|
|
||||||
|
- **OOS-1.** User-facing UNIQUE constraint surface (`add
|
||||||
|
unique <T>: <c>`, `drop unique`, naming, multi-column unique).
|
||||||
|
Stays as C3-track work.
|
||||||
|
- **OOS-2.** Strict-monotonic AUTOINCREMENT semantics — we
|
||||||
|
retain plain `INTEGER PRIMARY KEY` for PK serial. Rebuild-
|
||||||
|
reset of the high-water mark is acceptable for a teaching
|
||||||
|
tool; users who care can be taught the distinction in a
|
||||||
|
later iteration.
|
||||||
|
- **OOS-3.** Custom serial start values, custom step sizes, or
|
||||||
|
multi-column composite serial.
|
||||||
|
- **OOS-4.** Non-PK serial when the table has no PK at all
|
||||||
|
(caught by Open Question 1's resolution).
|
||||||
|
- **OOS-5.** A `[client-side]` note on the empty-table case
|
||||||
|
(`add column` on an empty table). No rows means nothing to
|
||||||
|
auto-fill — the operation is a structural change with no
|
||||||
|
pedagogical "the tool did this for you" content.
|
||||||
|
- **OOS-6.** Reading and emitting CHECK constraints — only
|
||||||
|
UNIQUE is required for this ADR.
|
||||||
|
|
||||||
|
## Consequences
|
||||||
|
|
||||||
|
- The "serial only on PK" mental model is replaced with
|
||||||
|
"serial works anywhere". Pedagogically richer: students see
|
||||||
|
auto-incrementing columns as a general feature, not as a
|
||||||
|
special PK-only quirk.
|
||||||
|
- One internal mechanism the user doesn't see (rowid alias vs
|
||||||
|
application MAX+1). The two paths converge to identical
|
||||||
|
user-facing behaviour, honouring ADR-0002's posture.
|
||||||
|
- `schema_to_ddl` and `read_schema` gain UNIQUE handling — a
|
||||||
|
partial pull-forward of C3 work. The user-facing constraint
|
||||||
|
surface stays deferred; this ADR only lands the internal
|
||||||
|
infrastructure required by serial / shortid contracts.
|
||||||
|
- `[client-side]` notes proliferate to cover auto-fill cases.
|
||||||
|
Strengthens the pedagogical lens: every place the playground
|
||||||
|
goes beyond what raw SQL does, the user is told.
|
||||||
|
- All four user-observed gaps from §Context closed. The
|
||||||
|
`int → serial → int` round-trip works (matching the existing
|
||||||
|
`text → shortid → text` round-trip from ADR-0017).
|
||||||
|
- Add-column-with-shortid producing a "valid" state aligns
|
||||||
|
with ADR-0005's design contract that shortids are unique
|
||||||
|
non-null identifiers.
|
||||||
|
|
||||||
|
## Relationship to earlier ADRs
|
||||||
|
|
||||||
|
- **ADR-0002** — User-facing posture honoured: the dual
|
||||||
|
serial implementation is hidden; the existing engine-name
|
||||||
|
leak in `do_add_column`'s refusal message is fixed
|
||||||
|
opportunistically.
|
||||||
|
- **ADR-0005** — Type vocabulary unchanged; `serial` definition
|
||||||
|
generalised. The keyword and the user model stay the same;
|
||||||
|
the implementation broadens.
|
||||||
|
- **ADR-0010** — Worker-thread serialisation is what makes the
|
||||||
|
non-PK serial MAX+1 path safe without explicit locks.
|
||||||
|
- **ADR-0011** — `fk_target_type` for serial unchanged
|
||||||
|
(`Serial → Int`); FK target compatibility remains as-is.
|
||||||
|
- **ADR-0013** — Rebuild-table primitive carries the auto-fill
|
||||||
|
cases for non-empty `add column` and `change column to
|
||||||
|
serial/shortid`.
|
||||||
|
- **ADR-0014** — INSERT-time auto-fill semantics extended to
|
||||||
|
non-PK serial. ADR-0014's auto-fill skip-list (which already
|
||||||
|
covers both serial and shortid symmetrically) is reused.
|
||||||
|
- **ADR-0015** — The text-format round-trip carries the new
|
||||||
|
UNIQUE constraints in metadata so a rebuild from
|
||||||
|
`project.yaml` reconstructs the database faithfully. Likely
|
||||||
|
needs a `__rdbms_playground_columns` schema additon (a
|
||||||
|
`unique` bool) — to be confirmed during implementation.
|
||||||
|
- **ADR-0017** — §3 transformer matrix amended: `int → serial`
|
||||||
|
joins per-cell-classified. §4.3 uniqueness check extended to
|
||||||
|
cover non-PK serial / shortid targets.
|
||||||
@@ -23,3 +23,4 @@ This directory contains the project's ADRs, recorded per
|
|||||||
- [ADR-0015 — Project storage runtime](0015-project-storage-runtime.md)
|
- [ADR-0015 — Project storage runtime](0015-project-storage-runtime.md)
|
||||||
- [ADR-0016 — Pretty table rendering for data and structure views](0016-pretty-table-rendering.md)
|
- [ADR-0016 — Pretty table rendering for data and structure views](0016-pretty-table-rendering.md)
|
||||||
- [ADR-0017 — Column type-change compatibility](0017-column-type-change-compatibility.md)
|
- [ADR-0017 — Column type-change compatibility](0017-column-type-change-compatibility.md)
|
||||||
|
- [ADR-0018 — Auto-fill contracts for `serial` and `shortid` columns](0018-auto-fill-contracts-for-serial-and-shortid.md)
|
||||||
|
|||||||
+47
-7
@@ -13,8 +13,8 @@ use tracing::{trace, warn};
|
|||||||
|
|
||||||
use crate::action::Action;
|
use crate::action::Action;
|
||||||
use crate::db::{
|
use crate::db::{
|
||||||
CascadeEffect, ChangeColumnTypeResult, DataResult, DeleteResult, InsertResult,
|
AddColumnResult, CascadeEffect, ChangeColumnTypeResult, DataResult, DeleteResult,
|
||||||
TableDescription, UpdateResult,
|
InsertResult, TableDescription, UpdateResult,
|
||||||
};
|
};
|
||||||
use crate::dsl::{Command, ParseError, parse_command};
|
use crate::dsl::{Command, ParseError, parse_command};
|
||||||
use crate::event::AppEvent;
|
use crate::event::AppEvent;
|
||||||
@@ -317,6 +317,10 @@ impl App {
|
|||||||
self.handle_dsl_change_column_success(&command, result);
|
self.handle_dsl_change_column_success(&command, result);
|
||||||
Vec::new()
|
Vec::new()
|
||||||
}
|
}
|
||||||
|
AppEvent::DslAddColumnSucceeded { command, result } => {
|
||||||
|
self.handle_dsl_add_column_success(&command, result);
|
||||||
|
Vec::new()
|
||||||
|
}
|
||||||
AppEvent::DslFailed { command, error } => {
|
AppEvent::DslFailed { command, error } => {
|
||||||
self.handle_dsl_failure(&command, &error);
|
self.handle_dsl_failure(&command, &error);
|
||||||
Vec::new()
|
Vec::new()
|
||||||
@@ -782,6 +786,26 @@ impl App {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
fn handle_dsl_add_column_success(
|
||||||
|
&mut self,
|
||||||
|
command: &Command,
|
||||||
|
result: AddColumnResult,
|
||||||
|
) {
|
||||||
|
let summary = format!("[ok] {} {}", command.verb(), command.display_subject());
|
||||||
|
self.note_system(summary);
|
||||||
|
// ADR-0018 §9: emit auto-fill note(s) before the
|
||||||
|
// structure render, so the pedagogical "the tool did
|
||||||
|
// this for you" line is in the user's eye-line next to
|
||||||
|
// the success summary.
|
||||||
|
for note in result.client_side_notes {
|
||||||
|
self.note_system(note);
|
||||||
|
}
|
||||||
|
for line in crate::output_render::render_structure(&result.description) {
|
||||||
|
self.note_system(line);
|
||||||
|
}
|
||||||
|
self.current_table = Some(result.description);
|
||||||
|
}
|
||||||
|
|
||||||
fn handle_dsl_change_column_success(
|
fn handle_dsl_change_column_success(
|
||||||
&mut self,
|
&mut self,
|
||||||
command: &Command,
|
command: &Command,
|
||||||
@@ -790,11 +814,14 @@ impl App {
|
|||||||
let summary = format!("[ok] {} {}", command.verb(), command.display_subject());
|
let summary = format!("[ok] {} {}", command.verb(), command.display_subject());
|
||||||
self.note_system(summary);
|
self.note_system(summary);
|
||||||
if let Some(note) = result.client_side {
|
if let Some(note) = result.client_side {
|
||||||
// ADR-0017 §6: pedagogical hook telling the learner
|
// ADR-0017 §6 + ADR-0018 §9: pedagogical hook
|
||||||
// "the tool did this for you; raw SQL would need a
|
// telling the learner "the tool did this for you;
|
||||||
// CAST or application-level code." Lossy variant
|
// raw SQL would need explicit CAST / UPDATE / etc."
|
||||||
// adds the lossy count when --force-conversion was
|
//
|
||||||
// used.
|
// When both transformations and auto-fills happen
|
||||||
|
// in the same operation, both note lines are
|
||||||
|
// emitted in order (ADR-0018 §9 explicit rule).
|
||||||
|
if note.transformed > 0 {
|
||||||
let line = if note.lossy > 0 {
|
let line = if note.lossy > 0 {
|
||||||
format!(
|
format!(
|
||||||
"[client-side] {n} row(s) transformed; {l} of those discarded \
|
"[client-side] {n} row(s) transformed; {l} of those discarded \
|
||||||
@@ -813,6 +840,19 @@ impl App {
|
|||||||
};
|
};
|
||||||
self.note_system(line);
|
self.note_system(line);
|
||||||
}
|
}
|
||||||
|
if note.auto_filled > 0 {
|
||||||
|
let kind = match note.auto_fill_kind {
|
||||||
|
Some(crate::db::AutoFillKind::Serial) => "serial",
|
||||||
|
Some(crate::db::AutoFillKind::ShortId) => "shortid",
|
||||||
|
None => "auto-generated",
|
||||||
|
};
|
||||||
|
self.note_system(format!(
|
||||||
|
"[client-side] {m} null cell(s) given auto-generated {kind} values. \
|
||||||
|
In raw SQL this would need an explicit UPDATE to populate.",
|
||||||
|
m = note.auto_filled,
|
||||||
|
));
|
||||||
|
}
|
||||||
|
}
|
||||||
for line in crate::output_render::render_structure(&result.description) {
|
for line in crate::output_render::render_structure(&result.description) {
|
||||||
self.note_system(line);
|
self.note_system(line);
|
||||||
}
|
}
|
||||||
|
|||||||
+9
-2
@@ -8,8 +8,8 @@
|
|||||||
use crossterm::event::KeyEvent;
|
use crossterm::event::KeyEvent;
|
||||||
|
|
||||||
use crate::db::{
|
use crate::db::{
|
||||||
ChangeColumnTypeResult, DataResult, DeleteResult, InsertResult, TableDescription,
|
AddColumnResult, ChangeColumnTypeResult, DataResult, DeleteResult, InsertResult,
|
||||||
UpdateResult,
|
TableDescription, UpdateResult,
|
||||||
};
|
};
|
||||||
use crate::dsl::Command;
|
use crate::dsl::Command;
|
||||||
|
|
||||||
@@ -49,6 +49,13 @@ pub enum AppEvent {
|
|||||||
command: Command,
|
command: Command,
|
||||||
result: ChangeColumnTypeResult,
|
result: ChangeColumnTypeResult,
|
||||||
},
|
},
|
||||||
|
/// An `add column …` succeeded. `result` carries the
|
||||||
|
/// post-add description plus any `[client-side]` notes
|
||||||
|
/// from the auto-fill paths (ADR-0018 §9).
|
||||||
|
DslAddColumnSucceeded {
|
||||||
|
command: Command,
|
||||||
|
result: AddColumnResult,
|
||||||
|
},
|
||||||
/// A DSL command failed. `error` is already a friendly
|
/// A DSL command failed. `error` is already a friendly
|
||||||
/// message produced via `DbError::friendly_message`.
|
/// message produced via `DbError::friendly_message`.
|
||||||
DslFailed {
|
DslFailed {
|
||||||
|
|||||||
@@ -360,7 +360,7 @@ mod tests {
|
|||||||
use crate::persistence::ColumnSchema;
|
use crate::persistence::ColumnSchema;
|
||||||
|
|
||||||
fn col(name: &str, ty: Type) -> ColumnSchema {
|
fn col(name: &str, ty: Type) -> ColumnSchema {
|
||||||
ColumnSchema { name: name.to_string(), user_type: ty }
|
ColumnSchema { name: name.to_string(), user_type: ty, unique: false }
|
||||||
}
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
|
|||||||
@@ -100,6 +100,13 @@ pub struct TableSchema {
|
|||||||
pub struct ColumnSchema {
|
pub struct ColumnSchema {
|
||||||
pub name: String,
|
pub name: String,
|
||||||
pub user_type: Type,
|
pub user_type: Type,
|
||||||
|
/// Whether this column carries a single-column UNIQUE
|
||||||
|
/// constraint (ADR-0018 §4). Stored explicitly in the
|
||||||
|
/// project YAML so that a `serial → int` round-trip
|
||||||
|
/// (which leaves UNIQUE in place) is preserved across a
|
||||||
|
/// save/load cycle. Defaults to `false` when missing in
|
||||||
|
/// older project files.
|
||||||
|
pub unique: bool,
|
||||||
}
|
}
|
||||||
|
|
||||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||||
@@ -318,6 +325,7 @@ mod tests {
|
|||||||
columns: vec![ColumnSchema {
|
columns: vec![ColumnSchema {
|
||||||
name: "Name".to_string(),
|
name: "Name".to_string(),
|
||||||
user_type: Type::Text,
|
user_type: Type::Text,
|
||||||
|
unique: false,
|
||||||
}],
|
}],
|
||||||
rows: vec![vec![CellValue::Text("Alice".to_string())]],
|
rows: vec![vec![CellValue::Text("Alice".to_string())]],
|
||||||
};
|
};
|
||||||
|
|||||||
+22
-6
@@ -71,6 +71,14 @@ fn write_table(out: &mut String, table: &TableSchema) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
fn write_column(out: &mut String, col: &ColumnSchema) {
|
fn write_column(out: &mut String, col: &ColumnSchema) {
|
||||||
|
if col.unique {
|
||||||
|
let _ = writeln!(
|
||||||
|
out,
|
||||||
|
" - {{ name: {}, type: {}, unique: true }}",
|
||||||
|
quote_if_needed(&col.name),
|
||||||
|
col.user_type.keyword(),
|
||||||
|
);
|
||||||
|
} else {
|
||||||
let _ = writeln!(
|
let _ = writeln!(
|
||||||
out,
|
out,
|
||||||
" - {{ name: {}, type: {} }}",
|
" - {{ name: {}, type: {} }}",
|
||||||
@@ -78,6 +86,7 @@ fn write_column(out: &mut String, col: &ColumnSchema) {
|
|||||||
col.user_type.keyword(),
|
col.user_type.keyword(),
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
}
|
||||||
|
|
||||||
fn write_relationship(out: &mut String, rel: &RelationshipSchema) {
|
fn write_relationship(out: &mut String, rel: &RelationshipSchema) {
|
||||||
let _ = writeln!(out, " - name: {}", quote_if_needed(&rel.name));
|
let _ = writeln!(out, " - name: {}", quote_if_needed(&rel.name));
|
||||||
@@ -181,6 +190,7 @@ pub(crate) fn parse_schema(body: &str) -> Result<SchemaSnapshot, YamlError> {
|
|||||||
columns.push(ColumnSchema {
|
columns.push(ColumnSchema {
|
||||||
name: c.name,
|
name: c.name,
|
||||||
user_type,
|
user_type,
|
||||||
|
unique: c.unique,
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
tables.push(TableSchema {
|
tables.push(TableSchema {
|
||||||
@@ -265,6 +275,11 @@ struct RawColumn {
|
|||||||
name: String,
|
name: String,
|
||||||
#[serde(rename = "type")]
|
#[serde(rename = "type")]
|
||||||
user_type: String,
|
user_type: String,
|
||||||
|
/// Optional flag introduced in ADR-0018 for single-column
|
||||||
|
/// UNIQUE constraints. Older project files without this
|
||||||
|
/// field default to `false`.
|
||||||
|
#[serde(default)]
|
||||||
|
unique: bool,
|
||||||
}
|
}
|
||||||
|
|
||||||
#[derive(Deserialize)]
|
#[derive(Deserialize)]
|
||||||
@@ -295,16 +310,16 @@ mod tests {
|
|||||||
name: "Customers".to_string(),
|
name: "Customers".to_string(),
|
||||||
primary_key: vec!["id".to_string()],
|
primary_key: vec!["id".to_string()],
|
||||||
columns: vec![
|
columns: vec![
|
||||||
ColumnSchema { name: "id".to_string(), user_type: Type::Serial },
|
ColumnSchema { name: "id".to_string(), user_type: Type::Serial, unique: false },
|
||||||
ColumnSchema { name: "Name".to_string(), user_type: Type::Text },
|
ColumnSchema { name: "Name".to_string(), user_type: Type::Text, unique: false },
|
||||||
],
|
],
|
||||||
},
|
},
|
||||||
TableSchema {
|
TableSchema {
|
||||||
name: "Orders".to_string(),
|
name: "Orders".to_string(),
|
||||||
primary_key: vec!["id".to_string()],
|
primary_key: vec!["id".to_string()],
|
||||||
columns: vec![
|
columns: vec![
|
||||||
ColumnSchema { name: "id".to_string(), user_type: Type::Serial },
|
ColumnSchema { name: "id".to_string(), user_type: Type::Serial, unique: false },
|
||||||
ColumnSchema { name: "CustId".to_string(), user_type: Type::Int },
|
ColumnSchema { name: "CustId".to_string(), user_type: Type::Int, unique: false },
|
||||||
],
|
],
|
||||||
},
|
},
|
||||||
],
|
],
|
||||||
@@ -359,6 +374,7 @@ mod tests {
|
|||||||
columns: vec![ColumnSchema {
|
columns: vec![ColumnSchema {
|
||||||
name: "yes".to_string(),
|
name: "yes".to_string(),
|
||||||
user_type: Type::Bool,
|
user_type: Type::Bool,
|
||||||
|
unique: false,
|
||||||
}],
|
}],
|
||||||
}],
|
}],
|
||||||
relationships: vec![],
|
relationships: vec![],
|
||||||
@@ -452,8 +468,8 @@ relationships:
|
|||||||
name: "Items".to_string(),
|
name: "Items".to_string(),
|
||||||
primary_key: vec!["a".to_string(), "b".to_string()],
|
primary_key: vec!["a".to_string(), "b".to_string()],
|
||||||
columns: vec![
|
columns: vec![
|
||||||
ColumnSchema { name: "a".to_string(), user_type: Type::Int },
|
ColumnSchema { name: "a".to_string(), user_type: Type::Int, unique: false },
|
||||||
ColumnSchema { name: "b".to_string(), user_type: Type::Int },
|
ColumnSchema { name: "b".to_string(), user_type: Type::Int, unique: false },
|
||||||
],
|
],
|
||||||
}],
|
}],
|
||||||
relationships: vec![],
|
relationships: vec![],
|
||||||
|
|||||||
+8
-3
@@ -29,8 +29,8 @@ use crate::action::Action;
|
|||||||
use crate::app::App;
|
use crate::app::App;
|
||||||
use crate::cli::Args;
|
use crate::cli::Args;
|
||||||
use crate::db::{
|
use crate::db::{
|
||||||
ChangeColumnTypeResult, DataResult, Database, DbError, DeleteResult, InsertResult,
|
AddColumnResult, ChangeColumnTypeResult, DataResult, Database, DbError, DeleteResult,
|
||||||
TableDescription, UpdateResult,
|
InsertResult, TableDescription, UpdateResult,
|
||||||
};
|
};
|
||||||
use crate::dsl::Command;
|
use crate::dsl::Command;
|
||||||
use crate::event::AppEvent;
|
use crate::event::AppEvent;
|
||||||
@@ -951,6 +951,10 @@ fn spawn_dsl_dispatch(
|
|||||||
command: command.clone(),
|
command: command.clone(),
|
||||||
result,
|
result,
|
||||||
},
|
},
|
||||||
|
Ok(CommandOutcome::AddColumn(result)) => AppEvent::DslAddColumnSucceeded {
|
||||||
|
command: command.clone(),
|
||||||
|
result,
|
||||||
|
},
|
||||||
Err(DbError::PersistenceFatal {
|
Err(DbError::PersistenceFatal {
|
||||||
operation,
|
operation,
|
||||||
path,
|
path,
|
||||||
@@ -988,6 +992,7 @@ enum CommandOutcome {
|
|||||||
Update(UpdateResult),
|
Update(UpdateResult),
|
||||||
Delete(DeleteResult),
|
Delete(DeleteResult),
|
||||||
ChangeColumn(ChangeColumnTypeResult),
|
ChangeColumn(ChangeColumnTypeResult),
|
||||||
|
AddColumn(AddColumnResult),
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Execute a parsed user command and return either a typed
|
/// Execute a parsed user command and return either a typed
|
||||||
@@ -1016,7 +1021,7 @@ async fn execute_command_typed(
|
|||||||
Command::AddColumn { table, column, ty } => database
|
Command::AddColumn { table, column, ty } => database
|
||||||
.add_column(table, column, ty, src)
|
.add_column(table, column, ty, src)
|
||||||
.await
|
.await
|
||||||
.map(|d| CommandOutcome::Schema(Some(d))),
|
.map(CommandOutcome::AddColumn),
|
||||||
Command::DropColumn { table, column } => database
|
Command::DropColumn { table, column } => database
|
||||||
.drop_column(table, column, src)
|
.drop_column(table, column, src)
|
||||||
.await
|
.await
|
||||||
|
|||||||
+44
-8
@@ -54,13 +54,14 @@ pub fn static_refusal(src: Type, target: Type) -> Option<String> {
|
|||||||
if src == target {
|
if src == target {
|
||||||
return Some(format!("column is already `{src}`."));
|
return Some(format!("column is already `{src}`."));
|
||||||
}
|
}
|
||||||
if matches!(target, Type::Serial) {
|
// ADR-0018 §8: `int → serial` is allowed via the matrix.
|
||||||
return Some(
|
// Other sources to serial are refused — the user routes
|
||||||
"the 'serial' type carries auto-increment primary-key semantics \
|
// via int first if needed.
|
||||||
that only apply at create-table time; pick a different target \
|
if matches!(target, Type::Serial) && !matches!(src, Type::Int) {
|
||||||
type (e.g. `int`)."
|
return Some(format!(
|
||||||
.to_string(),
|
"to convert from `{src}` to `serial`, change the column to `int` \
|
||||||
);
|
first; only `int → serial` is supported directly."
|
||||||
|
));
|
||||||
}
|
}
|
||||||
if matches!(src, Type::Blob) || matches!(target, Type::Blob) {
|
if matches!(src, Type::Blob) || matches!(target, Type::Blob) {
|
||||||
return Some(format!(
|
return Some(format!(
|
||||||
@@ -98,6 +99,14 @@ const fn is_in_matrix(src: Type, target: Type) -> bool {
|
|||||||
// stored values"). Storage stays INTEGER; we treat
|
// stored values"). Storage stays INTEGER; we treat
|
||||||
// it as an identity transformer.
|
// it as an identity transformer.
|
||||||
| (Serial, Int)
|
| (Serial, Int)
|
||||||
|
// int -> serial: ADR-0018 §8. Storage stays
|
||||||
|
// INTEGER, the metadata flips to "auto-generated"
|
||||||
|
// and the column gains UNIQUE if non-PK. The
|
||||||
|
// matrix entry is identity at the value level;
|
||||||
|
// uniqueness and auto-fill of nulls happen at the
|
||||||
|
// change-column orchestration layer, not in the
|
||||||
|
// per-cell transformer.
|
||||||
|
| (Int, Serial)
|
||||||
| (Bool, Int | Real | Decimal | Text)
|
| (Bool, Int | Real | Decimal | Text)
|
||||||
| (Decimal | Date | DateTime | ShortId | Real, Text)
|
| (Decimal | Date | DateTime | ShortId | Real, Text)
|
||||||
// Per-cell-classified
|
// Per-cell-classified
|
||||||
@@ -141,6 +150,16 @@ pub fn transform_cell(src: Type, target: Type, value: &Value) -> CellOutcome {
|
|||||||
Value::Integer(i) => CellOutcome::Clean(Value::Integer(*i)),
|
Value::Integer(i) => CellOutcome::Clean(Value::Integer(*i)),
|
||||||
other => unexpected_storage("serial", other),
|
other => unexpected_storage("serial", other),
|
||||||
},
|
},
|
||||||
|
// int -> serial: identity at the storage class level
|
||||||
|
// (both INTEGER); the conversion adds the auto-
|
||||||
|
// generated contract and (for non-PK columns) UNIQUE.
|
||||||
|
// Per-cell transformer is identity; the change-column
|
||||||
|
// orchestrator handles null auto-fill and uniqueness.
|
||||||
|
// ADR-0018 §8.
|
||||||
|
(Int, Serial) => match value {
|
||||||
|
Value::Integer(i) => CellOutcome::Clean(Value::Integer(*i)),
|
||||||
|
other => unexpected_storage("int", other),
|
||||||
|
},
|
||||||
|
|
||||||
// ---- Always-clean: bool source (stored as INTEGER 0/1) ----
|
// ---- Always-clean: bool source (stored as INTEGER 0/1) ----
|
||||||
(Bool, Int) => match value {
|
(Bool, Int) => match value {
|
||||||
@@ -568,12 +587,29 @@ mod tests {
|
|||||||
}
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn anything_to_serial_is_statically_refused() {
|
fn non_int_sources_to_serial_are_statically_refused() {
|
||||||
|
// ADR-0018 §8: only `int → serial` is allowed directly.
|
||||||
|
// Other sources have to route via int. (Same-type
|
||||||
|
// `serial → serial` is the no-op identity refusal.)
|
||||||
for &src in Type::all() {
|
for &src in Type::all() {
|
||||||
|
if matches!(src, Type::Int | Type::Serial) {
|
||||||
|
continue;
|
||||||
|
}
|
||||||
assert!(static_refusal(src, Type::Serial).is_some(), "{src:?}");
|
assert!(static_refusal(src, Type::Serial).is_some(), "{src:?}");
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn int_to_serial_is_allowed() {
|
||||||
|
// ADR-0018 §8: `int → serial` joins the matrix as
|
||||||
|
// always-clean identity. Non-null values pass through.
|
||||||
|
assert!(static_refusal(Type::Int, Type::Serial).is_none());
|
||||||
|
assert_eq!(
|
||||||
|
transform_cell(Type::Int, Type::Serial, &int(42)),
|
||||||
|
CellOutcome::Clean(int(42))
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn anything_involving_blob_is_statically_refused() {
|
fn anything_involving_blob_is_statically_refused() {
|
||||||
for &other in Type::all() {
|
for &other in Type::all() {
|
||||||
|
|||||||
Reference in New Issue
Block a user