# ADR-0005: Column type vocabulary ## Status Accepted ## Context Real RDBMS engines expose many type variants that exist for historical, performance, or platform reasons. A learner does not benefit from picking between `VARCHAR(255)`, `TEXT`, `CHAR(40)`, and `CLOB`. We control the user-facing surface and can present a small, semantically clear set of types that maps cleanly to the chosen backend (SQLite STRICT, ADR-0002). We also want to teach two distinct lessons about identifiers: 1. The default, easiest path: a simple auto-incrementing integer primary key. Used in 90% of intro examples. 2. Why integers aren't always the right answer: short random identifiers that survive merging data sets, sharing, or migration without collisions. Real UUIDs (36 characters) are too wide to display comfortably in TUI columns and exceed what learners actually need to understand the concept. ## Decision The user-facing column type vocabulary is: | User-facing type | SQLite STRICT mapping | Notes | |------------------|-----------------------|-------------------------------------------| | `text` | `TEXT` | Strings of any length. | | `int` | `INTEGER` | Plain integer. | | `real` | `REAL` | IEEE-754 double. | | `decimal` | `TEXT` | Stored as decimal string; rendered numeric. | | `bool` | `INTEGER` | 0/1 internally; `true`/`false` rendered. | | `date` | `TEXT` | ISO 8601 (`YYYY-MM-DD`). | | `datetime` | `TEXT` | ISO 8601 (`YYYY-MM-DDTHH:MM:SS[.fff][Z]`).| | `blob` | `BLOB` | Binary data. | | `serial` | `INTEGER PK AUTOINC.` | Auto-incrementing integer; PK by default. | | `shortid` | `TEXT` | 10–12 char base58 random; PK by default. | `shortid` uses base58 (no ambiguous `0`/`O`/`I`/`l`) and is generated client-side at insert time when the column has no value supplied. Decimal is stored as text to preserve precision — applications that need numeric comparison must use the engine's casts; this is acceptable for a teaching context and the constraint is documented. **Compound primary keys are supported.** They are essential for junction tables in m:n relationships (e.g. `OrderLines` keyed on `(order_id, product_id)`) and skipping them would teach the wrong lesson. The simplified DSL provides natural syntax for them (specifics in a later ADR). True UUIDs are intentionally **not** in the type set. ## Consequences - The type system is small enough to teach in five minutes. - Mapping to SQLite STRICT is mechanical and lossless for the intended use cases. - The shortid generator is a small, well-tested utility — bounded scope, no third-party dependency required. - Junction tables and other compound-key scenarios are first-class, reinforcing relational fundamentals. - Learners who later need a true UUID column will find that the app does not provide one; this is a deliberate trade-off in favour of TUI legibility. ## Amendment 1 — display rounding of coerced doubles (2026-06-12) Issue #32. The Decision keeps `decimal` exact by storing it as TEXT, noting that "numeric ops require casts" — the engine has no native decimal/BCD type (SQLite's storage classes are only NULL / INTEGER / REAL / TEXT / BLOB; `NUMERIC` is an affinity, not a type). What the original wording did not anticipate is that the engine performs that cast **implicitly**: `sum(price * qty)` over TEXT decimals coerces to an IEEE-754 double with no explicit cast, and the computed result carries no playground type (ADR-0030 §6), so it rendered with the double's full noise — `298.59999999999997` for `298.60`. For a teaching tool that is a confusing, off-topic lesson about float representation. ### Decision **Round floating-point values to 15 significant figures for display only.** A double carries ~15–17 significant decimal digits and the noise lives in the last one or two; rounding to 15 then taking the shortest round-tripping form of the rounded value collapses `298.59999999999997` → `298.6` and `0.30000000000000004` → `0.3`. A clean value rounds to itself, so the result is never longer than before; non-finite values pass through. Implemented as `format_real_display` in `db.rs`. The rounding is wired into **exactly one place — `format_cell`, the result-set / `show data` cell formatter** — because that is the only surface where the IEEE-754 noise actually appears: noise arises from *arithmetic/aggregation*, whose results flow through `format_cell`. Every other `f64`-to-string path deliberately keeps full precision, and the distinction is **semantic, not cosmetic**: - **Persistence stays exact.** The CSV encoder (`persistence::csv_io::format_real`) keeps the shortest round-tripping form so a stored `real` survives save/load byte-for-byte — rounding there would corrupt data. - **Uniqueness dry-runs key on exact values.** `render_value` (the diagnostic/echo formatter) is reused as a *canonical identity key* by `dry_run_unique` (ADR-0029 §5) and `check_uniqueness_collisions` (ADR-0017 §4.3): they group rows by this string to predict the duplicates the engine would reject. Rounding there would merge two distinct doubles into one key and report a collision the engine — which compares exact values — would not. So `render_value` keeps `format!("{r}")`. (It also never displays a *computed* value, so it has no noise to trim.) - **FK-key matching and EXPLAIN-SQL literals keep full precision** — neither is a data-cell display. Within `format_cell` the rounding applies to **all** REAL cells (stored `real` columns and computed results alike), for one consistent rule; the lost digits are at the double's precision limit, not real information, and a stored `real` typed by the user is itself noise-free so its display is unchanged in practice. Raw `decimal` columns are unaffected — they are TEXT and render verbatim, trailing zeros and all (`100.10`). Exact decimal *arithmetic* (a SQLite extension exposing `decimal_mul`/`decimal_sum`) was considered and rejected: it would require rewriting the user's standard-SQL operators into function calls, defeating both the "validated SQL runs verbatim" model and the goal of teaching ordinary SQL. ## Amendment 2 — drop `blob` from the vocabulary (2026-06-22) **This supersedes the `blob` row of the Decision table.** The type set is now **nine types**: `text`, `int`, `real`, `decimal`, `bool`, `date`, `datetime`, `serial`, `shortid`. ### Why `blob` is removed The original Decision listed `blob` purely to mirror the engine's storage classes (its only note was "Binary data."). Unlike the two other vocabulary judgements made here — compound PKs *defended* ("skipping them would teach the wrong lesson") and true UUIDs *excluded* ("in favour of TUI legibility") — `blob` never received the same pedagogical scrutiny. It rode along as the fourth storage class and was never re-examined. In practice `blob` is a **dead-end type** in this application: - **No value can be entered for it through the command surface.** There is no blob literal in either mode — neither the simplified syntax nor advanced SQL has one (ADR-0014: "DSL literal not supported this iteration"; ADR-0038: "blob has no literal … neither the DSL nor advanced SQL has a blob-literal syntax"). The binder refuses outright (`src/dsl/value.rs`). - **`seed` cannot generate it** (ADR-0048: `blob → unsupported`), and a `NOT NULL blob` column is a hard refusal. - So a blob column can only ever hold NULL via normal use. The only way bytes get in at all is hand-editing the base64 cell in a CSV — not something the app teaches or surfaces. The missing literal was filed as a perpetual "pre-existing gap" across many handoffs and never actioned. The right resolution is not to build a blob-entry affordance — **there is nothing natural to *do* with binary data in a terminal teaching tool, and nothing about the `blob` type that serves this app's learning goals** (pedagogy wins ties; this app is not the place to teach storage-class breadth). Removing it makes the type set smaller (still "teachable in five minutes", now with one less dead-end), removes a standing copy-rule bug (the binder's refusal message named the internal "DSL" term — flagged 2026-06-22), and simplifies the type-change matrix (ADR-0017's `↔ blob` static refusals vanish) and the seed generator. ### What this changes - `Type::Blob` and its keyword/aliases (`blob`, `binary`, `varbinary`) are removed. Because the variant set is the single source of truth, this clears both parse paths at once — `from_sql_name` (the alias-aware command path) *and* `FromStr`/`parse::()` (the keyword-only path the `.db` metadata read uses) — once the explicit `blob` arms are deleted (compiler-guided). The type slot offered by completion and the advanced-SQL grammar (`BLOB_SLOT`) loses `blob`; `output_render`/`describe` lose the `BLOB` rendering. The data-layer value plumbing it fed (`Value::Blob` / `CellValue::Blob(Vec)` and the base64 CSV encode/decode in `persistence::csv_io`) is removed with it. - **User-facing strings** lose their blob entries: the `help types` reference (catalog `help.types_reference`) drops `blob` from the ten-name list, and the blob value-slot hint ("Type a quoted blob literal or null", `hint.value_slot_blob`) — plus its `keys.rs` declaration and the walker mapping — is removed. The binder's blob refusal in `src/dsl/value.rs` (whose message named the internal "DSL" term — the copy-rule bug that surfaced this whole question) is deleted outright, so that fix needs no separate rewording. - The `base64` dependency **stays** — `clipboard.rs` uses it for OSC-52, independent of `blob`. - ADR-0011 (FK target-type compatibility) and ADR-0017 (type-change compatibility) lose their `blob` rows; ADR-0014/0038/0048's blob-literal/seed notes become moot. Prose that says "ten-type" (ADR-0030/0033/0035, `CLAUDE.md`, the website type reference) becomes "nine-type". ### Backward compatibility — a project migration `blob` was a creatable column type, so a saved project may name it. The type appears in **two artifacts**, and the migration must reconcile both: 1. **`project.yaml`** (authoritative) — a column declared `type: blob`. On load this is parsed by `from_sql_name`, which returns `None` once the type is gone → the project would fail to open. 2. **The derived `playground.db`** (ADR-0004/0015) — which carries the type *twice over*: the user-facing string `"blob"` in the `__rdbms_playground_columns` metadata table, **and** an actual `STRICT … BLOB` column in the SQLite schema. Crucially, load uses an existing `.db` **as-is** — `rebuild_from_text` runs only when the `.db` is *missing* (`runtime.rs`, ADR-0015 §7). Worse, the metadata read parses the stored type with `parse::().ok()` and falls back to `Type::Text` on failure — so a stale blob `.db` would *not* error; it would silently read the column as `text` while the engine column stays `BLOB`: an inconsistent, broken state. **So a YAML-only type-swap is insufficient.** The migration is: 1. **Migrate the YAML** via the ADR-0015 framework (`src/persistence/migrations.rs`): register the first real migrator (v1 → v2) that rewrites every `type: blob` to `type: text`. This is the framework's first production use — note its by-design consequence: bumping `latest_version` to 2 re-versions **every** project on next load (a `project.yaml.v1.bak` is written, the body rewritten) — harmless for blob-free projects (a pure version bump), but it does touch every file once. 2. **Force a rebuild of the `.db` from the migrated YAML** whenever the migration actually changed a column type (i.e. a `blob` was present) — reusing the existing `rebuild_from_text` path so the engine schema and metadata are regenerated as `text`, not left stale. A pure version bump with no schema change does **not** force a rebuild. **Conversion target: `text`** (decided 2026-06-22). Non-destructive — the CSV already stores blob cells as base64 *text*, so any bytes survive as a recoverable string, and an empty column (the overwhelmingly common case) simply becomes an empty `text` column. It is constraint-safe: `text` is a valid primary-key and FK-target type (`fk_target_type` is identity for both), so a former-blob PK or FK target stays valid. And it keeps the migration **CSV-free**: the column keeps its name and position, so every CSV header still matches the rebuilt schema (the rebuild's strict header==schema check, `do_rebuild_from_text`) and the existing cells decode as `text`. The considered alternatives were rejected: **drop the column** is cleaner semantically (there is no legitimately-entered content to keep) but the strict header check means it would force the migration to *also* rewrite every affected `data/.csv` to strip the column — not worth the machinery for a type that has existed unusably for a few days with very few users; and **refuse to load** strands the project. The rebuild-on-migration step (above) is required regardless of target, because the stale `.db` keeps a `STRICT … BLOB` *engine* column that can neither hold text nor be reported as anything usable until regenerated. ### Consequences - A smaller, fully-usable type set: every remaining type can be both created *and* populated through the command surface. - The type-change matrix and `seed` shed their blob special-cases; the binder sheds its blob refusal (and its copy-rule violation). - The ADR-0015 migration framework gets its first real exercise, validating that path end-to-end. - A breaking change for the rare project carrying a `blob` column, mitigated to a graceful, non-destructive load by the migration. - Re-introducing `blob` later (should a concrete teaching use appear) is a new ADR superseding this amendment — the same bar every other type change clears.