rdbms-playground/docs/adr/0005-column-type-vocabulary.md

ADR-0005: Column type vocabulary

Status

Accepted

Context

Real RDBMS engines expose many type variants that exist for historical, performance, or platform reasons. A learner does not benefit from picking between VARCHAR(255), TEXT, CHAR(40), and CLOB. We control the user-facing surface and can present a small, semantically clear set of types that maps cleanly to the chosen backend (SQLite STRICT, ADR-0002).

We also want to teach two distinct lessons about identifiers:

1. The default, easiest path: a simple auto-incrementing integer primary key. Used in 90% of intro examples.
2. Why integers aren't always the right answer: short random identifiers that survive merging data sets, sharing, or migration without collisions.

Real UUIDs (36 characters) are too wide to display comfortably in TUI columns and exceed what learners actually need to understand the concept.

Decision

The user-facing column type vocabulary is:

User-facing type	SQLite STRICT mapping	Notes
text	TEXT	Strings of any length.
int	INTEGER	Plain integer.
real	REAL	IEEE-754 double.
decimal	TEXT	Stored as decimal string; rendered numeric.
bool	INTEGER	0/1 internally; true/false rendered.
date	TEXT	ISO 8601 (YYYY-MM-DD).
datetime	TEXT	ISO 8601 (YYYY-MM-DDTHH:MM:SS[.fff][Z]).
blob	BLOB	Binary data.
serial	INTEGER PK AUTOINC.	Auto-incrementing integer; PK by default.
shortid	TEXT	10–12 char base58 random; PK by default.

shortid uses base58 (no ambiguous 0/O/I/l) and is generated client-side at insert time when the column has no value supplied.

Decimal is stored as text to preserve precision — applications that need numeric comparison must use the engine's casts; this is acceptable for a teaching context and the constraint is documented.

Compound primary keys are supported. They are essential for junction tables in m:n relationships (e.g. OrderLines keyed on (order_id, product_id)) and skipping them would teach the wrong lesson. The simplified DSL provides natural syntax for them (specifics in a later ADR).

True UUIDs are intentionally not in the type set.

Consequences

• The type system is small enough to teach in five minutes.
• Mapping to SQLite STRICT is mechanical and lossless for the intended use cases.
• The shortid generator is a small, well-tested utility — bounded scope, no third-party dependency required.
• Junction tables and other compound-key scenarios are first-class, reinforcing relational fundamentals.
• Learners who later need a true UUID column will find that the app does not provide one; this is a deliberate trade-off in favour of TUI legibility.

Amendment 1 — display rounding of coerced doubles (2026-06-12)

Issue #32. The Decision keeps decimal exact by storing it as TEXT, noting that "numeric ops require casts" — the engine has no native decimal/BCD type (SQLite's storage classes are only NULL / INTEGER / REAL / TEXT / BLOB; NUMERIC is an affinity, not a type). What the original wording did not anticipate is that the engine performs that cast implicitly: sum(price * qty) over TEXT decimals coerces to an IEEE-754 double with no explicit cast, and the computed result carries no playground type (ADR-0030 §6), so it rendered with the double's full noise — 298.59999999999997 for 298.60. For a teaching tool that is a confusing, off-topic lesson about float representation.

Decision

Round floating-point values to 15 significant figures for display only. A double carries ~15–17 significant decimal digits and the noise lives in the last one or two; rounding to 15 then taking the shortest round-tripping form of the rounded value collapses 298.59999999999997 → 298.6 and 0.30000000000000004 → 0.3. A clean value rounds to itself, so the result is never longer than before; non-finite values pass through. Implemented as format_real_display in db.rs.

The rounding is wired into exactly one place — format_cell, the result-set / show data cell formatter — because that is the only surface where the IEEE-754 noise actually appears: noise arises from arithmetic/aggregation, whose results flow through format_cell. Every other f64-to-string path deliberately keeps full precision, and the distinction is semantic, not cosmetic:

• Persistence stays exact. The CSV encoder (persistence::csv_io::format_real) keeps the shortest round-tripping form so a stored real survives save/load byte-for-byte — rounding there would corrupt data.
• Uniqueness dry-runs key on exact values. render_value (the diagnostic/echo formatter) is reused as a canonical identity key by dry_run_unique (ADR-0029 §5) and check_uniqueness_collisions (ADR-0017 §4.3): they group rows by this string to predict the duplicates the engine would reject. Rounding there would merge two distinct doubles into one key and report a collision the engine — which compares exact values — would not. So render_value keeps format!("{r}"). (It also never displays a computed value, so it has no noise to trim.)
• FK-key matching and EXPLAIN-SQL literals keep full precision — neither is a data-cell display.

Within format_cell the rounding applies to all REAL cells (stored real columns and computed results alike), for one consistent rule; the lost digits are at the double's precision limit, not real information, and a stored real typed by the user is itself noise-free so its display is unchanged in practice. Raw decimal columns are unaffected — they are TEXT and render verbatim, trailing zeros and all (100.10). Exact decimal arithmetic (a SQLite extension exposing decimal_mul/decimal_sum) was considered and rejected: it would require rewriting the user's standard-SQL operators into function calls, defeating both the "validated SQL runs verbatim" model and the goal of teaching ordinary SQL.

Amendment 2 — drop blob from the vocabulary (2026-06-22)

This supersedes the blob row of the Decision table. The type set is now nine types: text, int, real, decimal, bool, date, datetime, serial, shortid.

Why blob is removed

The original Decision listed blob purely to mirror the engine's storage classes (its only note was "Binary data."). Unlike the two other vocabulary judgements made here — compound PKs defended ("skipping them would teach the wrong lesson") and true UUIDs excluded ("in favour of TUI legibility") — blob never received the same pedagogical scrutiny. It rode along as the fourth storage class and was never re-examined.

In practice blob is a dead-end type in this application:

• No value can be entered for it through the command surface. There is no blob literal in either mode — neither the simplified syntax nor advanced SQL has one (ADR-0014: "DSL literal not supported this iteration"; ADR-0038: "blob has no literal … neither the DSL nor advanced SQL has a blob-literal syntax"). The binder refuses outright (src/dsl/value.rs).
• seed cannot generate it (ADR-0048: blob → unsupported), and a NOT NULL blob column is a hard refusal.
• So a blob column can only ever hold NULL via normal use. The only way bytes get in at all is hand-editing the base64 cell in a CSV — not something the app teaches or surfaces.

The missing literal was filed as a perpetual "pre-existing gap" across many handoffs and never actioned. The right resolution is not to build a blob-entry affordance — there is nothing natural to do with binary data in a terminal teaching tool, and nothing about the blob type that serves this app's learning goals (pedagogy wins ties; this app is not the place to teach storage-class breadth). Removing it makes the type set smaller (still "teachable in five minutes", now with one less dead-end), removes a standing copy-rule bug (the binder's refusal message named the internal "DSL" term — flagged 2026-06-22), and simplifies the type-change matrix (ADR-0017's ↔ blob static refusals vanish) and the seed generator.

What this changes

• Type::Blob and its keyword/aliases (blob, binary, varbinary) are removed. Because the variant set is the single source of truth, this clears both parse paths at once — from_sql_name (the alias-aware command path) and FromStr/parse::<Type>() (the keyword-only path the .db metadata read uses) — once the explicit blob arms are deleted (compiler-guided). The type slot offered by completion and the advanced-SQL grammar (BLOB_SLOT) loses blob; output_render/describe lose the BLOB rendering. The data-layer value plumbing it fed (Value::Blob / CellValue::Blob(Vec<u8>) and the base64 CSV encode/decode in persistence::csv_io) is removed with it.
• User-facing strings lose their blob entries: the help types reference (catalog help.types_reference) drops blob from the ten-name list, and the blob value-slot hint ("Type a quoted blob literal or null", hint.value_slot_blob) — plus its keys.rs declaration and the walker mapping — is removed. The binder's blob refusal in src/dsl/value.rs (whose message named the internal "DSL" term — the copy-rule bug that surfaced this whole question) is deleted outright, so that fix needs no separate rewording.
• The base64 dependency stays — clipboard.rs uses it for OSC-52, independent of blob.
• ADR-0011 (FK target-type compatibility) and ADR-0017 (type-change compatibility) lose their blob rows; ADR-0014/0038/0048's blob-literal/seed notes become moot. Prose that says "ten-type" (ADR-0030/0033/0035, CLAUDE.md, the website type reference) becomes "nine-type".

Backward compatibility — a project migration

blob was a creatable column type, so a saved project may name it. The type appears in two artifacts, and the migration must reconcile both:

1. project.yaml (authoritative) — a column declared type: blob. On load this is parsed by from_sql_name, which returns None once the type is gone → the project would fail to open.
2. The derived playground.db (ADR-0004/0015) — which carries the type twice over: the user-facing string "blob" in the __rdbms_playground_columns metadata table, and an actual STRICT … BLOB column in the SQLite schema. Crucially, load uses an existing .db as-is — rebuild_from_text runs only when the .db is missing (runtime.rs, ADR-0015 §7). Worse, the metadata read parses the stored type with parse::<Type>().ok() and falls back to Type::Text on failure — so a stale blob .db would not error; it would silently read the column as text while the engine column stays BLOB: an inconsistent, broken state.

So a YAML-only type-swap is insufficient. The migration is:

1. Migrate the YAML via the ADR-0015 framework (src/persistence/migrations.rs): register the first real migrator (v1 → v2) that rewrites every type: blob to type: text. This is the framework's first production use — note its by-design consequence: bumping latest_version to 2 re-versions every project on next load (a project.yaml.v1.bak is written, the body rewritten) — harmless for blob-free projects (a pure version bump), but it does touch every file once.
2. Force a rebuild of the .db from the migrated YAML whenever the migration actually changed a column type (i.e. a blob was present) — reusing the existing rebuild_from_text path so the engine schema and metadata are regenerated as text, not left stale. A pure version bump with no schema change does not force a rebuild.

Conversion target: text (decided 2026-06-22). Non-destructive — the CSV already stores blob cells as base64 text, so any bytes survive as a recoverable string, and an empty column (the overwhelmingly common case) simply becomes an empty text column. It is constraint-safe: text is a valid primary-key and FK-target type (fk_target_type is identity for both), so a former-blob PK or FK target stays valid. And it keeps the migration CSV-free: the column keeps its name and position, so every CSV header still matches the rebuilt schema (the rebuild's strict header==schema check, do_rebuild_from_text) and the existing cells decode as text.

The considered alternatives were rejected: drop the column is cleaner semantically (there is no legitimately-entered content to keep) but the strict header check means it would force the migration to also rewrite every affected data/<table>.csv to strip the column — not worth the machinery for a type that has existed unusably for a few days with very few users; and refuse to load strands the project. The rebuild-on-migration step (above) is required regardless of target, because the stale .db keeps a STRICT … BLOB engine column that can neither hold text nor be reported as anything usable until regenerated.

Consequences

• A smaller, fully-usable type set: every remaining type can be both created and populated through the command surface.
• The type-change matrix and seed shed their blob special-cases; the binder sheds its blob refusal (and its copy-rule violation).
• The ADR-0015 migration framework gets its first real exercise, validating that path end-to-end.
• A breaking change for the rare project carrying a blob column, mitigated to a graceful, non-destructive load by the migration.
• Re-introducing blob later (should a concrete teaching use appear) is a new ADR superseding this amendment — the same bar every other type change clears.