From 6b4c4dcea43cc40a82bbafde99f55f954eda8b8c Mon Sep 17 00:00:00 2001 From: "claude@clouddev1" Date: Mon, 22 Jun 2026 21:25:38 +0000 Subject: [PATCH] feat(types)!: drop the blob column type (ADR-0005 Amendment 2) blob was a dead-end: declarable but never fillable (no literal in either mode, seed-unsupported), so a blob column could only ever hold NULL. Remove Type::Blob + Value/CellValue::Blob + the base64 CSV path + the grammar slot + completion/render/type-change/seed handling + the binder refusal (whose message also carried a user-facing "DSL" copy-rule bug). The vocabulary is now nine types; base64 stays (clipboard OSC-52). Backward compat is the ADR-0015 migration framework's first real use: a v1->v2 format bump (CURRENT_SCHEMA_VERSION across serializer/parser/skeleton) with a migrator that rewrites `type: blob` -> `type: text`, and a forced .db rebuild from the migrated text when a blob column was actually converted (the stale .db keeps a STRICT BLOB engine column). Conversion to text is non-destructive and CSV-free. Covered by a full-stack integration test and a Tier-4 PTY test that opens a real legacy v1-blob project. Also sweeps the nine-type vocabulary through CLAUDE.md, requirements.md, the website (type reference, seed doc, highlight grammar), and the ADR-0030/0033/ 0035 cross-references; CHANGELOG Removed entry; handoff-79. BREAKING CHANGE: the `blob` column type is removed. Existing projects that declared a blob column are migrated on first open (the column becomes text; the original project.yaml is kept as a .v1.bak). --- CHANGELOG.md | 8 ++ CLAUDE.md | 4 +- docs/adr/0030-advanced-mode-sql-surface.md | 4 +- docs/adr/0033-sql-dml-grammar.md | 2 +- docs/adr/0035-advanced-mode-sql-ddl.md | 6 +- docs/handoff/20260622-handoff-79.md | 107 ++++++++++++++++ docs/requirements.md | 11 +- src/completion.rs | 5 +- src/db.rs | 60 +-------- src/dsl/grammar/shared.rs | 8 +- src/dsl/grammar/sql_create_table.rs | 5 +- src/dsl/types.rs | 48 +++---- src/dsl/value.rs | 11 -- src/dsl/walker/mod.rs | 2 - src/friendly/keys.rs | 1 - src/friendly/strings/en-US.yaml | 3 +- src/output_render.rs | 4 +- src/persistence/csv_io.rs | 29 ----- src/persistence/migrations.rs | 120 +++++++++++++++--- src/persistence/mod.rs | 3 +- src/persistence/yaml.rs | 36 ++++-- src/project/mod.rs | 7 +- src/runtime.rs | 28 +++- src/seed/generators.rs | 18 +-- src/type_change.rs | 25 +--- tests/e2e_pty.rs | 47 +++++++ tests/it/blob_removal_migration.rs | 107 ++++++++++++++++ tests/it/main.rs | 1 + tests/it/seed.rs | 63 --------- tests/it/sql_alter_table.rs | 8 -- tests/it/sql_dml_e2e.rs | 25 ++-- tests/it/sql_select.rs | 35 +---- tests/typing_surface/mod.rs | 1 - ...n_expects_type_candidates@after_paren.snap | 11 +- ...umn_says_yyyy_mm_dd_t@datetime_column.snap | 6 +- ...pe_candidates@change_after_open_paren.snap | 11 +- ...tween_expects_a_low_bound@between_low.snap | 6 +- ...n_paren_expects_an_item@in_open_paren.snap | 6 +- .../docs/reference/generating-sample-data.mdx | 3 - website/src/content/docs/reference/types.md | 9 +- website/src/grammars/rdbms.mjs | 2 +- 41 files changed, 506 insertions(+), 390 deletions(-) create mode 100644 docs/handoff/20260622-handoff-79.md create mode 100644 tests/it/blob_removal_migration.rs diff --git a/CHANGELOG.md b/CHANGELOG.md index 63ec520..5b7d4b5 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -21,6 +21,14 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 foreground/background contrast and its syntax-colour distinctness verified on each build. +### Removed +- The `blob` column type. It could be declared but never given a value + (there was no way to type a binary value in either mode), so it was a + dead end with nothing to teach. The vocabulary is now nine types. + Existing projects that used a `blob` column still open — the column is + converted to `text` on first load (a backup of the original project file + is kept alongside it). + ### Fixed - Pasting or scripting several commands at once no longer occasionally rejects a valid simple-mode `insert` — submitted right after adding a diff --git a/CLAUDE.md b/CLAUDE.md index 233cd79..4be7430 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -58,8 +58,8 @@ Current decisions at a glance (each backed by an ADR): `[temp]`-marker / contents-allowlist guards. Anything unexpected → refuse, never delete the wrong thing. - **Types:** `text`, `int`, `real`, `decimal`, `bool`, `date`, - `datetime`, `blob`, `serial`, `shortid`. Compound primary keys - supported. No real UUIDs (ADR-0005). FK column type + `datetime`, `serial`, `shortid`. Compound primary keys + supported. No real UUIDs; `blob` dropped (ADR-0005 Amendment 2). FK column type compatibility via `Type::fk_target_type()` — `serial → int`, `shortid → text`, others identity (ADR-0011). - **Safety:** append-only `history.log` for replay and scripting diff --git a/docs/adr/0030-advanced-mode-sql-surface.md b/docs/adr/0030-advanced-mode-sql-surface.md index 98b2c61..8928cb8 100644 --- a/docs/adr/0030-advanced-mode-sql-surface.md +++ b/docs/adr/0030-advanced-mode-sql-surface.md @@ -172,7 +172,7 @@ representation; the SQL surface does not round-trip through it. ### 5. Type vocabulary — the playground's, not the engine's -Advanced-mode DDL uses the playground's own ten-type +Advanced-mode DDL uses the playground's own nine-type vocabulary (ADR-0005). There is **no fallback to engine storage types**: a column created in advanced mode is a first-class `serial` / `decimal` / `date` / … exactly as a @@ -373,7 +373,7 @@ ADR when taken up (ADR-0026-style). - ADR-0002 — the engine is an implementation detail; "no engine name in user-facing strings" — §7 extends it. - ADR-0003 — the simple / advanced mode model this builds on. -- ADR-0005 — the ten-type vocabulary advanced DDL uses (§5). +- ADR-0005 — the nine-type vocabulary advanced DDL uses (§5). - ADR-0009 — the DSL conventions; the DSL stays usable in advanced mode. - ADR-0012 / ADR-0013 — the metadata tables the `Command` core diff --git a/docs/adr/0033-sql-dml-grammar.md b/docs/adr/0033-sql-dml-grammar.md index 73c973b..b629a4b 100644 --- a/docs/adr/0033-sql-dml-grammar.md +++ b/docs/adr/0033-sql-dml-grammar.md @@ -1649,7 +1649,7 @@ more. ## See also -- ADR-0005 — the ten-type vocabulary INSERT works with. +- ADR-0005 — the nine-type vocabulary INSERT works with. - ADR-0006 — the auto-snapshot before destructive ops. - ADR-0014 — the DSL DML model + cascade-summary precedent. - ADR-0015 — the persistence write-through path. diff --git a/docs/adr/0035-advanced-mode-sql-ddl.md b/docs/adr/0035-advanced-mode-sql-ddl.md index 4dcea1f..65b1796 100644 --- a/docs/adr/0035-advanced-mode-sql-ddl.md +++ b/docs/adr/0035-advanced-mode-sql-ddl.md @@ -58,7 +58,7 @@ Two things from the earlier phases shape this one: executed as-is, the engine would make the table, but the playground would lose what the user meant: that `id` is `serial`, that a `REFERENCES` clause is a *named relationship*, that `STRICT` - applies, that the ten-type vocabulary governs. Recovering that + applies, that the nine-type vocabulary governs. Recovering that needs the parsed statement either way. ADR-0030 §4 said "DDL → a `Command` … run the typed executor." That @@ -92,7 +92,7 @@ mode. Unlike the DML `Sql*` commands they **execute structurally**: the handler reads the parsed structure and performs the schema change through the playground's metadata-maintaining machinery — writing `__rdbms_playground_columns` / `__rdbms_playground_relationships`, -applying `STRICT`, using the ten-type vocabulary — so an +applying `STRICT`, using the nine-type vocabulary — so an advanced-mode-created object is a first-class playground object, identical to a simple-mode-created one (ADR-0030 §5). @@ -772,5 +772,5 @@ honouring it is not the lesson. - **ADR-0017** — the column type-change classification §7 shares. - **ADR-0029** — column constraints; **ADR-0025** — indexes; **ADR-0011** — FK column-type compatibility; **ADR-0005** — the - ten-type vocabulary. + nine-type vocabulary. - **ADR-0006** — undo; each DDL statement is one undo step (§10). diff --git a/docs/handoff/20260622-handoff-79.md b/docs/handoff/20260622-handoff-79.md new file mode 100644 index 0000000..5ab85c3 --- /dev/null +++ b/docs/handoff/20260622-handoff-79.md @@ -0,0 +1,107 @@ +# Session handoff — 2026-06-22 (79) + +A long session continuing the open-issue sweep (handoff-77/78). Cleared the +**help** issue (#36) and the **schema-cache** bug (#39) earlier; this note +mainly records the big piece: **dropping the `blob` column type** (ADR-0005 +Amendment 2), which grew out of a one-line copy-rule fix. + +## §1. State + +**Branch `main`.** Commits this session so far (all on `main`, **not pushed**): +`07575da` (#39 gate) · `e88fa79` (handoff-77 + changelog rule) · `3ad4aff` +(#36 help) · `64818c0` (handoff-78) · `1a2002d` (**ADR-0005 Amendment 2** — +the blob-drop decision record). + +**Uncommitted: the blob-removal implementation** (39 files) — staged for review, +commit proposed at session end (see §4). **Full suite green: 2520 passed / 0 +failed / 1 ignored** (1810 lib + 7 e2e_pty + 503 it + 200 typing). +`clippy --all-targets -D warnings` + `fmt --check` clean. + +**Closed this session:** #36, #39. **Open:** #37 (clause hints), #38 +(diagnostic route — needs a do/defer/close call), #40 (winget release notes). + +## §2. What shipped — drop `blob` (ADR-0005 Amendment 2) + +**Why** (recorded in the ADR): `blob` was a dead-end type — declarable but +never fillable (no literal in either mode, `seed`-unsupported), so a blob +column could only ever hold NULL. It rode along as the engine's fourth +storage class and never got the pedagogical scrutiny UUIDs (excluded) and +compound PKs (defended) did. The vocabulary is now **nine types**. + +**Discovered via:** the `value.rs:106` binder message *"…not supported in DSL +yet"* — a user-facing copy-rule violation (it `passthrough`es to the user via +`DbError::InvalidValue`). Rather than reword, we removed the type; the message +is deleted outright. + +**Implementation (test-first throughout):** +- **Removal** (compiler-guided): `Type::Blob` + its `keyword`/`sqlite`/`all()`/ + `from_sql_name` alias arms; `Value::Blob`; `CellValue::Blob(Vec)` + the + base64 CSV encode/decode; the `BLOB_SLOT` grammar slot; completion's `blob` + candidate; `output_render`'s `BLOB`; the type-change `↔ blob` static refusal; + the seed blob arms; the binder refusal. User-facing strings: dropped `blob` + from `help.types_reference` and removed the `hint.value_slot_blob` slot hint. + `base64` **stays** (clipboard OSC-52). +- **Backward-compat migration** (the ADR-0015 framework's **first real use**): + a **v1→v2 format bump**. `parse_schema` hard-pinned the version to 1, so this + rippled into a `CURRENT_SCHEMA_VERSION = 2` constant used by the serializer + + parser + skeleton, the first registered migrator (`migrate_v1_to_v2`: + rewrites `type: blob`→`type: text` + bumps the version), and ~30 `version: 1` + test fixtures. The runtime (already wired to `ensure_project_yaml_migrated`) + **forces a `.db` rebuild** from the migrated text **only when a blob column + was actually converted** (`body_declares_blob_column` pre-read at both open + sites) — the stale `.db` keeps a `STRICT … BLOB` engine column + `"blob"` + metadata it would otherwise use as-is. Conversion target **text** (not drop) + was the user's call (non-destructive, CSV-free, PK/FK-safe). +- **Tests:** migrator unit tests (incl. a column *named* `blob` + a `blob` + substring in a CHECK left untouched); a **full-stack** integration test + `tests/it/blob_removal_migration.rs` (v1 blob project → migrate → `text` + column + row data preserved + `.bak`); a **Tier-4 PTY** end-to-end test + `e2e_pty::opens_a_legacy_v1_blob_project_by_migrating_to_text` (the real + binary opens a seeded legacy v1-blob project via `--resume` → migrate + + rebuild + `text` column) — added in the implementation `/runda` pass to + cover the runtime's migrate-on-open glue; updated all-types tests to nine; + regenerated 5 typing-surface snapshots (reviewed — only the `blob`/`data` + candidate vanished). +- **Docs:** ADR-0005 Amendment 2 (committed `1a2002d`) + README index; swept + `ten-type`→`nine-type` / removed `blob` from CLAUDE.md, `requirements.md`, + the website (`reference/types.md`, the seed doc's stale `not null blob` + bullet, the `rdbms.mjs` highlight grammar), and the cross-reference counts in + ADR-0030/0033/0035. CHANGELOG `[Unreleased] → Removed`. + +## §3. Decisions & notes for next time + +- **convert-to-text, not drop-the-column** (user, 2026-06-22): drop would force + rewriting every `data/*.csv` past the rebuild's strict header check — not + worth it for a few-days-old, few-users tool. +- **Full v2 framework migration, not parse-time leniency** (user): the proper + recorded migration with `.bak`, exercising the dormant framework. +- **`BlobLit` grammar terminal deliberately left** — a pre-existing, + `#[allow(dead_code)]` speculative token never used by any grammar; orthogonal + to the type vocabulary. Its "blob literal" string is unreachable. Removing it + is a separate optional cleanup (touches the grammar core). +- **Known assumption:** the migrator + `body_declares_blob_column` match the + serializer's *inline* column format (`- { name: …, type: … }`). A + hand-written *block-style* `project.yaml` with `type: blob` wouldn't convert + (and would then fail to parse). `project.yaml` is always machine-serialized + inline, so this is fine in practice — noted in case a future format change + moves away from inline columns. +- **Other ADRs (0030/0035) keep their historical `blob` content** (type-change + matrix, `binary`/`varbinary` aliases) per the project's supersede-don't-rewrite + ADR philosophy; ADR-0005 Amendment 2 is the authoritative record. + +## §4. Process pins / next + +- Commits user-confirmed, no AI attribution, append-only, on `main`; **push is + the user's step** (everything this session is unpushed). +- **`/runda` was run on both the ADR (design) and the implementation.** The + implementation pass found one real gap — the runtime's migrate-on-open glue + was untested — closed with the new Tier-4 PTY test. It also verified the + import path migrates (extract→migrate→rebuild) and the non-migration + `parse_schema` callers fail-safe. +- Proposed commit split for the implementation: (1) `feat(types)!` — code + + tests + CHANGELOG; (2) `docs` — the CLAUDE.md/requirements/website/ADR-x-ref + sweep; (3) `docs(handoff-79)`. +- Consider a `cargo sweep` at this milestone. +- **Next open issues:** #37 (clause hints, on-mission), #38 (escalate + do/defer/close first), #40 (winget release notes). Plus: blob removal means a + `cargo sweep`-worthy build and a good moment to push. diff --git a/docs/requirements.md b/docs/requirements.md index 13c79aa..9901560 100644 --- a/docs/requirements.md +++ b/docs/requirements.md @@ -404,7 +404,7 @@ since ADR-0027.) referenced column), then `ALTER TABLE … ALTER COLUMN TYPE` (4f — runtime-decomposed to `change_column_type` with `ForceConversion`, the §7 advanced policy: lossy converts with a note, incompatible + static - refusals (`↔ blob`, non-`int → serial`) refuse, `int → serial` allowed; + refusals (non-`int → serial`) refuse, `int → serial` allowed; the internal-`__rdbms_*` guard folded into `do_change_column_type`), then `ALTER TABLE` add/drop constraint + add FK (4g — `ADD [CONSTRAINT ] (CHECK | UNIQUE | FOREIGN KEY)` + `DROP CONSTRAINT `; @@ -430,7 +430,7 @@ since ADR-0027.) refused with an engine-neutral message naming the construct. Implementation pending.)* - [x] **Q3** User-facing simplified types map transparently to - SQLite STRICT types in generated DDL. *(All ten types implemented + SQLite STRICT types in generated DDL. *(All nine types implemented and tested.)* - [x] **Q4** Supported SQL subset specification — **ADR-0030**. Advanced mode is a standard-SQL surface, engine-neutral; the @@ -462,10 +462,11 @@ since ADR-0027.) ## Type system (per ADR-0005) -- [x] **T1** All ten user-facing types implemented: `text`, - `int`, `real`, `decimal`, `bool`, `date`, `datetime`, `blob`, +- [x] **T1** All nine user-facing types implemented: `text`, + `int`, `real`, `decimal`, `bool`, `date`, `datetime`, `serial`, `shortid`. *(Mapping to SQLite STRICT covered by - ADR-0005; FK target type rule by ADR-0011.)* + ADR-0005; FK target type rule by ADR-0011; `blob` dropped by + ADR-0005 Amendment 2.)* - [x] **T2** `shortid` generation: base58, 10–12 characters, omits ambiguous characters; generated client-side at insert. *(Implemented per ADR-0014; auto-fills omitted shortid diff --git a/src/completion.rs b/src/completion.rs index ad8eb42..f6a269d 100644 --- a/src/completion.rs +++ b/src/completion.rs @@ -631,7 +631,7 @@ pub fn candidates_at_cursor_with_in_mode( // Source 1.5: type-name candidates when the walker expects // a column-type slot. Type names are a closed set sourced // from `Type::all()` (ADR-0005 declaration order: - // text/int/real/decimal/bool/date/datetime/blob/serial/ + // text/int/real/decimal/bool/date/datetime/serial/ // shortid). The walker surfaces this as // `Expectation::Ident { source: Types }`. let type_names: Vec = if expected.iter().any(|e| { @@ -2309,7 +2309,7 @@ mod tests { #[test] fn type_slot_offers_full_type_vocabulary_when_partial_empty() { // After `add column to T: Name (` the parser expects - // a column type. With no partial typed, all ten types + // a column type. With no partial typed, all nine types // from `Type::all()` are offered in declaration order. let cs = cands("add column to T: Name (", 23); assert_eq!( @@ -2322,7 +2322,6 @@ mod tests { "bool".to_string(), "date".to_string(), "datetime".to_string(), - "blob".to_string(), "serial".to_string(), "shortid".to_string(), ], diff --git a/src/db.rs b/src/db.rs index a230b55..f3dcdb0 100644 --- a/src/db.rs +++ b/src/db.rs @@ -3169,7 +3169,10 @@ fn row_value_to_cell(row: &rusqlite::Row<'_>, idx: usize) -> Result CellValue::Integer(n), ValueRef::Real(f) => CellValue::Real(f), ValueRef::Text(bytes) => CellValue::Text(String::from_utf8_lossy(bytes).into_owned()), - ValueRef::Blob(bytes) => CellValue::Blob(bytes.to_vec()), + // `blob` was removed from the vocabulary (ADR-0005 Amendment 2), so no + // column is BLOB-typed any more and a blob value-ref is not expected. + // Surface any stray bytes as lossy text rather than panicking. + ValueRef::Blob(bytes) => CellValue::Text(String::from_utf8_lossy(bytes).into_owned()), }) } @@ -3627,11 +3630,6 @@ fn do_drop_table( /// rows receive a generated value (1..N for serial, fresh /// shortids for shortid) and the column gains UNIQUE + /// NOT NULL. -/// -/// `blob` is statically refused as a column-add target by the -/// existing infrastructure (no DSL literal, downstream INSERT -/// would fail), but the path itself is allowed via the plain -/// branch — same as today. fn do_add_column( conn: &Connection, persistence: Option<&Persistence>, @@ -5043,7 +5041,7 @@ fn do_rename_table( /// /// 1. Target is `serial` (auto-increment is create-table-only). /// 2. Source ↔ target is statically refused per the matrix -/// (same-type, blob, date↔datetime, undefined cross-domain). +/// (same-type, date↔datetime, undefined cross-domain). /// 3. Column is the *child* side of any relationship (outbound /// FK) — drop the relationship first. /// 4. Column has any inbound FK (parent side) and the new type @@ -8641,9 +8639,7 @@ fn sample_parent_key_tuples( /// /// Foreign-key columns are filled by sampling existing parent rows /// (D14); a compound FK reads all its child columns from one sampled -/// parent row. An empty parent is refused with a friendly error. A -/// `NOT NULL blob` column (which seed cannot generate) is refused by -/// the block guard (D1); a nullable blob is omitted (→ NULL). +/// parent row. An empty parent is refused with a friendly error. /// /// **Phase 2 (SD2):** when `target_column` is `Some`, this delegates to /// [`do_seed_column_fill`] (fill one column across existing rows, D1 @@ -8719,17 +8715,6 @@ fn do_seed( if matches!(ty, Type::Serial) { continue; } - // blob has no DSL value path: refuse if required (D1), else omit. - if matches!(ty, Type::Blob) { - if c.notnull { - return Err(DbError::Unsupported(format!( - "cannot seed `{table}`: column `{}` is `NOT NULL` but has type `blob`, \ - which seed cannot generate. Add the rows another way or make it nullable.", - c.name, - ))); - } - continue; - } col_names.push(c.name.clone()); if let Some(&(fk_idx, pos)) = fk_child_pos.get(c.name.as_str()) { plans.push(SeedColPlan::ForeignKey { fk_idx, pos }); @@ -9122,7 +9107,7 @@ fn seed_override_literal(value: &Value, column: &str) -> Result /// Column-fill (ADR-0048 D1 form 2): fill one column across the table's /// **existing** rows (an UPDATE), the natural follow-up to `add column`. /// -/// Refuses PK and auto-generated (`serial`/`shortid`/`blob`) targets; +/// Refuses PK and auto-generated (`serial`/`shortid`) targets; /// an empty table is a friendly no-op. The `set` clause may only adjust /// the column being filled (the rest of the per-column heuristics do not /// apply — there is exactly one column). A UNIQUE / identifier target @@ -9178,12 +9163,6 @@ fn do_seed_column_fill( ty.keyword(), ))); } - if matches!(ty, Type::Blob) { - return Err(DbError::Unsupported(format!( - "cannot fill `{table}.{canonical_col}`: seed cannot generate `blob` values." - ))); - } - // The `set` clause may only adjust the filled column (user decision). for ov in overrides { if !ov.column.eq_ignore_ascii_case(&canonical_col) { @@ -11230,7 +11209,6 @@ fn cell_value_to_sqlite(cell: &CellValue) -> rusqlite::types::Value { CellValue::Integer(n) => Value::Integer(*n), CellValue::Real(f) => Value::Real(*f), CellValue::Text(s) => Value::Text(s.clone()), - CellValue::Blob(b) => Value::Blob(b.clone()), } } @@ -12912,30 +12890,6 @@ mod tests { } } - #[tokio::test] - async fn change_column_type_blob_target_refused_statically() { - let db = db(); - make_id_table(&db, "T").await; - db.add_column( - "T".to_string(), - ColumnSpec::new("Note".to_string(), Type::Text), - None, - ) - .await - .unwrap(); - let err = db - .change_column_type( - "T".to_string(), - "Note".to_string(), - Type::Blob, - ChangeColumnMode::Default, - None, - ) - .await - .unwrap_err(); - assert!(matches!(err, DbError::Unsupported(_)), "got {err:?}"); - } - #[tokio::test] async fn change_column_type_outbound_fk_refused() { // Child-side FK column: §4.2 says always refuse, diff --git a/src/dsl/grammar/shared.rs b/src/dsl/grammar/shared.rs index 10fc41f..baf862a 100644 --- a/src/dsl/grammar/shared.rs +++ b/src/dsl/grammar/shared.rs @@ -263,11 +263,6 @@ const DATETIME_SLOT: Node = Node::TypedValueSlot { column_name: None, inner: &TEXT_SLOT_INNER, }; -const BLOB_SLOT: Node = Node::TypedValueSlot { - ty: Type::Blob, - column_name: None, - inner: &TEXT_SLOT_INNER, -}; // shortid columns store base58 text (ADR-0011 fk_target_type // shortid → text); the slot accepts a quoted-text literal or // null. @@ -297,7 +292,6 @@ pub const fn slot_for_type(ty: Type) -> Node { Type::Text => TEXT_SLOT, Type::Date => DATE_SLOT, Type::DateTime => DATETIME_SLOT, - Type::Blob => BLOB_SLOT, } } @@ -310,7 +304,7 @@ const fn slot_inner_for_type(ty: Type) -> &'static Node { Type::Real => &REAL_SLOT_INNER, Type::Decimal => &DECIMAL_SLOT_INNER, Type::Bool => &BOOL_SLOT_INNER, - Type::Text | Type::Date | Type::DateTime | Type::Blob | Type::ShortId => &TEXT_SLOT_INNER, + Type::Text | Type::Date | Type::DateTime | Type::ShortId => &TEXT_SLOT_INNER, } } diff --git a/src/dsl/grammar/sql_create_table.rs b/src/dsl/grammar/sql_create_table.rs index df6b7e5..8ae7d4e 100644 --- a/src/dsl/grammar/sql_create_table.rs +++ b/src/dsl/grammar/sql_create_table.rs @@ -567,7 +567,7 @@ mod tests { fn standard_sql_type_aliases() { good("table t (a integer, b varchar, c boolean, d timestamp)"); good("table t (e bigint, f smallint, g char, h numeric)"); - good("table t (i binary, j varbinary, k float)"); + good("table t (k float)"); } #[test] @@ -734,7 +734,7 @@ mod builder_tests { fn standard_sql_aliases_map_to_playground_types() { let (_, cols, _, _) = sct( "create table t (a bigint, b varchar, c boolean, d timestamp, \ - e numeric, f float, g binary)", + e numeric, f float)", ); assert_eq!( cols, @@ -745,7 +745,6 @@ mod builder_tests { ("d".to_string(), Type::DateTime), ("e".to_string(), Type::Decimal), ("f".to_string(), Type::Real), - ("g".to_string(), Type::Blob), ] ); } diff --git a/src/dsl/types.rs b/src/dsl/types.rs index 3168af7..856c5dd 100644 --- a/src/dsl/types.rs +++ b/src/dsl/types.rs @@ -1,7 +1,8 @@ //! User-facing column types and their mapping to SQLite STRICT. //! -//! Implements the full ten-type vocabulary committed to in -//! ADR-0005. Storage choices for the text-backed types +//! Implements the nine-type vocabulary committed to in ADR-0005 +//! (ten as originally decided; `blob` dropped by Amendment 2). +//! Storage choices for the text-backed types //! (`decimal`, `date`, `datetime`) preserve precision and ISO //! readability; comparisons rely on lexicographic ordering or //! explicit casts at query time, which is acceptable for a @@ -27,8 +28,6 @@ pub enum Type { Date, /// ISO 8601 datetime stored as `YYYY-MM-DDTHH:MM:SS[.fff][Z]` (TEXT). DateTime, - /// Arbitrary binary data. - Blob, /// Auto-incrementing integer; intended as a default primary key. Serial, /// 10–12 character base58 random identifier (no ambiguous chars). @@ -47,7 +46,6 @@ impl Type { Self::Bool => "bool", Self::Date => "date", Self::DateTime => "datetime", - Self::Blob => "blob", Self::Serial => "serial", Self::ShortId => "shortid", } @@ -62,7 +60,6 @@ impl Type { Self::Text | Self::ShortId | Self::Decimal | Self::Date | Self::DateTime => "TEXT", Self::Int | Self::Serial | Self::Bool => "INTEGER", Self::Real => "REAL", - Self::Blob => "BLOB", } } @@ -79,8 +76,8 @@ impl Type { /// All types known in this iteration, in stable order. /// Ordering groups numeric types together, then boolean, - /// then temporal, then binary, then identity-flavoured - /// auto-generated types. + /// then temporal, then identity-flavoured auto-generated + /// types. #[must_use] pub const fn all() -> &'static [Self] { &[ @@ -91,15 +88,14 @@ impl Type { Self::Bool, Self::Date, Self::DateTime, - Self::Blob, Self::Serial, Self::ShortId, ] } /// True for the numeric types — `int`, `real`, `decimal`, - /// `serial`. `bool` (stored 0/1) and the text- / blob-backed - /// types are not numeric. Used to flag a `LIKE` text-pattern + /// `serial`. `bool` (stored 0/1) and the text-backed types + /// are not numeric. Used to flag a `LIKE` text-pattern /// match against a numeric column (ADR-0027, Amendment 1). #[must_use] pub const fn is_numeric(self) -> bool { @@ -129,7 +125,7 @@ impl Type { } /// Resolve a type name from the **advanced-mode SQL** type slot - /// (ADR-0035 §3). Accepts the ten playground keywords *and* the + /// (ADR-0035 §3). Accepts the nine playground keywords *and* the /// standard-SQL aliases mapped onto them. Case-insensitive; /// internal whitespace is collapsed so `double precision` resolves /// regardless of spacing. Returns `None` for an unrecognised name — @@ -137,7 +133,7 @@ impl Type { /// diagnostic. /// /// Deliberately distinct from [`FromStr`](std::str::FromStr), which - /// is the *simple-mode* parser and accepts only the ten keywords + /// is the *simple-mode* parser and accepts only the nine keywords /// (no aliases), so simple mode teaches the playground's own /// vocabulary. A length/precision argument (`varchar(255)`) is /// stripped by the grammar before the name reaches this resolver. @@ -157,8 +153,7 @@ impl Type { "timestamp" => Some(Self::DateTime), "numeric" => Some(Self::Decimal), "float" | "double precision" => Some(Self::Real), - "binary" | "varbinary" => Some(Self::Blob), - // Fall through to the canonical ten keywords. + // Fall through to the canonical nine keywords. other => other.parse::().ok(), } } @@ -270,20 +265,19 @@ mod tests { Type::Bool, Type::Date, Type::DateTime, - Type::Blob, ] { assert_eq!(ty.fk_target_type(), ty); } } #[test] - fn all_ten_types_are_present_and_distinct() { + fn all_nine_types_are_present_and_distinct() { let kws: Vec<&'static str> = Type::all().iter().map(|t| t.keyword()).collect(); - assert_eq!(kws.len(), 10); + assert_eq!(kws.len(), 9); let mut sorted = kws.clone(); sorted.sort_unstable(); sorted.dedup(); - assert_eq!(sorted.len(), 10, "keywords must be unique"); + assert_eq!(sorted.len(), 9, "keywords must be unique"); } #[test] @@ -294,16 +288,10 @@ mod tests { } #[test] - fn blob_type_maps_to_blob_storage() { - assert_eq!(Type::Blob.sqlite_strict_type(), "BLOB"); - } - - #[test] - fn unknown_type_message_lists_all_ten() { + fn unknown_type_message_lists_all_nine() { let err = "varchar".parse::().unwrap_err(); for kw in [ - "text", "int", "real", "decimal", "bool", "date", "datetime", "blob", "serial", - "shortid", + "text", "int", "real", "decimal", "bool", "date", "datetime", "serial", "shortid", ] { assert!( err.expected.contains(kw), @@ -314,12 +302,12 @@ mod tests { } // --- ADR-0035 §3: advanced-mode SQL type-name resolution --- - // `from_sql_name` accepts the ten playground keywords *and* the + // `from_sql_name` accepts the nine playground keywords *and* the // standard-SQL aliases. Simple-mode `FromStr` is unchanged (it // still rejects aliases — see `unknown_type_lists_expected_alternatives`). #[test] - fn sql_resolver_accepts_the_ten_canonical_keywords() { + fn sql_resolver_accepts_the_nine_canonical_keywords() { for &ty in Type::all() { assert_eq!(Type::from_sql_name(ty.keyword()), Some(ty)); } @@ -338,8 +326,6 @@ mod tests { ("numeric", Type::Decimal), ("float", Type::Real), ("double precision", Type::Real), - ("binary", Type::Blob), - ("varbinary", Type::Blob), ] { assert_eq!( Type::from_sql_name(alias), diff --git a/src/dsl/value.rs b/src/dsl/value.rs index be0a9b7..5d5cc23 100644 --- a/src/dsl/value.rs +++ b/src/dsl/value.rs @@ -101,10 +101,6 @@ impl Value { Type::Bool => self.bind_bool(column), Type::Date => self.bind_date(column), Type::DateTime => self.bind_datetime(column), - Type::Blob => Err(ValueError::Format { - column: column.to_string(), - message: "literal `blob` values are not supported in DSL yet".to_string(), - }), } } @@ -320,7 +316,6 @@ mod tests { #[test] fn null_binds_to_null_for_any_type() { for ty in Type::all() { - // Skip blob — null still works there too. assert_eq!(Value::Null.bind_for_column("c", *ty).unwrap(), Bound::Null); } } @@ -429,10 +424,4 @@ mod tests { let err = n("3..14").bind_for_column("c", Type::Decimal).unwrap_err(); assert!(matches!(err, ValueError::Format { .. })); } - - #[test] - fn blob_inserts_are_explicitly_unsupported_for_now() { - let err = t("0xdead").bind_for_column("c", Type::Blob).unwrap_err(); - assert!(matches!(err, ValueError::Format { message, .. } if message.contains("blob"))); - } } diff --git a/src/dsl/walker/mod.rs b/src/dsl/walker/mod.rs index d7b55c4..0f9fe45 100644 --- a/src/dsl/walker/mod.rs +++ b/src/dsl/walker/mod.rs @@ -240,7 +240,6 @@ const fn catalog_key_for_value_type(ty: crate::dsl::types::Type) -> &'static str Type::Text => "hint.value_slot_text", Type::Date => "hint.value_slot_date", Type::DateTime => "hint.value_slot_datetime", - Type::Blob => "hint.value_slot_blob", Type::Serial => "hint.value_slot_serial", Type::ShortId => "hint.value_slot_shortid", } @@ -4745,7 +4744,6 @@ mod tests { (Type::Text, "hint.value_slot_text"), (Type::Date, "hint.value_slot_date"), (Type::DateTime, "hint.value_slot_datetime"), - (Type::Blob, "hint.value_slot_blob"), ] { let schema = schema_with("T", &[("c", ty)]); let mode = hint_mode_at_input_with_schema("insert into T values (", &schema); diff --git a/src/friendly/keys.rs b/src/friendly/keys.rs index 64b65b9..4e5fd70 100644 --- a/src/friendly/keys.rs +++ b/src/friendly/keys.rs @@ -425,7 +425,6 @@ pub const KEYS_AND_PLACEHOLDERS: &[(&str, &[&str])] = &[ ("hint.value_literal_slot", &[]), ("hint.ambient_typing_name_then", &["next"]), // Per-column-type value-slot hints (ADR-0024 §Phase D). - ("hint.value_slot_blob", &[]), ("hint.value_slot_bool", &[]), ("hint.value_slot_date", &[]), ("hint.value_slot_datetime", &[]), diff --git a/src/friendly/strings/en-US.yaml b/src/friendly/strings/en-US.yaml index 66a925d..572c187 100644 --- a/src/friendly/strings/en-US.yaml +++ b/src/friendly/strings/en-US.yaml @@ -395,7 +395,7 @@ help: without running it (advanced SQL) # Type reference, appended after the command list. types_reference: | - Types: text, int, real, decimal, bool, date, datetime, blob, serial, shortid + Types: text, int, real, decimal, bool, date, datetime, serial, shortid Auto-generated types (serial, shortid): serial — integer that auto-fills with the next sequence value (MAX(col)+1) on insert. Outside a primary key it carries @@ -714,7 +714,6 @@ hint: value_slot_text: "Type a quoted string (e.g. 'Alice') or null" value_slot_date: "Type a quoted date as 'YYYY-MM-DD' or null" value_slot_datetime: "Type a quoted datetime as 'YYYY-MM-DD HH:MM:SS' or null" - value_slot_blob: "Type a quoted blob literal or null" # Serial / shortid in `values (…)` form: the user must enter # something at this position (no "skip column" syntax). `null` # is the auto-fill path (ADR-0018: serial / shortid columns diff --git a/src/output_render.rs b/src/output_render.rs index ac2654a..455f27b 100644 --- a/src/output_render.rs +++ b/src/output_render.rs @@ -345,7 +345,7 @@ const fn alignment_for(ty: Option) -> Alignment { match ty { Some(Type::Int | Type::Real | Type::Decimal | Type::Serial) => Alignment::Right, Some(Type::Text) | Some(Type::Bool) | Some(Type::Date) | Some(Type::DateTime) - | Some(Type::Blob) | Some(Type::ShortId) | None => Alignment::Left, + | Some(Type::ShortId) | None => Alignment::Left, } } @@ -1334,7 +1334,6 @@ mod tests { sqlite_type: match ty { Type::Int | Type::Serial | Type::Bool => "INTEGER".to_string(), Type::Real => "REAL".to_string(), - Type::Blob => "BLOB".to_string(), _ => "TEXT".to_string(), }, notnull, @@ -1361,7 +1360,6 @@ mod tests { Type::Bool, Type::Date, Type::DateTime, - Type::Blob, Type::ShortId, ] { assert_eq!(alignment_for(Some(ty)), Alignment::Left, "{ty:?}"); diff --git a/src/persistence/csv_io.rs b/src/persistence/csv_io.rs index 51ba8a3..db50e36 100644 --- a/src/persistence/csv_io.rs +++ b/src/persistence/csv_io.rs @@ -25,8 +25,6 @@ use std::io::Write as _; -use base64::Engine as _; - use crate::dsl::types::Type; use super::{CellValue, TableSnapshot}; @@ -149,12 +147,6 @@ fn encode_cell(ty: Type, value: &CellValue) -> Result { CellValue::Text(s) => Ok(Cell::Plain(s.clone())), other => Err(format!("expected date/datetime (text), got {other:?}")), }, - Type::Blob => match value { - CellValue::Blob(bytes) => Ok(Cell::Plain( - base64::engine::general_purpose::STANDARD.encode(bytes), - )), - other => Err(format!("expected blob, got {other:?}")), - }, Type::Serial => match value { CellValue::Integer(n) => Ok(Cell::Plain(n.to_string())), other => Err(format!("expected serial (int), got {other:?}")), @@ -362,10 +354,6 @@ pub(crate) fn decode_cell(ty: Type, cell: &RawCell) -> Result "false" => Ok(CellValue::Integer(0)), other => Err(format!("expected `true` or `false`, got `{other}`")), }, - Type::Blob => base64::engine::general_purpose::STANDARD - .decode(cell.content.as_bytes()) - .map(CellValue::Blob) - .map_err(|e| format!("invalid base64 blob: {e}")), } } @@ -467,18 +455,6 @@ mod tests { assert_eq!(s, "b\ntrue\nfalse\n"); } - #[test] - fn blobs_use_base64() { - let body = serialize_table(&TableSnapshot { - name: "T".to_string(), - columns: vec![col("blob", Type::Blob)], - rows: vec![vec![CellValue::Blob(b"hello".to_vec())]], - }) - .unwrap(); - let s = String::from_utf8(body).unwrap(); - assert!(s.contains("aGVsbG8=")); - } - #[test] fn dates_and_datetimes_pass_through() { let body = serialize_table(&TableSnapshot { @@ -548,13 +524,11 @@ mod tests { col("n", Type::Int), col("r", Type::Real), col("b", Type::Bool), - col("blob", Type::Blob), ], rows: vec![vec![ CellValue::Integer(42), CellValue::Real(std::f64::consts::PI), CellValue::Integer(1), - CellValue::Blob(b"hi".to_vec()), ]], }; let body = serialize_table(&table).unwrap(); @@ -572,9 +546,6 @@ mod tests { decode_cell(Type::Bool, &row[2]).unwrap(), CellValue::Integer(1) )); - assert!( - matches!(decode_cell(Type::Blob, &row[3]).unwrap(), CellValue::Blob(b) if b == b"hi") - ); } #[test] diff --git a/src/persistence/migrations.rs b/src/persistence/migrations.rs index cce9b59..fd1a162 100644 --- a/src/persistence/migrations.rs +++ b/src/persistence/migrations.rs @@ -44,6 +44,35 @@ use serde::Deserialize; /// migrator's job is purely the format transformation. pub type MigrateFn = fn(&str) -> Result; +/// The newest `project.yaml` format version this build writes and reads. +/// +/// Must equal `MigratorRegistry::production().latest_version()` (asserted +/// in tests) and is what the YAML serializer stamps and the parser +/// accepts. Bumped 1 → 2 by **ADR-0005 Amendment 2** (drop `blob`). +pub const CURRENT_SCHEMA_VERSION: u32 = 2; + +/// v1 → v2 migrator (ADR-0005 Amendment 2). `blob` was dropped from the +/// type vocabulary, so any `type: blob` column is rewritten to +/// `type: text` (the chosen conversion target — see the ADR). A pure, +/// line-oriented text transform: only column-definition lines (the +/// `- { name: …, type: … }` shape the serializer emits) are touched, so a +/// `blob` substring inside a CHECK expression or a default value is left +/// alone. Also bumps the `version:` field to 2 (the framework asserts the +/// advertised version, so this must stay in step). +fn migrate_v1_to_v2(body: &str) -> Result { + let mut out = String::with_capacity(body.len() + 1); + for line in body.split_inclusive('\n') { + if line.trim_end() == "version: 1" { + out.push_str(&line.replacen("version: 1", "version: 2", 1)); + } else if line.contains("{ name:") { + out.push_str(&line.replace(", type: blob", ", type: text")); + } else { + out.push_str(line); + } + } + Ok(out) +} + /// Ordered list of migrators. `migrators[i]` runs from /// version `i + 1` to version `i + 2` (so index 0 is v1→v2, /// index 1 is v2→v3, etc.). @@ -56,13 +85,12 @@ pub struct MigratorRegistry { } impl MigratorRegistry { - /// Production-default registry: empty. As new versions - /// land, register the migrators here in source-version - /// order. + /// Production-default registry. Migrators are listed in + /// source-version order (index 0 = v1→v2, …). #[must_use] - pub const fn production() -> Self { + pub fn production() -> Self { Self { - migrators: Vec::new(), + migrators: vec![migrate_v1_to_v2], } } @@ -156,6 +184,20 @@ pub struct MigrationOutcome { pub migrated_from: Option, } +/// Whether `body` declares at least one `type: blob` column. +/// +/// The runtime uses this as the signal to force a `.db` rebuild after the +/// v1→v2 migration: the migration rewrites the YAML, but the derived +/// `.db` keeps a `STRICT … BLOB` engine column + `"blob"` metadata that +/// load otherwise uses as-is (ADR-0005 Amendment 2). Scoped to +/// column-definition lines, mirroring [`migrate_v1_to_v2`], so a `blob` +/// substring inside a CHECK expression or default doesn't trip it. +#[must_use] +pub fn body_declares_blob_column(body: &str) -> bool { + body.lines() + .any(|l| l.contains("{ name:") && l.contains(", type: blob")) +} + /// Detect the version of `body` and migrate it to the /// registry's `latest_version()`. /// @@ -304,22 +346,68 @@ mod tests { .to_string() } + /// A project at the current (latest) version — nothing to migrate. + fn v2_body() -> String { + "version: 2\nproject:\n created_at: 2026-01-01T00:00:00Z\ntables: []\nrelationships: []\n" + .to_string() + } + #[test] - fn production_registry_latest_version_is_1() { + fn production_registry_latest_version_matches_current_schema_version() { let r = MigratorRegistry::production(); - assert_eq!(r.latest_version(), 1); + assert_eq!(r.latest_version(), CURRENT_SCHEMA_VERSION); + assert_eq!(r.latest_version(), 2); } #[test] fn no_migration_runs_when_body_already_latest() { let tmp = tempdir(); let outcome = - migrate_to_latest(&v1_body(), &MigratorRegistry::production(), tmp.path()).unwrap(); - assert_eq!(outcome.body, v1_body()); + migrate_to_latest(&v2_body(), &MigratorRegistry::production(), tmp.path()).unwrap(); + assert_eq!(outcome.body, v2_body()); assert_eq!(outcome.migrated_from, None); // No .bak written when nothing migrated. - let bak = tmp.path().join("project.yaml.v1.bak"); - assert!(!bak.exists(), "no .bak when no migration"); + assert!( + !tmp.path().join("project.yaml.v2.bak").exists(), + "no .bak when no migration" + ); + } + + /// ADR-0005 Amendment 2: the real production v1→v2 migrator rewrites a + /// `blob` column to `text` and bumps the version, leaving everything + /// else (incl. a column literally named `blob`, and a `blob` substring + /// inside a CHECK) untouched. + #[test] + fn production_v1_to_v2_converts_blob_columns_to_text() { + let tmp = tempdir(); + let body = concat!( + "version: 1\n", + "project:\n created_at: x\n", + "tables:\n", + " - name: Files\n", + " primary_key: [id]\n", + " columns:\n", + " - { name: id, type: serial }\n", + " - { name: payload, type: blob }\n", + " - { name: blob, type: text }\n", + " check_constraints:\n", + " - \"note <> 'type: blob'\"\n", + "relationships: []\n", + ); + let outcome = + migrate_to_latest(body, &MigratorRegistry::production(), tmp.path()).unwrap(); + assert_eq!(outcome.migrated_from, Some(1)); + assert!(outcome.body.contains("version: 2")); + // The blob column became text. + assert!( + outcome.body.contains("{ name: payload, type: text }"), + "blob column converted: {}", + outcome.body + ); + // A column NAMED blob is untouched; the CHECK string is untouched. + assert!(outcome.body.contains("{ name: blob, type: text }")); + assert!(outcome.body.contains("note <> 'type: blob'")); + assert!(!outcome.body.contains("type: blob }")); } #[test] @@ -332,7 +420,7 @@ mod tests { err, MigrateError::NewerThanSupported { file: 99, - latest: 1 + latest: 2 } ), "got: {err:?}", @@ -393,17 +481,17 @@ mod tests { } #[test] - fn ensure_yaml_migrated_no_op_on_v1_with_empty_registry() { + fn ensure_yaml_migrated_no_op_when_already_latest() { let tmp = tempdir(); let yaml_path = tmp.path().join("project.yaml"); - std::fs::write(&yaml_path, v1_body()).unwrap(); + std::fs::write(&yaml_path, v2_body()).unwrap(); let outcome = ensure_project_yaml_migrated(tmp.path(), &MigratorRegistry::production()).unwrap(); assert_eq!(outcome.migrated_from, None); // File unchanged. let on_disk = std::fs::read_to_string(&yaml_path).unwrap(); - assert_eq!(on_disk, v1_body()); - assert!(!tmp.path().join("project.yaml.v1.bak").exists()); + assert_eq!(on_disk, v2_body()); + assert!(!tmp.path().join("project.yaml.v2.bak").exists()); } #[test] diff --git a/src/persistence/mod.rs b/src/persistence/mod.rs index 3d5234e..ce240be 100644 --- a/src/persistence/mod.rs +++ b/src/persistence/mod.rs @@ -280,7 +280,6 @@ pub enum CellValue { Integer(i64), Real(f64), Text(String), - Blob(Vec), } impl Persistence { @@ -561,7 +560,7 @@ mod tests { }; p.write_schema(&schema).unwrap(); let body = fs::read_to_string(dir.path().join(PROJECT_YAML)).unwrap(); - assert!(body.contains("version: 1")); + assert!(body.contains("version: 2")); assert!(body.contains("created_at:")); } diff --git a/src/persistence/yaml.rs b/src/persistence/yaml.rs index 3a4fcd6..490b088 100644 --- a/src/persistence/yaml.rs +++ b/src/persistence/yaml.rs @@ -32,7 +32,11 @@ use super::{ #[must_use] pub(super) fn serialize_schema(schema: &SchemaSnapshot) -> String { let mut out = String::new(); - let _ = writeln!(out, "version: 1"); + let _ = writeln!( + out, + "version: {}", + crate::persistence::migrations::CURRENT_SCHEMA_VERSION + ); let _ = writeln!(out, "project:"); let _ = writeln!(out, " created_at: {}", quote_if_needed(&schema.created_at)); // ADR-0015 mode-restore amendment (issue #14): the input mode @@ -283,7 +287,11 @@ const fn is_safe_yaml_char(c: char) -> bool { pub(crate) fn parse_schema(body: &str) -> Result { let raw: RawProject = serde_norway::from_str(body).map_err(|e| YamlError::Syntax(e.to_string()))?; - if raw.version != 1 { + // The migration framework (ADR-0015) upgrades older project files to + // `CURRENT_SCHEMA_VERSION` before `parse_schema` ever runs, so the + // parser only accepts the current version (a stale version reaching + // here means migration was skipped — a bug worth surfacing). + if raw.version != crate::persistence::migrations::CURRENT_SCHEMA_VERSION { return Err(YamlError::UnsupportedVersion(raw.version)); } let mut tables: Vec = Vec::with_capacity(raw.tables.len()); @@ -617,7 +625,7 @@ mod tests { let body = serialize_schema(&snapshot()); // Spot-check structural lines rather than asserting on // the whole blob — easier to read in failure output. - assert!(body.contains("version: 1")); + assert!(body.contains("version: 2")); assert!(body.contains("created_at: 2026-05-07T14:30:12Z")); assert!(body.contains("- name: Customers")); assert!(body.contains("primary_key: [id]")); @@ -752,7 +760,7 @@ mod tests { // Older project files (written before unique indexes) omit the // `unique` field; the `#[serde(default)]` makes it `false`. let body = "\ -version: 1 +version: 2 project: created_at: 2026-05-25T00:00:00Z tables: @@ -922,7 +930,7 @@ indexes: // Back-compat: a project file written before §4g (bare-string // check_constraints) parses with name = None. let body = "\ -version: 1 +version: 2 project: created_at: \"2026-05-25T00:00:00Z\" tables: @@ -949,7 +957,7 @@ indexes: [] // A project file written before table-level CHECK existed (no // `check_constraints:` key) parses with an empty list. let body = "\ -version: 1 +version: 2 project: created_at: 2026-05-25T00:00:00Z tables: @@ -966,7 +974,7 @@ relationships: [] #[test] fn parses_minimal_yaml_with_no_tables() { let body = "\ -version: 1 +version: 2 project: created_at: 2026-05-07T14:30:12Z tables: [] @@ -993,7 +1001,7 @@ relationships: [] #[test] fn rejects_unknown_column_type() { let body = "\ -version: 1 +version: 2 project: created_at: x tables: @@ -1012,7 +1020,7 @@ relationships: [] #[test] fn rejects_unknown_action() { let body = "\ -version: 1 +version: 2 project: created_at: x tables: [] @@ -1090,7 +1098,7 @@ relationships: fn parse_schema_defaults_mode_to_simple_when_field_absent() { // A pre-#14 project file carries no `mode:` field; it must // parse with the default mode, not fail. - let body = "version: 1\nproject:\n created_at: x\ntables: []\nrelationships: []\n"; + let body = "version: 2\nproject:\n created_at: x\ntables: []\nrelationships: []\n"; let parsed = parse_schema(body).expect("legacy file parses"); assert_eq!(parsed.mode, Mode::Simple); } @@ -1100,13 +1108,13 @@ relationships: // `None` (no stored preference) must be distinct from an // explicit `simple`, so restore-on-open precedence can tell // "fall back to default" from "the user chose simple". - let absent = "version: 1\nproject:\n created_at: x\ntables: []\n"; + let absent = "version: 2\nproject:\n created_at: x\ntables: []\n"; assert_eq!(parse_stored_mode(absent), None); - let explicit_simple = "version: 1\nproject:\n created_at: x\n mode: simple\ntables: []\n"; + let explicit_simple = "version: 2\nproject:\n created_at: x\n mode: simple\ntables: []\n"; assert_eq!(parse_stored_mode(explicit_simple), Some(Mode::Simple)); - let advanced = "version: 1\nproject:\n created_at: x\n mode: advanced\ntables: []\n"; + let advanced = "version: 2\nproject:\n created_at: x\n mode: advanced\ntables: []\n"; assert_eq!(parse_stored_mode(advanced), Some(Mode::Advanced)); } @@ -1114,7 +1122,7 @@ relationships: fn parse_stored_mode_falls_back_to_none_on_unknown_value() { // An unrecognised mode keyword degrades to "no preference" // rather than rejecting the whole file over a UI hint. - let body = "version: 1\nproject:\n created_at: x\n mode: expert\ntables: []\n"; + let body = "version: 2\nproject:\n created_at: x\n mode: expert\ntables: []\n"; assert_eq!(parse_stored_mode(body), None); } } diff --git a/src/project/mod.rs b/src/project/mod.rs index d007f6e..9772e01 100644 --- a/src/project/mod.rs +++ b/src/project/mod.rs @@ -384,7 +384,7 @@ impl Project { /// Build the on-disk skeleton for a fresh project: the /// directory itself, an empty `data/`, an empty /// `history.log`, a placeholder `project.yaml` with just - /// `version: 1` and `created_at`, and a `.gitignore`. + /// the current schema `version` and `created_at`, and a `.gitignore`. /// /// `playground.db` is not created here; it's created the /// first time `Database::open` runs against the path @@ -401,7 +401,8 @@ impl Project { // schema mutation; for now we just ensure the file // exists and carries the version + creation timestamp. let yaml = format!( - "version: 1\nproject:\n created_at: {}\ntables: []\nrelationships: []\n", + "version: {}\nproject:\n created_at: {}\ntables: []\nrelationships: []\n", + crate::persistence::migrations::CURRENT_SCHEMA_VERSION, iso8601_now(), ); write_if_missing(&path.join(PROJECT_YAML), &yaml)?; @@ -830,7 +831,7 @@ mod tests { // YAML carries version + created_at. let yaml = fs::read_to_string(path.join(PROJECT_YAML)).unwrap(); - assert!(yaml.contains("version: 1")); + assert!(yaml.contains("version: 2")); assert!(yaml.contains("created_at:")); // .gitignore matches ADR-0015. diff --git a/src/runtime.rs b/src/runtime.rs index 41d887f..4d8c0a8 100644 --- a/src/runtime.rs +++ b/src/runtime.rs @@ -176,6 +176,13 @@ pub async fn run(args: Args) -> Result<()> { // `project.yaml.v.bak` breadcrumb on disk; that's // sufficient v1 UX and lets us defer dedicated event // plumbing until a real migrator demands it. + // ADR-0005 Amendment 2: note a pre-migration `blob` column so we can + // force a `.db` rebuild after the v1→v2 migration converts it to text + // (the stale `.db` keeps a `STRICT … BLOB` engine column + `"blob"` + // metadata that load otherwise uses as-is). + let had_blob_column = std::fs::read_to_string(project.path().join(crate::project::PROJECT_YAML)) + .map(|b| crate::persistence::migrations::body_declares_blob_column(&b)) + .unwrap_or(false); let migrate_registry = crate::persistence::migrations::MigratorRegistry::production(); let migration_outcome = crate::persistence::migrations::ensure_project_yaml_migrated( project.path(), @@ -235,7 +242,12 @@ pub async fn run(args: Args) -> Result<()> { Database::open_with_persistence_and_undo(db_path.as_path(), persistence, undo_enabled) .context("open database")?; let mut initial_events: Vec = Vec::new(); - if !db_existed { + // ADR-0005 Amendment 2: a blob→text migration leaves the existing + // `.db` with a stale `STRICT … BLOB` engine column, so rebuild it from + // the migrated text. A pure version bump with no schema change does not. + let force_rebuild_after_migration = + migration_outcome.migrated_from.is_some() && had_blob_column; + if !db_existed || force_rebuild_after_migration { match database.rebuild_from_text(project_path.clone(), None).await { Ok(()) => { // Surface the silent rebuild as a system note @@ -927,7 +939,13 @@ async fn perform_switch( // state momentarily, but the next user action will // surface the error and they can retry). let migrate_registry = crate::persistence::migrations::MigratorRegistry::production(); - crate::persistence::migrations::ensure_project_yaml_migrated( + // ADR-0005 Amendment 2: see the matching note in `run()` — a blob→text + // migration needs the `.db` rebuilt from the migrated text. + let had_blob_column = + std::fs::read_to_string(new_project.path().join(crate::project::PROJECT_YAML)) + .map(|b| crate::persistence::migrations::body_declares_blob_column(&b)) + .unwrap_or(false); + let migration_outcome = crate::persistence::migrations::ensure_project_yaml_migrated( new_project.path(), &migrate_registry, ) @@ -949,7 +967,11 @@ async fn perform_switch( let new_database = Database::open_with_persistence_and_undo(&db_path, persistence, undo_enabled) .map_err(|e| e.to_string())?; - if !db_existed && let Err(e) = new_database.rebuild_from_text(new_path.clone(), None).await { + let force_rebuild_after_migration = + migration_outcome.migrated_from.is_some() && had_blob_column; + if (!db_existed || force_rebuild_after_migration) + && let Err(e) = new_database.rebuild_from_text(new_path.clone(), None).await + { return Err(e.friendly_message()); } diff --git a/src/seed/generators.rs b/src/seed/generators.rs index 9d09be1..acfce5a 100644 --- a/src/seed/generators.rs +++ b/src/seed/generators.rs @@ -138,7 +138,7 @@ fn range_value(low: &str, high: &str, ty: Type, rng: &mut SeedRng) -> Value { Type::DateTime => parse_datetime_range(low, high) .map(|(lo, hi)| Value::Text(random_datetime_between(rng, lo, hi))) .unwrap_or_else(|| generic_for_type(ty, rng)), - // text / bool / blob / shortid have no range meaning. + // text / bool / shortid have no range meaning. _ => generic_for_type(ty, rng), } } @@ -155,8 +155,8 @@ pub fn range_bounds_reason(ty: Type, low: &str, high: &str) -> Option { Type::Real | Type::Decimal => parse_real_range(low, high).is_some(), Type::Date => parse_date_range(low, high).is_some(), Type::DateTime => parse_datetime_range(low, high).is_some(), - // text / bool / blob / shortid have no range meaning. - Type::Text | Type::Bool | Type::Blob | Type::ShortId => false, + // text / bool / shortid have no range meaning. + Type::Text | Type::Bool | Type::ShortId => false, }; if ok { return None; @@ -169,7 +169,7 @@ pub fn range_bounds_reason(ty: Type, low: &str, high: &str) -> Option { "expected two quoted datetimes, e.g. `between '2023-01-01T00:00:00' and '2024-12-31T23:59:59'`" .to_string() } - Type::Text | Type::Bool | Type::Blob | Type::ShortId => { + Type::Text | Type::Bool | Type::ShortId => { "a `between` range only applies to numeric and date/datetime columns".to_string() } }) @@ -242,9 +242,8 @@ fn random_datetime_between( } /// Type-based fallback generation (D8). Never produces NULL for a -/// generatable type; `blob`/`serial`/`shortid` are handled by the -/// executor (autogen / block guard) and yield NULL here only as a -/// last resort. +/// generatable type; `serial`/`shortid` are handled by the executor +/// (autogen) and yield NULL here only as a last resort. fn generic_for_type(ty: Type, rng: &mut SeedRng) -> Value { use fake::faker::lorem::en as lorem; match ty { @@ -267,7 +266,6 @@ fn generic_for_type(ty: Type, rng: &mut SeedRng) -> Value { Type::Bool => Value::Bool(rng.random_range(0..2) == 1), Type::Date => Value::Text(format_date(random_past_date(rng, 0, RECENT_WINDOW_DAYS))), Type::DateTime => Value::Text(random_recent_datetime(rng)), - Type::Blob => Value::Null, } } @@ -705,10 +703,6 @@ mod tests { generate_value(&Generator::Generic, Type::Bool, &mut rng), Value::Bool(_) )); - assert!(matches!( - generate_value(&Generator::Generic, Type::Blob, &mut rng), - Value::Null - )); // shortid fallback is a valid base58 id. let Value::Text(sid) = generate_value(&Generator::Generic, Type::ShortId, &mut rng) else { panic!("shortid not text") diff --git a/src/type_change.rs b/src/type_change.rs index fa3a2d9..6797ab9 100644 --- a/src/type_change.rs +++ b/src/type_change.rs @@ -17,8 +17,8 @@ //! //! Pairs not present in the matrix are statically refused via //! [`static_refusal`] before any per-cell pass runs. Same-type -//! identity, anything → `serial`, anything ↔ `blob`, and -//! `date` ↔ `datetime` direct are all statically refused. +//! identity, anything → `serial`, and `date` ↔ `datetime` direct +//! are all statically refused. use rusqlite::types::Value; @@ -46,9 +46,8 @@ pub enum CellOutcome { /// refused; `None` when the per-cell matrix should be consulted. /// /// Static refusals cover: same-type identity, anything → -/// `serial`, anything ↔ `blob`, `date` ↔ `datetime` direct, and -/// any cross-domain pair not present in the matrix -/// (e.g. `bool` → `date`). +/// `serial`, `date` ↔ `datetime` direct, and any cross-domain +/// pair not present in the matrix (e.g. `bool` → `date`). #[must_use] pub fn static_refusal(src: Type, target: Type) -> Option { if src == target { @@ -63,12 +62,6 @@ pub fn static_refusal(src: Type, target: Type) -> Option { first; only `int → serial` is supported directly." )); } - if matches!(src, Type::Blob) || matches!(target, Type::Blob) { - return Some(format!( - "conversion between `{src}` and `{target}` is not supported \ - in this version." - )); - } if matches!( (src, target), (Type::Date, Type::DateTime) | (Type::DateTime, Type::Date) @@ -601,16 +594,6 @@ mod tests { ); } - #[test] - fn anything_involving_blob_is_statically_refused() { - for &other in Type::all() { - if other != Type::Blob { - assert!(static_refusal(Type::Blob, other).is_some(), "{other:?}"); - assert!(static_refusal(other, Type::Blob).is_some(), "{other:?}"); - } - } - } - #[test] fn date_to_datetime_direct_is_statically_refused() { assert!(static_refusal(Type::Date, Type::DateTime).is_some()); diff --git a/tests/e2e_pty.rs b/tests/e2e_pty.rs index 7eadab6..8bfaaae 100644 --- a/tests/e2e_pty.rs +++ b/tests/e2e_pty.rs @@ -32,6 +32,7 @@ // (the guard *is* the serialization) or pointless here. #![allow(clippy::significant_drop_tightening)] +use std::fs; use std::io::{Read, Write}; use std::path::Path; use std::sync::{Arc, Mutex, MutexGuard}; @@ -404,6 +405,52 @@ fn back_to_back_insert_after_ddl_still_succeeds() { app.quit(); } +/// Flow 6 — ADR-0005 Amendment 2 backward-compat: opening a **legacy v1 +/// project that declares a `blob` column** migrates it to `text` and opens +/// cleanly. This drives the real runtime's migrate-on-open + rebuild path +/// end-to-end — the post-removal binary would otherwise reject the v1/blob +/// project file. (The integration test covers the migrator + rebuild via +/// direct calls; this covers the runtime glue that decides to run them.) +#[test] +fn opens_a_legacy_v1_blob_project_by_migrating_to_text() { + let dir = TempDir::new().expect("data dir"); + let root = dir.path(); + + // Seed a v1 project (the old format) with a `blob` column, under the + // data root's projects/ dir, and point `--resume` at it. No + // playground.db is shipped, so the open rebuilds from the migrated text. + let proj = root.join("projects").join("Legacy"); + fs::create_dir_all(proj.join("data")).expect("create project dirs"); + fs::write( + proj.join("project.yaml"), + concat!( + "version: 1\n", + "project:\n created_at: 2026-01-01T00:00:00Z\n mode: simple\n", + "tables:\n", + " - name: Files\n", + " primary_key: [id]\n", + " columns:\n", + " - { name: id, type: serial }\n", + " - { name: payload, type: blob }\n", + "relationships: []\n", + "indexes: []\n", + ), + ) + .expect("write project.yaml"); + fs::write(proj.join("data").join("Files.csv"), "id,payload\n1,aGVsbG8=\n").expect("write csv"); + fs::write(root.join("last_project"), format!("{}\n", proj.display())).expect("write resume"); + + // Open via --resume: the runtime migrates blob→text and rebuilds, so + // the project opens and the table shows in the sidebar. + let mut app = PtyApp::launch_in(root, &["--resume"]); + app.wait_for_table("Files"); + // The former blob column is now a normal text column — `show data` + // renders it (header + the base64 value preserved as text). + app.submit("show data Files"); + app.wait_for("payload"); + app.quit(); +} + // ===================== NFR perf (measured, generous) =================== // // These run against the DEBUG binary, so the bounds are loose diff --git a/tests/it/blob_removal_migration.rs b/tests/it/blob_removal_migration.rs new file mode 100644 index 0000000..344767b --- /dev/null +++ b/tests/it/blob_removal_migration.rs @@ -0,0 +1,107 @@ +//! Full-stack test for **ADR-0005 Amendment 2** (drop `blob`): a saved +//! v1 project that declares a `blob` column is migrated to v2 (`blob` → +//! `text`) on open, and a rebuild from the migrated text reconstructs the +//! column as `text` with the row data preserved — the base64 cell survives +//! as a recoverable string. This exercises the real migrator + the +//! `rebuild_from_text` path the runtime forces when a blob column was +//! converted (the runtime loop itself isn't booted in integration tests). + +use std::fs; + +use rdbms_playground::db::Database; +use rdbms_playground::dsl::Type; +use rdbms_playground::persistence::Persistence; +use rdbms_playground::persistence::migrations::{ + MigratorRegistry, body_declares_blob_column, ensure_project_yaml_migrated, +}; + +fn rt() -> tokio::runtime::Runtime { + tokio::runtime::Builder::new_current_thread() + .enable_all() + .build() + .expect("tokio rt") +} + +#[test] +fn v1_blob_project_migrates_to_text_and_rebuilds() { + let dir = tempfile::tempdir().expect("tempdir"); + let root = dir.path(); + + // A hand-written v1 project: a serial PK + a `blob` column carrying a + // (hand-edited, base64) value — the only way bytes ever reached a blob + // column, since no command can insert one. + let yaml = concat!( + "version: 1\n", + "project:\n created_at: 2026-01-01T00:00:00Z\n mode: simple\n", + "tables:\n", + " - name: Files\n", + " primary_key: [id]\n", + " columns:\n", + " - { name: id, type: serial }\n", + " - { name: data, type: blob }\n", + "relationships: []\n", + "indexes: []\n", + ); + fs::write(root.join("project.yaml"), yaml).expect("write yaml"); + fs::create_dir(root.join("data")).expect("data dir"); + fs::write(root.join("data").join("Files.csv"), "id,data\n1,aGVsbG8=\n").expect("write csv"); + + // The runtime's blob-detection signal sees the column… + assert!(body_declares_blob_column(yaml), "blob column detected"); + + // 1. Migration runs (as the runtime does on open). + let outcome = ensure_project_yaml_migrated(root, &MigratorRegistry::production()) + .expect("migration succeeds"); + assert_eq!(outcome.migrated_from, Some(1), "v1 project was migrated"); + + let migrated = fs::read_to_string(root.join("project.yaml")).expect("read migrated"); + assert!(migrated.contains("version: 2"), "version bumped: {migrated}"); + assert!( + migrated.contains("{ name: data, type: text }"), + "blob column rewritten to text: {migrated}" + ); + assert!(!migrated.contains("type: blob"), "no blob type remains: {migrated}"); + // The pre-migration original is preserved as a .bak. + let bak = fs::read_to_string(root.join("project.yaml.v1.bak")).expect("read bak"); + assert!(bak.contains("type: blob"), "bak keeps the original blob column"); + + // 2. Rebuild from the migrated text (as the runtime forces when a blob + // column was converted): the column is now `text` and the row data + // survives as a recoverable string. + let db = Database::open_with_persistence( + root.join("playground.db"), + Persistence::new(root.to_path_buf()), + ) + .expect("open db"); + rt().block_on(async { + db.rebuild_from_text(root.to_path_buf(), None) + .await + .expect("rebuild"); + + let desc = db + .describe_table("Files".to_string()) + .await + .expect("describe"); + let data_col = desc + .columns + .iter() + .find(|c| c.name == "data") + .expect("data column present"); + assert_eq!( + data_col.user_type, + Some(Type::Text), + "the former blob column is now text", + ); + + let rows = db + .query_data("Files".to_string(), None, None) + .await + .expect("query"); + assert_eq!(rows.rows.len(), 1, "row preserved through the migration"); + assert_eq!( + rows.rows[0][1].as_deref(), + Some("aGVsbG8="), + "the base64 cell survives as recoverable text", + ); + }); +} diff --git a/tests/it/main.rs b/tests/it/main.rs index 8383378..521ff53 100644 --- a/tests/it/main.rs +++ b/tests/it/main.rs @@ -7,6 +7,7 @@ //! `tests/typing_surface_matrix.rs` stays a separate binary (it is //! already a consolidated `mod`-based target). +mod blob_removal_migration; mod case_insensitive_names; mod column_op_guards; mod compound_fk; diff --git a/tests/it/seed.rs b/tests/it/seed.rs index 7e2e41c..5a4b8f4 100644 --- a/tests/it/seed.rs +++ b/tests/it/seed.rs @@ -617,69 +617,6 @@ fn seed_refuses_when_a_parent_table_is_empty() { ); } -#[test] -fn seed_refuses_a_not_null_blob_column() { - let (_project, db, _dir) = open_project_db(); - let rt = rt(); - let mut payload = ColumnSpec::new("payload", Type::Blob); - payload.not_null = true; - rt.block_on(db.create_table( - "Files".to_string(), - vec![ColumnSpec::new("id", Type::Serial), payload], - vec!["id".to_string()], - None, - )) - .expect("create Files"); - - let err = rt - .block_on(db.seed( - "Files".into(), - None, - Some(2), - Vec::new(), - Some(1), - Some("seed Files 2".into()), - )) - .expect_err("seed must refuse a NOT NULL blob"); - let msg = err.to_string(); - assert!( - msg.contains("payload") && msg.to_lowercase().contains("blob"), - "error should name the un-generatable blob column: {msg}" - ); -} - -#[test] -fn seed_omits_a_nullable_blob_column() { - let (project, db, _dir) = open_project_db(); - let rt = rt(); - rt.block_on(db.create_table( - "Files".to_string(), - vec![ - ColumnSpec::new("id", Type::Serial), - ColumnSpec::new("name", Type::Text), - // nullable blob → omitted (→ NULL), seed still succeeds. - ColumnSpec::new("payload", Type::Blob), - ], - vec!["id".to_string()], - None, - )) - .expect("create Files"); - - let res = rt - .block_on(db.seed( - "Files".into(), - None, - Some(3), - Vec::new(), - Some(1), - Some("seed Files 3".into()), - )) - .expect("seed succeeds despite the nullable blob"); - assert_eq!(res.produced, 3); - let csv = read_csv(&project, "Files").expect("Files CSV"); - assert_eq!(data_row_count(&csv), 3); -} - // — uniqueness, junction distinct-combos, IN-CHECK (D10 / D14 / D17) — /// The `n`th comma-separated field of each data row (the generated diff --git a/tests/it/sql_alter_table.rs b/tests/it/sql_alter_table.rs index c262a34..189793c 100644 --- a/tests/it/sql_alter_table.rs +++ b/tests/it/sql_alter_table.rs @@ -356,14 +356,6 @@ fn e2e_alter_column_type_static_refusals() { ), "text→serial is refused (only int→serial is allowed)" ); - assert!( - replay_is_refused( - "create table T with pk id(int)\n\ - add column T: v (text)\n\ - alter table T alter column v type blob\n", - ), - "↔ blob is statically refused" - ); } #[test] diff --git a/tests/it/sql_dml_e2e.rs b/tests/it/sql_dml_e2e.rs index 9cf74fd..5a82b3d 100644 --- a/tests/it/sql_dml_e2e.rs +++ b/tests/it/sql_dml_e2e.rs @@ -208,13 +208,12 @@ fn e2e_insert_select_cross_table_copies_rows_and_persists_both() { // =============================================================== #[test] -fn e2e_multirow_insert_all_ten_types_roundtrips_and_returning_recovers_each_type() { +fn e2e_multirow_insert_all_nine_types_roundtrips_and_returning_recovers_each_type() { let (project, db, _dir) = open_project_db(); let rt = rt(); - // serial PK + shortid auto-fill; the other eight columns are - // user-supplied. `blob` has no value-literal grammar yet - // (see src/dsl/value.rs), so it is inserted NULL — its *type* - // still round-trips through the RETURNING column-origin path. + // serial PK + shortid auto-fill; the other seven columns are + // user-supplied. Their *types* round-trip through the RETURNING + // column-origin path. create_cols( &db, &rt, @@ -228,7 +227,6 @@ fn e2e_multirow_insert_all_ten_types_roundtrips_and_returning_recovers_each_type ("flag", Type::Bool), ("d", Type::Date), ("ts", Type::DateTime), - ("bl", Type::Blob), ("sid", Type::ShortId), ], &["ser"], @@ -237,17 +235,17 @@ fn e2e_multirow_insert_all_ten_types_roundtrips_and_returning_recovers_each_type let result = run_insert( &db, &rt, - "insert into allten (txt, i, r, dec, flag, d, ts, bl) values \ - ('hi', 42, 1.5, 9.50, true, '2026-05-23', '2026-05-23 10:00:00', null), \ - ('yo', 7, 2.5, 3.25, false, '2025-01-01', '2025-01-01 00:00:00', null) \ - returning ser, txt, i, r, dec, flag, d, ts, bl, sid", + "insert into allten (txt, i, r, dec, flag, d, ts) values \ + ('hi', 42, 1.5, 9.50, true, '2026-05-23', '2026-05-23 10:00:00'), \ + ('yo', 7, 2.5, 3.25, false, '2025-01-01', '2025-01-01 00:00:00') \ + returning ser, txt, i, r, dec, flag, d, ts, sid", ) .expect("multi-row INSERT … RETURNING runs"); assert_eq!(result.rows_affected, 2, "two rows inserted"); assert_eq!(result.data.rows.len(), 2, "RETURNING yields both rows"); - // Every one of the ten playground types is recovered via the + // Every one of the nine playground types is recovered via the // RETURNING column-origin path (matrix R5). assert_eq!( result.data.column_types, @@ -260,10 +258,9 @@ fn e2e_multirow_insert_all_ten_types_roundtrips_and_returning_recovers_each_type Some(Type::Bool), Some(Type::Date), Some(Type::DateTime), - Some(Type::Blob), Some(Type::ShortId), ], - "RETURNING recovers each of the ten playground types; got {:?}", + "RETURNING recovers each of the nine playground types; got {:?}", result.data.column_types, ); @@ -281,7 +278,7 @@ fn e2e_multirow_insert_all_ten_types_roundtrips_and_returning_recovers_each_type "dates round-trip: {csv:?}" ); - let sids: Vec<&str> = rows.iter().filter_map(|r| r[9].as_deref()).collect(); + let sids: Vec<&str> = rows.iter().filter_map(|r| r[8].as_deref()).collect(); assert_eq!(sids.len(), 2, "both shortids present"); assert!( sids.iter().all(|s| !s.is_empty()), diff --git a/tests/it/sql_select.rs b/tests/it/sql_select.rs index 6d6aa35..3b516c3 100644 --- a/tests/it/sql_select.rs +++ b/tests/it/sql_select.rs @@ -575,11 +575,7 @@ fn database_run_select_type_recovery_works_on_empty_table() { // when no row matches. // // This test pins that invariant: a fresh table with no - // rows still yields the right `column_types` entry. It - // also justifies the all-types test below using NULL for - // col_blob (the DSL Value enum has no Blob variant, but - // since metadata doesn't read row values, a NULL cell - // doesn't compromise the recovery). + // rows still yields the right `column_types` entry. let (_p, db, _dir) = open_project_db(); let rt = rt(); rt.block_on(async { @@ -588,7 +584,6 @@ fn database_run_select_type_recovery_works_on_empty_table() { vec![ ColumnSpec::new("id", Type::Serial), ColumnSpec::new("col_text", Type::Text), - ColumnSpec::new("col_blob", Type::Blob), ], vec!["id".to_string()], None, @@ -602,32 +597,16 @@ fn database_run_select_type_recovery_works_on_empty_table() { .expect("SELECT runs even on empty table"); assert!(data_text.rows.is_empty()); assert_eq!(data_text.column_types, vec![Some(Type::Text)]); - - let data_blob = rt - .block_on(db.run_select("select col_blob from Empty".to_string())) - .expect("SELECT runs even on empty table"); - assert!(data_blob.rows.is_empty()); - assert_eq!( - data_blob.column_types, - vec![Some(Type::Blob)], - "Blob metadata must be recoverable even with no row data", - ); } #[test] -fn database_run_select_recovers_all_ten_playground_types() { +fn database_run_select_recovers_all_nine_playground_types() { // ADR-0032 §12 + Amendment 1 — every playground type // round-trips through column-origin metadata on a bare // projection ref. One table holds one column of each // type; a SELECT of that column produces the right - // `column_types[0]` entry. - // - // `serial` and `shortid` are auto-generated. `col_blob` - // is left NULL in the inserted row because the DSL Value - // enum has no Blob variant — but per - // `database_run_select_type_recovery_works_on_empty_table` - // above, column-origin metadata is row-independent, so - // the NULL cell doesn't compromise this test's correctness. + // `column_types[0]` entry. `serial` and `shortid` are + // auto-generated. let (_p, db, _dir) = open_project_db(); let rt = rt(); rt.block_on(async { @@ -642,7 +621,6 @@ fn database_run_select_recovers_all_ten_playground_types() { ColumnSpec::new("col_bool", Type::Bool), ColumnSpec::new("col_date", Type::Date), ColumnSpec::new("col_datetime", Type::DateTime), - ColumnSpec::new("col_blob", Type::Blob), ColumnSpec::new("col_shortid", Type::ShortId), ], vec!["pk".to_string()], @@ -650,10 +628,6 @@ fn database_run_select_recovers_all_ten_playground_types() { ) .await .expect("create table"); - // Blob has no DSL literal form, so col_blob takes the - // default NULL on insert. Column-origin metadata is - // based on the column DEFINITION, not the row value - // (Amendment 1), so the type recovery still succeeds. db.insert( "AllTypes".to_string(), Some(vec![ @@ -691,7 +665,6 @@ fn database_run_select_recovers_all_ten_playground_types() { ("col_bool", Type::Bool), ("col_date", Type::Date), ("col_datetime", Type::DateTime), - ("col_blob", Type::Blob), ("col_shortid", Type::ShortId), ]; for (col, expected_type) in cases { diff --git a/tests/typing_surface/mod.rs b/tests/typing_surface/mod.rs index dae318e..e645339 100644 --- a/tests/typing_surface/mod.rs +++ b/tests/typing_surface/mod.rs @@ -117,7 +117,6 @@ pub fn schema_every_type() -> SchemaCache { ("b", Type::Bool), ("dt", Type::Date), ("ts", Type::DateTime), - ("data", Type::Blob), ("sid", Type::ShortId), ("auto", Type::Serial), ("note", Type::Text), diff --git a/tests/typing_surface/snapshots/typing_surface_matrix__typing_surface__create_table__after_paren_expects_type_candidates@after_paren.snap b/tests/typing_surface/snapshots/typing_surface_matrix__typing_surface__create_table__after_paren_expects_type_candidates@after_paren.snap index 9aefae2..4e8a56f 100644 --- a/tests/typing_surface/snapshots/typing_surface_matrix__typing_surface__create_table__after_paren_expects_type_candidates@after_paren.snap +++ b/tests/typing_surface/snapshots/typing_surface_matrix__typing_surface__create_table__after_paren_expects_type_candidates@after_paren.snap @@ -1,5 +1,6 @@ --- source: tests/typing_surface/create_table.rs +assertion_line: 89 description: "input=\"create table Customers with pk Code(\" cursor=36" expression: "& a" --- @@ -45,11 +46,6 @@ Assessment { kind: Keyword, mode: Both, }, - Candidate { - text: "blob", - kind: Keyword, - mode: Both, - }, Candidate { text: "serial", kind: Keyword, @@ -107,11 +103,6 @@ Assessment { kind: Keyword, mode: Both, }, - Candidate { - text: "blob", - kind: Keyword, - mode: Both, - }, Candidate { text: "serial", kind: Keyword, diff --git a/tests/typing_surface/snapshots/typing_surface_matrix__typing_surface__delete_with_where__delete_with_datetime_column_says_yyyy_mm_dd_t@datetime_column.snap b/tests/typing_surface/snapshots/typing_surface_matrix__typing_surface__delete_with_where__delete_with_datetime_column_says_yyyy_mm_dd_t@datetime_column.snap index 97aeafb..1cadd1d 100644 --- a/tests/typing_surface/snapshots/typing_surface_matrix__typing_surface__delete_with_where__delete_with_datetime_column_says_yyyy_mm_dd_t@datetime_column.snap +++ b/tests/typing_surface/snapshots/typing_surface_matrix__typing_surface__delete_with_where__delete_with_datetime_column_says_yyyy_mm_dd_t@datetime_column.snap @@ -1,5 +1,6 @@ --- source: tests/typing_surface/delete_with_where.rs +assertion_line: 76 description: "input=\"delete from Things where ts=\" cursor=28" expression: "& a" --- @@ -35,11 +36,6 @@ Assessment { kind: Identifier, mode: Both, }, - Candidate { - text: "data", - kind: Identifier, - mode: Both, - }, Candidate { text: "dt", kind: Identifier, diff --git a/tests/typing_surface/snapshots/typing_surface_matrix__typing_surface__rename_change_column__change_after_open_paren_offers_type_candidates@change_after_open_paren.snap b/tests/typing_surface/snapshots/typing_surface_matrix__typing_surface__rename_change_column__change_after_open_paren_offers_type_candidates@change_after_open_paren.snap index 7832b9a..44d70ec 100644 --- a/tests/typing_surface/snapshots/typing_surface_matrix__typing_surface__rename_change_column__change_after_open_paren_offers_type_candidates@change_after_open_paren.snap +++ b/tests/typing_surface/snapshots/typing_surface_matrix__typing_surface__rename_change_column__change_after_open_paren_offers_type_candidates@change_after_open_paren.snap @@ -1,5 +1,6 @@ --- source: tests/typing_surface/rename_change_column.rs +assertion_line: 69 description: "input=\"change column in Customers: Email (\" cursor=35" expression: "& a" --- @@ -45,11 +46,6 @@ Assessment { kind: Keyword, mode: Both, }, - Candidate { - text: "blob", - kind: Keyword, - mode: Both, - }, Candidate { text: "serial", kind: Keyword, @@ -107,11 +103,6 @@ Assessment { kind: Keyword, mode: Both, }, - Candidate { - text: "blob", - kind: Keyword, - mode: Both, - }, Candidate { text: "serial", kind: Keyword, diff --git a/tests/typing_surface/snapshots/typing_surface_matrix__typing_surface__where_expression__between_expects_a_low_bound@between_low.snap b/tests/typing_surface/snapshots/typing_surface_matrix__typing_surface__where_expression__between_expects_a_low_bound@between_low.snap index 621829e..e65e4c1 100644 --- a/tests/typing_surface/snapshots/typing_surface_matrix__typing_surface__where_expression__between_expects_a_low_bound@between_low.snap +++ b/tests/typing_surface/snapshots/typing_surface_matrix__typing_surface__where_expression__between_expects_a_low_bound@between_low.snap @@ -1,5 +1,6 @@ --- source: tests/typing_surface/where_expression.rs +assertion_line: 50 description: "input=\"delete from Things where k between \" cursor=35" expression: "& a" --- @@ -35,11 +36,6 @@ Assessment { kind: Identifier, mode: Both, }, - Candidate { - text: "data", - kind: Identifier, - mode: Both, - }, Candidate { text: "dt", kind: Identifier, diff --git a/tests/typing_surface/snapshots/typing_surface_matrix__typing_surface__where_expression__in_list_open_paren_expects_an_item@in_open_paren.snap b/tests/typing_surface/snapshots/typing_surface_matrix__typing_surface__where_expression__in_list_open_paren_expects_an_item@in_open_paren.snap index f83a641..e55d940 100644 --- a/tests/typing_surface/snapshots/typing_surface_matrix__typing_surface__where_expression__in_list_open_paren_expects_an_item@in_open_paren.snap +++ b/tests/typing_surface/snapshots/typing_surface_matrix__typing_surface__where_expression__in_list_open_paren_expects_an_item@in_open_paren.snap @@ -1,5 +1,6 @@ --- source: tests/typing_surface/where_expression.rs +assertion_line: 58 description: "input=\"delete from Things where k in (\" cursor=31" expression: "& a" --- @@ -35,11 +36,6 @@ Assessment { kind: Identifier, mode: Both, }, - Candidate { - text: "data", - kind: Identifier, - mode: Both, - }, Candidate { text: "dt", kind: Identifier, diff --git a/website/src/content/docs/reference/generating-sample-data.mdx b/website/src/content/docs/reference/generating-sample-data.mdx index f0e401a..3b92ef0 100644 --- a/website/src/content/docs/reference/generating-sample-data.mdx +++ b/website/src/content/docs/reference/generating-sample-data.mdx @@ -306,9 +306,6 @@ automatically — no override needed. (a guard against a typo like `seed members 1000000`). Seed in smaller batches if you genuinely need more. - `seed members 0` does nothing. -- A `not null` column seed cannot produce a value for — the only real case is - a `not null blob` — makes seed refuse the whole command and name the column, - rather than fail partway through. A whole `seed` is a **single step** in the history: one [`undo`](/using-the-playground/undo-and-history/) removes every row it added, not one row at a time. diff --git a/website/src/content/docs/reference/types.md b/website/src/content/docs/reference/types.md index b4bf073..b4057ab 100644 --- a/website/src/content/docs/reference/types.md +++ b/website/src/content/docs/reference/types.md @@ -1,14 +1,14 @@ --- title: Types -description: The ten column types, what they store, and the auto-generating serial and shortid types. +description: The nine column types, what they store, and the auto-generating serial and shortid types. sidebar: order: 1 --- -Every column has a type. The playground offers ten, chosen to cover what a +Every column has a type. The playground offers nine, chosen to cover what a learner needs without the sprawl of a production database. -## The ten types +## The nine types | Type | Stores | |---|---| @@ -19,7 +19,6 @@ learner needs without the sprawl of a production database. | `bool` | A truth value, shown as `true` / `false`. | | `date` | A calendar date, `YYYY-MM-DD`. | | `datetime` | A date and time, `YYYY-MM-DDTHH:MM:SS`. | -| `blob` | Arbitrary binary data. | | `serial` | An auto-incrementing whole number — see below. | | `shortid` | A short random identifier — see below. | @@ -68,5 +67,5 @@ In advanced mode you may also use familiar standard-SQL spellings, which map onto the types above — for example `integer`, `bigint`, and `smallint` are all `int`; `varchar` and `char` are `text`; `boolean` is `bool`; `timestamp` is `datetime`; `numeric` is `decimal`; `float` and `double precision` are -`real`. Simple mode uses only the ten names in the table, so it teaches one +`real`. Simple mode uses only the nine names in the table, so it teaches one clear vocabulary. diff --git a/website/src/grammars/rdbms.mjs b/website/src/grammars/rdbms.mjs index e28b66b..44ee865 100644 --- a/website/src/grammars/rdbms.mjs +++ b/website/src/grammars/rdbms.mjs @@ -48,7 +48,7 @@ const repository = { name: 'constant.language.rdbms', }, type: { - match: '(?i)\\b(text|int|real|decimal|bool|date|datetime|blob|serial|shortid)\\b', + match: '(?i)\\b(text|int|real|decimal|bool|date|datetime|serial|shortid)\\b', name: 'support.type.rdbms', }, keyword: {