feat(types)!: drop the blob column type (ADR-0005 Amendment 2)
ci / gate (push) Failing after 35s
ci / manifests (push) Successful in 3s
website / deploy (push) Successful in 35s

blob was a dead-end: declarable but never fillable (no literal in either
mode, seed-unsupported), so a blob column could only ever hold NULL. Remove
Type::Blob + Value/CellValue::Blob + the base64 CSV path + the grammar slot
+ completion/render/type-change/seed handling + the binder refusal (whose
message also carried a user-facing "DSL" copy-rule bug). The vocabulary is
now nine types; base64 stays (clipboard OSC-52).

Backward compat is the ADR-0015 migration framework's first real use: a
v1->v2 format bump (CURRENT_SCHEMA_VERSION across serializer/parser/skeleton)
with a migrator that rewrites `type: blob` -> `type: text`, and a forced .db
rebuild from the migrated text when a blob column was actually converted
(the stale .db keeps a STRICT BLOB engine column). Conversion to text is
non-destructive and CSV-free. Covered by a full-stack integration test and a
Tier-4 PTY test that opens a real legacy v1-blob project.

Also sweeps the nine-type vocabulary through CLAUDE.md, requirements.md, the
website (type reference, seed doc, highlight grammar), and the ADR-0030/0033/
0035 cross-references; CHANGELOG Removed entry; handoff-79.

BREAKING CHANGE: the `blob` column type is removed. Existing projects that
declared a blob column are migrated on first open (the column becomes text;
the original project.yaml is kept as a .v1.bak).
This commit is contained in:
claude@clouddev1
2026-06-22 21:25:38 +00:00
parent 1a2002dbf6
commit 6b4c4dcea4
41 changed files with 506 additions and 390 deletions
+47
View File
@@ -32,6 +32,7 @@
// (the guard *is* the serialization) or pointless here.
#![allow(clippy::significant_drop_tightening)]
use std::fs;
use std::io::{Read, Write};
use std::path::Path;
use std::sync::{Arc, Mutex, MutexGuard};
@@ -404,6 +405,52 @@ fn back_to_back_insert_after_ddl_still_succeeds() {
app.quit();
}
/// Flow 6 — ADR-0005 Amendment 2 backward-compat: opening a **legacy v1
/// project that declares a `blob` column** migrates it to `text` and opens
/// cleanly. This drives the real runtime's migrate-on-open + rebuild path
/// end-to-end — the post-removal binary would otherwise reject the v1/blob
/// project file. (The integration test covers the migrator + rebuild via
/// direct calls; this covers the runtime glue that decides to run them.)
#[test]
fn opens_a_legacy_v1_blob_project_by_migrating_to_text() {
let dir = TempDir::new().expect("data dir");
let root = dir.path();
// Seed a v1 project (the old format) with a `blob` column, under the
// data root's projects/ dir, and point `--resume` at it. No
// playground.db is shipped, so the open rebuilds from the migrated text.
let proj = root.join("projects").join("Legacy");
fs::create_dir_all(proj.join("data")).expect("create project dirs");
fs::write(
proj.join("project.yaml"),
concat!(
"version: 1\n",
"project:\n created_at: 2026-01-01T00:00:00Z\n mode: simple\n",
"tables:\n",
" - name: Files\n",
" primary_key: [id]\n",
" columns:\n",
" - { name: id, type: serial }\n",
" - { name: payload, type: blob }\n",
"relationships: []\n",
"indexes: []\n",
),
)
.expect("write project.yaml");
fs::write(proj.join("data").join("Files.csv"), "id,payload\n1,aGVsbG8=\n").expect("write csv");
fs::write(root.join("last_project"), format!("{}\n", proj.display())).expect("write resume");
// Open via --resume: the runtime migrates blob→text and rebuilds, so
// the project opens and the table shows in the sidebar.
let mut app = PtyApp::launch_in(root, &["--resume"]);
app.wait_for_table("Files");
// The former blob column is now a normal text column — `show data`
// renders it (header + the base64 value preserved as text).
app.submit("show data Files");
app.wait_for("payload");
app.quit();
}
// ===================== NFR perf (measured, generous) ===================
//
// These run against the DEBUG binary, so the bounds are loose
+107
View File
@@ -0,0 +1,107 @@
//! Full-stack test for **ADR-0005 Amendment 2** (drop `blob`): a saved
//! v1 project that declares a `blob` column is migrated to v2 (`blob` →
//! `text`) on open, and a rebuild from the migrated text reconstructs the
//! column as `text` with the row data preserved — the base64 cell survives
//! as a recoverable string. This exercises the real migrator + the
//! `rebuild_from_text` path the runtime forces when a blob column was
//! converted (the runtime loop itself isn't booted in integration tests).
use std::fs;
use rdbms_playground::db::Database;
use rdbms_playground::dsl::Type;
use rdbms_playground::persistence::Persistence;
use rdbms_playground::persistence::migrations::{
MigratorRegistry, body_declares_blob_column, ensure_project_yaml_migrated,
};
fn rt() -> tokio::runtime::Runtime {
tokio::runtime::Builder::new_current_thread()
.enable_all()
.build()
.expect("tokio rt")
}
#[test]
fn v1_blob_project_migrates_to_text_and_rebuilds() {
let dir = tempfile::tempdir().expect("tempdir");
let root = dir.path();
// A hand-written v1 project: a serial PK + a `blob` column carrying a
// (hand-edited, base64) value — the only way bytes ever reached a blob
// column, since no command can insert one.
let yaml = concat!(
"version: 1\n",
"project:\n created_at: 2026-01-01T00:00:00Z\n mode: simple\n",
"tables:\n",
" - name: Files\n",
" primary_key: [id]\n",
" columns:\n",
" - { name: id, type: serial }\n",
" - { name: data, type: blob }\n",
"relationships: []\n",
"indexes: []\n",
);
fs::write(root.join("project.yaml"), yaml).expect("write yaml");
fs::create_dir(root.join("data")).expect("data dir");
fs::write(root.join("data").join("Files.csv"), "id,data\n1,aGVsbG8=\n").expect("write csv");
// The runtime's blob-detection signal sees the column…
assert!(body_declares_blob_column(yaml), "blob column detected");
// 1. Migration runs (as the runtime does on open).
let outcome = ensure_project_yaml_migrated(root, &MigratorRegistry::production())
.expect("migration succeeds");
assert_eq!(outcome.migrated_from, Some(1), "v1 project was migrated");
let migrated = fs::read_to_string(root.join("project.yaml")).expect("read migrated");
assert!(migrated.contains("version: 2"), "version bumped: {migrated}");
assert!(
migrated.contains("{ name: data, type: text }"),
"blob column rewritten to text: {migrated}"
);
assert!(!migrated.contains("type: blob"), "no blob type remains: {migrated}");
// The pre-migration original is preserved as a .bak.
let bak = fs::read_to_string(root.join("project.yaml.v1.bak")).expect("read bak");
assert!(bak.contains("type: blob"), "bak keeps the original blob column");
// 2. Rebuild from the migrated text (as the runtime forces when a blob
// column was converted): the column is now `text` and the row data
// survives as a recoverable string.
let db = Database::open_with_persistence(
root.join("playground.db"),
Persistence::new(root.to_path_buf()),
)
.expect("open db");
rt().block_on(async {
db.rebuild_from_text(root.to_path_buf(), None)
.await
.expect("rebuild");
let desc = db
.describe_table("Files".to_string())
.await
.expect("describe");
let data_col = desc
.columns
.iter()
.find(|c| c.name == "data")
.expect("data column present");
assert_eq!(
data_col.user_type,
Some(Type::Text),
"the former blob column is now text",
);
let rows = db
.query_data("Files".to_string(), None, None)
.await
.expect("query");
assert_eq!(rows.rows.len(), 1, "row preserved through the migration");
assert_eq!(
rows.rows[0][1].as_deref(),
Some("aGVsbG8="),
"the base64 cell survives as recoverable text",
);
});
}
+1
View File
@@ -7,6 +7,7 @@
//! `tests/typing_surface_matrix.rs` stays a separate binary (it is
//! already a consolidated `mod`-based target).
mod blob_removal_migration;
mod case_insensitive_names;
mod column_op_guards;
mod compound_fk;
-63
View File
@@ -617,69 +617,6 @@ fn seed_refuses_when_a_parent_table_is_empty() {
);
}
#[test]
fn seed_refuses_a_not_null_blob_column() {
let (_project, db, _dir) = open_project_db();
let rt = rt();
let mut payload = ColumnSpec::new("payload", Type::Blob);
payload.not_null = true;
rt.block_on(db.create_table(
"Files".to_string(),
vec![ColumnSpec::new("id", Type::Serial), payload],
vec!["id".to_string()],
None,
))
.expect("create Files");
let err = rt
.block_on(db.seed(
"Files".into(),
None,
Some(2),
Vec::new(),
Some(1),
Some("seed Files 2".into()),
))
.expect_err("seed must refuse a NOT NULL blob");
let msg = err.to_string();
assert!(
msg.contains("payload") && msg.to_lowercase().contains("blob"),
"error should name the un-generatable blob column: {msg}"
);
}
#[test]
fn seed_omits_a_nullable_blob_column() {
let (project, db, _dir) = open_project_db();
let rt = rt();
rt.block_on(db.create_table(
"Files".to_string(),
vec![
ColumnSpec::new("id", Type::Serial),
ColumnSpec::new("name", Type::Text),
// nullable blob → omitted (→ NULL), seed still succeeds.
ColumnSpec::new("payload", Type::Blob),
],
vec!["id".to_string()],
None,
))
.expect("create Files");
let res = rt
.block_on(db.seed(
"Files".into(),
None,
Some(3),
Vec::new(),
Some(1),
Some("seed Files 3".into()),
))
.expect("seed succeeds despite the nullable blob");
assert_eq!(res.produced, 3);
let csv = read_csv(&project, "Files").expect("Files CSV");
assert_eq!(data_row_count(&csv), 3);
}
// — uniqueness, junction distinct-combos, IN-CHECK (D10 / D14 / D17) —
/// The `n`th comma-separated field of each data row (the generated
-8
View File
@@ -356,14 +356,6 @@ fn e2e_alter_column_type_static_refusals() {
),
"text→serial is refused (only int→serial is allowed)"
);
assert!(
replay_is_refused(
"create table T with pk id(int)\n\
add column T: v (text)\n\
alter table T alter column v type blob\n",
),
"↔ blob is statically refused"
);
}
#[test]
+11 -14
View File
@@ -208,13 +208,12 @@ fn e2e_insert_select_cross_table_copies_rows_and_persists_both() {
// ===============================================================
#[test]
fn e2e_multirow_insert_all_ten_types_roundtrips_and_returning_recovers_each_type() {
fn e2e_multirow_insert_all_nine_types_roundtrips_and_returning_recovers_each_type() {
let (project, db, _dir) = open_project_db();
let rt = rt();
// serial PK + shortid auto-fill; the other eight columns are
// user-supplied. `blob` has no value-literal grammar yet
// (see src/dsl/value.rs), so it is inserted NULL — its *type*
// still round-trips through the RETURNING column-origin path.
// serial PK + shortid auto-fill; the other seven columns are
// user-supplied. Their *types* round-trip through the RETURNING
// column-origin path.
create_cols(
&db,
&rt,
@@ -228,7 +227,6 @@ fn e2e_multirow_insert_all_ten_types_roundtrips_and_returning_recovers_each_type
("flag", Type::Bool),
("d", Type::Date),
("ts", Type::DateTime),
("bl", Type::Blob),
("sid", Type::ShortId),
],
&["ser"],
@@ -237,17 +235,17 @@ fn e2e_multirow_insert_all_ten_types_roundtrips_and_returning_recovers_each_type
let result = run_insert(
&db,
&rt,
"insert into allten (txt, i, r, dec, flag, d, ts, bl) values \
('hi', 42, 1.5, 9.50, true, '2026-05-23', '2026-05-23 10:00:00', null), \
('yo', 7, 2.5, 3.25, false, '2025-01-01', '2025-01-01 00:00:00', null) \
returning ser, txt, i, r, dec, flag, d, ts, bl, sid",
"insert into allten (txt, i, r, dec, flag, d, ts) values \
('hi', 42, 1.5, 9.50, true, '2026-05-23', '2026-05-23 10:00:00'), \
('yo', 7, 2.5, 3.25, false, '2025-01-01', '2025-01-01 00:00:00') \
returning ser, txt, i, r, dec, flag, d, ts, sid",
)
.expect("multi-row INSERT … RETURNING runs");
assert_eq!(result.rows_affected, 2, "two rows inserted");
assert_eq!(result.data.rows.len(), 2, "RETURNING yields both rows");
// Every one of the ten playground types is recovered via the
// Every one of the nine playground types is recovered via the
// RETURNING column-origin path (matrix R5).
assert_eq!(
result.data.column_types,
@@ -260,10 +258,9 @@ fn e2e_multirow_insert_all_ten_types_roundtrips_and_returning_recovers_each_type
Some(Type::Bool),
Some(Type::Date),
Some(Type::DateTime),
Some(Type::Blob),
Some(Type::ShortId),
],
"RETURNING recovers each of the ten playground types; got {:?}",
"RETURNING recovers each of the nine playground types; got {:?}",
result.data.column_types,
);
@@ -281,7 +278,7 @@ fn e2e_multirow_insert_all_ten_types_roundtrips_and_returning_recovers_each_type
"dates round-trip: {csv:?}"
);
let sids: Vec<&str> = rows.iter().filter_map(|r| r[9].as_deref()).collect();
let sids: Vec<&str> = rows.iter().filter_map(|r| r[8].as_deref()).collect();
assert_eq!(sids.len(), 2, "both shortids present");
assert!(
sids.iter().all(|s| !s.is_empty()),
+4 -31
View File
@@ -575,11 +575,7 @@ fn database_run_select_type_recovery_works_on_empty_table() {
// when no row matches.
//
// This test pins that invariant: a fresh table with no
// rows still yields the right `column_types` entry. It
// also justifies the all-types test below using NULL for
// col_blob (the DSL Value enum has no Blob variant, but
// since metadata doesn't read row values, a NULL cell
// doesn't compromise the recovery).
// rows still yields the right `column_types` entry.
let (_p, db, _dir) = open_project_db();
let rt = rt();
rt.block_on(async {
@@ -588,7 +584,6 @@ fn database_run_select_type_recovery_works_on_empty_table() {
vec![
ColumnSpec::new("id", Type::Serial),
ColumnSpec::new("col_text", Type::Text),
ColumnSpec::new("col_blob", Type::Blob),
],
vec!["id".to_string()],
None,
@@ -602,32 +597,16 @@ fn database_run_select_type_recovery_works_on_empty_table() {
.expect("SELECT runs even on empty table");
assert!(data_text.rows.is_empty());
assert_eq!(data_text.column_types, vec![Some(Type::Text)]);
let data_blob = rt
.block_on(db.run_select("select col_blob from Empty".to_string()))
.expect("SELECT runs even on empty table");
assert!(data_blob.rows.is_empty());
assert_eq!(
data_blob.column_types,
vec![Some(Type::Blob)],
"Blob metadata must be recoverable even with no row data",
);
}
#[test]
fn database_run_select_recovers_all_ten_playground_types() {
fn database_run_select_recovers_all_nine_playground_types() {
// ADR-0032 §12 + Amendment 1 — every playground type
// round-trips through column-origin metadata on a bare
// projection ref. One table holds one column of each
// type; a SELECT of that column produces the right
// `column_types[0]` entry.
//
// `serial` and `shortid` are auto-generated. `col_blob`
// is left NULL in the inserted row because the DSL Value
// enum has no Blob variant — but per
// `database_run_select_type_recovery_works_on_empty_table`
// above, column-origin metadata is row-independent, so
// the NULL cell doesn't compromise this test's correctness.
// `column_types[0]` entry. `serial` and `shortid` are
// auto-generated.
let (_p, db, _dir) = open_project_db();
let rt = rt();
rt.block_on(async {
@@ -642,7 +621,6 @@ fn database_run_select_recovers_all_ten_playground_types() {
ColumnSpec::new("col_bool", Type::Bool),
ColumnSpec::new("col_date", Type::Date),
ColumnSpec::new("col_datetime", Type::DateTime),
ColumnSpec::new("col_blob", Type::Blob),
ColumnSpec::new("col_shortid", Type::ShortId),
],
vec!["pk".to_string()],
@@ -650,10 +628,6 @@ fn database_run_select_recovers_all_ten_playground_types() {
)
.await
.expect("create table");
// Blob has no DSL literal form, so col_blob takes the
// default NULL on insert. Column-origin metadata is
// based on the column DEFINITION, not the row value
// (Amendment 1), so the type recovery still succeeds.
db.insert(
"AllTypes".to_string(),
Some(vec![
@@ -691,7 +665,6 @@ fn database_run_select_recovers_all_ten_playground_types() {
("col_bool", Type::Bool),
("col_date", Type::Date),
("col_datetime", Type::DateTime),
("col_blob", Type::Blob),
("col_shortid", Type::ShortId),
];
for (col, expected_type) in cases {
-1
View File
@@ -117,7 +117,6 @@ pub fn schema_every_type() -> SchemaCache {
("b", Type::Bool),
("dt", Type::Date),
("ts", Type::DateTime),
("data", Type::Blob),
("sid", Type::ShortId),
("auto", Type::Serial),
("note", Type::Text),
@@ -1,5 +1,6 @@
---
source: tests/typing_surface/create_table.rs
assertion_line: 89
description: "input=\"create table Customers with pk Code(\" cursor=36"
expression: "& a"
---
@@ -45,11 +46,6 @@ Assessment {
kind: Keyword,
mode: Both,
},
Candidate {
text: "blob",
kind: Keyword,
mode: Both,
},
Candidate {
text: "serial",
kind: Keyword,
@@ -107,11 +103,6 @@ Assessment {
kind: Keyword,
mode: Both,
},
Candidate {
text: "blob",
kind: Keyword,
mode: Both,
},
Candidate {
text: "serial",
kind: Keyword,
@@ -1,5 +1,6 @@
---
source: tests/typing_surface/delete_with_where.rs
assertion_line: 76
description: "input=\"delete from Things where ts=\" cursor=28"
expression: "& a"
---
@@ -35,11 +36,6 @@ Assessment {
kind: Identifier,
mode: Both,
},
Candidate {
text: "data",
kind: Identifier,
mode: Both,
},
Candidate {
text: "dt",
kind: Identifier,
@@ -1,5 +1,6 @@
---
source: tests/typing_surface/rename_change_column.rs
assertion_line: 69
description: "input=\"change column in Customers: Email (\" cursor=35"
expression: "& a"
---
@@ -45,11 +46,6 @@ Assessment {
kind: Keyword,
mode: Both,
},
Candidate {
text: "blob",
kind: Keyword,
mode: Both,
},
Candidate {
text: "serial",
kind: Keyword,
@@ -107,11 +103,6 @@ Assessment {
kind: Keyword,
mode: Both,
},
Candidate {
text: "blob",
kind: Keyword,
mode: Both,
},
Candidate {
text: "serial",
kind: Keyword,
@@ -1,5 +1,6 @@
---
source: tests/typing_surface/where_expression.rs
assertion_line: 50
description: "input=\"delete from Things where k between \" cursor=35"
expression: "& a"
---
@@ -35,11 +36,6 @@ Assessment {
kind: Identifier,
mode: Both,
},
Candidate {
text: "data",
kind: Identifier,
mode: Both,
},
Candidate {
text: "dt",
kind: Identifier,
@@ -1,5 +1,6 @@
---
source: tests/typing_surface/where_expression.rs
assertion_line: 58
description: "input=\"delete from Things where k in (\" cursor=31"
expression: "& a"
---
@@ -35,11 +36,6 @@ Assessment {
kind: Identifier,
mode: Both,
},
Candidate {
text: "data",
kind: Identifier,
mode: Both,
},
Candidate {
text: "dt",
kind: Identifier,