feat(types)!: drop the blob column type (ADR-0005 Amendment 2)
ci / gate (push) Failing after 35s
ci / manifests (push) Successful in 3s
website / deploy (push) Successful in 35s

blob was a dead-end: declarable but never fillable (no literal in either
mode, seed-unsupported), so a blob column could only ever hold NULL. Remove
Type::Blob + Value/CellValue::Blob + the base64 CSV path + the grammar slot
+ completion/render/type-change/seed handling + the binder refusal (whose
message also carried a user-facing "DSL" copy-rule bug). The vocabulary is
now nine types; base64 stays (clipboard OSC-52).

Backward compat is the ADR-0015 migration framework's first real use: a
v1->v2 format bump (CURRENT_SCHEMA_VERSION across serializer/parser/skeleton)
with a migrator that rewrites `type: blob` -> `type: text`, and a forced .db
rebuild from the migrated text when a blob column was actually converted
(the stale .db keeps a STRICT BLOB engine column). Conversion to text is
non-destructive and CSV-free. Covered by a full-stack integration test and a
Tier-4 PTY test that opens a real legacy v1-blob project.

Also sweeps the nine-type vocabulary through CLAUDE.md, requirements.md, the
website (type reference, seed doc, highlight grammar), and the ADR-0030/0033/
0035 cross-references; CHANGELOG Removed entry; handoff-79.

BREAKING CHANGE: the `blob` column type is removed. Existing projects that
declared a blob column are migrated on first open (the column becomes text;
the original project.yaml is kept as a .v1.bak).
This commit is contained in:
claude@clouddev1
2026-06-22 21:25:38 +00:00
parent 1a2002dbf6
commit 6b4c4dcea4
41 changed files with 506 additions and 390 deletions
+2 -3
View File
@@ -631,7 +631,7 @@ pub fn candidates_at_cursor_with_in_mode(
// Source 1.5: type-name candidates when the walker expects
// a column-type slot. Type names are a closed set sourced
// from `Type::all()` (ADR-0005 declaration order:
// text/int/real/decimal/bool/date/datetime/blob/serial/
// text/int/real/decimal/bool/date/datetime/serial/
// shortid). The walker surfaces this as
// `Expectation::Ident { source: Types }`.
let type_names: Vec<String> = if expected.iter().any(|e| {
@@ -2309,7 +2309,7 @@ mod tests {
#[test]
fn type_slot_offers_full_type_vocabulary_when_partial_empty() {
// After `add column to T: Name (` the parser expects
// a column type. With no partial typed, all ten types
// a column type. With no partial typed, all nine types
// from `Type::all()` are offered in declaration order.
let cs = cands("add column to T: Name (", 23);
assert_eq!(
@@ -2322,7 +2322,6 @@ mod tests {
"bool".to_string(),
"date".to_string(),
"datetime".to_string(),
"blob".to_string(),
"serial".to_string(),
"shortid".to_string(),
],
+7 -53
View File
@@ -3169,7 +3169,10 @@ fn row_value_to_cell(row: &rusqlite::Row<'_>, idx: usize) -> Result<CellValue, D
ValueRef::Integer(n) => CellValue::Integer(n),
ValueRef::Real(f) => CellValue::Real(f),
ValueRef::Text(bytes) => CellValue::Text(String::from_utf8_lossy(bytes).into_owned()),
ValueRef::Blob(bytes) => CellValue::Blob(bytes.to_vec()),
// `blob` was removed from the vocabulary (ADR-0005 Amendment 2), so no
// column is BLOB-typed any more and a blob value-ref is not expected.
// Surface any stray bytes as lossy text rather than panicking.
ValueRef::Blob(bytes) => CellValue::Text(String::from_utf8_lossy(bytes).into_owned()),
})
}
@@ -3627,11 +3630,6 @@ fn do_drop_table(
/// rows receive a generated value (1..N for serial, fresh
/// shortids for shortid) and the column gains UNIQUE +
/// NOT NULL.
///
/// `blob` is statically refused as a column-add target by the
/// existing infrastructure (no DSL literal, downstream INSERT
/// would fail), but the path itself is allowed via the plain
/// branch — same as today.
fn do_add_column(
conn: &Connection,
persistence: Option<&Persistence>,
@@ -5043,7 +5041,7 @@ fn do_rename_table(
///
/// 1. Target is `serial` (auto-increment is create-table-only).
/// 2. Source ↔ target is statically refused per the matrix
/// (same-type, blob, date↔datetime, undefined cross-domain).
/// (same-type, date↔datetime, undefined cross-domain).
/// 3. Column is the *child* side of any relationship (outbound
/// FK) — drop the relationship first.
/// 4. Column has any inbound FK (parent side) and the new type
@@ -8641,9 +8639,7 @@ fn sample_parent_key_tuples(
///
/// Foreign-key columns are filled by sampling existing parent rows
/// (D14); a compound FK reads all its child columns from one sampled
/// parent row. An empty parent is refused with a friendly error. A
/// `NOT NULL blob` column (which seed cannot generate) is refused by
/// the block guard (D1); a nullable blob is omitted (→ NULL).
/// parent row. An empty parent is refused with a friendly error.
///
/// **Phase 2 (SD2):** when `target_column` is `Some`, this delegates to
/// [`do_seed_column_fill`] (fill one column across existing rows, D1
@@ -8719,17 +8715,6 @@ fn do_seed(
if matches!(ty, Type::Serial) {
continue;
}
// blob has no DSL value path: refuse if required (D1), else omit.
if matches!(ty, Type::Blob) {
if c.notnull {
return Err(DbError::Unsupported(format!(
"cannot seed `{table}`: column `{}` is `NOT NULL` but has type `blob`, \
which seed cannot generate. Add the rows another way or make it nullable.",
c.name,
)));
}
continue;
}
col_names.push(c.name.clone());
if let Some(&(fk_idx, pos)) = fk_child_pos.get(c.name.as_str()) {
plans.push(SeedColPlan::ForeignKey { fk_idx, pos });
@@ -9122,7 +9107,7 @@ fn seed_override_literal(value: &Value, column: &str) -> Result<String, DbError>
/// Column-fill (ADR-0048 D1 form 2): fill one column across the table's
/// **existing** rows (an UPDATE), the natural follow-up to `add column`.
///
/// Refuses PK and auto-generated (`serial`/`shortid`/`blob`) targets;
/// Refuses PK and auto-generated (`serial`/`shortid`) targets;
/// an empty table is a friendly no-op. The `set` clause may only adjust
/// the column being filled (the rest of the per-column heuristics do not
/// apply — there is exactly one column). A UNIQUE / identifier target
@@ -9178,12 +9163,6 @@ fn do_seed_column_fill(
ty.keyword(),
)));
}
if matches!(ty, Type::Blob) {
return Err(DbError::Unsupported(format!(
"cannot fill `{table}.{canonical_col}`: seed cannot generate `blob` values."
)));
}
// The `set` clause may only adjust the filled column (user decision).
for ov in overrides {
if !ov.column.eq_ignore_ascii_case(&canonical_col) {
@@ -11230,7 +11209,6 @@ fn cell_value_to_sqlite(cell: &CellValue) -> rusqlite::types::Value {
CellValue::Integer(n) => Value::Integer(*n),
CellValue::Real(f) => Value::Real(*f),
CellValue::Text(s) => Value::Text(s.clone()),
CellValue::Blob(b) => Value::Blob(b.clone()),
}
}
@@ -12912,30 +12890,6 @@ mod tests {
}
}
#[tokio::test]
async fn change_column_type_blob_target_refused_statically() {
let db = db();
make_id_table(&db, "T").await;
db.add_column(
"T".to_string(),
ColumnSpec::new("Note".to_string(), Type::Text),
None,
)
.await
.unwrap();
let err = db
.change_column_type(
"T".to_string(),
"Note".to_string(),
Type::Blob,
ChangeColumnMode::Default,
None,
)
.await
.unwrap_err();
assert!(matches!(err, DbError::Unsupported(_)), "got {err:?}");
}
#[tokio::test]
async fn change_column_type_outbound_fk_refused() {
// Child-side FK column: §4.2 says always refuse,
+1 -7
View File
@@ -263,11 +263,6 @@ const DATETIME_SLOT: Node = Node::TypedValueSlot {
column_name: None,
inner: &TEXT_SLOT_INNER,
};
const BLOB_SLOT: Node = Node::TypedValueSlot {
ty: Type::Blob,
column_name: None,
inner: &TEXT_SLOT_INNER,
};
// shortid columns store base58 text (ADR-0011 fk_target_type
// shortid → text); the slot accepts a quoted-text literal or
// null.
@@ -297,7 +292,6 @@ pub const fn slot_for_type(ty: Type) -> Node {
Type::Text => TEXT_SLOT,
Type::Date => DATE_SLOT,
Type::DateTime => DATETIME_SLOT,
Type::Blob => BLOB_SLOT,
}
}
@@ -310,7 +304,7 @@ const fn slot_inner_for_type(ty: Type) -> &'static Node {
Type::Real => &REAL_SLOT_INNER,
Type::Decimal => &DECIMAL_SLOT_INNER,
Type::Bool => &BOOL_SLOT_INNER,
Type::Text | Type::Date | Type::DateTime | Type::Blob | Type::ShortId => &TEXT_SLOT_INNER,
Type::Text | Type::Date | Type::DateTime | Type::ShortId => &TEXT_SLOT_INNER,
}
}
+2 -3
View File
@@ -567,7 +567,7 @@ mod tests {
fn standard_sql_type_aliases() {
good("table t (a integer, b varchar, c boolean, d timestamp)");
good("table t (e bigint, f smallint, g char, h numeric)");
good("table t (i binary, j varbinary, k float)");
good("table t (k float)");
}
#[test]
@@ -734,7 +734,7 @@ mod builder_tests {
fn standard_sql_aliases_map_to_playground_types() {
let (_, cols, _, _) = sct(
"create table t (a bigint, b varchar, c boolean, d timestamp, \
e numeric, f float, g binary)",
e numeric, f float)",
);
assert_eq!(
cols,
@@ -745,7 +745,6 @@ mod builder_tests {
("d".to_string(), Type::DateTime),
("e".to_string(), Type::Decimal),
("f".to_string(), Type::Real),
("g".to_string(), Type::Blob),
]
);
}
+17 -31
View File
@@ -1,7 +1,8 @@
//! User-facing column types and their mapping to SQLite STRICT.
//!
//! Implements the full ten-type vocabulary committed to in
//! ADR-0005. Storage choices for the text-backed types
//! Implements the nine-type vocabulary committed to in ADR-0005
//! (ten as originally decided; `blob` dropped by Amendment 2).
//! Storage choices for the text-backed types
//! (`decimal`, `date`, `datetime`) preserve precision and ISO
//! readability; comparisons rely on lexicographic ordering or
//! explicit casts at query time, which is acceptable for a
@@ -27,8 +28,6 @@ pub enum Type {
Date,
/// ISO 8601 datetime stored as `YYYY-MM-DDTHH:MM:SS[.fff][Z]` (TEXT).
DateTime,
/// Arbitrary binary data.
Blob,
/// Auto-incrementing integer; intended as a default primary key.
Serial,
/// 1012 character base58 random identifier (no ambiguous chars).
@@ -47,7 +46,6 @@ impl Type {
Self::Bool => "bool",
Self::Date => "date",
Self::DateTime => "datetime",
Self::Blob => "blob",
Self::Serial => "serial",
Self::ShortId => "shortid",
}
@@ -62,7 +60,6 @@ impl Type {
Self::Text | Self::ShortId | Self::Decimal | Self::Date | Self::DateTime => "TEXT",
Self::Int | Self::Serial | Self::Bool => "INTEGER",
Self::Real => "REAL",
Self::Blob => "BLOB",
}
}
@@ -79,8 +76,8 @@ impl Type {
/// All types known in this iteration, in stable order.
/// Ordering groups numeric types together, then boolean,
/// then temporal, then binary, then identity-flavoured
/// auto-generated types.
/// then temporal, then identity-flavoured auto-generated
/// types.
#[must_use]
pub const fn all() -> &'static [Self] {
&[
@@ -91,15 +88,14 @@ impl Type {
Self::Bool,
Self::Date,
Self::DateTime,
Self::Blob,
Self::Serial,
Self::ShortId,
]
}
/// True for the numeric types — `int`, `real`, `decimal`,
/// `serial`. `bool` (stored 0/1) and the text- / blob-backed
/// types are not numeric. Used to flag a `LIKE` text-pattern
/// `serial`. `bool` (stored 0/1) and the text-backed types
/// are not numeric. Used to flag a `LIKE` text-pattern
/// match against a numeric column (ADR-0027, Amendment 1).
#[must_use]
pub const fn is_numeric(self) -> bool {
@@ -129,7 +125,7 @@ impl Type {
}
/// Resolve a type name from the **advanced-mode SQL** type slot
/// (ADR-0035 §3). Accepts the ten playground keywords *and* the
/// (ADR-0035 §3). Accepts the nine playground keywords *and* the
/// standard-SQL aliases mapped onto them. Case-insensitive;
/// internal whitespace is collapsed so `double precision` resolves
/// regardless of spacing. Returns `None` for an unrecognised name —
@@ -137,7 +133,7 @@ impl Type {
/// diagnostic.
///
/// Deliberately distinct from [`FromStr`](std::str::FromStr), which
/// is the *simple-mode* parser and accepts only the ten keywords
/// is the *simple-mode* parser and accepts only the nine keywords
/// (no aliases), so simple mode teaches the playground's own
/// vocabulary. A length/precision argument (`varchar(255)`) is
/// stripped by the grammar before the name reaches this resolver.
@@ -157,8 +153,7 @@ impl Type {
"timestamp" => Some(Self::DateTime),
"numeric" => Some(Self::Decimal),
"float" | "double precision" => Some(Self::Real),
"binary" | "varbinary" => Some(Self::Blob),
// Fall through to the canonical ten keywords.
// Fall through to the canonical nine keywords.
other => other.parse::<Self>().ok(),
}
}
@@ -270,20 +265,19 @@ mod tests {
Type::Bool,
Type::Date,
Type::DateTime,
Type::Blob,
] {
assert_eq!(ty.fk_target_type(), ty);
}
}
#[test]
fn all_ten_types_are_present_and_distinct() {
fn all_nine_types_are_present_and_distinct() {
let kws: Vec<&'static str> = Type::all().iter().map(|t| t.keyword()).collect();
assert_eq!(kws.len(), 10);
assert_eq!(kws.len(), 9);
let mut sorted = kws.clone();
sorted.sort_unstable();
sorted.dedup();
assert_eq!(sorted.len(), 10, "keywords must be unique");
assert_eq!(sorted.len(), 9, "keywords must be unique");
}
#[test]
@@ -294,16 +288,10 @@ mod tests {
}
#[test]
fn blob_type_maps_to_blob_storage() {
assert_eq!(Type::Blob.sqlite_strict_type(), "BLOB");
}
#[test]
fn unknown_type_message_lists_all_ten() {
fn unknown_type_message_lists_all_nine() {
let err = "varchar".parse::<Type>().unwrap_err();
for kw in [
"text", "int", "real", "decimal", "bool", "date", "datetime", "blob", "serial",
"shortid",
"text", "int", "real", "decimal", "bool", "date", "datetime", "serial", "shortid",
] {
assert!(
err.expected.contains(kw),
@@ -314,12 +302,12 @@ mod tests {
}
// --- ADR-0035 §3: advanced-mode SQL type-name resolution ---
// `from_sql_name` accepts the ten playground keywords *and* the
// `from_sql_name` accepts the nine playground keywords *and* the
// standard-SQL aliases. Simple-mode `FromStr` is unchanged (it
// still rejects aliases — see `unknown_type_lists_expected_alternatives`).
#[test]
fn sql_resolver_accepts_the_ten_canonical_keywords() {
fn sql_resolver_accepts_the_nine_canonical_keywords() {
for &ty in Type::all() {
assert_eq!(Type::from_sql_name(ty.keyword()), Some(ty));
}
@@ -338,8 +326,6 @@ mod tests {
("numeric", Type::Decimal),
("float", Type::Real),
("double precision", Type::Real),
("binary", Type::Blob),
("varbinary", Type::Blob),
] {
assert_eq!(
Type::from_sql_name(alias),
-11
View File
@@ -101,10 +101,6 @@ impl Value {
Type::Bool => self.bind_bool(column),
Type::Date => self.bind_date(column),
Type::DateTime => self.bind_datetime(column),
Type::Blob => Err(ValueError::Format {
column: column.to_string(),
message: "literal `blob` values are not supported in DSL yet".to_string(),
}),
}
}
@@ -320,7 +316,6 @@ mod tests {
#[test]
fn null_binds_to_null_for_any_type() {
for ty in Type::all() {
// Skip blob — null still works there too.
assert_eq!(Value::Null.bind_for_column("c", *ty).unwrap(), Bound::Null);
}
}
@@ -429,10 +424,4 @@ mod tests {
let err = n("3..14").bind_for_column("c", Type::Decimal).unwrap_err();
assert!(matches!(err, ValueError::Format { .. }));
}
#[test]
fn blob_inserts_are_explicitly_unsupported_for_now() {
let err = t("0xdead").bind_for_column("c", Type::Blob).unwrap_err();
assert!(matches!(err, ValueError::Format { message, .. } if message.contains("blob")));
}
}
-2
View File
@@ -240,7 +240,6 @@ const fn catalog_key_for_value_type(ty: crate::dsl::types::Type) -> &'static str
Type::Text => "hint.value_slot_text",
Type::Date => "hint.value_slot_date",
Type::DateTime => "hint.value_slot_datetime",
Type::Blob => "hint.value_slot_blob",
Type::Serial => "hint.value_slot_serial",
Type::ShortId => "hint.value_slot_shortid",
}
@@ -4745,7 +4744,6 @@ mod tests {
(Type::Text, "hint.value_slot_text"),
(Type::Date, "hint.value_slot_date"),
(Type::DateTime, "hint.value_slot_datetime"),
(Type::Blob, "hint.value_slot_blob"),
] {
let schema = schema_with("T", &[("c", ty)]);
let mode = hint_mode_at_input_with_schema("insert into T values (", &schema);
-1
View File
@@ -425,7 +425,6 @@ pub const KEYS_AND_PLACEHOLDERS: &[(&str, &[&str])] = &[
("hint.value_literal_slot", &[]),
("hint.ambient_typing_name_then", &["next"]),
// Per-column-type value-slot hints (ADR-0024 §Phase D).
("hint.value_slot_blob", &[]),
("hint.value_slot_bool", &[]),
("hint.value_slot_date", &[]),
("hint.value_slot_datetime", &[]),
+1 -2
View File
@@ -395,7 +395,7 @@ help:
without running it (advanced SQL)
# Type reference, appended after the command list.
types_reference: |
Types: text, int, real, decimal, bool, date, datetime, blob, serial, shortid
Types: text, int, real, decimal, bool, date, datetime, serial, shortid
Auto-generated types (serial, shortid):
serial — integer that auto-fills with the next sequence value
(MAX(col)+1) on insert. Outside a primary key it carries
@@ -714,7 +714,6 @@ hint:
value_slot_text: "Type a quoted string (e.g. 'Alice') or null"
value_slot_date: "Type a quoted date as 'YYYY-MM-DD' or null"
value_slot_datetime: "Type a quoted datetime as 'YYYY-MM-DD HH:MM:SS' or null"
value_slot_blob: "Type a quoted blob literal or null"
# Serial / shortid in `values (…)` form: the user must enter
# something at this position (no "skip column" syntax). `null`
# is the auto-fill path (ADR-0018: serial / shortid columns
+1 -3
View File
@@ -345,7 +345,7 @@ const fn alignment_for(ty: Option<Type>) -> Alignment {
match ty {
Some(Type::Int | Type::Real | Type::Decimal | Type::Serial) => Alignment::Right,
Some(Type::Text) | Some(Type::Bool) | Some(Type::Date) | Some(Type::DateTime)
| Some(Type::Blob) | Some(Type::ShortId) | None => Alignment::Left,
| Some(Type::ShortId) | None => Alignment::Left,
}
}
@@ -1334,7 +1334,6 @@ mod tests {
sqlite_type: match ty {
Type::Int | Type::Serial | Type::Bool => "INTEGER".to_string(),
Type::Real => "REAL".to_string(),
Type::Blob => "BLOB".to_string(),
_ => "TEXT".to_string(),
},
notnull,
@@ -1361,7 +1360,6 @@ mod tests {
Type::Bool,
Type::Date,
Type::DateTime,
Type::Blob,
Type::ShortId,
] {
assert_eq!(alignment_for(Some(ty)), Alignment::Left, "{ty:?}");
-29
View File
@@ -25,8 +25,6 @@
use std::io::Write as _;
use base64::Engine as _;
use crate::dsl::types::Type;
use super::{CellValue, TableSnapshot};
@@ -149,12 +147,6 @@ fn encode_cell(ty: Type, value: &CellValue) -> Result<Cell, String> {
CellValue::Text(s) => Ok(Cell::Plain(s.clone())),
other => Err(format!("expected date/datetime (text), got {other:?}")),
},
Type::Blob => match value {
CellValue::Blob(bytes) => Ok(Cell::Plain(
base64::engine::general_purpose::STANDARD.encode(bytes),
)),
other => Err(format!("expected blob, got {other:?}")),
},
Type::Serial => match value {
CellValue::Integer(n) => Ok(Cell::Plain(n.to_string())),
other => Err(format!("expected serial (int), got {other:?}")),
@@ -362,10 +354,6 @@ pub(crate) fn decode_cell(ty: Type, cell: &RawCell) -> Result<CellValue, String>
"false" => Ok(CellValue::Integer(0)),
other => Err(format!("expected `true` or `false`, got `{other}`")),
},
Type::Blob => base64::engine::general_purpose::STANDARD
.decode(cell.content.as_bytes())
.map(CellValue::Blob)
.map_err(|e| format!("invalid base64 blob: {e}")),
}
}
@@ -467,18 +455,6 @@ mod tests {
assert_eq!(s, "b\ntrue\nfalse\n");
}
#[test]
fn blobs_use_base64() {
let body = serialize_table(&TableSnapshot {
name: "T".to_string(),
columns: vec![col("blob", Type::Blob)],
rows: vec![vec![CellValue::Blob(b"hello".to_vec())]],
})
.unwrap();
let s = String::from_utf8(body).unwrap();
assert!(s.contains("aGVsbG8="));
}
#[test]
fn dates_and_datetimes_pass_through() {
let body = serialize_table(&TableSnapshot {
@@ -548,13 +524,11 @@ mod tests {
col("n", Type::Int),
col("r", Type::Real),
col("b", Type::Bool),
col("blob", Type::Blob),
],
rows: vec![vec![
CellValue::Integer(42),
CellValue::Real(std::f64::consts::PI),
CellValue::Integer(1),
CellValue::Blob(b"hi".to_vec()),
]],
};
let body = serialize_table(&table).unwrap();
@@ -572,9 +546,6 @@ mod tests {
decode_cell(Type::Bool, &row[2]).unwrap(),
CellValue::Integer(1)
));
assert!(
matches!(decode_cell(Type::Blob, &row[3]).unwrap(), CellValue::Blob(b) if b == b"hi")
);
}
#[test]
+104 -16
View File
@@ -44,6 +44,35 @@ use serde::Deserialize;
/// migrator's job is purely the format transformation.
pub type MigrateFn = fn(&str) -> Result<String, MigrateError>;
/// The newest `project.yaml` format version this build writes and reads.
///
/// Must equal `MigratorRegistry::production().latest_version()` (asserted
/// in tests) and is what the YAML serializer stamps and the parser
/// accepts. Bumped 1 → 2 by **ADR-0005 Amendment 2** (drop `blob`).
pub const CURRENT_SCHEMA_VERSION: u32 = 2;
/// v1 → v2 migrator (ADR-0005 Amendment 2). `blob` was dropped from the
/// type vocabulary, so any `type: blob` column is rewritten to
/// `type: text` (the chosen conversion target — see the ADR). A pure,
/// line-oriented text transform: only column-definition lines (the
/// `- { name: …, type: … }` shape the serializer emits) are touched, so a
/// `blob` substring inside a CHECK expression or a default value is left
/// alone. Also bumps the `version:` field to 2 (the framework asserts the
/// advertised version, so this must stay in step).
fn migrate_v1_to_v2(body: &str) -> Result<String, MigrateError> {
let mut out = String::with_capacity(body.len() + 1);
for line in body.split_inclusive('\n') {
if line.trim_end() == "version: 1" {
out.push_str(&line.replacen("version: 1", "version: 2", 1));
} else if line.contains("{ name:") {
out.push_str(&line.replace(", type: blob", ", type: text"));
} else {
out.push_str(line);
}
}
Ok(out)
}
/// Ordered list of migrators. `migrators[i]` runs from
/// version `i + 1` to version `i + 2` (so index 0 is v1→v2,
/// index 1 is v2→v3, etc.).
@@ -56,13 +85,12 @@ pub struct MigratorRegistry {
}
impl MigratorRegistry {
/// Production-default registry: empty. As new versions
/// land, register the migrators here in source-version
/// order.
/// Production-default registry. Migrators are listed in
/// source-version order (index 0 = v1→v2, …).
#[must_use]
pub const fn production() -> Self {
pub fn production() -> Self {
Self {
migrators: Vec::new(),
migrators: vec![migrate_v1_to_v2],
}
}
@@ -156,6 +184,20 @@ pub struct MigrationOutcome {
pub migrated_from: Option<u32>,
}
/// Whether `body` declares at least one `type: blob` column.
///
/// The runtime uses this as the signal to force a `.db` rebuild after the
/// v1→v2 migration: the migration rewrites the YAML, but the derived
/// `.db` keeps a `STRICT … BLOB` engine column + `"blob"` metadata that
/// load otherwise uses as-is (ADR-0005 Amendment 2). Scoped to
/// column-definition lines, mirroring [`migrate_v1_to_v2`], so a `blob`
/// substring inside a CHECK expression or default doesn't trip it.
#[must_use]
pub fn body_declares_blob_column(body: &str) -> bool {
body.lines()
.any(|l| l.contains("{ name:") && l.contains(", type: blob"))
}
/// Detect the version of `body` and migrate it to the
/// registry's `latest_version()`.
///
@@ -304,22 +346,68 @@ mod tests {
.to_string()
}
/// A project at the current (latest) version — nothing to migrate.
fn v2_body() -> String {
"version: 2\nproject:\n created_at: 2026-01-01T00:00:00Z\ntables: []\nrelationships: []\n"
.to_string()
}
#[test]
fn production_registry_latest_version_is_1() {
fn production_registry_latest_version_matches_current_schema_version() {
let r = MigratorRegistry::production();
assert_eq!(r.latest_version(), 1);
assert_eq!(r.latest_version(), CURRENT_SCHEMA_VERSION);
assert_eq!(r.latest_version(), 2);
}
#[test]
fn no_migration_runs_when_body_already_latest() {
let tmp = tempdir();
let outcome =
migrate_to_latest(&v1_body(), &MigratorRegistry::production(), tmp.path()).unwrap();
assert_eq!(outcome.body, v1_body());
migrate_to_latest(&v2_body(), &MigratorRegistry::production(), tmp.path()).unwrap();
assert_eq!(outcome.body, v2_body());
assert_eq!(outcome.migrated_from, None);
// No .bak written when nothing migrated.
let bak = tmp.path().join("project.yaml.v1.bak");
assert!(!bak.exists(), "no .bak when no migration");
assert!(
!tmp.path().join("project.yaml.v2.bak").exists(),
"no .bak when no migration"
);
}
/// ADR-0005 Amendment 2: the real production v1→v2 migrator rewrites a
/// `blob` column to `text` and bumps the version, leaving everything
/// else (incl. a column literally named `blob`, and a `blob` substring
/// inside a CHECK) untouched.
#[test]
fn production_v1_to_v2_converts_blob_columns_to_text() {
let tmp = tempdir();
let body = concat!(
"version: 1\n",
"project:\n created_at: x\n",
"tables:\n",
" - name: Files\n",
" primary_key: [id]\n",
" columns:\n",
" - { name: id, type: serial }\n",
" - { name: payload, type: blob }\n",
" - { name: blob, type: text }\n",
" check_constraints:\n",
" - \"note <> 'type: blob'\"\n",
"relationships: []\n",
);
let outcome =
migrate_to_latest(body, &MigratorRegistry::production(), tmp.path()).unwrap();
assert_eq!(outcome.migrated_from, Some(1));
assert!(outcome.body.contains("version: 2"));
// The blob column became text.
assert!(
outcome.body.contains("{ name: payload, type: text }"),
"blob column converted: {}",
outcome.body
);
// A column NAMED blob is untouched; the CHECK string is untouched.
assert!(outcome.body.contains("{ name: blob, type: text }"));
assert!(outcome.body.contains("note <> 'type: blob'"));
assert!(!outcome.body.contains("type: blob }"));
}
#[test]
@@ -332,7 +420,7 @@ mod tests {
err,
MigrateError::NewerThanSupported {
file: 99,
latest: 1
latest: 2
}
),
"got: {err:?}",
@@ -393,17 +481,17 @@ mod tests {
}
#[test]
fn ensure_yaml_migrated_no_op_on_v1_with_empty_registry() {
fn ensure_yaml_migrated_no_op_when_already_latest() {
let tmp = tempdir();
let yaml_path = tmp.path().join("project.yaml");
std::fs::write(&yaml_path, v1_body()).unwrap();
std::fs::write(&yaml_path, v2_body()).unwrap();
let outcome =
ensure_project_yaml_migrated(tmp.path(), &MigratorRegistry::production()).unwrap();
assert_eq!(outcome.migrated_from, None);
// File unchanged.
let on_disk = std::fs::read_to_string(&yaml_path).unwrap();
assert_eq!(on_disk, v1_body());
assert!(!tmp.path().join("project.yaml.v1.bak").exists());
assert_eq!(on_disk, v2_body());
assert!(!tmp.path().join("project.yaml.v2.bak").exists());
}
#[test]
+1 -2
View File
@@ -280,7 +280,6 @@ pub enum CellValue {
Integer(i64),
Real(f64),
Text(String),
Blob(Vec<u8>),
}
impl Persistence {
@@ -561,7 +560,7 @@ mod tests {
};
p.write_schema(&schema).unwrap();
let body = fs::read_to_string(dir.path().join(PROJECT_YAML)).unwrap();
assert!(body.contains("version: 1"));
assert!(body.contains("version: 2"));
assert!(body.contains("created_at:"));
}
+22 -14
View File
@@ -32,7 +32,11 @@ use super::{
#[must_use]
pub(super) fn serialize_schema(schema: &SchemaSnapshot) -> String {
let mut out = String::new();
let _ = writeln!(out, "version: 1");
let _ = writeln!(
out,
"version: {}",
crate::persistence::migrations::CURRENT_SCHEMA_VERSION
);
let _ = writeln!(out, "project:");
let _ = writeln!(out, " created_at: {}", quote_if_needed(&schema.created_at));
// ADR-0015 mode-restore amendment (issue #14): the input mode
@@ -283,7 +287,11 @@ const fn is_safe_yaml_char(c: char) -> bool {
pub(crate) fn parse_schema(body: &str) -> Result<SchemaSnapshot, YamlError> {
let raw: RawProject =
serde_norway::from_str(body).map_err(|e| YamlError::Syntax(e.to_string()))?;
if raw.version != 1 {
// The migration framework (ADR-0015) upgrades older project files to
// `CURRENT_SCHEMA_VERSION` before `parse_schema` ever runs, so the
// parser only accepts the current version (a stale version reaching
// here means migration was skipped — a bug worth surfacing).
if raw.version != crate::persistence::migrations::CURRENT_SCHEMA_VERSION {
return Err(YamlError::UnsupportedVersion(raw.version));
}
let mut tables: Vec<TableSchema> = Vec::with_capacity(raw.tables.len());
@@ -617,7 +625,7 @@ mod tests {
let body = serialize_schema(&snapshot());
// Spot-check structural lines rather than asserting on
// the whole blob — easier to read in failure output.
assert!(body.contains("version: 1"));
assert!(body.contains("version: 2"));
assert!(body.contains("created_at: 2026-05-07T14:30:12Z"));
assert!(body.contains("- name: Customers"));
assert!(body.contains("primary_key: [id]"));
@@ -752,7 +760,7 @@ mod tests {
// Older project files (written before unique indexes) omit the
// `unique` field; the `#[serde(default)]` makes it `false`.
let body = "\
version: 1
version: 2
project:
created_at: 2026-05-25T00:00:00Z
tables:
@@ -922,7 +930,7 @@ indexes:
// Back-compat: a project file written before §4g (bare-string
// check_constraints) parses with name = None.
let body = "\
version: 1
version: 2
project:
created_at: \"2026-05-25T00:00:00Z\"
tables:
@@ -949,7 +957,7 @@ indexes: []
// A project file written before table-level CHECK existed (no
// `check_constraints:` key) parses with an empty list.
let body = "\
version: 1
version: 2
project:
created_at: 2026-05-25T00:00:00Z
tables:
@@ -966,7 +974,7 @@ relationships: []
#[test]
fn parses_minimal_yaml_with_no_tables() {
let body = "\
version: 1
version: 2
project:
created_at: 2026-05-07T14:30:12Z
tables: []
@@ -993,7 +1001,7 @@ relationships: []
#[test]
fn rejects_unknown_column_type() {
let body = "\
version: 1
version: 2
project:
created_at: x
tables:
@@ -1012,7 +1020,7 @@ relationships: []
#[test]
fn rejects_unknown_action() {
let body = "\
version: 1
version: 2
project:
created_at: x
tables: []
@@ -1090,7 +1098,7 @@ relationships:
fn parse_schema_defaults_mode_to_simple_when_field_absent() {
// A pre-#14 project file carries no `mode:` field; it must
// parse with the default mode, not fail.
let body = "version: 1\nproject:\n created_at: x\ntables: []\nrelationships: []\n";
let body = "version: 2\nproject:\n created_at: x\ntables: []\nrelationships: []\n";
let parsed = parse_schema(body).expect("legacy file parses");
assert_eq!(parsed.mode, Mode::Simple);
}
@@ -1100,13 +1108,13 @@ relationships:
// `None` (no stored preference) must be distinct from an
// explicit `simple`, so restore-on-open precedence can tell
// "fall back to default" from "the user chose simple".
let absent = "version: 1\nproject:\n created_at: x\ntables: []\n";
let absent = "version: 2\nproject:\n created_at: x\ntables: []\n";
assert_eq!(parse_stored_mode(absent), None);
let explicit_simple = "version: 1\nproject:\n created_at: x\n mode: simple\ntables: []\n";
let explicit_simple = "version: 2\nproject:\n created_at: x\n mode: simple\ntables: []\n";
assert_eq!(parse_stored_mode(explicit_simple), Some(Mode::Simple));
let advanced = "version: 1\nproject:\n created_at: x\n mode: advanced\ntables: []\n";
let advanced = "version: 2\nproject:\n created_at: x\n mode: advanced\ntables: []\n";
assert_eq!(parse_stored_mode(advanced), Some(Mode::Advanced));
}
@@ -1114,7 +1122,7 @@ relationships:
fn parse_stored_mode_falls_back_to_none_on_unknown_value() {
// An unrecognised mode keyword degrades to "no preference"
// rather than rejecting the whole file over a UI hint.
let body = "version: 1\nproject:\n created_at: x\n mode: expert\ntables: []\n";
let body = "version: 2\nproject:\n created_at: x\n mode: expert\ntables: []\n";
assert_eq!(parse_stored_mode(body), None);
}
}
+4 -3
View File
@@ -384,7 +384,7 @@ impl Project {
/// Build the on-disk skeleton for a fresh project: the
/// directory itself, an empty `data/`, an empty
/// `history.log`, a placeholder `project.yaml` with just
/// `version: 1` and `created_at`, and a `.gitignore`.
/// the current schema `version` and `created_at`, and a `.gitignore`.
///
/// `playground.db` is not created here; it's created the
/// first time `Database::open` runs against the path
@@ -401,7 +401,8 @@ impl Project {
// schema mutation; for now we just ensure the file
// exists and carries the version + creation timestamp.
let yaml = format!(
"version: 1\nproject:\n created_at: {}\ntables: []\nrelationships: []\n",
"version: {}\nproject:\n created_at: {}\ntables: []\nrelationships: []\n",
crate::persistence::migrations::CURRENT_SCHEMA_VERSION,
iso8601_now(),
);
write_if_missing(&path.join(PROJECT_YAML), &yaml)?;
@@ -830,7 +831,7 @@ mod tests {
// YAML carries version + created_at.
let yaml = fs::read_to_string(path.join(PROJECT_YAML)).unwrap();
assert!(yaml.contains("version: 1"));
assert!(yaml.contains("version: 2"));
assert!(yaml.contains("created_at:"));
// .gitignore matches ADR-0015.
+25 -3
View File
@@ -176,6 +176,13 @@ pub async fn run(args: Args) -> Result<()> {
// `project.yaml.v<N>.bak` breadcrumb on disk; that's
// sufficient v1 UX and lets us defer dedicated event
// plumbing until a real migrator demands it.
// ADR-0005 Amendment 2: note a pre-migration `blob` column so we can
// force a `.db` rebuild after the v1→v2 migration converts it to text
// (the stale `.db` keeps a `STRICT … BLOB` engine column + `"blob"`
// metadata that load otherwise uses as-is).
let had_blob_column = std::fs::read_to_string(project.path().join(crate::project::PROJECT_YAML))
.map(|b| crate::persistence::migrations::body_declares_blob_column(&b))
.unwrap_or(false);
let migrate_registry = crate::persistence::migrations::MigratorRegistry::production();
let migration_outcome = crate::persistence::migrations::ensure_project_yaml_migrated(
project.path(),
@@ -235,7 +242,12 @@ pub async fn run(args: Args) -> Result<()> {
Database::open_with_persistence_and_undo(db_path.as_path(), persistence, undo_enabled)
.context("open database")?;
let mut initial_events: Vec<AppEvent> = Vec::new();
if !db_existed {
// ADR-0005 Amendment 2: a blob→text migration leaves the existing
// `.db` with a stale `STRICT … BLOB` engine column, so rebuild it from
// the migrated text. A pure version bump with no schema change does not.
let force_rebuild_after_migration =
migration_outcome.migrated_from.is_some() && had_blob_column;
if !db_existed || force_rebuild_after_migration {
match database.rebuild_from_text(project_path.clone(), None).await {
Ok(()) => {
// Surface the silent rebuild as a system note
@@ -927,7 +939,13 @@ async fn perform_switch(
// state momentarily, but the next user action will
// surface the error and they can retry).
let migrate_registry = crate::persistence::migrations::MigratorRegistry::production();
crate::persistence::migrations::ensure_project_yaml_migrated(
// ADR-0005 Amendment 2: see the matching note in `run()` — a blob→text
// migration needs the `.db` rebuilt from the migrated text.
let had_blob_column =
std::fs::read_to_string(new_project.path().join(crate::project::PROJECT_YAML))
.map(|b| crate::persistence::migrations::body_declares_blob_column(&b))
.unwrap_or(false);
let migration_outcome = crate::persistence::migrations::ensure_project_yaml_migrated(
new_project.path(),
&migrate_registry,
)
@@ -949,7 +967,11 @@ async fn perform_switch(
let new_database =
Database::open_with_persistence_and_undo(&db_path, persistence, undo_enabled)
.map_err(|e| e.to_string())?;
if !db_existed && let Err(e) = new_database.rebuild_from_text(new_path.clone(), None).await {
let force_rebuild_after_migration =
migration_outcome.migrated_from.is_some() && had_blob_column;
if (!db_existed || force_rebuild_after_migration)
&& let Err(e) = new_database.rebuild_from_text(new_path.clone(), None).await
{
return Err(e.friendly_message());
}
+6 -12
View File
@@ -138,7 +138,7 @@ fn range_value(low: &str, high: &str, ty: Type, rng: &mut SeedRng) -> Value {
Type::DateTime => parse_datetime_range(low, high)
.map(|(lo, hi)| Value::Text(random_datetime_between(rng, lo, hi)))
.unwrap_or_else(|| generic_for_type(ty, rng)),
// text / bool / blob / shortid have no range meaning.
// text / bool / shortid have no range meaning.
_ => generic_for_type(ty, rng),
}
}
@@ -155,8 +155,8 @@ pub fn range_bounds_reason(ty: Type, low: &str, high: &str) -> Option<String> {
Type::Real | Type::Decimal => parse_real_range(low, high).is_some(),
Type::Date => parse_date_range(low, high).is_some(),
Type::DateTime => parse_datetime_range(low, high).is_some(),
// text / bool / blob / shortid have no range meaning.
Type::Text | Type::Bool | Type::Blob | Type::ShortId => false,
// text / bool / shortid have no range meaning.
Type::Text | Type::Bool | Type::ShortId => false,
};
if ok {
return None;
@@ -169,7 +169,7 @@ pub fn range_bounds_reason(ty: Type, low: &str, high: &str) -> Option<String> {
"expected two quoted datetimes, e.g. `between '2023-01-01T00:00:00' and '2024-12-31T23:59:59'`"
.to_string()
}
Type::Text | Type::Bool | Type::Blob | Type::ShortId => {
Type::Text | Type::Bool | Type::ShortId => {
"a `between` range only applies to numeric and date/datetime columns".to_string()
}
})
@@ -242,9 +242,8 @@ fn random_datetime_between(
}
/// Type-based fallback generation (D8). Never produces NULL for a
/// generatable type; `blob`/`serial`/`shortid` are handled by the
/// executor (autogen / block guard) and yield NULL here only as a
/// last resort.
/// generatable type; `serial`/`shortid` are handled by the executor
/// (autogen) and yield NULL here only as a last resort.
fn generic_for_type(ty: Type, rng: &mut SeedRng) -> Value {
use fake::faker::lorem::en as lorem;
match ty {
@@ -267,7 +266,6 @@ fn generic_for_type(ty: Type, rng: &mut SeedRng) -> Value {
Type::Bool => Value::Bool(rng.random_range(0..2) == 1),
Type::Date => Value::Text(format_date(random_past_date(rng, 0, RECENT_WINDOW_DAYS))),
Type::DateTime => Value::Text(random_recent_datetime(rng)),
Type::Blob => Value::Null,
}
}
@@ -705,10 +703,6 @@ mod tests {
generate_value(&Generator::Generic, Type::Bool, &mut rng),
Value::Bool(_)
));
assert!(matches!(
generate_value(&Generator::Generic, Type::Blob, &mut rng),
Value::Null
));
// shortid fallback is a valid base58 id.
let Value::Text(sid) = generate_value(&Generator::Generic, Type::ShortId, &mut rng) else {
panic!("shortid not text")
+4 -21
View File
@@ -17,8 +17,8 @@
//!
//! Pairs not present in the matrix are statically refused via
//! [`static_refusal`] before any per-cell pass runs. Same-type
//! identity, anything → `serial`, anything ↔ `blob`, and
//! `date` ↔ `datetime` direct are all statically refused.
//! identity, anything → `serial`, and `date` ↔ `datetime` direct
//! are all statically refused.
use rusqlite::types::Value;
@@ -46,9 +46,8 @@ pub enum CellOutcome {
/// refused; `None` when the per-cell matrix should be consulted.
///
/// Static refusals cover: same-type identity, anything →
/// `serial`, anything ↔ `blob`, `date` ↔ `datetime` direct, and
/// any cross-domain pair not present in the matrix
/// (e.g. `bool` → `date`).
/// `serial`, `date` ↔ `datetime` direct, and any cross-domain
/// pair not present in the matrix (e.g. `bool` → `date`).
#[must_use]
pub fn static_refusal(src: Type, target: Type) -> Option<String> {
if src == target {
@@ -63,12 +62,6 @@ pub fn static_refusal(src: Type, target: Type) -> Option<String> {
first; only `int serial` is supported directly."
));
}
if matches!(src, Type::Blob) || matches!(target, Type::Blob) {
return Some(format!(
"conversion between `{src}` and `{target}` is not supported \
in this version."
));
}
if matches!(
(src, target),
(Type::Date, Type::DateTime) | (Type::DateTime, Type::Date)
@@ -601,16 +594,6 @@ mod tests {
);
}
#[test]
fn anything_involving_blob_is_statically_refused() {
for &other in Type::all() {
if other != Type::Blob {
assert!(static_refusal(Type::Blob, other).is_some(), "{other:?}");
assert!(static_refusal(other, Type::Blob).is_some(), "{other:?}");
}
}
}
#[test]
fn date_to_datetime_direct_is_statically_refused() {
assert!(static_refusal(Type::Date, Type::DateTime).is_some());