DSL parser, async DB worker, types, history, metadata, polish
Track 1 implementation plus polish round. Parser (chumsky): - Grammar-based DSL producing a typed Command AST. - create table X with pk [name:type[,name:type...]] supports arbitrary names, any user type, compound PKs natively. Bare form errors with a friendly hint pointing at `with pk`. - add column to table X: Name (type); drop table X. - Required clauses use keyword grammar; -- reserved for opt-in flags (ADR-0009). Custom Rich reasons preferred when surfacing chumsky errors so unknown-type messages list valid alternatives. Database (ADR-0010, ADR-0012): - rusqlite + STRICT tables + foreign_keys=ON. - Dedicated worker thread; mpsc Request inbox, oneshot replies. - Typed DbError with friendly_message() hook for H1. - Internal __rdbms_playground_columns metadata table preserves user-facing types across schema reads, atomically maintained alongside DDL via Connection transactions. list_tables hides it via the new __rdbms_ internal-table convention. Types (ADR-0005, ADR-0011): - All ten user-facing types: text, int, real, decimal, bool, date, datetime, blob, serial, shortid. - Type::fk_target_type() for FK-side column-type rule (Serial->Int, ShortId->Text, others identity) -- foundation for the FK iteration. App / Runtime / UI: - update() stays pure-sync; runtime dispatches DSL via spawned tasks, results post back as AppEvent::Dsl*. - Items panel renders live tables list; output panel shows the user-facing structure of the current table after each DDL. - In-memory command history (Up/Down, draft preservation, consecutive-duplicate dedup) -- I2 partial. - Mouse capture removed; terminal native text selection restored (toggle approach revisited when scroll/click features land). Docs: - ADRs 0009 (DSL syntax conventions), 0010 (DB worker), 0011 (FK type compat), 0012 (internal metadata table). - requirements.md progress notes; new V4 entry for the scrollable session-log + inline rich rendering + Markdown export direction. Tests: 103 passing (91 lib + 12 integration), 0 skipped. Clippy clean with nursery enabled.
This commit is contained in:
claude@clouddev1
@@ -0,0 +1,87 @@
|
||||
# ADR-0009: DSL command syntax conventions
|
||||
|
||||
## Status
|
||||
|
||||
Accepted
|
||||
|
||||
## Context
|
||||
|
||||
As the DSL grows, its commands need consistent surface
|
||||
conventions. Without an explicit rule, every command would
|
||||
invent its own way of expressing optional vs. required parts,
|
||||
and the surface would drift toward an unreadable soup.
|
||||
|
||||
The decision is informed by experience from this iteration:
|
||||
when we initially proposed `create table X --pk` the most
|
||||
common form (a basic table with a primary key) required a `--`
|
||||
flag, which is cosmetically wrong — `--` reads as "extra
|
||||
option," and the most-used form should not look like one.
|
||||
|
||||
## Decision
|
||||
|
||||
The DSL surface follows three rules.
|
||||
|
||||
### 1. Required clauses use keyword grammar
|
||||
|
||||
Required parts of a command are written in plain words and
|
||||
read like English. Examples:
|
||||
|
||||
- `create table <Name> with pk <name>:<type>`
|
||||
- `add column to table <Name>: <Name> (<Type>)`
|
||||
- `drop table <Name>`
|
||||
|
||||
The `with` clause format is the canonical pattern for
|
||||
attaching required structural information to an entity-creating
|
||||
command, and is reusable: future iterations may add `with
|
||||
index`, `with check`, etc. Multiple `with` clauses on the same
|
||||
command are allowed in principle.
|
||||
|
||||
### 2. Optional flags use `--prefix`
|
||||
|
||||
Flags signal "I am asking for an extra capability or
|
||||
non-default behaviour." Examples planned for later iterations:
|
||||
|
||||
- `add 1:n relationship on Customers.Id=Orders.CustId --create-fk`
|
||||
(auto-creates the FK column instead of requiring it to exist)
|
||||
- *(future)* `--rename-on-clash`, `--no-strict`, etc.
|
||||
|
||||
A user reading "with pk id:serial" sees only what's needed; a
|
||||
user reading "...with pk id:serial --some-flag" sees that they
|
||||
have asked for something beyond default. The visual distinction
|
||||
is intentional.
|
||||
|
||||
### 3. One sigil only — `:` for the simple-mode advanced escape
|
||||
|
||||
Per ADR-0003, prefixing a single line with `:` in simple mode
|
||||
treats that one submission as if it were entered in advanced
|
||||
mode. This is the only sigil in the system. App-level commands,
|
||||
DSL commands, and SQL all use plain words.
|
||||
|
||||
### Lexical rules
|
||||
|
||||
- **Keywords are case-insensitive.** `CREATE TABLE Customers
|
||||
WITH PK email:TEXT` is equivalent to `create table Customers
|
||||
with pk email:text`.
|
||||
- **Identifiers are case-preserving.** `Customers` and
|
||||
`customers` are different identifiers if a backend would
|
||||
treat them as such (we follow SQLite's case-insensitive
|
||||
identifier rules at the schema level but preserve the user's
|
||||
written casing in display).
|
||||
- **Whitespace is liberal.** Any amount of horizontal whitespace
|
||||
between tokens is accepted, including around punctuation
|
||||
(`,`, `:`, `(`, `)`).
|
||||
|
||||
## Consequences
|
||||
|
||||
- The basic, most-common form of any command remains readable
|
||||
and free of cosmetic punctuation. New users see only words.
|
||||
- Optional adornments are visually distinct, encouraging
|
||||
discoverability of advanced features without forcing them on
|
||||
beginners.
|
||||
- New commands inherit a uniform shape: keyword-based clauses
|
||||
for required parts, `--` flags for opt-ins. Drift is bounded
|
||||
by this rule.
|
||||
- The grammar implementation (`chumsky`) maps cleanly onto this
|
||||
structure: a `with_clause` rule can be reused across
|
||||
commands, and flag parsing has a single representation when
|
||||
it lands.
|
||||
@@ -0,0 +1,93 @@
|
||||
# ADR-0010: Database access via a dedicated worker thread
|
||||
|
||||
## Status
|
||||
|
||||
Accepted
|
||||
|
||||
## Context
|
||||
|
||||
`rusqlite::Connection` is `Send` but `!Sync`, and its API is
|
||||
synchronous. Our application loop is asynchronous (Tokio,
|
||||
ADR-0001). We need a clean async boundary that keeps:
|
||||
|
||||
- The `App::update` function pure — same input, same state
|
||||
transition, same outputs (per ADR-0001's Elm-style design);
|
||||
this is what makes ADR-0008's Tier 1 and Tier 3 tests
|
||||
tractable.
|
||||
- The path to future requirements (B3 query timeout/cancellation,
|
||||
U1 snapshot capture, P3 auto-save) free of architectural
|
||||
refactors.
|
||||
- Errors surfaceable in a way that the future H1 friendly-error
|
||||
layer can plug into without callsite changes.
|
||||
|
||||
Considered alternatives:
|
||||
|
||||
1. **Sync calls from async tasks** — call rusqlite directly
|
||||
inside `update`. Blocks the event loop on every DB call.
|
||||
Unworkable as soon as queries grow.
|
||||
2. **`tokio::task::spawn_blocking` per call** — uses tokio's
|
||||
blocking thread pool. Acceptable for short ops but the pool
|
||||
is sized for short tasks, and a long-lived "this is the
|
||||
database connection" task is not its intended use.
|
||||
3. **Dedicated worker thread with typed request/response
|
||||
channels** — the connection lives on a thread we own; the
|
||||
App holds a sender; replies come back via per-request
|
||||
`oneshot` channels.
|
||||
|
||||
## Decision
|
||||
|
||||
A single dedicated OS thread (`std::thread::Builder::new().name("rdbms-db-worker")`)
|
||||
owns the `rusqlite::Connection`. The App talks to it through:
|
||||
|
||||
- A bounded `tokio::sync::mpsc::Sender<Request>` carrying typed
|
||||
request enum variants, each of which embeds a
|
||||
`tokio::sync::oneshot::Sender` for the reply.
|
||||
- A typed `DbError` (`thiserror`-derived) wrapping the union of
|
||||
database-side failure modes, with a `friendly_message()`
|
||||
method whose body today is a passthrough — this is the hook
|
||||
point for the future H1 friendly-error layer.
|
||||
|
||||
The worker:
|
||||
|
||||
- Receives requests via `Receiver::blocking_recv` (no Tokio
|
||||
context required on the worker thread).
|
||||
- Performs the operation synchronously against the connection.
|
||||
- Sends the result to the per-request `oneshot` reply channel.
|
||||
- Exits when the last `Sender` is dropped (clean shutdown via
|
||||
Drop, no shutdown protocol).
|
||||
|
||||
The public `Database` handle is `Clone` (it's a wrapper around
|
||||
a `Sender`), so multiple components can hold it.
|
||||
|
||||
## Consequences
|
||||
|
||||
- `App::update` stays sync and pure. The runtime, on receiving
|
||||
an `Action::ExecuteDsl`, spawns a `tokio::spawn` task that
|
||||
awaits the DB worker and posts the result back as a new
|
||||
`AppEvent`. The Elm pattern is preserved.
|
||||
- New database operations are mechanical to add: define a
|
||||
`Request` variant with its `oneshot` reply type, add a method
|
||||
on `Database`, add a handler in the worker.
|
||||
- B3 (query timeout/cancellation) lands on top of this without
|
||||
architecture change: a cancellation token can be stored
|
||||
alongside the in-flight request, and the worker can interrupt
|
||||
via `rusqlite::Connection::interrupt`.
|
||||
- U1 (snapshot capture) and P3 (auto-save) are also additive:
|
||||
more `Request` variants, possibly using SQLite's online
|
||||
backup API.
|
||||
- Errors carry a typed kind (`UniqueViolation`, `NoSuchTable`,
|
||||
etc.). When H1's friendly-error layer lands, the body of
|
||||
`friendly_message()` becomes the translation table; callsites
|
||||
do not change.
|
||||
- Cost: an extra OS thread per database. For a desktop tool
|
||||
this is negligible; this would be different for a server
|
||||
application that hosts many databases.
|
||||
|
||||
## Implementation notes
|
||||
|
||||
- Channel capacity 64 is sufficient bursts head-room for an
|
||||
interactive tool.
|
||||
- The connection is opened in the spawning thread and *moved*
|
||||
into the worker. This gives us fail-fast behaviour: a bad
|
||||
path or permissions issue surfaces immediately to the caller
|
||||
before any worker is started.
|
||||
@@ -0,0 +1,78 @@
|
||||
# ADR-0011: Foreign-key column type compatibility
|
||||
|
||||
## Status
|
||||
|
||||
Accepted
|
||||
|
||||
## Context
|
||||
|
||||
Two of our user-facing types (per ADR-0005) carry insert-time
|
||||
auto-generation semantics that only apply on the primary-key
|
||||
side of a relationship:
|
||||
|
||||
- `serial` is `INTEGER PRIMARY KEY` with rowid-alias /
|
||||
auto-increment behaviour.
|
||||
- `shortid` is `TEXT` populated client-side with a 10–12 char
|
||||
base58 random value when no value is supplied.
|
||||
|
||||
A foreign-key column referencing such a primary key does *not*
|
||||
auto-generate values — it stores the looked-up value of the
|
||||
primary key. Asking the user to declare the FK column with the
|
||||
same user-facing type as the PK would be wrong: `serial` on the
|
||||
FK side would imply the FK column has its own auto-increment
|
||||
counter (it doesn't), and similar for `shortid`.
|
||||
|
||||
Without this rule, the future FK declaration grammar (C3, C4)
|
||||
would either generate incorrect DDL or rely on the user to
|
||||
remember to use a different type on the FK side — an easy
|
||||
foot-gun.
|
||||
|
||||
## Decision
|
||||
|
||||
Define `Type::fk_target_type(self) -> Type` returning the
|
||||
appropriate user-facing type for a foreign-key column that
|
||||
references a primary key of `self`:
|
||||
|
||||
| User-facing type | `fk_target_type()` |
|
||||
|------------------|--------------------|
|
||||
| `text` | `text` |
|
||||
| `int` | `int` |
|
||||
| `real` | `real` |
|
||||
| `decimal` | `decimal` |
|
||||
| `bool` | `bool` |
|
||||
| `date` | `date` |
|
||||
| `datetime` | `datetime` |
|
||||
| `blob` | `blob` |
|
||||
| `serial` | **`int`** |
|
||||
| `shortid` | **`text`** |
|
||||
|
||||
All types other than `serial` and `shortid` are identity
|
||||
mappings. The two exceptions strip the auto-generation
|
||||
semantics by mapping to the underlying value type.
|
||||
|
||||
## Consequences
|
||||
|
||||
- The future FK declaration grammar uses `fk_target_type()` in
|
||||
one of two ways:
|
||||
- **Auto-typing** — when the FK column does not yet exist and
|
||||
`--create-fk` is given (or whatever the equivalent flag
|
||||
becomes), the column is created with
|
||||
`pk_column.ty.fk_target_type()`.
|
||||
- **Validation** — when the FK column already exists, its
|
||||
type is compared against `fk_target_type()`. A mismatch
|
||||
yields a clear diagnostic ("Customers.id is `serial`; the
|
||||
FK column should be `int`, not `text`"). This is the
|
||||
teaching moment ADR-0009's design philosophy targets.
|
||||
- Adding a new user-facing type forces an explicit decision
|
||||
about its `fk_target_type()`. The type system is therefore
|
||||
closed under FK declaration.
|
||||
- The advisory feedback module (foreshadowed in V4 and the
|
||||
hint surface) can use the same mapping to surface
|
||||
recommendations during command typing — e.g. "you used
|
||||
`serial` for an FK; conventionally `int` is the right fit
|
||||
here." This is *advice* not gating, consistent with
|
||||
ADR-0009's separation of required grammar from optional
|
||||
guidance.
|
||||
- `fk_target_type()` is implemented and tested *before* the FK
|
||||
declaration grammar lands, so the FK iteration is grammar
|
||||
work only.
|
||||
@@ -0,0 +1,115 @@
|
||||
# ADR-0012: Internal metadata for user-facing column types
|
||||
|
||||
## Status
|
||||
|
||||
Accepted
|
||||
|
||||
## Context
|
||||
|
||||
ADR-0005 commits to ten user-facing types (`text`, `int`, `real`,
|
||||
`decimal`, `bool`, `date`, `datetime`, `blob`, `serial`,
|
||||
`shortid`) and to mapping them transparently onto SQLite STRICT
|
||||
types so that learners do not see SQLite's erased forms (Q3 in
|
||||
the requirements checklist). The mapping is many-to-few:
|
||||
`shortid`, `decimal`, `date`, `datetime` all map to `TEXT`;
|
||||
`serial` and `bool` both map to `INTEGER`.
|
||||
|
||||
Reading the schema back via `PRAGMA table_info` therefore loses
|
||||
the original user-facing type. Rendering "id INTEGER" instead of
|
||||
"id serial", or "created TEXT" instead of "created datetime",
|
||||
breaks the Q3 promise even though we generated correct DDL.
|
||||
|
||||
The same information is needed by track 2 (project file format,
|
||||
ADR-0004): when serialising the schema to YAML, we need the
|
||||
user-facing types, not the SQLite-erased ones.
|
||||
|
||||
## Decision
|
||||
|
||||
Maintain an internal SQLite table that records the user-facing
|
||||
type each column was declared with. The table is part of the
|
||||
database itself, so it travels with the `playground.db` file
|
||||
and is the single source of truth for round-tripping types.
|
||||
|
||||
```sql
|
||||
CREATE TABLE __rdbms_playground_columns (
|
||||
table_name TEXT NOT NULL,
|
||||
column_name TEXT NOT NULL,
|
||||
user_type TEXT NOT NULL,
|
||||
PRIMARY KEY (table_name, column_name)
|
||||
) STRICT;
|
||||
```
|
||||
|
||||
- **Bootstrap.** The table is created on every connection open
|
||||
with `CREATE TABLE IF NOT EXISTS`, alongside `PRAGMA
|
||||
foreign_keys = ON`.
|
||||
- **Writes.** `create_table` and `add_column` insert one row per
|
||||
user column, atomically with the DDL via
|
||||
`Connection::unchecked_transaction`. `drop_table` deletes all
|
||||
rows for the dropped table. Every DDL operation that adds or
|
||||
removes user columns must keep this table in sync.
|
||||
- **Reads.** `describe_table` LEFT-JOINs `pragma_table_info`
|
||||
with this table to populate `ColumnDescription::user_type`.
|
||||
Callers prefer `user_type` for display; `sqlite_type` is
|
||||
retained for diagnostics and fallback.
|
||||
- **Visibility.** `list_tables` filters out any name starting
|
||||
with `__rdbms_` so the metadata table is hidden from the user.
|
||||
|
||||
### Naming convention for internal tables
|
||||
|
||||
All future internal tables use the `__rdbms_` prefix (double
|
||||
underscore plus `rdbms_`). This serves two purposes:
|
||||
|
||||
1. **Avoiding collisions.** SQLite reserves `sqlite_` for its
|
||||
own use; we reserve `__rdbms_` for ours. A user cannot
|
||||
create a table whose name conflicts with our internals
|
||||
without explicitly typing the prefix, which is unlikely by
|
||||
accident.
|
||||
2. **Single filter rule.** `list_tables` and any future
|
||||
schema-introspection code excludes names matching
|
||||
`sqlite_%` and `__rdbms_%`. New internal tables are
|
||||
automatically hidden as long as they follow the convention.
|
||||
|
||||
## Consequences
|
||||
|
||||
- The Q3 / ADR-0005 transparency promise is delivered. Users
|
||||
see `id serial`, not `id INTEGER`.
|
||||
- The DSL parser, type system, and database layer collectively
|
||||
control all user-facing type information; SQLite's erased
|
||||
view is never the authoritative source.
|
||||
- Track 2 (project file format) round-trips user types via
|
||||
this metadata. The YAML schema dump reads from
|
||||
`__rdbms_playground_columns`; loading reconstructs the
|
||||
metadata alongside the DDL.
|
||||
- DDL operations carry a small additional cost (one transaction
|
||||
per operation, one row per user column). Negligible for
|
||||
interactive use; would matter only for bulk schema operations,
|
||||
which are not in scope.
|
||||
- Adding new user-column-creating operations must extend this
|
||||
metadata in step. The single-source-of-truth invariant is
|
||||
worth defending — a future code review should reject any new
|
||||
DDL that touches columns without updating
|
||||
`__rdbms_playground_columns`.
|
||||
- The convention `__rdbms_<name>` for internal tables is
|
||||
established for any future internal-state needs (snapshot
|
||||
bookkeeping, replay log indexing, etc.).
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
- **In-memory cache in `App`.** Simpler, but doesn't survive
|
||||
a restart, and would have to be rebuilt by parsing past
|
||||
command history. Loses the single-source-of-truth property.
|
||||
- **Sentinel comments in DDL.** SQLite's
|
||||
`sqlite_master.sql` retains the original CREATE TABLE text;
|
||||
we could embed `/* user_type: serial */` markers and parse
|
||||
them back. Brittle and hard to keep correct under ALTER.
|
||||
- **Lossy reverse mapping.** Heuristically guess `serial` from
|
||||
`INTEGER PRIMARY KEY`, `text` from `TEXT`, etc. Cannot
|
||||
distinguish `text`/`shortid`/`decimal`/`date`/`datetime`,
|
||||
all backed by `TEXT`. Wrong by construction.
|
||||
|
||||
## See also
|
||||
|
||||
- ADR-0002 (database engine, STRICT tables, `foreign_keys` ON)
|
||||
- ADR-0005 (column type vocabulary, mapping commitment)
|
||||
- ADR-0010 (database access via dedicated worker thread —
|
||||
describes where this metadata is read and written from)
|
||||
@@ -14,3 +14,7 @@ This directory contains the project's ADRs, recorded per
|
||||
- [ADR-0006 — Undo snapshots and replay log](0006-undo-snapshots-and-replay-log.md)
|
||||
- [ADR-0007 — Sharing and export](0007-sharing-and-export.md)
|
||||
- [ADR-0008 — Testing approach](0008-testing-approach.md)
|
||||
- [ADR-0009 — DSL command syntax conventions](0009-dsl-command-syntax-conventions.md)
|
||||
- [ADR-0010 — Database access via a dedicated worker thread](0010-database-access-via-worker-thread.md)
|
||||
- [ADR-0011 — Foreign-key column type compatibility](0011-fk-column-type-compatibility.md)
|
||||
- [ADR-0012 — Internal metadata for user-facing column types](0012-internal-metadata-for-user-facing-types.md)
|
||||
|
||||
Reference in New Issue
Block a user