rdbms-playground/docs/adr/0008-testing-approach.md

ADR-0008: Testing approach

Status

Accepted

Context

The project's working standards (CLAUDE.md, plus the user's global testing rules) require:

• Test coverage established before changes.
• Bugs reproduced with failing tests before fixes.
• Integration tests that exercise the full stack a user touches.
• "All green, zero skips" as the only acceptable end state.

A TUI application needs a credible testing story for those rules to be enforceable. Naïvely, "TUI testing" sounds like an afterthought of manual screenshots. In practice, the Rust + Ratatui ecosystem supports automation across every level that matters, and we want to commit to that explicitly rather than discover it ad-hoc.

Decision

Testing is structured in four tiers. Tiers 1–3 run on every commit in CI; tier 4 runs on every commit for a focused subset of critical flows and on a nightly schedule for broader coverage.

Tier 1 — Pure-logic unit tests

Standard cargo test against modules with no terminal dependency:

• DSL parser and command dispatcher
• Type mapping (user-facing → SQLite STRICT types)
• Project I/O (project.yaml + data/<table>.csv round-tripping)
• Database rebuild from authoritative text sources
• Snapshot ring buffer logic (ADR-0006)
• Replay log writer/reader

These are the bulk of the test count. Every behavioural unit has unit tests; modules with non-trivial logic also have property tests via proptest where the input space justifies it (parser inputs, type coercion, CSV escaping, etc.).

Tier 2 — Render assertions via Ratatui TestBackend

Ratatui's in-tree TestBackend renders into a Buffer (a 2D cell grid). Tests build an app state, render a frame, and assert on the resulting buffer.

• Cell-level assertions for narrow tests ("the status bar shows mode label Simple in the expected style").
• Snapshot tests via insta for whole-frame coverage of representative views (default screen, query result table, schema view, undo confirmation prompt). Snapshots are checked in and reviewed on diff.

Snapshot discipline:

• Snapshots cover stable, intentional UI surfaces; they are not added reflexively to every component.
• A snapshot diff is treated as a real review item, not a rubber-stamp. Reviewers must confirm the change is intended.

Tier 3 — Synthetic event-loop integration tests

The application's update function consumes crossterm::event::Event values. Tests feed sequences of synthetic events (KeyEvent, MouseEvent, resize) to the update function, then render via TestBackend and assert on both state and buffer.

This tier is the equivalent of react-testing-library for our TUI: it exercises the full input → state → render path without a real terminal, and is where the most valuable behavioural tests live. Examples:

• Typing a DSL command in simple mode, submitting with Ctrl-Enter, asserting the table list updates and the schema view re-renders.
• Triggering undo, asserting the confirmation prompt appears with the expected timestamp and change summary, confirming, asserting state restoration.
• Switching modes and verifying the prompt label and border colour change.

Tier 4 — PTY-based black-box end-to-end

A small number of critical flows are exercised against the actual built binary in a pseudo-terminal:

• Tooling: portable-pty for the PTY, expectrl for expect-style scripting, vt100 to parse the terminal output stream into an inspectable cell grid.
• These tests catch issues the lower tiers miss: TTY setup, signal handling, terminal mode transitions, real I/O timing.

Tier 4 is reserved for the highest-value flows, not blanket coverage. The initial scope is:

• Cold launch → first DDL command → graceful quit.
• Project save → process restart → reopen → identical state.
• Project export → import in a fresh project → rebuilt database matches the source.
• undo immediately after a DROP TABLE, including the confirmation prompt.

Tier 4 tests run in CI on every commit (the focused list above) and on a nightly schedule for any extended coverage.

Tooling commitments

• cargo test — Tier 1, Tier 2, Tier 3.
• proptest — property-based testing for parser and conversion layers.
• Ratatui's TestBackend — frame rendering for tests.
• insta — snapshot testing of rendered buffers.
• portable-pty, expectrl, vt100 — Tier 4 PTY-based tests.
• CI matrix covers Linux, macOS, and Windows on stable Rust.

Honest limits

• No cross-terminal-emulator regression coverage. Tier 4 exercises a PTY but not real terminal emulators (xterm, Alacritty, Windows Terminal, etc.). Crossterm abstracts these well in practice; if a real-emulator regression ever surfaces, we will revisit.
• No visual aesthetic checks. Tests assert cell contents and styles, not "this layout is pretty". Visual polish is reviewed manually by humans.
• Snapshot brittleness is a known failure mode. We mitigate by being selective about what gets a snapshot and by treating snapshot diffs as real review items.

Consequences

• Test discipline from CLAUDE.md is enforceable on every layer: parser bugs caught at Tier 1, UI flow bugs caught at Tier 3, real-binary regressions caught at Tier 4.
• Module boundaries are designed for testability from the start (pure logic modules separate from rendering, an explicit update function consuming events).
• CI cost is real but bounded: Tiers 1–3 are fast; Tier 4 is the only slow tier and is kept narrow.
• Adding a feature implies adding tests at the appropriate tier (or tiers); coverage is not retrofitted later.