docs(tt4,nfr): record Tier-4 + NFR verification (ADR-0008 A1, ADR-0057)
Document the regression-hardening work from this session: - ADR-0008 Amendment 1 — Tier 4 realized: the portable-pty + vt100 harness, expectrl dropped, serial/fail-fast design, the four flows, sidebar-region reads, command pacing (issue #39), CI default-target (advances TT5). - ADR-0057 (new) — NFR verification strategy: contrast + ΔE2000 gates (NFR-5/7), startup/RSS measured (NFR-1/3: release ~29 ms / ~10 MB), NFR-2 by architecture, NFR-4/6 reviewer note. Records the two shipped light-theme contrast defects this work caught and fixed. - requirements.md: TT4 [~]->[x], TT5 [/] (only a Windows exec runner left), NFR-1/2/3/5/7 ->[x], NFR-4/6 ->[/], each with evidence. - CHANGELOG.md (new): Keep a Changelog + SemVer, [Unreleased] + [0.2.0]. - README ADR index updated; handoff-76; the working plan. Docs-only (CI gate skips via paths-ignore).
This commit is contained in:
@@ -146,4 +146,57 @@ and on a nightly schedule for any extended coverage.
|
||||
- CI cost is real but bounded: Tiers 1–3 are fast; Tier 4 is the
|
||||
only slow tier and is kept narrow.
|
||||
- Adding a feature implies adding tests at the appropriate tier
|
||||
(or tiers); coverage is not retrofitted later.
|
||||
(or tiers); coverage is not retrofitted later.
|
||||
|
||||
## Amendment 1 — 2026-06-22: Tier 4 realized (TT4)
|
||||
|
||||
Tier 4 was a specification only (no PTY deps, no tests) until this
|
||||
amendment. It is now implemented in `tests/e2e_pty.rs`, and the realized
|
||||
shape refines the original decision in three ways.
|
||||
|
||||
**Tooling — `expectrl` dropped.** The decision named
|
||||
`portable-pty` + `expectrl` + `vt100`. We kept **`portable-pty`** (0.9, to
|
||||
spawn the real binary in a PTY at a fixed size) and **`vt100`** (0.16, to
|
||||
parse the output stream into an inspectable cell grid), both current and
|
||||
maintained. We **dropped `expectrl`**: it bundles its own PTY abstraction
|
||||
(which conflicts with `portable-pty`) and matches line-by-line, a poor fit
|
||||
for a full-screen TUI. A small hand-rolled `wait_for(predicate, timeout)`
|
||||
that polls the vt100 screen replaces it — fewer dependencies, and assertions
|
||||
read the actual rendered grid.
|
||||
|
||||
**Harness shape.** Each test spawns the binary
|
||||
(`env!("CARGO_BIN_EXE_rdbms-playground")`) under a fresh PTY at **100×30**
|
||||
(the wide three-region layout, ADR-0046), with its **own temp `--data-dir`**
|
||||
so it never touches real projects or resume state, and `--theme dark` for
|
||||
deterministic styling. Tests run **serially** (one PTY + child at a time)
|
||||
via a poison-tolerant global lock, so timing stays predictable on the
|
||||
low-parallelism self-hosted runner. Waits are **tight and fail-fast** (3 s):
|
||||
a slow wait is a genuine hang, and a timeout panics with a full screen dump
|
||||
for debuggability.
|
||||
|
||||
Two non-obvious robustness rules the harness encodes:
|
||||
- **Read table presence from the Tables *sidebar* region, not the whole
|
||||
screen** — the Output panel echoes every command, so a table name typed
|
||||
in a command would pollute a whole-screen match.
|
||||
- **Pace each command to completion before the next** (real-user / cast
|
||||
cadence). A command submitted while the previous one's worker rebuild is
|
||||
still in flight can be misread against a stale schema cache (issue #39,
|
||||
discovered here — interactive use is unaffected).
|
||||
|
||||
**The four flows** mirror this ADR's Tier-4 scope: (1) cold launch → first
|
||||
DDL → graceful quit (Ctrl-C); (2) create → quit → reopen via `--resume`,
|
||||
asserting the *column* (not just the table name) round-trips; (3) export →
|
||||
import into a fresh project → schema **and** data rebuilt; (4) `undo` after
|
||||
`DROP TABLE`, through the Y/N confirm modal.
|
||||
|
||||
**CI.** The target runs by default in `cargo test`, so the existing Linux
|
||||
gate exercises it on every push — this **advances TT5** (Tier 4 now runs in
|
||||
CI on Linux). PTY-in-container was validated (openpty + child spawn work
|
||||
under the Docker runtime the Gitea runner uses); the first real CI run is
|
||||
the final confirmation. **Windows execution remains out of scope** (no
|
||||
Windows runner), and the original Tier-4 note about a *nightly broader
|
||||
coverage* schedule is **not** implemented — the focused four flows are the
|
||||
committed TT4 scope; broader coverage is deferred.
|
||||
|
||||
**Piggybacked NFR measurement.** The same harness measures NFR-1 (startup
|
||||
to first frame) and NFR-3 (idle RSS, Linux) — see ADR-0057.
|
||||
Reference in New Issue
Block a user