Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Testing Strategy

Status: proposed — consolidated 2026-09-01 from material already designed in 004-implementation-reference.md, 070-roadmap.md, and .github/workflows/ci.yml. No new design decisions were made here.

Scope

Test pyramid for the workspace: unit (pam-core), Store parity (PgStore vs MemStore), PostgreSQL dialect/migration tests against an ephemeral container, worker/executor integration against a real tmux, partition-path tests (orchestrator stop/start with signal spool replay), API contract tests, and the runnable phase gates.

Quality Gates

Four gates, run in this order, all must exit 0 — enforced locally before every commit and as the CI test job (005-engineering-rules.md § Commit Gate, 070-roadmap.md § Gate conventions):

cargo fmt --all -- --check
cargo clippy --workspace --all-targets -- -D warnings
cargo build --workspace
cargo test --workspace

No coverage percentage is gated — cargo-llvm-cov output is generated as an informational artifact only (see § CI Workflow below), not enforced as a pass/fail threshold.

CI Workflow (.github/workflows/ci.yml)

Three independent jobs on every push/PR to main (004-implementation-reference.md § CI):

  • test — the four Quality Gates above as separate steps, toolchain 1.93.1, Cargo.lock-keyed cache. From Phase 1 this job additionally needs an ephemeral PostgreSQL service container for PgStore tests — not yet added, since the scaffold is still pre-ADR-012-pivot (SQLite-backed); code catches up starting Phase 1 (004-implementation-reference.md § What Exists).
  • docs — frontmatter presence (implementation: key, README.md navigation files exempt) and relative-link / .mermaid-reference resolution, checked with a stdlib-Python script; a best-effort mermaid-cli parse step runs continue-on-error (Node flakiness must not redden the job, but a genuine syntax error still shows in the step log).
  • coverage — cargo-llvm-cov --workspace --lcov uploaded as a 30-day-retained lcov artifact, plus a per-crate summary printed to the log. Informational only (see § Quality Gates above).

Test Pyramid by Phase

Each roadmap phase (070-roadmap.md) ends at a gate expressed as a runnable command sequence or an observable outcome — never a vague quality adjective. Summarized here; the roadmap doc is the source of truth for exact gate steps.

  • Phase 0 (scaffold) — 5 crates, each compiling with a smoke test; the four Quality Gates pass workspace-wide.
  • Phase 1 (PM domain) — Store parity: the shared, parameterized Store trait test suite runs against both PgStore and MemStore and passes identically; pam task add/list/move/done round-trips against a real PostgreSQL instance (ephemeral container acceptable).
  • Phase 2 (worker + tmux executor) — the widest integration surface, five paths against a real tmux: crash-path reconciliation (no duplicate or orphaned sessions after a worker restart), partition path (stop the orchestrator mid-run, restart it — spooled run.started/run.finished signals replay and the run record converges with no manual repair), duplicate-delivery idempotency (the same one-shot signal POSTed twice yields exactly one transition and one journalled event), stale-attempt rejection (a signal stamped with an old attempt changes neither runs nor events), and the tmux safety-boundary path (decoy sessions on both the pam and default sockets survive pam run stop/cleanup untouched).
  • Phase 3 (web SPA) — an embedding test (the built binary serves the SPA with web/dist renamed away on disk, proving embedding rather than static-file serving) plus a browser-observable check (board + live pane render with real data).
  • Phase 4 (provider adapters) — end-to-end: two real providers each complete a scripted task with distinct completion-signal sources (wrapper-journal vs provider-native hook) visibly different in the run record; pam provider doctor exits 0 on the dev machine.
  • Phase 5 (interactive attach) — the single-writer lease contract: one browser’s keystrokes reach the tmux pane and stream to a read-only second viewer; a second concurrent write attempt is rejected with a clear lease error and contributes no input.
  • Phase 6 (desktop shell) — the built .app/dmg launches, reaches the same core, and passes the Phase 3 gate through the shell webview; cargo-binstall metadata validates locally (dry run).

Gate Conventions

  • All commands run from the repository root on the dev machine (macOS — the primary worker platform per ADR-011; Linux is verified per OQ-9 when an environment is available).
  • From Phase 1 on, every gate assumes a reachable PostgreSQL (a local ephemeral container is acceptable) and a locally running orchestrator
    • worker (pam serve + pam worker on the same machine is the dev shape of the multi-host topology).
  • CI runs the headless subset only — fmt, clippy, build, test, docs checks. Browser- and tmux-interactive checks (Phases 2, 3, 5, 6) are verified manually at phase exit, not automated in CI.
  • A phase flips to done only when every item in its gate has passed; partial completion keeps it in-progress with a note.

Open Questions