Keyboard shortcuts

Press or to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

HTTP API Contract

Authentication & Transport

Bind Configuration

  • Bind address: PAM_BIND env var (default: 0.0.0.0) — the central orchestrator is reachable from worker hosts and remote browsers (ADR-012)
  • Port: PAM_PORT env var (default: 7898)
  • TLS: served via TLS in deployment (termination strategy — reverse proxy vs native — is a deployment choice; the token model below assumes TLS protects the transport)
  • CORS: Not required (embedded SPA is same-origin)

Dev mode exception: the trunk serve dev server proxies /api127.0.0.1:PAM_PORT for local development.

Authentication Scheme

  • Method: Bearer token in Authorization header
  • Token format: 32-character hex string
  • Storage: platform data dir token file (~/Library/Application Support/pam/token on macOS, ~/.local/share/pam/token on Linux) on the central host
  • CLI behavior: Automatically reads and injects token
  • Generation: Auto-generated by the central orchestrator on first start if missing
  • Worker authentication is separate: workers register with a pre-provisioned worker token (PAM_WORKER_TOKEN), stored in the worker host’s platform data dir; whether to upgrade to mutual TLS is tracked as OQ-10

Token injection flow: The orchestrator generates the token file in its platform data dir on first start; the CLI reads that file automatically; PAM_TOKEN env var takes precedence over the file.

Error Envelope

All error responses follow this structure:

{
  "error": {
    "code": "string",
    "message": "string",
    "details": {} // optional, context-specific
  }
}

Standard HTTP status codes:

  • 400 Bad Request (validation errors)
  • 401 Unauthorized (missing/invalid token)
  • 404 Not Found (resource missing)
  • 409 Conflict (state conflicts, lease acquisition)
  • 500 Internal Server Error
  • 503 Service Unavailable (database/executor issues)

REST API Endpoints

All endpoints prefixed with /api/v1.

Projects

Notes:

  • slug auto-generated from name if not provided
  • Delete performs soft-archive (status → archived)

Work Items

Notes:

  • kind values: milestone, epic, task, subtask
  • status values: todo, doing, done, cancelled
  • parent_id references work_items.id (self-FK for hierarchy)
  • Board move = PATCH status field only
  • path field computed from parent chain (read-only)

Reparent semantics (PATCH parent_id):

  • Nullable — passing null promotes the item to the project root
  • A move whose target parent belongs to a different project is rejected 400 (the project_id invariant is re-checked on every reparent)
  • A move that would place an item inside its own subtree is rejected 400 (cycle guard)
  • The move rewrites path for the item and all of its descendants in one transaction, and journals work_item.reparented once, for the moved item only — descendants emit no event of their own
  • Transition legality, the hierarchy guards, and the path-rewrite rule are owned by 002-data-model.md; this endpoint only exposes them

Runs

Notes:

  • provider values: claude, codex, grok, gemini
  • session_name format: pam-<8hex> (tmux socket pam on the executing host; uniqueness is scoped per host)
  • host_id on run objects identifies the executing worker host
  • status values: pending, spawning, running, exited, failed, stopped
  • keys allowlist: alphanumeric, Enter, Ctrl+C (executor-side validation)
  • Pane snapshots arrive over the executing worker’s uplink stream (workers accept no inbound connections — the orchestrator never pulls from a worker); the orchestrator caches the latest snapshot and serves it
  • Logs endpoint streams the run’s log file, relayed by the orchestrator from the executing host (path per tmux-executor: <platform data dir>/runs/<run_id>.log on that host)
  • Runs may only target work items of kind task or subtask (application-level validation)
  • POST /runs always creates the run in pending and returns 202 — it never fails synchronously because no host is currently eligible. Dispatch is asynchronous; the response confirms admission, not placement
  • An unclaimed run fails at the dispatch timeout (default 5 minutes, configurable, measured from runs.pending_since — not created_at, which does not reset on requeue; confirmed OQ-15 fix, 071-risks-and-open-questions.md). Both terminal outcomes are recorded at that timeout, never at claim time: failure_reason = 'no_eligible_host' when no online, non-disabled host ever matched the requested target — whether or not the request named a specific host — and failure_reason = 'dispatch_timeout' when a host matched but never claimed the run. A target host going offline or being disabled does not immediately fail the run. The pending → failed transition itself is owned by 002-data-model.md
  • A run requeued by the restart policy re-enters pending with attempt incremented, and MAY be claimed by a different host than the previous attempt — host affinity is not preserved across attempts unless the run pinned a host at creation

Hosts (Worker Registry)

Notes:

  • status values: registered, online, offline, disabled
  • Worker registration and heartbeat are internal endpoints (/api/internal/worker/*), authenticated with the worker token — not part of the public surface
  • Hosts are created by the operator (pam host add) and receive a worker token at provisioning time; a worker cannot self-register an unknown host
  • disabled is sticky: a heartbeat from a disabled host updates last_seen_at but never status. Only an explicit POST /hosts/{id}/enable leaves disabled — an operator exclusion must survive the host continuing to run
  • enable’s target status is derived, not fixed: it clears the disabled flag and resolves to online when last_seen_at falls inside the liveness window, otherwise offline
  • disable is drain-only: runs already executing on the host continue, keep streaming pane output, and relay signals to completion. Only new dispatch is withheld. Disabling never kills a session and never triggers the graceful stop ladder
  • registered means the operator created the row and no worker has heartbeated yet; online and offline are heartbeat-derived. A host never returns to registered
  • The active-run count from GET /hosts/{id} is how an operator judges drain completion after a disable

Provider Catalog

Events

Event topics: See 002-data-model.md for canonical enumeration (project.*, work_item.*, run.*, worker.*).

Server-Sent Events (SSE)

Event Stream Endpoint

GET /api/v1/events/stream

Query Parameters:

  • topic: Optional topic filter (supports glob patterns: run.*)
  • Last-Event-ID: Resume cursor (maps to events.seq)

Payload Envelope

id: <seq>
event: <topic>
data: <json_payload>

Example:

id: 1234
event: run.started
data: {"run_id": "f8d9e2a1-4b7c-4e21-9a03-5c6d8e1f2a3b", "attempt": 1, "task_id": "3c9a7f10-2d84-4f6e-b1c5-90ab7d4e6f22", "provider": "claude", "session_name": "pam-f8d9e2a1"}

Payload identifier rules (normative):

  • run_id and task_id are UUID strings, not integers — they carry runs.id and work_items.id verbatim (see 002-data-model.md)
  • Every run.* event payload carries attempt alongside run_id, so a consumer can tell which incarnation of the run the event describes. Without it, a restarted run’s events are indistinguishable from the dead incarnation’s replayed ones (runs.attempt, see 002-data-model.md)
  • session_name’s 8-hex suffix is the leading 8 characters of run_id; it is a display/lookup convenience, never an identifier in its own right
  • The canonical topic enumeration is owned by 002-data-model.md §Event Topics

Reconnect Semantics

  • Client includes Last-Event-ID: <seq> header on reconnect
  • Server resumes from seq + 1 (exclusive cursor)
  • If seq not found (purged), server sends full snapshot then resumes live
  • No heartbeats: rely on TCP keepalive

Note: Events are kept 30 days by default, with best-effort purge at orchestrator startup (see Data Retention policy in 002-data-model.md).

WebSocket Protocols

Live Pane WebSocket (Read-Only)

WS /api/v1/runs/{id}/pane/ws

Purpose: Server → client streaming of terminal pane updates.

Protocol Design: Full snapshot on change with sequence numbers.

Server Frame Format:

{
  "type": "pane_snapshot",
  "seq": 123,
  "run_id": "abc123",
  "capture": "<terminal content>",
  "timestamp": "2026-08-29T12:34:56Z"
}

Sequence namespace note: the pane-snapshot seq above orders WebSocket snapshot frames for the terminal view. It is a different sequence from the seq carried by repeated run signals (progress) at the signal-ingest edge (see Provider Hook Receiver below) — the two occupy separate namespaces and must not be conflated.

Justification: Full snapshot chosen over delta encoding because:

  • Terminal panes are relatively small (actual spawn size 220×50 ≈ 11,000 chars, ~11 KB per full snapshot)
  • Avoids delta frame synchronization complexity
  • Simplifies client state management (no diff application)
  • Modern xterm.js handles full replacements efficiently

Throttling: Source capture polling is 1 Hz (snapshots emitted only on change); the 10 fps cap applies to the interactive attach WebSocket relay’s frame coalescing.

Attach WebSocket (Interactive)

WS /api/v1/runs/{id}/attach/ws

Purpose: Bidirectional PTY relay for interactive terminal session.

Authentication: Token passed as query parameter ?token= (browser WebSocket clients cannot set headers). Note: token appears in URLs, so avoid logging full WebSocket URLs in logs.

Lease Enforcement:

  • Single-writer lease prevents concurrent attach conflicts
  • First client acquires lease automatically
  • Subsequent clients receive lease-acquired error

Server → Client Frames:

  1. Lease Acquired (confirmation):
{
  "type": "lease_acquired",
  "run_id": "abc123",
  "attached_at": "2026-08-29T12:34:56Z"
}
  1. Lease Held (error to others):
{
  "type": "error",
  "code": "lease_held",
  "message": "Session already attached by another client"
}
  1. PTY Data (terminal output):
{
  "type": "data",
  "bytes": "<base64-encoded binary PTY output>"
}
  1. Resize (client should sync dimensions):
{
  "type": "resize",
  "rows": 24,
  "cols": 80
}

Client → Server Frames:

  1. Input (keystrokes):
{
  "type": "input",
  "bytes": "<base64-encoded input bytes>"
}
  1. Resize (request PTY resize):
{
  "type": "resize",
  "rows": 24,
  "cols": 80
}
  1. Lease Release (explicit detach):
{
  "type": "release_lease"
}

Single-Writer Enforcement:

  • Server tracks lease holder per run
  • Non-holder clients receive lease_held on any frame attempt
  • Lease auto-releases on WebSocket close
  • Server terminates attach WS when run exits

Note: The 409 Conflict status code in the REST status-code table applies to REST conflicts, not the WebSocket attach handshake (which uses the {"type":"error","code":"lease_held"} frame for lease rejection after upgrade).

Internal Endpoints

Transport (resolved, OQ-11): persistent bidirectional WebSocket, one resident connection per worker process. A worker opens WS /api/internal/worker/uplink immediately after a successful POST /api/internal/worker/register and keeps it open for the process lifetime, reconnecting with backoff on drop. Heartbeat, run-claim dispatch, and the snapshot/signal/log uplink all ride this single connection — periodic polling is not the primary path; it remains a documented fallback only if the persistent connection proves brittle in the field (071-risks-and-open-questions.md).

Registration stays a one-shot REST call (simple request/response, happens once per worker process start); everything recurring moves to the WS frames below.

Registration

POST /api/internal/worker/register

Authenticated with the worker token. Resolves the caller against an operator-provisioned hosts row — a worker cannot self-register an unknown host (002-data-model.md).

Request:

{
  "host_name": "mac-studio-01",
  "platform": "macos",
  "instance_id": "a1b2c3d4-5e6f-4789-9abc-def012345678"
}

instance_id (OQ-14, confirmed, 071-risks-and-open-questions.md): generated once by the worker process and persisted in its local data dir, then sent unchanged on every registration and heartbeat for the life of that persisted value. Stored on hosts.instance_id (002-data-model.md). When an incoming instance_id disagrees with the pinned value, the server compares now against the pinned instance’s last_seen_at (captured before the update) to the existing 90-second offline threshold: past the threshold reads as a legitimate reinstall (the value updates silently, an info-level worker.instance_reinstalled event is journaled); within it reads as a suspected clone (the pinned instance_id is left unchanged, hosts.instance_suspect is set, and a worker.instance_suspect anomaly event is journaled once on the false→true transition). See 002-data-model.md for the full column and transition detail.

Response:

{
  "host_id": "3c9a7f10-2d84-4f6e-b1c5-90ab7d4e6f22",
  "status": "registered" | "online"
}

WS /api/internal/worker/uplink

Authentication: Authorization bearer header with the worker token — unlike the browser-facing Attach WebSocket, the worker is not a browser and can set headers on the handshake, so no query-param workaround is needed.

Server → Client (orchestrator → worker) frames:

  1. Heartbeat ack:
{ "type": "heartbeat_ack", "timestamp": "2026-09-01T12:00:00Z" }
  1. Run dispatch (a claimed run pushed to this host):
{
  "type": "run_dispatch",
  "run_id": "f8d9e2a1-4b7c-4e21-9a03-5c6d8e1f2a3b",
  "task_id": "3c9a7f10-2d84-4f6e-b1c5-90ab7d4e6f22",
  "provider": "claude",
  "session_name": "pam-f8d9e2a1",
  "attempt": 1,
  "cwd": "/Users/op/projects/pam"
}
  1. Error (protocol-level, e.g. malformed frame):
{ "type": "error", "code": "string", "message": "string" }

Client → Server (worker → orchestrator) frames:

  1. Heartbeat — sent every 30 s:
{ "type": "heartbeat", "host_id": "3c9a7f10-2d84-4f6e-b1c5-90ab7d4e6f22", "instance_id": "a1b2c3d4-5e6f-4789-9abc-def012345678", "timestamp": "2026-09-01T12:00:00Z" }
  1. Run dispatch ack:
{ "type": "run_dispatch_ack", "run_id": "f8d9e2a1-...", "attempt": 1, "accepted": true }
  1. Snapshot — pane capture relay; the orchestrator caches the latest and serves it via GET /runs/{id}/pane and the Live Pane WebSocket:
{ "type": "snapshot", "run_id": "f8d9e2a1-...", "capture": "<terminal content>", "timestamp": "2026-09-01T12:00:00Z" }
  1. Signal — lifecycle/progress relay; the frame wraps the same body documented under Provider Hook Receiver below (kind/attempt/seq/source/payload/timestamp), and is subject to the identical ingest-ordering, dedup, and epoch rules — this uplink frame and the POST /api/internal/runs/{id}/signals path are two transports for the same contract, never two contracts:
{ "type": "signal", "run_id": "f8d9e2a1-...", "kind": "progress", "attempt": 1, "seq": 42, "source": "wrapper", "payload": {}, "timestamp": "2026-09-01T12:00:00Z" }
  1. Log chunk — appended run-log text relay, backing GET /runs/{id}/logs:
{ "type": "log_chunk", "run_id": "f8d9e2a1-...", "text": "...", "timestamp": "2026-09-01T12:00:00Z" }

Notes:

  • Claim serialization is unchanged by this transport: the orchestrator still performs the SELECT ... FOR UPDATE-guarded claim (002-data-model.md) before pushing run_dispatch; the WS only changes how the claimed run reaches the worker — a push down the worker’s own outbound connection, never a connection the orchestrator opens toward the worker.
  • 30 s heartbeat cadence; the orchestrator marks a host offline after 90 s without one (3 missed beats). This is a distinct timer from the worker’s own local reconciliation loop (30 s) and liveness poll (10 s) documented in 010-tmux-executor.md — those never cross the wire.
  • worker.registered fires when a host’s derived status flips to online (first heartbeat after registration, or reconnection after an offline gap); worker.offline fires when the 90 s threshold is crossed (002-data-model.md §Event Topics).
  • On reconnect after a drop, the worker replays its on-disk signal spool verbatim (attempt/seq preserved, never renumbered) before resuming live heartbeat — the same at-least-once delivery model documented under Provider Hook Receiver applies here.
  • runs.attempt is allocated centrally only; the worker never assigns it. run_dispatch carries the attempt the orchestrator already incremented; the worker’s run_dispatch_ack and every subsequent signal for that run echo it verbatim.
  • OQ-14 (worker instance identity, 071-risks-and-open-questions.md): confirmed 2026-09-01 — instance_id is carried on registration and every heartbeat and stored on hosts.instance_id (002-data-model.md). The mismatch-detection algorithm is the recency-based heuristic described above (§ Worker Uplink Registration): an incoming instance_id that disagrees with the pinned value is judged a reinstall or a suspected clone by comparing the pinned instance’s last-seen gap against the 90-second offline threshold. A suspect host is excluded from new dispatch (alongside disabled) but its already-running runs are left untouched; an operator clears the flag with pam host clear-suspect (040-cli.md), which also accepts the new instance_id as the pinned value.

Provider Hook Receiver

POST /api/internal/runs/{id}/signals

Purpose: Receive completion signals from provider-native hooks (claude-hooks, etc.).

Trust Model:

  • Callers are wrappers and provider hooks on worker hosts, plus the workers themselves — all remote relative to the orchestrator
  • Authenticated with the worker token (same credential class as the registration uplink; a dedicated signal token remains under review — OQ-8)

Request Shape:

{
  "kind": "started" | "finished" | "completed" | "failed" | "progress",
  "attempt": 1,
  "seq": 42,
  "source": "wrapper" | "provider-hook" | "manual",
  "payload": {},
  "timestamp": "2026-08-29T12:34:56Z"
}

Notes:

  • kind values are lowercase, mapped 1:1 to provider-adapters’ RunSignal enum
  • attempt (required, integer): the incarnation epoch of the emitting process — the run’s current attempt number (runs.attempt, see 002-data-model.md). It is allocated centrally (incremented on each restart-policy requeue) and injected into the spawn environment; the emitting wrapper (pam exec run) stamps every signal with it. Numbers are stamped at emission time and ride the worker’s on-disk spool verbatim on replay — the worker never renumbers.
  • seq (optional, int64): carried ONLY by repeated kinds (progress); the one-shot kinds (started, finished, completed, failed) omit it — they are exactly-once via the central dedup table. Monotonic within (run_id, attempt); a progress signal is applied only when seq > runs.last_progress_seq.
  • source values: wrapper (from pam exec run), provider-hook (from provider-native hooks), manual (normalized internally from public-API actions such as POST /runs/{id}/stop — manual sources do not call this endpoint directly; users authenticate with bearer tokens, not worker tokens)

Ingest ordering (normative) — the signal-ingest edge applies, in order:

  1. Epoch / dedup / ordering guard. Compare the signal’s (attempt, seq) pair against the run row:
    • attempt < runs.attempt → the signal comes from a dead incarnation: discard — no status write, no event emission.
    • attempt > runs.attempt → impossible under central attempt allocation: protocol error (see response table).
    • attempt == runs.attempt → pass the processed_signals dedup table (defined in 002-data-model.md; PRIMARY KEY (run_id, attempt, kind), ingest does INSERT … ON CONFLICT DO NOTHING — a conflict means the signal was already applied and is dropped), then, for repeated kinds, the seq > runs.last_progress_seq ordering check.
  2. Run status write. The guarded transition itself, per the status-transition rules in 020-provider-adapters.md.
  3. Event journal + SSE fan-out. Journal the resulting run.* event and push it to SSE subscribers.

Step 1 sits strictly upstream of both step 2 and step 3. A guard placed after either one would let a phantom event reach the dashboard for a signal whose status write was skipped — e.g., a replayed duplicate failed emitting run.failed to SSE while the run stays running. The two axes are orthogonal: attempt says which incarnation is speaking; seq says whether this signal is newer within that incarnation.

Response semantics:

Protocol-error envelope examples:

{
  "error": {
    "code": "invalid_attempt",
    "message": "signal attempt exceeds the run's current attempt",
    "details": { "signal_attempt": 3, "current_attempt": 1 }
  }
}
{
  "error": {
    "code": "missing_attempt",
    "message": "signal payload did not include an attempt field",
    "details": { "current_attempt": 1 }
  }
}

Other errors follow the standard status-code table: 400 (malformed payload), 404 (unknown run), 500 (internal failure).

Integration: Hooks configured in provider adapters (see 020-provider-adapters.md).

Versioning

Current API version: /api/v1

Future versioning strategy:

  • Major version bump (/api/v2) for breaking changes
  • Backward-compatible additions within /api/v1
  • Deprecation headers (Sunset) for phased removal

OpenAPI generation: Deferred to post-MVP (tooling selection TBD: utoipa vs hand-written).

Completion Signal Ladder

For run termination detection, override authority (conflict winner: manual > provider hook > wrapper journal) is distinct from signal trust (task-level PRIMARY = provider hooks; session-level = wrapper journal; manual lowest; pane-idle UI hint only). See the canonical specification in 020-provider-adapters.md.