WP-05 · Multi-agent orchestration

Agent Board: coordinating concurrent AI coding agents with a shared task ledger

Abstract. When several autonomous coding agents run at once, the operator loses the one thing that matters: a true picture of who is doing what. Agent Board is a local supervision layer for concurrent Claude and Codex agents. Every agent writes to one append-only SQLite ledger through one of three producers: a launch wrapper that registers the task before the process exists, Claude lifecycle hooks with a hard latency budget, and a reconciler that diffs the ledger against the process table every five seconds. Health is derived, never stored. Admission is enforced by kernel file locks, one writer and two slots per repository. Work passes a bounded, adversarial review loop, and anything that needs a person lands in a deduplicated, self-expiring human-blocked queue. The invariant throughout: unknown stays unknown.

1. The problem: invisible concurrency

I run long agent jobs in parallel: Codex builds launched as "Astra", Claude Code sessions, and the helper agents those sessions spawn. Without a shared record, three questions have no answer: is it still working, is it stuck, and how much of the subscription is it burning. The original requirement was blunt: one always-on page that is completely accurate. If an agent is working, it is on the page, with its live progress.

Accuracy here is a distributed systems problem. Agents start and die outside any single process, report asynchronously, and lie by omission when they crash. Agent Board treats the board as a projection of a ledger, and the ledger as the only source of truth.

2. Design invariants

1. Register before spawn. A task row exists before its process does, so no agent can run unrecorded through the supported path.
2. Unknown stays unknown. No estimated percentages, no inferred exit codes, no guessed usage. A Claude session without a reliable process ID shows its process status as unknown, and its Stop button is disabled.
3. Derive, do not store, health. Status is a fact recorded by producers; health is computed at read time from status and idle time.
4. Never block the agent. Instrumentation must fail open: a busy ledger drops an event rather than slow an agent down.
5. Local and dependency-free. Python standard library and SQLite only, no package installs, no CDN, and no network calls except the usage reader. The server binds to the local machine only.

3. The ledger

The ledger is one SQLite database in write-ahead logging mode, so the wrapper, the hooks, the reconciler and the server can write concurrently, with a 30 second busy timeout. It has four tables. tasks holds one row per unit of work: source (astra, claude or claude-subagent), status, process ID, parent edge, thread ID, model, effort, timestamps, exit code, last message, last command, files changed and progress counters. events is append-only and totally ordered by a sequence number, so any timeline can be reconstructed and streamed incrementally. kv holds derived snapshots such as usage readings and the last reconcile pass. items holds human-level work items in four lanes: idea, doing, waiting and done.

Two details carry weight. First, schema evolution is additive: new columns are added on connect, and old ledgers are backfilled in place. Second, the ledger keeps a byte-level cost proxy per task, maintained by triggers on every event insert and prompt update. It is explicitly a proxy, prompt plus event payload bytes, and is never presented as tokens or dollars. An early bug shaped the payload policy: truncating serialized JSON broke parsing, so oversized payloads are compacted by shrinking string fields instead of cutting the document.

4. Three producers

4.1 The launch wrapper

All Codex work goes through one wrapper. It writes the task row, then spawns the agent in JSON event mode, streams every event into the ledger, heartbeats every 15 seconds while the child lives, and records the exit code. A spawn failure is recorded as failed with exit code 127 instead of disappearing. Signal forwarding is installed before the prompt is sent, so a stop request reaches the child. The test suite proves the ordering: a fake agent waits before its first event while the test polls the ledger and asserts the row already exists.

4.2 Claude lifecycle hooks

Claude sessions are captured by hooks: session start and end, prompt submit, helper start and stop, and tool events before and after each call. Lifecycle hooks run synchronously; tool hooks run asynchronously so they never add latency to the agent. Each hook has a 25 millisecond lock wait and a 150 millisecond work budget, targeting an exit under 200 milliseconds. On timeout it logs and drops the event. It always exits zero and never writes to standard output, because anything printed would be injected into the agent's context.

4.3 The reconciler

Every five seconds a reconciler compares the ledger with reality. A running task whose process is gone and has no recorded exit becomes lost. A running task idle for more than 600 seconds becomes stalled, and flips back on its next event. It also scans the process table for Codex processes not launched through the wrapper, walks up to three parent levels to avoid double counting wrapper children, and records each one as untracked, keyed by process ID so it is never duplicated. When that process exits, it becomes lost.

5. Liveness, health and progress

Health is computed per read: working under two minutes idle, quiet from two to ten, stalled beyond ten, plus lost, done, failed, queued and untracked. Heartbeats count as activity, which makes health a liveness signal, not a progress signal. The board says so in plain words.

Percent complete exists only with a declared denominator. A run launched with a step count asks the agent to emit PROGRESS n/total lines, which the ledger parses from messages. Claude tasks report completed checklist items over total items, cleared at each new turn. Everything else shows "Progress not reported". Even then the count is the agent's own claim; the timeline is the evidence.

6. Admission control

Concurrency on one repository is bounded by a lease built on kernel advisory locks. The lock root is derived from a hash of the resolved checkout path. A writing run first takes an exclusive writer lock, so there is one writer per checkout. Every run then needs one of two slot locks, tried without blocking; if both are held, the run blocks on the first. The effective cap is two wrapper-controlled runs per repository. There are no sentinel files, sleeps or polling loops, and a crashed holder releases its lock when the kernel closes the file.

The cap came from evidence, not taste. On the first full day, with every task at high effort, the Codex weekly allowance dropped to 11% left, and a third concurrent stream stalled while two ran. Release work (full test suites, browser suites, batched deploys) takes a separate global browser lock and a source-bound receipt, so focused tests stay inside tasks and aggregate verification runs once per release window. Runs also carry a token budget, 40 million by default, counted from the agent's own token events; a run over budget is stopped and exits 125.

7. The review loop

Implementation runs as a role, worker, followed by an isolated review. The reviewer gets a packet, not a conversation: the task-local diff, at most 200 lines of test output, the worker report and the path of the original brief, inside a 256 KB budget. When the budget is tight, unchanged context and non-test file bodies are dropped with recovery paths the reviewer can open. Prompts over 8,000 bytes are stored as files and passed by reference.

The verdict grammar is strict: exactly APPROVE, or a numbered list of fixes. Anything else is classified INVALID. A FIX verdict returns to the worker once, then a second review runs. If it is still unresolved the run exits 2 and becomes eligible for an architect, but nothing escalates automatically. Reviewers can be Astra, Kimi, or both, where both must approve and disagreements are labeled. The ledger holds 584 verdicts: 160 APPROVE, 337 FIX and 87 INVALID. Fix requests outnumber approvals two to one, which is the point of the loop.

8. The human-blocked queue

"Needs you" is the queue of things only I can unblock. It is computed, not curated, from five sources: tasks that failed in the last two hours, reviews still asking for changes, needs raised manually in the last two hours, needs raised by a live Claude session in the last 15 minutes, and work items in the waiting lane assigned to me.

Three rules keep it short and true. A need is suppressed when it was acknowledged after its latest signal. A failure is suppressed when a later run in the same working directory, or with the same title, succeeded after it. And needs are deduplicated per task, keeping the most recent signal. The board shows at most five per lane, with the rest behind an overflow. My rule is explicit: needs you, maximum five, real and current only.

9. Presentation: one row per project

Raw tasks are the wrong unit for a human. A single Claude session can spawn dozens of helpers, and an early office view showed 109 jobs at once, many of them stale helpers that had never closed. My verdict: "There is too many helpers... this isn't supposed to be like a visual stimulus thing, it's supposed to be an actual tool."

The fix was aggregation. Tasks are routed to a project by an explicit tag, then by a single product name in the brief or branch, then by working-directory rules, falling back to the main room. Each project renders as one row, or one lead worker in its room, with helpers shown as a count. Jobs idle more than 15 minutes leave the office. The office itself is the fun part: a pixel hub where the orchestrators sit in suits, doors with green, yellow or red lights, and one themed room per project, each with a whiteboard. It obeys one hard rule of mine: the camera never moves by itself.

10. Operating record

A read-only count of the live ledger on 2026-10-23, covering 2026-09-08 to 2026-10-23:

The largest number is the most instructive. Most Codex rows were never launched by the wrapper at all: they are processes the reconciler caught running on their own, recorded as untracked, then lost when they ended. That is the untracked detector doing exactly its job, and it is also a measure of how much agent activity would otherwise be invisible.

11. Failure analysis

• The kill chain reaction. Killing a reviewer returned a signal exit, which the verdict parser read as INVALID, which triggered the fix loop, which spawned a new worker and a new review. Each kill produced two or three more tasks. The patch: a reviewer stopped by a signal ends the chain with a STOPPED verdict.
• Budget burn. The first day ran 36 tasks and about 24 agent hours in 36 hours, all at high effort. A breakdown of where usage went found re-reading context at about 25%, oversized command output at 18%, repeated aggregate verification at 12% and high effort on routine work at 12%. Roles, bounded packets, the release window and the two-run cap came out of that analysis.
• Phantom helpers. Helper tasks that never closed showed as stalled for hundreds of hours and flooded the view. Aggregation by project and a 15 minute idle cutoff replaced one sprite per helper.
• Lock contention. Builds in the board's own repository failed with "database is locked" while a separate rendering service held the ledger. The operating rule is to pause that service during builds there.
• Ready before verified. Interface work was reported ready before anyone opened it in a real browser with real data. The rule now: no UI is ready until a screenshot of it has been looked at.

12. Limitations

• Heartbeats prove liveness, not useful progress.
• Claude sessions have no reliable process ID through hooks, so they cannot be stopped from the board.
• The lease only covers wrapper-controlled runs. Direct agent calls do not participate.
• Progress counts are self-reported by the agent.
• The board is local to one machine by design.

13. Claim table

14. Sources

• Agent Board source code: ledger, wrapper, hooks, reconciler, server and policy modules, October 2026.
• The original build specification, product notes and project-rooms specification.
• The test suite, counted 2026-10-23.
• A read-only count of the live ledger, 2026-10-23.
• My project notes, 2026-09-08 to 2026-10-19.