psf-memo/doc/swarmforge.md

SwarmForge — Description, protocol, and migration requirements

Working document. Project source: https://github.com/unclebob/swarm-forge Goal: understand SwarmForge's architecture and evaluate adapting it to pi as the agent, on Linux, with DeepSeek / GLM / Qwen models.

---

1. Project description

SwarmForge is a tmux-based agent orchestration platform that turns a swarm of AI agents into a coordinated software engineering team. It was created by Robert C. Martin and applies his own engineering discipline (TDD, Gherkin/acceptance testing, mutation testing, CRAP/DRY analysis) to the problem of coordinating agents.

Core idea: each agent lives in its own git worktree and its own tmux session, and agents communicate via a file-based handoff protocol delivered by a daemon. There are no direct messages between agents and no direct access to the tmux socket by them.

Branch structure

Branch	Description	Roles
main	Documentary: shared operational scripts + default constitution articles	—
two-pack	Fast backend workflow (TDD + hardening, no Gherkin)	coder → cleaner → coder
four-pack	Compact workflow with Gherkin specification	specifier → coder → refactorer → architect → specifier
six-pack	Full workflow with all quality gates separated	specifier → coder → cleaner → architect → hardender → QA → end

Each executable branch contains the project config: swarmforge.conf (topology), roles/<role>.prompt (per-role prompts) and constitution.prompt + articles (shared rules). On startup, the ./swarm wrapper downloads the shared operational scripts from main (first time only) and launches the orchestrator.

How it works at a high level

1. Declarative configuration: swarmforge.conf defines the swarm window by window:

   window <role> <agent> <worktree> [task|batch] [extra-args...]
   

2. Launcher (swarmforge.bb, Babashka): validates the config, initializes the git repo if needed, creates a worktree per role under .worktrees/, creates a tmux session per role on a project-owned socket and launches each agent with its initial prompt.
3. Agents: each runs as an interactive TUI in its tmux pane, inside its worktree, with the handoff scripts on its PATH.
4. Daemon (handoffd.bb): owner of the tmux socket. Watches agent outboxes, delivers handoffs to recipient inboxes and wakes agents with a message typed into their pane.
5. Handoff protocol: agents create validated drafts, receive them as tasks or batches (task/batch), and report completion with done_with_current.sh.
6. Optional viewer: terminal adapters (terminal-adapters/*.sh) open one window per role for real-time observation, with a watchdog that reopens closed windows without losing agent state.

Key features

• Config-driven topology: swarm shape comes from swarmforge.conf, not from code.
• Per-project roles: swarmforge/roles/<role>.prompt per branch/backlog.
• Layered constitution: constitution.prompt directs agents to read articles under swarmforge/constitution/articles/ (shared engineering, handoff and workflow rules + local per-branch rules).
• Per-role backends: each role can use a different agent CLI (claude, codex, copilot, grok).
• Observable: one terminal window per role, or headless in tmux.
• Self-hosted and light: only needs tmux, git, zsh and Babashka; all state lives in .swarmforge/ inside the project.
• Operational robustness: host sleep prevention (caffeinate/systemd-inhibit), task resumption after restart, file-based audit (new → in_process → completed).

---

2. Handoff protocol (summary)

The protocol separates state (files on the filesystem, durable and auditable) from control (tmux, only for notification and liveness).

Messages

Only two types, both strictly validated:

type: git_handoff          type: note
to: <role>[,<role>...]     to: <role>[,<role>...]
priority: NN (00-99)       priority: NN (00-99)
task: <stable-name>        message: <1 line, max 80 chars>
commit: <10 hex>

• git_handoff: the sender has committed work; the receiver does merge_and_process <role> <commit>.
• note: short message; only when the constitution or role explicitly authorizes it.

Flow

1. The agent commits and writes a draft with headers only.
2. swarm_handoff.sh is the validation gate: rejects reserved fields, unknown roles, invalid priorities, ambiguous commits (canonicalizes the hash with git rev-parse --disambiguate) and bodies that are not generated.
3. The helper generates the payload (id, from, role, task, created_at, body) and installs it atomically in outbox/.
4. The daemon polls (1s), copies the handoff to each recipient's inbox/new/ (adding recipient and enqueued_at) and wakes the receiver.
5. The receiver runs ready_for_next.sh → moves to inbox/in_process/ (adds dequeued_at) and prints TASK:/BATCH: with the payload.
6. On completion, done_with_current.sh moves to inbox/completed/ (adds completed_at) and picks up the next task if one exists.
7. The daemon moves the sender's original to sent/ or failed/.

Wake-up (control plane)

The daemon "wakes" an agent by typing into its tmux pane:

tmux send-keys -t <session> -l "You have new handoff mail. If idle, run ready_for_next.sh."
tmux send-keys -t <session> C-m    # Enter
tmux send-keys -t <session> C-j    # robustness LF

The agent receives it as a user message. Protocol rules: if it is already working, ignore the wake-up; done_with_current.sh picks up the next task when finished. In practice, an agent with a message queue (like pi) enqueues the wake-up and delivers it when the turn ends.

Chain rules

• Intermediate roles always forward a git_handoff to the next role in the chain, no matter what (even if the change is non-functional).
• The final handoff of the chain (broadcast) is merge-only: recipients merge and do not forward.
• Task names (task:) are preserved along the chain.

---

3. Migration requirements

3.1 Agent contract (necessary condition)

SwarmForge requires the agent to be a long-lived interactive process in a tmux pane that satisfies:

1. Interactive CLI (TUI/REPL) that keeps running — wake-ups arrive as typed text + Enter; a one-shot CLI cannot receive work.
2. Initial prompt via command line (or injectable via tmux send-keys after startup).
3. Work in the worktree directory (cd <worktree> && <agent> ...).
4. Ability to run commands (the helpers swarm_handoff.sh, ready_for_next.sh, done_with_current.sh are shell/bb on PATH — they are model-agnostic).

Everything else in the protocol (handoffs, worktrees, daemon, wake-ups, watchdog) does not know about the model: the only integration point is the launch arm in swarmforge.bb and the validated backend list in parse-config.

3.2 Validation: pi as agent

Fits out of the box. Verified in docs and installed binary:

• pi "<prompt>" starts the TUI, sends the initial message and stays interactive (confirmed in dist/modes/interactive/interactive-mode.js).
• tmux officially supported (docs/tmux.md). Recommendation: tmux ≥ 3.5 with extended-keys-format csi-u for modified keys; the basic protocol (Enter) works with any version.
• Compatible wake-up: in pi Enter = send, Ctrl+J = new line (the daemon's C-j is harmless).
• Message queue: a message typed while pi is working is enqueued and delivered when the turn ends — ideal for the protocol's wake-up semantics.
• Sessions: pi -c (continue), --session, --name "SwarmForge <Role>" (session name).

Operational requirements with pi:

Requirement	Detail
Trust prompt	pi asks on startup in a new project and would block the agent. Use -a/--approve in the launch arm or pre-seed ~/.pi/agent/trust.json.
Fixed model	Use --model <provider>/<model> so each role does not start at the login selector.
Runtime	Node.js ≥ 22 (npm install) or standalone script; Linux supported natively.

Proposed launch arm in swarmforge.bb:

"pi" (str "pi -a --name " (sq (str "SwarmForge " display))
           " --model " (sq model) " "
           (extra-args-prefix row)
           "\"$(cat " (sq (str prompt-file)) ")\"")

3.3 Validation: opencode as agent

Fits with a mandatory adaptation. Verified on the real v1.18.15 binary and in source:

• opencode (no args) starts the persistent interactive TUI.
• ⚠️ The TUI does not accept an initial message via CLI: --prompt in TUI mode calls a Node rl.question (waits for stdin input); only --mini --prompt sends it as a message, but with interactive: false (runs and exits). opencode run "<msg>" is headless one-shot — not usable as a swarm agent.
• Solution: launch the TUI (opencode --auto) and inject the prompt with tmux send-keys after startup — the same mechanism the daemon already uses to wake. About ~10 lines in launch-role! (launch → sleep → send-keys -l "$(cat prompt)" + Enter).

opencode --auto -m <provider>/<model>    # in the role's tmux session
# after ~2s:
tmux send-keys -t <target> -l "<initial prompt>" ; tmux send-keys -t <target> C-m

Operational requirements with opencode:

Requirement	Detail
Permissions	--auto (auto-approve; also the hidden aliases --yolo / --dangerously-skip-permissions) — equivalent to the swarm's autonomous mode.
Sessions	-c/--continue, -s/--session for the restart flow ("on restart, run ready_for_next.sh").
Runtime	Static binary (npm opencode-ai or GitHub release); Linux supported.
Future	opencode serve + attach/SDK/ACP would allow a native message queue without keyboard wake-ups (would require changing the architecture, not adapting it).

3.4 Models: DeepSeek / GLM / Qwen

Model	pi	opencode
DeepSeek	Native: DEEPSEEK_API_KEY, provider deepseek, --model deepseek/...	Native in catalog (models.dev): deepseek-*
Qwen	Native: QWEN_TOKEN_PLAN_API_KEY, providers qwen-token-plan / -individual / -cn (China)	Native: qwen3.x-*, alibaba-*/qwen*
GLM (Zhipu)	Not native: needs a custom provider extension (OpenAI-compatible, api: "openai-completions", thinkingFormat: "zai") or OpenAI-compatible proxy	Native: glm-4.x/glm-5.x (opencode-go/glm-*, alibaba-*/glm-*)

Note: pi already implements the thinking formats of all three families (thinkingFormat: "deepseek" | "zai" | "qwen" in docs/custom-provider.md), which simplifies GLM integration: you only need to register the endpoint and models with that extension.

3.5 Linux (runtime)

Requirement	Status	Detail
zsh	Hard requirement	Scripts use #!/usr/bin/env zsh. Arch: pacman -S zsh.
tmux	Required	Recommended ≥ 3.5 (pi with extended keys).
git	Required	Worktrees and commit protocol.
Babashka (bb)	Required	Launcher and all helpers are Babashka (cross-platform).
Node.js ≥ 22	pi only	npm install of pi (or standalone script).
Terminal	Headless works	By default on Linux (no osascript/wt.exe) the launcher falls back to none: attaches the current shell to the first role's session and the rest stay detached (tmux -S <socket> attach -t swarmforge-<role>). The swarm runs fully without windows.
Automatic windows (optional)	To build	Write a terminal-adapters/wezterm.sh (or kitty) for Linux: 5-function contract (~40 lines). WezTerm is the most scriptable (wezterm cli); Ghostty on Linux has no remote control.
Shutdown	Plan for	The close-swarm script lives on the main branch; executable branches do not carry it — copy it into the project or use shutdown by "closing the first window".
Sleep prevention	Works	systemd-inhibit on Linux (systemd running). Disable with SWARMFORGE_PREVENT_SLEEP=0.

3.6 Necessary code changes (minimal)

In swarmforge/scripts/swarmforge.bb (the working branch, e.g. four-pack):

1. parse-config: add the backend to the validated list, e.g. #{"claude" "codex" "copilot" "grok" "pi"}.
2. launch-command: add the new backend's arm (pi: section 3.2; opencode: section 3.3).
3. check-backend-dependencies!: no changes — already checks that the binary exists on PATH.

In the project config:

• swarmforge.conf: window coder pi master (or opencode), with [task|batch] and extra args per role.

Optional depending on goal:

• Shared constitution articles in swarmforge/constitution/articles/ of the branch (the wrapper only stages them in scripts/shared-articles/; confirm agents read what the branch needs).
• Linux terminal adapter (section 3.5).
• close-swarm in the project.

---

4. Architecture diagram

flowchart TB
    subgraph Config["Configuration (per project/branch)"]
        CONF["swarmforge.conf<br/>window role agent worktree [task|batch] [args]"]
        ROLES["swarmforge/roles/&lt;role&gt;.prompt"]
        CONST["swarmforge/constitution.prompt<br/>+ constitution/articles/"]
    end

    subgraph Launcher["Launcher — swarmforge.bb (Babashka)"]
        PARSE["Validate config and prompts"]
        WT["Git worktrees<br/>.worktrees/&lt;role&gt; (branch per role)"]
        TMUX["tmux sessions<br/>swarmforge-&lt;role&gt; · project-owned socket"]
        LAUNCH["send-keys: export SWARMFORGE_ROLE<br/>+ PATH helpers + cd worktree<br/>+ &lt;agent&gt; '$(cat prompt)'"]
    end

    subgraph Swarm["Swarm (1 agent per role)"]
        A1["Agent TUI<br/>(tmux pane)"]
        A2["Agent TUI<br/>(tmux pane)"]
        A3["Agent TUI<br/>(tmux pane)"]
    end

    subgraph State["Durable state — filesystem (.swarmforge/handoffs)"]
        OUT["outbox/ · sent/ · failed/"]
        IN["inbox/ new · in_process · completed"]
    end

    subgraph Control["Control — daemon handoffd.bb"]
        DAEMON["Poll outbox → deliver to inbox<br/>→ wake-up via tmux send-keys"]
    end

    subgraph Viewer["Viewer (optional)"]
        ADAPT["terminal-adapters/*.sh"]
        WATCH["swarm-window-watchdog"]
    end

    CONF --> PARSE
    ROLES --> PARSE
    CONST --> PARSE
    PARSE --> WT --> LAUNCH
    PARSE --> TMUX --> LAUNCH
    LAUNCH --> A1 & A2 & A3
    A1 & A2 & A3 -->|"helpers on PATH:<br/>swarm_handoff.sh"| OUT
    OUT --> DAEMON
    DAEMON -->|"deliver .handoff"| IN
    DAEMON -->|"wake-up: text + Enter"| A1 & A2 & A3
    A1 & A2 & A3 -->|"ready_for_next.sh<br/>done_with_current.sh"| IN
    A1 & A2 & A3 -->|"work (git)"| WT
    A1 & A2 & A3 -->|"observe: tmux attach"| TMUX
    TMUX --> ADAPT --> WATCH

5. Protocol diagram (one handoff cycle)

sequenceDiagram
    autonumber
    participant S as Sender agent (e.g. coder)
    participant V as swarm_handoff.sh (gate)
    participant O as outbox/ (sender)
    participant D as Daemon handoffd.bb
    participant I as inbox/ (receiver)
    participant R as Receiver agent (e.g. cleaner)

    Note over S: git commit (message with role byline)
    S->>S: Write draft (type/to/priority/task/commit)
    S->>V: swarm_handoff.sh `<draft>`
    V->>V: Validate: known roles, priority 00-99,<br/>canonical commit (10 hex, --disambiguate),<br/>reserved fields, body forbidden
    V->>O: Install generated .handoff (id, from, role,<br/>task, created_at, merge_and_process payload)
    V-->>S: HANDOFF QUEUED
    O->>D: Poll (1 s)
    D->>I: Copy to each recipient inbox/new/<br/>+ recipient, enqueued_at headers
    D->>R: tmux send-keys -l 'You have new handoff mail...'<br/>+ C-m (Enter) + C-j (robustness)
    Note over R: If busy → ignore (queue or next<br/>done_with_current will pick it up)
    R->>I: ready_for_next.sh → move to in_process/<br/>+ dequeued_at header
    I-->>R: TASK: `<path>` / BATCH: `<items>` + PAYLOAD
    R->>R: merge_and_process `<sender>` `<commit>`<br/>+ process the task in its worktree
    R->>I: done_with_current.sh → completed/<br/>+ completed_at header
    I-->>R: Next task or NO_TASK
    D->>O: Move original to sent/ (or failed/)

Inbox task lifecycle

stateDiagram-v2
    [*] --> new: daemon delivers .handoff
    new --> in_process: ready_for_next.sh (dequeued_at)
    in_process --> completed: done_with_current.sh (completed_at)
    in_process --> in_process: next queued task
    new --> [*]: NO_TASK (empty queue)
    failed --> [*]: delivery error (sender outbox)

---

6. Executive summary

1. The architecture is model-agnostic: the only integration point for a new backend is the launch arm + the validated backend list in swarmforge.bb; the handoff protocol, worktrees, daemon and wake-ups do not know about the agent.
2. pi fits directly: interactive initial message via CLI, tmux supported, message queue aligned with wake-up semantics, native DeepSeek/Qwen and GLM with a small extension.
3. opencode fits with an adaptation: the TUI does not accept an initial prompt via CLI → inject via tmux send-keys after startup (mechanism already in the system). Native DeepSeek/GLM/Qwen.
4. Linux is a first-class citizen by design: the swarm lives in tmux, not in windows; headless runs fully. Automatic windows are only an optional terminal adapter.
5. Minimum requirements: zsh + tmux (≥3.5 recommended) + git + Babashka + (Node.js for pi) + ~15 lines of changes in swarmforge.bb + provider config.