Four Agents Polypack

Four-Agents-Polypack

Repository: imattau/Four-Agents-Polypack

A research experiment: four independent LLM agent processes collaborating solely through a shared polypack-mcp memory store — no direct agent-to-agent messaging, no central orchestrator. Each agent reads shared memory, decides on an action (propose / support / challenge / revise / summarise / consensus), and writes its contribution back, entirely on its own loop.

The question it tests: does shared adaptive memory plus independent LLM iteration produce real multi-agent collaboration — genuine information sharing, disagreement, and consensus-seeking — without any coordination logic telling the agents what to do or when?

How it works

                 Polypack MCP (shared store)
                            |
             ┌──────────────┼──────────────┐
        Agent A          Agent B          Agent C
                            |
                         Agent D

Each agent is a separate process (experiment/agent.py) with a unique id, its own LLM connection, and the same task and Polypack context. experiment/runner.py only starts and stops the four processes — it never assigns roles, picks who acts next, or resolves disputes. Consensus is something the agents have to construct themselves, by linking their contributions in the shared graph (using memory_link's RESPONDS_TO relationship).

Works against either Groq (cloud) or a local Ollama instance — same code, selected by one env var — so the whole thing can run with no API key and no rate limit.

Experiments

Three tasks, same architecture, one variable changed at a time:

Task id	What it tests
hiker-kit	Baseline coordination: all four agents start with identical information (choose 5 items for an emergency kit) — any consensus gap is purely about coordination behavior.
comms-choice	Distributed evidence under real disagreement: each agent is privately briefed with different, genuinely relevant (and true) evidence favoring different conclusions — reaching a good decision requires combining information no single agent starts with.
evac-route	Misinformation rejection: three agents get true, consistent evidence; the fourth gets a fabricated claim that directly contradicts it on a checkable point. Tests whether the group catches and rejects bad information rather than absorbing it.

Full results, methodology, and a cross-cutting review of how well polypack itself performed as infrastructure are in experiment/REPORT.md.

Selected findings (Experiment 1A, hiker-kit)

Three live runs (4 agents, same model via Groq, shared polypack-mcp store, no orchestrator), surfacing and fixing two real implementation bugs along the way:

• Run 1 — 3 of 4 agents crashed on unhandled groq.RateLimitError (shared org-wide rate limit across four keys). Fixed with retry/backoff plus a turn-skip guard instead of a hard crash.
• Run 2 (post-fix) — all 4 agents survived 16 turns, zero crashes. The group converged in substance (near-identical 5-item kits chosen independently), but the graph under-reported it: of 4 support actions, only 1 RESPONDS_TO edge was genuinely cross-agent — the other 3 agents linked support back to their own restated proposal. Checking actual content (not just the graph) showed real cross-agent information flow was happening even though formal credit-giving via target_memory_id mostly wasn't. Fixed with an explicit "CONSENSUS RULE" prompt instruction: scan shared memory for an existing consensus proposal and respond to that exact id before proposing a new one.
• Run 3 (post prompt-fix) — all 4 agents again survived 14 turns, zero crashes. Cross-agent response ratio jumped to 70% (7 of 10 support edges targeted another agent's memory), with one proposal picking up explicit support from 3 of 4 distinct agents — one short of the spec's strict 4/4 threshold. The fourth agent sustained a genuine, repeated minority position on a different, also-converged proposal, rather than fragmenting or going silent.

A 10-run batch replication against local Ollama (gpt-oss:20b) followed, to check whether Run 3's near-consensus was characteristic or a fluke — see the full report for those results and a second crash bug it surfaced (an empty-completion edge case under response_format=json_object).

Quick start

pip install -r experiment/requirements.txt

# run all four agents against a local Ollama instance (no API key, no rate limit)
LLM_PROVIDER=ollama POLYPACK_MCP_URL=http://127.0.0.1:8765/mcp/ python experiment/runner.py

# or against Groq (needs experiment/api_keys.json -- see experiment/api_keys.example.json)
POLYPACK_MCP_URL=http://127.0.0.1:8765/mcp/ python experiment/runner.py

# pick a task and replicate it N times with aggregated metrics
python experiment/batch_run.py --runs 10 --provider ollama --task evac-route

See CLAUDE.md in the repo for the full command reference, task/provider selection details, and implementation notes.

---

Back to Home. See also polypack-mcp, the shared memory server this experiment runs against, and Adaptive memory, the underlying polypack feature both depend on.