Feature + orchestration benchmark / not a model leaderboard / 2026-10-19

There's no universal winner.

We tested how agent orchestrators run Claude Code, Codex and friends in parallel. On Linux, with a stub agent, Orca launched six agents in under 2 s against 11 to 12 s for Claude Squad, and used about ten times the memory. Which matters depends on your machine and workflow.

Measured / stub-agent test on Linux / 3 runs eachMeasured
Verified stub tasks
18/18Orca
18/18Claude Squad
Six-agent launch, worst of 3 (s). Lower is faster; highlighted bar wins

Orca 1.52 to 1.90 s / Claude Squad 11.18 to 11.89 s (includes about 1.1 s per session of tmux keystroke delays from our driver)

Peak memory, worst of 3 (MB RSS). Lower is lighter; highlighted bar wins

Orca 1124 to 1140 MB (about 1.1 GB) / Claude Squad 102 to 118 MB

The stub agent is a deterministic script, not an LLM. It measures the orchestrator, not code quality.

Key takeaways

What the data says

  1. 01

    Fast or light, not both

    In the stub-agent test Orca launched six agents in 1.52 to 1.90 s but peaked at 1124 to 1140 MB. Claude Squad took 11.18 to 11.89 s and peaked at 102 to 118 MB. Both verified 18/18.

  2. 02

    Orca: most validated

    The only GUI orchestrator we drove end-to-end on Linux, headless via orca serve. Highest weighted feature assessment (93) with full evidence coverage.

  3. 03

    T3 Code: deepest documented features

    Cross-provider delegate_task, an MCP for agents, PR flow and mobile. Its server booted headless in 1.01 s, but no agent session ran through it, so its 90 rests on docs and a server probe.

  4. 04

    Claude Squad: the light terminal option

    About 100 MB under load, scriptable through tmux, and sessions came back paused and resumable after we killed tmux. Weak on PR and remote features (assessed 65).

  5. 05

    Cloudroom: different model

    David Ondrej's tool runs sessions on cloud VMs. GUI launched in 3.3 s; the core needs root on a fresh VM so it was not installed. Assessed 72.0, range 62.0 to 82.0.

  6. 06

    No real-model results yet

    No Claude, Codex, Cursor or Grok CLI was logged in. A ready-to-run real-agent round is in the repo; nothing from it is reported as a result.

Secondary research / weighted feature assessment

Top 3 by assessed features

Read the scores as a feature assessment. Orca was run end-to-end with stub agents; T3 Code was not run through an equivalent session, so Orca 93 vs T3 Code 90 is a difference in assessed features and evidence, not an experimentally proven gap.

Assessment from docs, repos and feature testing. The range shows the score if every unevidenced criterion were 0 or 5. All 13 tools.

Tier list / feature assessment

Tiers at a glance

Feature-assessment tiers, not a performance ranking. Cutoffs and notes.

How the stub test works

6 parallel worktrees, one merge

6 parallel worktrees: 5 clean merges, 1 designed conflict (t5b). Ordered Bayer dither.

Each orchestrator launches six sessions of a deterministic stub agent in isolated worktrees. The stub waits, then writes a reference solution; hidden verifiers decide pass or fail. Two tasks edit the same function on purpose, so one merge conflicts by design. This isolates what the orchestrator controls: launch, isolation, crash containment and footprint.

Raw logs for every number are in the repo under runs/.

Rerun it yourself
Category matrix / assessment

Where each tool earns its points

Each cell is a 0 to 5 judgement; denser dither means a higher score. Hatched cells had no evidence and are unranked.

Real-agent round

Coming next

A real-agent comparison is built and ready to run: 3 orchestrators (plain git worktree as control, Orca, Claude Squad) x 1 model x the same 3 coding tasks x 3 repetitions, recording verified success, wall time, tokens and resource use. It has not been run with a real model yet because no agent CLI was logged in on the box. No results from it are shown anywhere on this site.

What it needs to run