agentic-review-swarm

agentic-review-swarm runs independent agent reviewers over a committed repo snapshot plus paper artifacts. Unlike review-swarm, which sends a static bundle to LLM APIs, this tool gives each reviewer its own disposable checkout so it can inspect files, run commands, and verify that paper claims are actually derived from repository contents.

The intended use case is paper reproducibility review:

• copy a clean snapshot of the repo under review
• attach the manuscript PDF/source and optional extra context
• run several agent harnesses in parallel
• collect one Markdown report per agent/role

Install

For local development:

cd /Users/danielc/agentic-review-swarm
python3 -m venv .venv
. .venv/bin/activate
pip install -e .

Quick Start

Dry-run the plan:

agentic-review-swarm run \
  --subject /Users/danielc/refactor-bench \
  --paper paper.pdf \
  --role reproducibility \
  --agent codex='codex exec --cwd {sandbox} --output-last-message {report} < {prompt}' \
  --dry-run

Run it for real:

agentic-review-swarm run \
  --subject /Users/danielc/refactor-bench \
  --paper paper.pdf \
  --role reproducibility \
  --role data-lineage \
  --agent codex='codex exec --cwd {sandbox} --output-last-message {report} < {prompt}' \
  --agent claude='claude -p "$(cat {prompt})"' \
  --out /tmp/agentic-paper-review

Each {sandbox} is a separate directory. The command template receives:

• {sandbox}: disposable checkout root
• {prompt}: prompt file path
• {report}: report path to write
• {log}: per-job log path
• {role}: role name
• {agent}: agent name

Templates are shell commands, so quote them as a single CLI argument. If the agent writes its final answer to stdout, the harness writes that stdout to {report} and tees combined stdout/stderr to {log}. If the agent has its own report-output flag, point it at {report}; the harness will still keep lifecycle and stderr output in {log}.

Why This Is Separate From review-swarm

review-swarm is a direct provider API caller. It is excellent for broad static critique, but it cannot run make, inspect git history, query generated result files, or compare source data to a PDF table.

This repo is intentionally agent-oriented instead:

• reviewers run in isolated checkouts
• reports are grounded in commands and file paths
• the input snapshot is recorded in a manifest
• uncommitted local state is excluded by default

Snapshot Modes

By default, agentic-review-swarm copies the committed HEAD snapshot using git archive. That is the right mode for checking whether a paper is supported by committed reproducibility artifacts.

Use --dirty only when you explicitly want reviewers to inspect the current working tree.

Output

The output directory contains:

• manifest.json: subject commit, roles, agents, and command templates
• run.log: top-level lifecycle log for monitoring the swarm
• prompts/*.md: exact prompts given to each reviewer
• reports/*.md: reviewer outputs
• logs/*.log: per-job combined stdout/stderr, written while each agent runs
• sandboxes/*: kept only with --keep-sandboxes

For long-running reviews, monitor progress with:

tail -f /tmp/agentic-paper-review/run.log
tail -f /tmp/agentic-paper-review/logs/claude-reproducibility.log

Current Status

This is a harness scaffold. It does not vendor Codex, Claude Code, or any other agent; you provide the command template for the agent CLI you want to run.