AdaMAST adds a diagnostic feedback layer to an agent. It checks work at meaningful boundaries, records evidence about recurring failures, and learns a project-specific taxonomy from completed traces. Your existing agent or harness keeps owning the task.
Paper: Adaptive Failure Taxonomies as Feedback for LLM-Agent Improvement Procedures
Documentation: Website · Interactive setup · Architecture
Requirements: Python 3.10 or newer.
python -m pip install --upgrade adamastInstall AdaMAST once for the host you use:
Codex
adamast-codex-install --user-level
adamast-doctor --codexClaude Code
adamast-claude-install --user-level
adamast-doctor --claude-codeStart a new conversation after installation. AdaMAST opens the local taxonomy library and lets you choose:
- MAST to begin with the built-in general taxonomy;
- a stored taxonomy to share the project's learned vocabulary;
- No taxonomy to disable AdaMAST for that conversation.
No adamast.json, external model API key, standalone host CLI, or second login
is required for this interactive path. Run both installers when you want Codex
and Claude Code to share taxonomy state for the same Git project.
- AdaMAST resolves the Git project and task group, then pins one taxonomy.
- The agent continues normal work. Checkpoints inspect only recent activity.
- A completed assistant episode becomes one canonical trace.
- At five eligible traces, AdaMAST queues taxonomy generation by default.
- One native host subagent proposes a candidate while the main agent keeps working. After exact-span checks, a separate support-review subagent must approve every replacement code before foreground activation.
- The first refinement review occurs after ten additional traces; later reviews occur every twenty traces by default.
If a project already has a learned taxonomy, MAST remains available as a
numbered choice. Selecting it creates an isolated fresh-* task group for that
conversation and leaves the shared project taxonomy unchanged.
Learn how this is kept durable and race-safe in Native taxonomy learning.
| Goal | Start here |
|---|---|
| Use AdaMAST in every Codex task | adamast-codex-install --user-level · Codex guide |
| Use AdaMAST in every Claude Code session | adamast-claude-install --user-level · Claude Code guide |
| Configure hooks for one repository | Project setup |
| Wrap one direct model call | adamast-single-run · Single LLM guide |
| Learn from an existing trace folder | adamast-import-traces · Taxonomies |
| Integrate a custom agent harness | from adamast_runtime import start_session · Runtime API |
| Inspect an example without configuring a provider | python -m examples.dashboard_demo · Example run |
At a checkpoint, the agent follows a fixed sequence:
Observe: What concretely happened or was omitted?
Correlate: Which evidence-supported cause explains it?
Map: Which active failure code applies, if any?
Decide: Continue, or make one focused repair.
none apply is valid. AdaMAST does not manufacture a failure just to force a
change. Blocking hosts can hold completion for bounded repair rounds; Codex
uses a compact single-pass Stop checkpoint because some desktop builds do not
redeliver hook continuations.
Improvement procedures need feedback that preserves why a trajectory failed. Scalar rewards discard the reason. Free-form reflection is difficult to aggregate. A fixed taxonomy cannot know the target agent's roles, tools, or domain before observing it.
AdaMAST learns a compact set of evidence-grounded failure codes from the target system's own traces. Until a learned taxonomy is active, runs start from the built-in 14-code adaptation of MAST from "Why Do Multi-Agent LLM Systems Fail?" (Cemri et al., 2025).
Generated codes are organized along three stable axes:
| Axis | Scope | Example |
|---|---|---|
| System-level | Can arise in any agent system | Context exhaustion |
| Role-specific | Tied to a discovered component role | Checker rubber-stamps solver output |
| Domain-specific | Requires task knowledge | Algorithm mismatch |
The paper evaluates this vocabulary as feedback for best-of-N selection, evolutionary agent optimization, and runtime reflection. On TRAIL, induced codes align with expert annotations at Cohen's kappa 0.725.
| Path | Responsibility |
|---|---|
adamast_runtime/ |
Harness-neutral sessions, gates, traces, learning, validation, activation, and evidence |
adamast_integration/interactive/ |
Shared selector, browser, routes, native jobs, and receipt protocol |
adamast_integration/codex/ |
Codex hooks and transcript adapter |
adamast_integration/claude_code/ |
Claude Code hooks, gates, and transcript adapter |
adamast_integration/single_llm/ |
Direct single-model adapter |
finding/ |
MAST, taxonomy registry, display metadata, and local views |
judge_types/ |
Taxonomy and reflection judges |
AdaMAST_as_a_Judge/ |
Judge-focused evaluation checks |
vendor/adamast/ |
Maintained in-tree fork of the research generation pipeline |
examples/ |
Runnable demonstrations |
runs/ |
Evaluation artifacts and reproduction notes |
The complete ownership rules are in Architecture. Each package has its own README with a file-level map.
Reported summaries, exact taxonomies, and reproduction instructions live in
runs/. Per-question rows and raw scorer output are not included, so
the headline numbers below cannot be independently recomputed from this
repository alone.
| Experiment | Headline |
|---|---|
| OfficeQA Pro | 44.4% → 51.9% official scorer, same 133-question harness in both arms |
| Circle packing, n=26 | AdaMAST-guided search reaches 0.997 of the AlphaEvolve record in 20 evaluations |
The paper reports AdaMAST-Judge at 89.9% accuracy on Terminal-Bench 2.0 and an 87.9% to 91.9% held-out improvement for evolutionary optimization on a 655-problem set.
| Need | Page |
|---|---|
| First interactive install | Interactive setup |
| See a complete run | Example run |
| Understand terms | Concepts |
| Understand code ownership | Architecture |
| Understand native workers | Native taxonomy learning |
| Configure one project | Getting started |
| Look up every field | Configuration reference |
| Debug setup | Troubleshooting |
| Browse all docs | Documentation index |
| Command | Purpose |
|---|---|
adamast-doctor |
Validate paths, configuration, hooks, and host contracts. |
adamast-status |
Show the active taxonomy, traces, learning state, and recent decisions. |
adamast-find |
List or select stored taxonomies. |
adamast-dashboard |
Open the read-only localhost runtime dashboard. |
adamast-traces |
Inspect trace state. |
adamast-import-traces |
Generate a taxonomy from existing traces. |
adamast-codex-install / adamast-codex-uninstall |
Manage Codex hooks. |
adamast-claude-install / adamast-claude-uninstall |
Manage Claude Code hooks. |
adamast-single-run |
Wrap one direct model task with AdaMAST. |
Development setup, verification commands, and package boundaries are in CONTRIBUTING.md. Release steps are in RELEASING.md.
The original research pipeline is available on the
paper-pipeline
branch. A maintained, locally patched fork is included under
vendor/adamast/; its provenance and change categories are
documented in VENDORED.md.
Apache-2.0. See LICENSE.