Architecture · LikeC4

mcp-ax

Evaluates an MCP tool from the agent's perspective — discovers findings, generates patches, tracks regressions against author baselines

Open the interactive explorer → View on GitHub

built evolving planned research risk

The whole picture

System landscape

The system in context, then opened up into its containers.

mcp-ax — system landscape — mcp-ax — system landscape
index

mcp-ax — containers (built vs planned) — mcp-ax — containers (built vs planned)
mcpaxSystem

Evaluates an MCP tool from the agent's perspective — discovers findings, generates patches, tracks regressions against author baselines

Risks & open questions — Risks & open questions
risks

Planned & research work — Planned & research work
planned

Inside each box

Containers & components

Each part decomposed into the components that implement it. Every box links to its source in the interactive explorer.

mcp-ax CLI (cli/) — pipeline stages — mcp-ax CLI (cli/) — pipeline stages
cliContainer

cli/ — deterministic, cassette-backed pipeline emitting typed JSON artifacts; dispatched by bin/mcp-ax → python3 -m cli

Eval harness (harness/) — components — Eval harness (harness/) — components
harnessContainer

harness/ — the data-driven core the CLI consumes: AX#### registry, typed-artifact schemas, scenarios, trace runtime, assertions

Trace runtime (harness/runtime/) — replay & live recorder — Trace runtime (harness/runtime/) — replay & live recorder
runtimeContainer

Cassette replay, live agent/MCP session, budget cap, run.json writer, capability profiles

Static lint rules (lints/) — AX#### implementations — Static lint rules (lints/) — AX#### implementations
lintsContainer

lints/ — the AX#### rule implementations the lint stage runs; consume the registry, never hardcode finding logic

Description-override proxy (proxy/) — R9 components — Description-override proxy (proxy/) — R9 components
proxyContainer

R9 middleware that rewrites only the description field of a tools/list response so authors A/B a wording without touching the server

CI gate tooling (tools/) — sentinel + schema + prompt + freshness — CI gate tooling (tools/) — sentinel + schema + prompt + freshness
ciContainer

tools/ — the linters and checks wired into make + the sentinel-gate workflow

How it runs

Walkthrough flows

Dynamic views — the narrative spine of the system, step by step.

Walkthrough — static lint → explain → fix loop — Walkthrough — static lint → explain → fix loop
lintFix

The $0 tier: no model, no network. Lint a tools/list manifest, explain any AX#### that fires, and emit an idempotent auto-fix diff (dry-run by default).

Walkthrough — a trace run (replay by default) — Walkthrough — a trace run (replay by default)
traceRun

Budget is a hard precondition checked before any work. Cassettes replay byte-for-byte; --record/--no-cassette go live through the recorder. Every agent message is scanned for self-scoring (R8).

Walkthrough — live recording (--record) — Walkthrough — live recording (--record)
liveRecord

EVOLVING. The recorder drives the agent loop over a real LLM + MCP session, shapes schema-valid trace records, enforces the budget the moment cost crosses the cap, and writes a fresh cassette. All reasoning is the model's (ZFC).

Walkthrough — R9 description-override try cycle — Walkthrough — R9 description-override try cycle
tryCycle

The author inner loop (≤60s, ≤$0.10): rewrite only the description, prove nothing else changed, then lint + smoke-trace the rewritten catalog and diff. Live proxy stdio is gated on ph3.14; the deterministic core runs today.

Where it runs

Deployment

What runs where, and the process & data boundaries between the pieces.

Deployment — where each piece runs — Deployment — where each piece runs
deployment

Explore it live

The figures above are static exports. The interactive explorer lets you pan, zoom, follow relationships, and jump from any box to the source.

Open the interactive explorer →