Coco
Build agents

Built-in middleware

Add call limits, retries and fallbacks, PII redaction, summarization, context editing, and trace scrubbing from coco.agents.middleware.

Add the middleware the framework ships, from call limits to PII redaction.

Each middleware below comes from coco.agents.middleware. Pass it to create_agent(..., middleware=[...]), in the order you want it to run. See Middleware for the order.

Prerequisites: The framework, installed as in the Quickstart, with OPENAI_API_KEY set.


Call limits

Stop runaway loops and bound spend per run or across a whole thread:

from coco.agents.middleware import ModelCallLimitMiddleware, ToolCallLimitMiddleware

model_limiter = ModelCallLimitMiddleware(
    run_limit=10,
    thread_limit=50,
    exit_behavior="end",  # or "error"
)

tool_limiter = ToolCallLimitMiddleware(
    tool_name="web_search",  # None applies to all tools
    run_limit=5,
    thread_limit=20,
    exit_behavior="continue",
)

Retries and fallbacks

from coco.agents import create_agent
from coco.agents.middleware import (
    ModelFallbackMiddleware,
    ModelRetryMiddleware,
    ToolErrorMiddleware,
    ToolRetryMiddleware,
)
from coco_core.tools import tool


@tool
def get_exchange_rate(currency: str) -> str:
    """Return the exchange rate for a currency against AUD."""
    return "0.65"


def on_tool_error(exc, request):
    if isinstance(exc, TimeoutError):
        return f"`{request.tool_call['name']}` timed out. Try again later."
    return None  # let every other exception propagate


agent = create_agent(
    model="openai:gpt-4o",
    tools=[get_exchange_rate],
    middleware=[
        ModelRetryMiddleware(max_retries=3, initial_delay=1.0, backoff_factor=2.0),
        ModelFallbackMiddleware("anthropic:claude-sonnet-4-5-20250929"),
        ToolErrorMiddleware(on_tool_error),
        ToolRetryMiddleware(max_retries=2, on_failure="error"),
    ],
)

ModelRetryMiddleware retries a failed model call with exponential backoff, and ModelFallbackMiddleware moves to the next model when the primary fails. ToolRetryMiddleware retries a failed tool.

ToolErrorMiddleware turns the tool exceptions you choose into an error ToolMessage the model can read. It needs an on_error function: return the message for an exception you want the model to see, or None to let it propagate. Put ToolRetryMiddleware after it in the list, with on_failure="error", so an exception that outlasts the retries reaches it.

PII protection

Detect personal information and "block", "redact", "mask" or "hash" it. Built-in detectors cover "email", "credit_card", "ip", "mac_address" and "url", and you can pass your own regex or detector function.

from coco.agents.middleware import PIIMiddleware

pii_guard = PIIMiddleware(
    "credit_card",
    strategy="redact",      # "block" | "redact" | "mask" | "hash"
    apply_to_input=True,
    apply_to_output=True,
    apply_to_tool_results=True,
)

Context window management

Keep long threads within a token budget:

from coco.agents.middleware import (
    ClearToolUsesEdit,
    ContextEditingMiddleware,
    SummarizationMiddleware,
)

summarizer = SummarizationMiddleware(
    model="openai:gpt-4o-mini",
    trigger=("tokens", 32_000),
    keep=("messages", 10),
)

context_cleaner = ContextEditingMiddleware(
    edits=[ClearToolUsesEdit(trigger=20_000, keep=5)]
)

SummarizationMiddleware summarizes older messages once the thread passes the trigger, and ContextEditingMiddleware clears stale tool results.

Trace payload privacy

Keep timing and structure in CocoTracer, but leave message and state payloads out of middleware spans:

from coco.agents.middleware import TracePolicy, configure_trace_policy, omit_payload

# Apply process-wide to all middleware spans
configure_trace_policy(
    TracePolicy(
        process_inputs=omit_payload,
        process_outputs=omit_payload,
    )
)

All built-in middleware

MiddlewareWhat it does
HumanInTheLoopMiddlewarePauses before the tools you name for a person to approve, edit, or reject. See Human in the loop.
ModelCallLimitMiddlewareLimits model calls per run and per thread.
ToolCallLimitMiddlewareLimits tool calls per run and per thread, for all tools or one.
ModelRetryMiddlewareRetries failed model calls with exponential backoff.
ModelFallbackMiddlewareFalls back to other models when the primary fails.
ToolRetryMiddlewareRetries failed tool calls.
ToolErrorMiddlewareTurns the tool exceptions its on_error function chooses into error ToolMessage results.
PIIMiddlewareDetects and blocks, redacts, masks, or hashes personal information.
SummarizationMiddlewareSummarizes older history past a token or message threshold.
ContextEditingMiddlewareClears stale tool uses with ClearToolUsesEdit.
LLMToolSelectorMiddlewareUses a fast model to pick the relevant tools from a large catalog each turn.
ProviderToolSearchMiddlewareConfigures the provider's own tool search for large toolsets.
LLMToolEmulatorEmulates tools with a model, for dry runs.
TodoListMiddlewareGives the agent tools to keep a task list in its state.
ShellToolMiddlewareRuns shell commands under an execution policy, with redaction rules.
FilesystemFileSearchMiddlewareGives the agent scoped glob and content search over files.

On this page