Built-in middleware
Add call limits, retries and fallbacks, PII redaction, summarization, context editing, and trace scrubbing from coco.agents.middleware.
Add the middleware the framework ships, from call limits to PII redaction.
Each middleware below comes from coco.agents.middleware. Pass it to create_agent(..., middleware=[...]), in the order you want it to run. See Middleware for the order.
Prerequisites: The framework, installed as in the Quickstart, with OPENAI_API_KEY set.
Call limits
Stop runaway loops and bound spend per run or across a whole thread:
from coco.agents.middleware import ModelCallLimitMiddleware, ToolCallLimitMiddleware
model_limiter = ModelCallLimitMiddleware(
run_limit=10,
thread_limit=50,
exit_behavior="end", # or "error"
)
tool_limiter = ToolCallLimitMiddleware(
tool_name="web_search", # None applies to all tools
run_limit=5,
thread_limit=20,
exit_behavior="continue",
)Retries and fallbacks
from coco.agents import create_agent
from coco.agents.middleware import (
ModelFallbackMiddleware,
ModelRetryMiddleware,
ToolErrorMiddleware,
ToolRetryMiddleware,
)
from coco_core.tools import tool
@tool
def get_exchange_rate(currency: str) -> str:
"""Return the exchange rate for a currency against AUD."""
return "0.65"
def on_tool_error(exc, request):
if isinstance(exc, TimeoutError):
return f"`{request.tool_call['name']}` timed out. Try again later."
return None # let every other exception propagate
agent = create_agent(
model="openai:gpt-4o",
tools=[get_exchange_rate],
middleware=[
ModelRetryMiddleware(max_retries=3, initial_delay=1.0, backoff_factor=2.0),
ModelFallbackMiddleware("anthropic:claude-sonnet-4-5-20250929"),
ToolErrorMiddleware(on_tool_error),
ToolRetryMiddleware(max_retries=2, on_failure="error"),
],
)ModelRetryMiddleware retries a failed model call with exponential backoff, and ModelFallbackMiddleware moves to the next model when the primary fails. ToolRetryMiddleware retries a failed tool.
ToolErrorMiddleware turns the tool exceptions you choose into an error ToolMessage the model can read. It needs an on_error function: return the message for an exception you want the model to see, or None to let it propagate. Put ToolRetryMiddleware after it in the list, with on_failure="error", so an exception that outlasts the retries reaches it.
PII protection
Detect personal information and "block", "redact", "mask" or "hash" it. Built-in detectors cover "email", "credit_card", "ip", "mac_address" and "url", and you can pass your own regex or detector function.
from coco.agents.middleware import PIIMiddleware
pii_guard = PIIMiddleware(
"credit_card",
strategy="redact", # "block" | "redact" | "mask" | "hash"
apply_to_input=True,
apply_to_output=True,
apply_to_tool_results=True,
)Context window management
Keep long threads within a token budget:
from coco.agents.middleware import (
ClearToolUsesEdit,
ContextEditingMiddleware,
SummarizationMiddleware,
)
summarizer = SummarizationMiddleware(
model="openai:gpt-4o-mini",
trigger=("tokens", 32_000),
keep=("messages", 10),
)
context_cleaner = ContextEditingMiddleware(
edits=[ClearToolUsesEdit(trigger=20_000, keep=5)]
)SummarizationMiddleware summarizes older messages once the thread passes the trigger, and ContextEditingMiddleware clears stale tool results.
Trace payload privacy
Keep timing and structure in CocoTracer, but leave message and state payloads out of middleware spans:
from coco.agents.middleware import TracePolicy, configure_trace_policy, omit_payload
# Apply process-wide to all middleware spans
configure_trace_policy(
TracePolicy(
process_inputs=omit_payload,
process_outputs=omit_payload,
)
)All built-in middleware
| Middleware | What it does |
|---|---|
HumanInTheLoopMiddleware | Pauses before the tools you name for a person to approve, edit, or reject. See Human in the loop. |
ModelCallLimitMiddleware | Limits model calls per run and per thread. |
ToolCallLimitMiddleware | Limits tool calls per run and per thread, for all tools or one. |
ModelRetryMiddleware | Retries failed model calls with exponential backoff. |
ModelFallbackMiddleware | Falls back to other models when the primary fails. |
ToolRetryMiddleware | Retries failed tool calls. |
ToolErrorMiddleware | Turns the tool exceptions its on_error function chooses into error ToolMessage results. |
PIIMiddleware | Detects and blocks, redacts, masks, or hashes personal information. |
SummarizationMiddleware | Summarizes older history past a token or message threshold. |
ContextEditingMiddleware | Clears stale tool uses with ClearToolUsesEdit. |
LLMToolSelectorMiddleware | Uses a fast model to pick the relevant tools from a large catalog each turn. |
ProviderToolSearchMiddleware | Configures the provider's own tool search for large toolsets. |
LLMToolEmulator | Emulates tools with a model, for dry runs. |
TodoListMiddleware | Gives the agent tools to keep a task list in its state. |
ShellToolMiddleware | Runs shell commands under an execution policy, with redaction rules. |
FilesystemFileSearchMiddleware | Gives the agent scoped glob and content search over files. |