I Cut Claude API Costs by 50% Using a Self-Modifying Agentic System

My Claude API bill hit $340 in a single month. The system I was running was doing what it was supposed to do—processing thousands of tasks per day, generating content, classifying inputs, drafting responses—but the cost curve was pointing straight up and I had no clear way to bend it without sacrificing quality.

So I built something that fights back: a self-modifying agentic loop that analyzes each incoming task, routes it to the cheapest model capable of handling it, compresses the context window before every call, and caches repeated prompt structures using Anthropic’s native caching API. The result was a 50% reduction in monthly API spend with zero degradation in output quality on the tasks that mattered.

This article walks through the entire architecture, the specific code patterns, and the three levers that drive the most savings. If you’re running any kind of production Claude workload, this is worth your time.

Why Claude API Costs Spiral Out of Control

Before getting into the solution, it helps to understand exactly where the money goes.

Claude charges per token: input tokens and output tokens, priced separately. Claude 3.5 Sonnet costs $3.00 per million input tokens and $15.00 per million output tokens (as of mid-2026). Claude 3 Haiku sits at $0.25 and $1.25 respectively. That’s a 12x difference on inputs and a 12x difference on outputs.

Most people building agents make four expensive mistakes:

1. They use Sonnet for everything. Classification, routing, formatting, summarization—tasks a much cheaper model handles just as well.
2. They pass the same bloated system prompt on every call. A 2,000-token system prompt sent 10,000 times per month is 20 million tokens before you’ve done a single unit of useful work.
3. They let context windows grow unchecked. Multi-turn agents accumulate conversation history that often exceeds what’s actually necessary for the current task.
4. They never cache. Anthropic’s prompt caching can cut costs on repeated prefixes by up to 90%, and most developers don’t use it at all.

The self-modifying system I built addresses all four of these.

The Architecture: A Three-Layer Cost Control Stack

The system has three layers that operate before every Claude API call:

Layer 1: Task Classifier (Haiku) A fast, cheap Haiku call reads the incoming task and assigns it a complexity score (1-5) and a task type (classification, generation, reasoning, coding, summarization). This costs about 0.003 cents per call. It returns a JSON object used by Layer 2.

Layer 2: Model Router Based on the classifier output, the router picks the optimal model. Complexity 1-2 tasks go to Haiku. Complexity 3 tasks go to Haiku with extended thinking disabled. Complexity 4-5 tasks go to Sonnet. The router also selects one of three system prompt variants: minimal (200 tokens), standard (800 tokens), or full (2,000 tokens).

Layer 3: Context Compressor Before the final call, the compressor trims the conversation history. It keeps the last N turns where N is determined by task type: classification tasks get zero history, generation tasks get 2 turns, reasoning tasks get 5 turns. Older turns are replaced with a summary generated (by Haiku, again cheaply) during a previous pass.

This is the “self-modifying” piece. The system prompt sent to Claude on any given call is not static. It is selected dynamically based on what the task actually needs. The agent modifies its own instructions before each execution.

Building the Task Classifier

Here is the core classifier function:

import anthropic
import json

client = anthropic.Anthropic()

CLASSIFIER_SYSTEM = """You are a task complexity classifier. 
Respond ONLY with valid JSON: {"complexity": 1-5, "type": "classification|generation|reasoning|coding|summarization"}
1 = trivial (yes/no, single label), 5 = complex (multi-step reasoning, code generation)."""

def classify_task(task_text: str) -> dict:
    response = client.messages.create(
        model="claude-3-haiku-20240307",
        max_tokens=64,
        system=CLASSIFIER_SYSTEM,
        messages=[{"role": "user", "content": f"Classify this task: {task_text[:500]}"}]
    )
    return json.loads(response.content[0].text)

The classifier costs roughly $0.00003 per call. On a 10,000 call/day system that’s $0.30/day in classifier overhead. You’ll recover that immediately from routing savings.

Intelligent Model Routing: Where Most of the Savings Come From

With complexity scores in hand, routing is straightforward:

MODEL_MAP = {
    "low": "claude-3-haiku-20240307",       # complexity 1-2
    "medium": "claude-3-haiku-20240307",    # complexity 3
    "high": "claude-3-5-sonnet-20241022",   # complexity 4-5
}

SYSTEM_PROMPTS = {
    "minimal": "You are a helpful assistant. Be concise.",  # ~20 tokens
    "standard": open("prompts/standard.txt").read(),       # ~800 tokens
    "full": open("prompts/full.txt").read(),               # ~2000 tokens
}

def route_task(task_text: str, classification: dict) -> dict:
    complexity = classification["complexity"]
    task_type = classification["type"]
    
    if complexity <= 2:
        model = MODEL_MAP["low"]
        prompt_key = "minimal"
    elif complexity == 3:
        model = MODEL_MAP["medium"]
        prompt_key = "standard"
    else:
        model = MODEL_MAP["high"]
        prompt_key = "full"
    
    # Classification tasks never need history
    max_history_turns = 0 if task_type == "classification" else (2 if complexity <= 3 else 5)
    
    return {
        "model": model,
        "system_prompt": SYSTEM_PROMPTS[prompt_key],
        "max_history_turns": max_history_turns
    }

In my workload, approximately 60% of tasks were complexity 1-3. Routing those to Haiku instead of Sonnet produced a 38% reduction in token spend before any other optimization.

For a deeper look at how Claude compares to other APIs on a cost-per-task basis, see our breakdown of Claude API vs OpenAI API costs for developers.

Prompt Caching: The Multiplier Nobody Uses

Anthropic’s prompt caching lets you mark a portion of your prompt as cacheable. On cache hits, input tokens cost 10% of the standard rate. That’s a 90% discount on the cached portion.

The catch: you need at least 1,024 tokens in the cached prefix, and the cache lasts 5 minutes (extended to 1 hour if you use cache_control with ephemeral type). For most agent systems with a fixed system prompt, this is trivially easy to set up.

def make_cached_request(system_prompt: str, messages: list, model: str) -> str:
    response = client.messages.create(
        model=model,
        max_tokens=1024,
        system=[
            {
                "type": "text",
                "text": system_prompt,
                "cache_control": {"type": "ephemeral"}
            }
        ],
        messages=messages
    )
    return response.content[0].text

If your system prompt is 2,000 tokens and you’re making 500 calls per hour on Sonnet, uncached cost is 500 x 2,000 x $0.000003 = $3.00/hour just for the system prompt. With caching, after the first call in each 5-minute window, that drops to $0.30/hour. On a 24-hour day that’s $72 saved versus $7.20.

Context Compression: Stopping History Bloat

Multi-turn agents are the worst offenders for token waste. A conversation that started at 500 tokens can hit 15,000 tokens after 20 turns, even when only the last 3 turns are relevant to the current task.

The compressor runs a summarization pass using Haiku when history exceeds a threshold:

COMPRESS_THRESHOLD = 6  # turns before compression kicks in

def compress_history(history: list) -> list:
    if len(history) <= COMPRESS_THRESHOLD:
        return history
    
    # Summarize everything except the last 3 turns
    to_summarize = history[:-3]
    recent = history[-3:]
    
    summary_prompt = "Summarize this conversation history in 3-5 bullet points. Preserve key decisions, facts established, and user preferences. Be concise."
    
    summary_response = client.messages.create(
        model="claude-3-haiku-20240307",
        max_tokens=256,
        system=summary_prompt,
        messages=[{
            "role": "user",
            "content": json.dumps(to_summarize)
        }]
    )
    
    summary_text = summary_response.content[0].text
    
    compressed_history = [
        {"role": "user", "content": f"[Previous conversation summary]\n{summary_text}"},
        {"role": "assistant", "content": "Understood. I have the context from our previous conversation."}
    ] + recent
    
    return compressed_history

This keeps history semantically intact while slashing token count. In testing, conversations that would have hit 12,000 input tokens by turn 15 stayed under 3,500 tokens with compression active.

Putting It All Together: The Self-Modifying Loop

Here is the full orchestration loop:

def run_agent(task_text: str, history: list = None) -> str:
    history = history or []
    
    # Layer 1: Classify
    classification = classify_task(task_text)
    
    # Layer 2: Route
    routing = route_task(task_text, classification)
    
    # Layer 3: Compress context
    compressed_history = compress_history(history)
    
    # Trim to max allowed turns for this task type
    max_turns = routing["max_history_turns"]
    trimmed_history = compressed_history[-max_turns * 2:] if max_turns > 0 else []
    
    # Add current task
    messages = trimmed_history + [{"role": "user", "content": task_text}]
    
    # Execute with caching
    result = make_cached_request(
        system_prompt=routing["system_prompt"],
        messages=messages,
        model=routing["model"]
    )
    
    # Update history
    history.append({"role": "user", "content": task_text})
    history.append({"role": "assistant", "content": result})
    
    return result, history

Each call now automatically selects the right model, the right system prompt size, and the right history length. The agent modifies its own context before every execution based on what the task actually needs. That’s the self-modifying part.

If you’re still at the stage of building your first Claude-powered agent, our guide on building your first AI agent with the Claude API covers the foundation you’ll need before layering this on top.

Real Numbers: What the Savings Actually Look Like

Here are the before and after numbers from my production system over a 30-day period:

Output quality on the tasks I cared about (complexity 4-5, routed to Sonnet) was unchanged. Haiku tasks were fine because by definition those were tasks where nuance didn’t matter.

The latency improvement was a bonus. Haiku is significantly faster than Sonnet, so routing simpler tasks there also sped up the overall system throughput.

Tuning the System for Your Workload

A few notes on customizing this for your specific use case:

Adjust complexity thresholds based on your error tolerance. If misrouting a task to Haiku that needed Sonnet causes downstream problems, raise the threshold. Route complexity 3 to Sonnet instead of Haiku. You’ll give back some savings but improve reliability.

Vary max_tokens by task type. Classification tasks rarely need more than 50 output tokens. Capping max_tokens tightly on those tasks prevents accidental verbose responses that inflate output costs. Add a max_tokens_map to the router.

Monitor cache miss rate. If your cache hit rate is below 50%, your system prompt is changing too often before calls. Audit what’s dynamic in your prompt and move it to the user message instead of the system message.

Use the Stop Claude Being Lazy guide for Haiku tasks. Haiku can sometimes produce underdeveloped responses on tasks it finds trivial. The prompting techniques in that guide apply to Haiku too and are worth applying to your standard and minimal prompt variants.

If the core issue is that usage-based pricing has already become painful at scale, it’s also worth evaluating whether some of your workloads could shift to local inference entirely. We covered that option in detail in Usage-Based Pricing Killing Your Vibe? Run Local AI.

Extending the System: What to Build Next

This architecture is a foundation. Here are the high-value extensions worth building:

Feedback loop for classifier accuracy. Log every classification decision and track downstream quality signals (user corrections, retry rates, downstream error rates). Feed this back into the classifier as few-shot examples over time. The classifier gets smarter, routing gets more accurate.

Dynamic cache warming. For workloads where you know certain system prompts will be used heavily, pre-warm the cache at the start of every 5-minute window with a dummy call. This ensures the cache is hot when real traffic hits.

Per-user context budgets. In multi-tenant systems, give each user a token budget per session. When they approach the limit, trigger compression automatically. This prevents power users from running up costs that your pricing model doesn’t account for.

Cost attribution by task type. Instrument the system to log actual spend per task type. Within a month you’ll have a clear picture of which task categories drive the most cost, which lets you prioritize where to optimize next.

For broader architectural thinking on how to structure prompt engineering at scale, the Prompt Engineering Guide for Claude and GPT-4o in 2026 is a solid companion read to this article.

Conclusion: Build the System That Builds the Prompts

The mistake most developers make is trying to optimize prompts manually, one call at a time. That approach doesn’t scale. Costs grow with usage and there’s no ceiling on how bad it gets.

The better approach is to build a layer of intelligence that sits in front of every Claude call and makes the cost/quality tradeoff automatically. That’s what this system does. It classifies, routes, caches, and compresses without you having to think about it per-call.

The implementation I’ve shared here is production-ready and took about two days to build and tune. If your Claude API costs are already above $100/month, you’ll recover that engineering time within the first billing cycle.

Start with model routing. That alone will cut your bill in the first week. Then layer in prompt caching and context compression. Track your numbers at each step.

Ready to go deeper? If you haven’t set up your Claude project structure yet, start with Inside the .claude/ Folder: What Every AI Developer Needs to Know before wiring up this system. Getting your config and tooling right first will save you debugging time later.

Affiliate disclosure: Some links in this article are affiliate links. I earn a commission when you sign up using my link, at no extra cost to you.