Anthropic Prompt Caching and Batch API

Overview

Anthropic's Prompt Caching and Batch API represent two significant infrastructure improvements that enable cost-effective, high-throughput agent deployments. Released in 2026, these features address the two biggest constraints facing production agent systems: the cost and latency of repeated context, and the ability to process large batches of agent tasks asynchronously.

Prompt Caching allows Anthropic models to cache prefix content that appears repeatedly across requests. For agent systems that use long system prompts, tool definitions, or shared context across many calls, caching can reduce both latency and cost by up to 90%. The cache operates at the token level, with automatic detection of common prefixes across requests.

The Batch API enables submitting large volumes of requests for asynchronous processing with a 50% cost discount. Ideal for agent pipelines that don't require real-time responses — report generation, data analysis, content creation, and batch processing — the Batch API can process thousands of requests per hour at half the cost of real-time API calls.

Together, these features make it economically viable to deploy high-volume agent systems that were previously cost-prohibitive, enabling new categories of agent applications at enterprise scale.

Features

Prompt Caching

• Automatic prefix caching: tokens appearing at the start of requests are cached automatically
• Multi-level caching: system prompts, tool definitions, and conversation history cached independently
• 90% cost reduction: cached tokens cost only a fraction of uncached tokens
• Latency improvement: cached requests return significantly faster
• No code changes: caching is automatic — no special API parameters required
• Cross-session sharing: cached content shared across all requests from the same API key

Batch API

• 50% cost discount: batch requests cost half of real-time API pricing
• High throughput: process thousands of requests per batch submission
• Asynchronous processing: submit, poll for completion, download results
• 24-hour SLA: all batch requests completed within 24 hours
• JSON Lines format: simple, standard input/output format
• Automatic retries: failed requests are automatically retried

Installation

# Install Anthropic SDK (supports both caching and batch API)
pip install anthropic

# Or via npm
npm install @anthropic-ai/sdk

Quick Start — Prompt Caching

from anthropic import Anthropic

client = Anthropic()

# Long system prompt — will be cached on first use
SYSTEM_PROMPT = """
You are a specialized AI research assistant. You have access to the following tools:
[... 50,000 tokens of tool definitions, context, and instructions ...]
"""

# First call — cache miss, full cost
response = client.messages.create(
    model="claude-opus-4-7",
    max_tokens=1024,
    system=SYSTEM_PROMPT,
    messages=[{"role": "user", "content": "What is LangGraph?"}],
)

# Second call — cache hit, ~90% cost reduction for system prompt
response = client.messages.create(
    model="claude-opus-4-7",
    max_tokens=1024,
    system=SYSTEM_PROMPT,
    messages=[{"role": "user", "content": "What is CrewAI?"}],
)
# usage.cache_creation_input_tokens: 50000 (first call)
# usage.cache_read_input_tokens: 50000 (second call)

import Anthropic from "@anthropic-ai/sdk";

const client = new Anthropic();

const systemPrompt = `[... long shared context ...]`;

// First call — creates cache
const response1 = await client.messages.create({
  model: "claude-opus-4-7",
  max_tokens: 1024,
  system: systemPrompt,
  messages: [{ role: "user", content: "First question" }],
});
// response1.usage.cacheCreationInputTokens = 50000

// Second call — reads from cache
const response2 = await client.messages.create({
  model: "claude-opus-4-7",
  max_tokens: 1024,
  system: systemPrompt,
  messages: [{ role: "user", content: "Second question" }],
});
// response2.usage.cacheReadInputTokens = 50000

Quick Start — Batch API

import json
from anthropic import Anthropic

client = Anthropic()

# Prepare batch input (JSON Lines format)
requests = [
    {
        "custom_id": f"request-{i}",
        "method": "POST",
        "url": "/v1/messages",
        "body": {
            "model": "claude-sonnet-4-6",
            "max_tokens": 500,
            "messages": [
                {"role": "user", "content": f"Summarize document {i}"}
            ],
        },
    }
    for i in range(1000)
]

# Write to JSON Lines file
with open("batch_input.jsonl", "w") as f:
    for req in requests:
        f.write(json.dumps(req) + "\n")

# Upload and create batch
with open("batch_input.jsonl", "rb") as f:
    file = client.files.create(file=f)

batch = client.messages.batches.create(input_file_id=file.id)

# Poll for completion
while batch.processing_status == "in_progress":
    import time
    time.sleep(60)
    batch = client.messages.batches.retrieve(batch.id)

# Download results
if batch.processing_status == "completed":
    results_file = client.files.content(batch.output_file_id)
    # Parse results from JSON Lines

Agent Use Cases

Caching for Agent Systems

Scenario	Savings
Multi-agent with shared system prompt	90% on system prompt tokens
Tool-heavy agents (50+ tools)	90% on tool definition tokens
RAG agents with shared knowledge base	90% on retrieved context
Long-running conversations	90% on conversation history prefix

Batch API for Agent Pipelines

Scenario	Description
Document processing	Batch-summarize 10,000 documents overnight
Content generation	Generate 500 blog posts with 50% cost savings
Data extraction	Extract structured data from 1000+ web pages
Code review	Batch-review 500 pull requests
Translation	Translate documents at scale

Pros

• ✅ Prompt Caching: up to 90% cost reduction on repeated context
• ✅ Prompt Caching: automatic — no code changes needed
• ✅ Batch API: 50% cost discount for non-real-time processing
• ✅ Batch API: high throughput for large-scale operations
• ✅ Both features work with all Claude models
• ✅ Compatible with all agent frameworks (LangGraph, CrewAI, AutoGen)

Cons

• ❌ Cache has a TTL (time-to-live) — cached content expires
• ❌ Cache key must match exactly — even whitespace changes invalidate cache
• ❌ Batch API has 24-hour SLA — not suitable for real-time applications
• ❌ Batch results are downloadable files — no streaming support

When to Use

• Prompt Caching: Any agent system with repeated context — long system prompts, tool definitions, RAG contexts, or conversation histories
• Batch API: Any agent pipeline that processes large volumes of requests asynchronously — document processing, content generation, data extraction, and batch analysis

Resources

• Anthropic Prompt Caching Documentation
• Anthropic Batch API Documentation
• Anthropic SDK Reference