Building RAG with Claude Opus 4.7's 2M Context

Overview

Building RAG with Claude Opus 4.7's 2M Context walks through a pattern shift in production RAG architectures triggered by Claude Opus 4.7's 2 million token context window, which shipped on September 5, 2026. Prior to Opus 4.7, RAG pipelines were embedding-heavy: chunk documents, embed, store in a vector database, retrieve top-k chunks, feed to the model. With a 2M context window, several teams have discovered that their entire corpus fits inside a single request — eliminating the chunk-and-retrieve pipeline entirely for a meaningful subset of workloads.

This tutorial covers both worlds: full-context workflows for corpora that fit, and semantic tool-calling for corpora that don't. The shift is not "RAG is dead" — it's "RAG-as-an-architecture is being reconsidered for workloads where it doesn't need to exist."

Prerequisites

• Anthropic API key with Opus 4.7 access
• A corpus of documents (e.g., 100k-2M tokens total)
• Python 3.10+
• An agent framework (LangGraph, OpenAI Agents SDK, PydanticAI, or raw Anthropic SDK)

Architecture Decision

Before building, decide which pattern fits your corpus:

Corpus Size	Pattern	Tools
< 500K tokens	Full-context workflow	Anthropic SDK only
500K - 2M tokens	Full-context with prompt caching	Anthropic SDK + caching
2M - 20M tokens	Semantic tool-calling	Anthropic SDK + structured tool calls
> 20M tokens	Traditional RAG (with retrieval)	Vector DB + Anthropic SDK

Approach 1: Full-Context Workflow

For corpora under ~2M tokens, skip RAG entirely and send the whole corpus in one request.

from anthropic import Anthropic

client = Anthropic()

# Load the entire corpus
corpus = open("docs/corpus.md").read()  # up to 2M tokens

response = client.messages.create(
    model="claude-opus-4-7",
    max_tokens=1024,
    messages=[{
        "role": "user",
        "content": f"""
{corpus}

Now answer: What are the three key takeaways from this document?
""",
    }],
)

print(response.content[0].text)

Prompt Caching

Full-context requests are expensive. Use prompt caching to reduce cost when the corpus is stable:

response = client.messages.create(
    model="claude-opus-4-7",
    max_tokens=1024,
    cache_control={"type": "ephemeral"},
    messages=[{
        "role": "user",
        "content": [
            {
                "type": "text",
                "text": corpus,
                "cache_control": {"type": "ephemeral"},
            },
            {
                "type": "text",
                "text": "Answer: What are the three key takeaways?",
            },
        ],
    }],
)
# First call: $3.75 (2M tokens input)
# Subsequent calls with same corpus prefix: $0.375 (cached)

Approach 2: Semantic Tool-Calling

For corpora between 2M and 20M tokens, expose the corpus as structured tools instead of vector-search chunks.

from anthropic import Anthropic
from pydantic import BaseModel, Field

client = Anthropic()

class SearchTool(BaseModel):
    query: str = Field(..., description="Search query")
    filter: dict | None = Field(default=None, description="Structured filter")

tools = [
    {
        "name": "search_docs",
        "description": "Search the documentation corpus with natural language queries",
        "input_schema": {
            "type": "object",
            "properties": {
                "query": {"type": "string"},
                "filter": {"type": "object"},
            },
            "required": ["query"],
        },
    },
]

# Tool implementation
def search_docs(query: str, filter: dict | None = None):
    # Structured query against indexed corpus, not vector search
    return search_corpus_structured(query, filter)

response = client.messages.create(
    model="claude-opus-4-7",
    max_tokens=1024,
    tools=tools,
    messages=[{"role": "user", "content": "What does the pricing page say about enterprise plans?"}],
)

The semantic tool-calling pattern treats the corpus as a queryable data source rather than something to chunk and embed. Retrieval is done through structured queries, not vector similarity.

Approach 3: Traditional RAG (with Retrieval)

For corpora over 20M tokens, traditional RAG is still necessary. The 2M context window helps here too — you can retrieve larger chunks (50K-100K tokens per retrieval) and still fit them in context.

from anthropic import Anthropic
from langchain_core.vectorstores import Qdrant

client = Anthropic()
vectorstore = Qdrant(...)

# Retrieve larger chunks
def retrieve(query: str, k: int = 5):
    # 5 chunks of ~30K tokens each = 150K tokens of context
    return vectorstore.similarity_search(query, k=k)

def answer(query: str):
    chunks = retrieve(query, k=5)
    context = "\n\n".join(chunk.page_content for chunk in chunks)
    
    response = client.messages.create(
        model="claude-opus-4-7",
        max_tokens=1024,
        messages=[{
            "role": "user",
            "content": f"Context:\n{context}\n\nQuery: {query}",
        }],
    )
    return response.content[0].text

Multi-Step Agent Workflow

For larger questions, agents explore the corpus iteratively:

from anthropic import Anthropic

client = Anthropic()

def research_agent(question: str, corpus: dict):
    # Agent iteratively explores the corpus
    message_history = [{"role": "user", "content": question}]
    
    for _ in range(10):  # up to 10 exploration steps
        response = client.messages.create(
            model="claude-opus-4-7",
            max_tokens=2048,
            messages=message_history,
            tools=[
                {"name": "search", "description": "Search the corpus", "input_schema": {...}},
                {"name": "read_section", "description": "Read a specific section", "input_schema": {...}},
            ],
        )
        
        # Add response to history and check if tool was called
        message_history.append(response.content[-1])
        if response.stop_reason == "end_turn":
            return response.content[-1].text
    
    return None

Evaluation

Build a small eval set (~50 questions) and test all three approaches:

eval_questions = [
    {"q": "What is the pricing for enterprise plans?", "expected": "starts at $10k"},
    {"q": "How does the auth system handle OAuth?", "expected": "..."},
    # ...
]

for approach in ["full_context", "semantic_tool", "traditional_rag"]:
    for q in eval_questions:
        answer = approaches[approach](q["q"])
        print(f"{approach}: {evaluate(answer, q['expected'])}")

Cost Considerations

Pattern	Cost per query (100K corpus)	Cost per query (2M corpus)
Full-context	$0.375 (cached)	$3.75 (cached)
Semantic tool	$0.15 (with embedding)	$0.40
Traditional RAG	$0.05	$0.08

Full-context is more expensive per query but produces dramatically better answers on long-horizon questions where traditional RAG misses context.

Pros

• ✅ Dramatically simpler architecture for corpora that fit
• ✅ No embedding pipelines to maintain
• ✅ No chunking decisions to make
• ✅ Better answers on long-horizon questions
• ✅ Fewer failure modes (no chunk boundary issues)

Cons

• ❌ Higher per-query cost on large corpora
• ❌ Doesn't work for corpora over 2M tokens
• ❌ Requires careful prompt caching to keep costs manageable
• ❌ Some frameworks' tool-calling support is still catching up

When to Use

• Full-context: corpora under 500K tokens where simplicity beats cost
• Semantic tool-calling: corpora between 2M and 20M tokens
• Traditional RAG: corpora over 20M tokens where embedding retrieval is necessary

Resources

• Anthropic 2M Context Docs
• Prompt Caching Guide
• OpenAI Agents SDK