LangChain RAG Template

Overview

A production-ready RAG (Retrieval-Augmented Generation) pipeline template built with LangChain. This template provides a complete, configurable foundation for building question-answering systems over your own data.

Features

• Multi-source ingestion: Support for PDF, DOCX, TXT, Markdown, Web URLs
• Smart chunking: Multiple chunking strategies (character, recursive, semantic)
• Vector store options: Chroma, FAISS, Pinecone, Weaviate, Qdrant
• Embedding models: OpenAI, HuggingFace, Cohere, Google Vertex
• Retrieval strategies: Similarity search, MMR, self-query, parent document
• Chain types: Stuff, Refine, Map-Reduce, Map-Rerank
• Memory support: Conversation history with vector store memory
• Evaluation: Built-in evaluation framework

Installation

pip install langchain langchain-openai langchain-community langchain-chroma

Template Structure

rag-pipeline-template/
├── config/
│   ├── embeddings.yaml      # Embedding model configuration
│   ├── chunking.yaml        # Chunking strategy configuration
│   └── retrieval.yaml       # Retrieval configuration
├── ingestion/
│   ├── loaders.py           # Document loaders
│   ├── chunkers.py          # Chunking strategies
│   └── pipeline.py          # Ingestion pipeline
├── retrieval/
│   ├── vector_store.py      # Vector store setup
│   ├── retrievers.py        # Retrieval strategies
│   └── ranking.py           # Re-ranking (optional)
├── chains/
│   ├── qa_chain.py          # Main QA chain
│   ├── conversation.py      # Conversational QA
│   └── summarization.py     # Document summarization
├── evaluation/
│   ├── metrics.py           # Evaluation metrics
│   └── eval_pipeline.py     # Evaluation pipeline
├── api/
│   ├── fastapi_server.py    # FastAPI API server
│   └── routes.py            # API endpoints
├── tests/
│   ├── test_ingestion.py
│   ├── test_retrieval.py
│   └── test_chains.py
├── main.py                  # Entry point
└── requirements.txt

Quick Start

1. Configuration

# config/embeddings.yaml
model: openai
model_name: text-embedding-3-small
dimensions: 1536

# config/chunking.yaml
strategy: recursive_character
chunk_size: 1000
chunk_overlap: 200
selectors:
  - MarkdownHeaderTextSplitter
  - SentenceTransformersTokenTextSplitter

# config/retrieval.yaml
vector_store: chroma
search_type: similarity
search_kwargs:
  k: 4
score_threshold: 0.7

2. Ingest Documents

from ingestion.pipeline import ingest_documents
from config import load_config

config = load_config()
documents = ingest_documents(
    sources=["./data/*.pdf", "./data/*.md"],
    config=config["ingestion"]
)

print(f"Ingested {len(documents)} documents")

3. Create Vector Store

from retrieval.vector_store import create_vector_store
from config import load_config

config = load_config()
vectorstore = create_vector_store(
    documents=documents,
    config=config["retrieval"]
)

print(f"Created vector store with {vectorstore.count()} vectors")

4. Build QA Chain

from chains.qa_chain import create_qa_chain
from langchain_openai import ChatOpenAI

llm = ChatOpenAI(model="gpt-4o", temperature=0)
chain = create_qa_chain(
    llm=llm,
    retriever=vectorstore.as_retriever(),
    chain_type="stuff",
    return_source_documents=True
)

# Test the chain
result = chain.invoke("What is the return policy?")
print(result["answer"])
print("Sources:", result["source_documents"])

5. Conversational QA

from chains.conversation import create_conversational_chain
from langchain_community.chat_message_histories import ChatMessageHistory

chain = create_conversational_chain(
    llm=llm,
    retriever=vectorstore.as_retriever(),
    memory=ChatMessageHistory()
)

response = chain.invoke({
    "input": "What is the return policy?",
    "chat_history": []
})
print(response["answer"])

Advanced Features

Multi-Vector Retrieval

from retrieval.retrievers import create_multi_vector_retriever

retriever = create_multi_vector_retriever(
    vectorstore=vectorstore,
    store_summaries=True,
    summary_llm=ChatOpenAI(model="gpt-4o-mini")
)

Self-Querying Retriever

from retrieval.retrievers import create_self_query_retriever

retriever = create_self_query_retriever(
    vectorstore=vectorstore,
    metadata_field_info=[
        {"name": "source", "type": "string", "description": "Document source"},
        {"name": "page", "type": "integer", "description": "Page number"},
        {"name": "category", "type": "string", "description": "Content category"}
    ],
    document_content_description="Content from technical documentation"
)

# Query with metadata filter
results = retriever.invoke("What's on page 5 about authentication?")

Hybrid Search (BM25 + Vector)

from retrieval.retrievers import create_hybrid_retriever

retriever = create_hybrid_retriever(
    vectorstore=vectorstore,
    bm25_index=documents,
    alpha=0.7  # Weight for vector search
)

Re-ranking

from retrieval.ranking import add_reranker
from langchain_cohere import CohereRerank

reranker = CohereRerank(model="rerank-english-v3.0")
retriever = add_reranker(
    retriever=vectorstore.as_retriever(),
    reranker=reranker,
    top_n=5
)

API Server

from api.fastapi_server import app

# Run server
uvicorn.run(app, host="0.0.0.0", port=8000)

API Endpoints

# Ingest documents
curl -X POST http://localhost:8000/ingest \
  -F "file=@document.pdf"

# Ask a question
curl -X POST http://localhost:8000/ask \
  -H "Content-Type: application/json" \
  -d '{"query": "What is the return policy?"}'

# Conversational question
curl -X POST http://localhost:8000/chat \
  -H "Content-Type: application/json" \
  -d '{"message": "What is the return policy?", "conversation_id": "abc123"}'

# Health check
curl http://localhost:8000/health

Evaluation

from evaluation.eval_pipeline import evaluate_rag
from evaluation.metrics import calculate_metrics

results = evaluate_rag(
    chain=chain,
    test_dataset="data/evaluation/test_set.json",
    metrics=["faithfulness", "answer_relevancy", "context_precision"]
)

metrics = calculate_metrics(results)
print(f"Faithfulness: {metrics['faithfulness']:.2f}")
print(f"Answer Relevancy: {metrics['answer_relevancy']:.2f}")

Customization

Add Custom Document Loader

from langchain_community.document_loaders import BaseLoader

class MyCustomLoader(BaseLoader):
    def lazy_load(self):
        # Your custom loading logic
        yield Document(page_content="...", metadata={"source": "..."})

# Use in pipeline
loader = MyCustomLoader(config={"api_key": "..."})
documents = loader.load()

Add Custom Chunking Strategy

from langchain_text_splitters import RecursiveCharacterTextSplitter

class CustomChunker:
    def chunk(self, documents):
        splitter = RecursiveCharacterTextSplitter(
            chunk_size=500,
            chunk_overlap=100,
            separators=["\n\n", "\n", "。", "！", "？", ".", " "]
        )
        return splitter.split_documents(documents)

Add Custom Retrieval Prompt

from langchain_core.prompts import ChatPromptTemplate

custom_prompt = ChatPromptTemplate.from_messages([
    ("system", """You are an assistant for question-answering tasks. 
    Use the following pieces of retrieved context to answer the question. 
    If you don't know the answer, say that you don't know. 
    Use three sentences maximum and keep the answer concise.

    {context}"""),
    ("human", "{input}"),
])

Troubleshooting

Low Retrieval Quality

1. Check embedding model matches between indexing and retrieval
2. Adjust chunk size (try 500-2000)
3. Increase number of retrieved documents (k)
4. Add re-ranking for better precision
5. Use hybrid search instead of pure vector search

Slow Performance

1. Use smaller embedding model (text-embedding-3-small)
2. Enable caching for embeddings
3. Use FAISS instead of Chroma for faster local search
4. Implement async processing for ingestion
5. Use batch operations for vector store

Memory Issues

1. Process documents in batches
2. Use smaller chunk sizes
3. Clear vector store between runs
4. Use streaming for long documents
5. Implement document preprocessing to reduce size

Resources

• LangChain RAG Guide: https://python.langchain.com/docs/tutorials/rag/
• LangChain Documentation: https://python.langchain.com/docs/
• Vector Store Comparison: https://python.langchain.com/docs/integrations/vectorstores/
• Evaluation Framework: https://docs.smith.langchain.com/evaluation/

---

Last updated: May 2026