LangGraph Introduces Rubrics: Agents That Evaluate and Correct Their Own Work

Overview

LangGraph has introduced "rubrics"—a new feature that enables agents to evaluate and correct their own work. This capability brings self-reflection and quality assurance directly into the agent loop, reducing the need for external validation and enabling autonomous improvement.

What Are Rubrics?

Rubrics are structured evaluation criteria that agents use to:

Assess Output Quality: Check if results meet defined standards
Identify Issues: Detect errors, inconsistencies, or missing information
Propose Corrections: Generate improvements based on evaluation findings
Iterate Until Satisfied: Continue refining until quality thresholds are met

How Rubrics Work

1. Define Evaluation Criteria

Rubrics specify what "good" looks like for a given task:

from langgraph.graph import StateGraph

rubric = {
    "accuracy": {
        "description": "Output must be factually correct",
        "weight": 0.4,
        "check": verify_facts,
    },
    "completeness": {
        "description": "All required information must be present",
        "weight": 0.3,
        "check": check_completeness,
    },
    "clarity": {
        "description": "Output must be clear and well-structured",
        "weight": 0.3,
        "check": assess_clarity,
    },
}

2. Automatic Evaluation Loop

The agent evaluates its own work:

def evaluation_node(state: State):
    score = evaluate_rubric(state.output, rubric)
    if score < THRESHOLD:
        return "needs_improvement"
    return "approved"

builder.add_node("evaluate", evaluation_node)
builder.add_node("improve", improvement_node)
builder.add_edge("evaluate", "improve", lambda s: s == "needs_improvement")
builder.add_edge("improve", "evaluate")

3. Self-Correction

When evaluation fails, the agent:

Identifies specific rubric criteria that failed
Generates targeted improvements
Re-evaluates until all criteria pass

Benefits

Benefit	Description
Reduced Human Oversight	Agents self-validate, reducing manual review burden
Consistent Quality	Standardized criteria ensure consistent output quality
Continuous Improvement	Agents learn from evaluation feedback
Transparent Quality Metrics	Rubric scores provide objective quality measures

Use Cases

Content Generation: Ensure articles meet style and accuracy standards
Code Generation: Validate code against style guides and tests
Research Summaries: Check completeness and factual accuracy
Customer Responses: Verify tone, accuracy, and compliance

Integration with Existing Patterns

Rubrics integrate with:

Evaluator-Optimizer Pattern: Natural fit for the evaluation-feedback loop
Human-in-the-Loop: Low-quality outputs can trigger human review
Multi-Agent Systems: Different agents can specialize in generation vs. evaluation
Persistence: Evaluation results can be stored for audit and improvement

Best Practices

Start Simple: Begin with 3-5 key criteria, expand as needed
Define Clear Metrics: Make criteria objective and measurable
Weight Appropriately: Prioritize criteria based on task importance
Test Rubric Quality: Validate that rubrics correlate with human judgment
Iterate on Rubrics: Refine criteria based on failure patterns