Agent Observability with OpenTelemetry 2026

Overview

Agent observability became a first-class discipline in 2026, driven by the recognition that tracing agent executions with traditional application-monitoring tools is insufficient. This workflow shows how to instrument any agent stack (LangGraph, CrewAI, OpenAI Agents SDK, PydanticAI, AgentGuard) with OpenTelemetry so agent executions produce standard OTel traces that plug into any observability platform (Jaeger, Grafana Tempo, Datadog, New Relic).

Difficulty

Hard

Tools Required

• OpenTelemetry SDK — Python 1.25+, TypeScript 1.25+, or Go 1.25+
• OpenTelemetry Collector — for trace forwarding
• Observability backend — Jaeger, Grafana Tempo, Datadog, New Relic, or AgentOps
• Agent framework — LangGraph, CrewAI, OpenAI Agents SDK, PydanticAI, or AgentGuard

Workflow Steps

Step 1: Install the OTel SDK

# Python
pip install opentelemetry-api opentelemetry-sdk opentelemetry-exporter-otlp

# TypeScript
npm install @opentelemetry/api @opentelemetry/sdk-node @opentelemetry/exporter-trace-otlp-http

# Go
go get go.opentelemetry.io/otel

Step 2: Initialize the Tracer

from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
from opentelemetry.sdk.trace.export import BatchSpanExporter

trace.set_tracer_provider(TracerProvider())
exporter = OTLPSpanExporter(endpoint="http://localhost:4318/v1/traces")
trace.get_tracer_provider().add_span_processor(BatchSpanExporter(exporter))

tracer = trace.get_tracer("my-agent")

Step 3: Instrument the Agent

Most agent frameworks have OTel integrations. For LangGraph:

from langchain_core.tracers.langsmith import LangSmithTracer
from opentelemetry.instrumentation.langchain import LangChainInstrumentor

LangChainInstrumentor().instrument()

For OpenAI Agents SDK:

from openai.agents import Agent, Runner
from opentelemetry.instrumentation.openai import OpenAIInstrumentor

OpenAIInstrumentor().instrument()

Step 4: Add Custom Agent Spans

For custom logic between agent steps:

with tracer.start_as_current_span("planner.step") as span:
    span.set_attribute("agent.id", "agent:planner-v1")
    span.set_attribute("planner.intent", "code_review")
    
    result = await planner.run(task)
    
    span.set_attribute("planner.output.tokens", len(result))
    span.set_attribute("planner.tool_calls", result.tool_calls_count)

Step 5: Tag Tool Calls

with tracer.start_as_current_span("tool.call") as span:
    span.set_attribute("tool.name", "read_file")
    span.set_attribute("tool.server", "github-mcp")
    span.set_attribute("agent.id", current_agent_id)
    
    result = await call_tool("read_file", args)
    
    span.set_attribute("tool.result.size", len(result))
    span.set_attribute("tool.result.hash", hash(result))

Step 6: Forward to Your Backend

Deploy the OTel Collector:

# collector-config.yaml
receivers:
  otlp:
    protocols:
      http:
        endpoint: 0.0.0.0:4318

processors:
  batch:
    timeout: 1s
    send_batch_size: 1000

exporters:
  otlp/jaeger:
    endpoint: "jaeger-collector:4317"
    tls:
      insecure: true
  otlp/datadog:
    endpoint: "https://trace-api.datadoghq.com"
    api_key: "${DD_API_KEY}"

service:
  pipelines:
    traces:
      receivers: [otlp]
      processors: [batch]
      exporters: [otlp/jaeger, otlp/datadog]

Step 7: Build Agent-Specific Dashboards

Key metrics for agent observability:

# Grafana dashboard variables
- agent_id (filter by agent)
- model (filter by model)
- tool_name (filter by tool)
- time_range

Visualize:

• Agent step latency — p50, p95, p99 by agent
• Tool call latency — per tool, per server
• Token throughput — input vs output tokens per agent
• Cost per run — dollars per execution
• Failure rate — by agent, by tool, by error type
• Cross-agent coordination — which agents call which other agents

Step 8: Alert on Anomalies

Set up alerts for:

alerts:
  - name: agent_high_latency
    condition: p95_step_latency > 5s for 5m
    notify: #agent-alerts
    
  - name: agent_error_spike
    condition: error_rate > 5% for 3m
    notify: #agent-alerts
    
  - name: agent_cost_spike
    condition: cost_per_run > $0.50 for 10m
    notify: #finance
    
  - name: agent_tool_failure
    condition: tool_failure_rate > 10% for 5m
    notify: #agent-alerts

Example Usage

A production agent deployment with full observability:

from openai.agents import Agent, Runner
from opentelemetry.instrumentation.openai import OpenAIInstrumentor

OpenAIInstrumentor().instrument()

agent = Agent(
    name="code-reviewer",
    model="claude-opus-4-7",
    instructions="You are a security-focused code reviewer.",
    tools=[read_file, run_tests, post_comment],
)

# Each Runner.run() produces a root trace
# Each tool call produces a child span
# Cross-agent coordination produces linked traces

await Runner.run(agent, input=pr.diff)

The resulting trace looks like:

[code-reviewer.run]
├── [tool.read_file] github-mcp @ src/auth.py
├── [tool.run_tests] local @ auth tests
│   ├── [subspan] pytest collection
│   └── [subspan] pytest execution
├── [tool.post_comment] github-mcp @ PR #42
└── [agent.planner.spawn] agent:summarizer-v1
    ├── [summarizer.run]
    └── [tool.summarize] claude-opus-4-7

Pros

• ✅ Standard observability — no custom telemetry format
• ✅ Works with every major observability backend
• ✅ Framework-agnostic — instrument any agent framework
• ✅ Distributed tracing across agent + tool boundaries
• ✅ Industry-standard metrics, logs, and traces (the OpenTelemetry signal model)

Cons

• ❌ Steeper setup than purpose-built agent observability tools (AgentOps, LangSmith)
• ❌ Custom span tags need careful schema design
• ❌ Backend cost grows with trace volume
• ❌ Some frameworks' OTel instrumentation is still catching up

When to Use

• Production agent deployments needing enterprise observability
• Teams already invested in OpenTelemetry
• Multi-framework agent stacks needing unified observability
• Environments requiring audit-grade trace retention

Resources

• OpenTelemetry docs
• OpenTelemetry Python SDK
• AgentOps docs
• LangSmith OTel integration