agents-libagents-lib
AgentsMCP ServersWorkflowsTutorialsComparisonsTemplatesNewsAbout
🔌

LangSmith MCP

Observability•⭐ 850

Monitor, debug, and evaluate LangChain applications with LangSmith observability platform.

GitHubWebsite →
Claude DesktopCursor

Overview

Monitor, debug, and evaluate LangChain applications with LangSmith observability platform.

Setup

Run with npx:

npx -y @modelcontextprotocol/server-langsmith

Configuration

LANGCHAIN_API_KEY and LANGCHAIN_PROJECT environment variables

Documentation

LangSmith MCP

Overview

LangSmith MCP is a Model Context Protocol server that provides integration with LangSmith, LangChain's production observability platform. It enables AI agents to monitor, debug, and evaluate LangChain applications directly from within their workflow.

LangSmith provides comprehensive tracing, evaluation, and monitoring for LangChain applications, making it easier to understand how your agents are performing and identify issues.

Features

  • Trace Inspection — View detailed traces of LangChain runs
  • Dataset Management — Manage evaluation datasets
  • Evaluation Results — Access evaluation metrics and results
  • Feedback Collection — View and manage user feedback
  • Annotation Queue — Review and annotate traces
  • Project Monitoring — Monitor multiple projects and environments

Installation

npx -y @modelcontextprotocol/server-langsmith

Configuration

{
  "mcpServers": {
    "langsmith": {
      "command": "npx",
      "args": ["-y", "@modelcontextprotocol/server-langsmith"],
      "env": {
        "LANGCHAIN_API_KEY": "your-api-key",
        "LANGCHAIN_PROJECT": "your-project-name"
      }
    }
  }
}

Available Tools

ToolDescription
list_projectsList all LangSmith projects
get_project_statsGet statistics for a project
list_runsList runs in a project with filters
get_run_detailsGet detailed information about a run
create_datasetCreate a new evaluation dataset
create_exampleAdd an example to a dataset
list_feedbackList feedback on runs

Usage Examples

List Projects

projects = langsmith.list_projects()
for project in projects:
    print(f"{project.name}: {project.run_count} runs")

Get Run Details

run = langsmith.get_run_details(run_id="your-run-id")
print(run.inputs, run.outputs, run.error)

Create Evaluation Dataset

dataset = langsmith.create_dataset(
    name="my-evaluation-dataset",
    description="Dataset for evaluating agent performance"
)

Claude Desktop Setup

Add to claude_desktop_config.json:

{
  "mcpServers": {
    "langsmith": {
      "command": "npx",
      "args": ["-y", "@modelcontextprotocol/server-langsmith"],
      "env": {
        "LANGCHAIN_API_KEY": "ls__your-api-key"
      }
    }
  }
}

Pros

  • ✅ Deep integration with LangChain ecosystem
  • ✅ Comprehensive trace visualization
  • ✅ Built-in evaluation framework
  • ✅ Production monitoring capabilities
  • ✅ Team collaboration features
  • ✅ Active development by LangChain team

Cons

  • ❌ Requires LangSmith account and API key
  • ❌ Some features require paid tier
  • ❌ Primarily focused on LangChain applications
  • ❌ Cloud-dependent for full functionality

When to Use

LangSmith MCP is ideal for:

  • Monitoring production LangChain applications
  • Debugging complex agent workflows
  • Evaluating agent performance on datasets
  • Collecting and reviewing user feedback
  • Team collaboration on agent development

Consider alternatives when:

  • You need framework-agnostic monitoring (use AgentOps)
  • You want self-hosted observability (use Helicone)
  • You only need basic cost tracking (use built-in tools)

Resources