Anthropic Computer Use MCP Server

Overview

The Anthropic Computer Use MCP Server exposes Claude's Computer Use capabilities as MCP tools, enabling AI agents to control computers through visual perception and direct GUI manipulation. With Claude 4.6 Opus's enhanced Computer Use 2.0, this server provides pixel-perfect desktop automation, multi-app orchestration, and long-horizon task execution.

Features

• Computer control — Keyboard, mouse, and screenshot commands
• Multi-app support — Control across multiple desktop applications
• Visual grounding — Pixel-perfect action targeting
• Accessibility tree — Structured UI element detection
• Long-horizon tasks — Up to 128K reasoning tokens
• Safety guardrails — Policy-based action control
• MCP 2.0 support — Latest MCP protocol features
• Cross-platform — Windows, macOS, and Linux

Installation

Quick Setup

# Install via pip
pip install anthropic-computer-use-mcp

# Or install from source
git clone https://github.com/anthropics/computer-use-mcp.git
cd computer-use-mcp
pip install -e .

Claude Desktop Configuration

Add to your claude_desktop_config.json:

{
  "mcpServers": {
    "computer-use": {
      "command": "anthropic-computer-use-mcp",
      "args": ["serve"],
      "env": {
        "ANTHROPIC_API_KEY": "your-api-key-here",
        "COMPUTER_USE_DISPLAY": "0"
      }
    }
  }
}

Docker Setup

docker run -d \
  -e ANTHROPIC_API_KEY=your-api-key \
  -e DISPLAY=:0 \
  -v /tmp/.X11-unix:/tmp/.X11-unix \
  anthropic/computer-use-mcp:latest

Setup

1. Get an Anthropic API Key

   • Sign up at console.anthropic.com
   • Create an API key in your account settings

2. Set Environment Variables

   export ANTHROPIC_API_KEY="your-api-key-here"
   export COMPUTER_USE_DISPLAY="0"  # Display number
   

3. Start the Server

   anthropic-computer-use-mcp serve
   

Available Tools

Tool	Description
screenshot	Capture current screen state
key	Press a single key
type	Type text
mouse_move	Move cursor to coordinates
left_click	Click left mouse button
right_click	Click right mouse button
double_click	Double-click at current position
scroll	Scroll up/down
switch_app	Switch to another application
get_window_info	Get information about active window

Usage Examples

Take Screenshot

{
  "name": "screenshot",
  "arguments": {
    "resolution": "1920x1080"
  }
}

Type Text

{
  "name": "type",
  "arguments": {
    "text": "Hello World",
    "slowly": true
  }
}

Click at Coordinates

{
  "name": "mouse_move",
  "arguments": {
    "x": 800,
    "y": 600
  }
}

{
  "name": "left_click",
  "arguments": {}
}

Press Key Combination

{
  "name": "key",
  "arguments": {
    "keys": ["Command", "Shift", "4"]
  }
}

Switch Application

{
  "name": "switch_app",
  "arguments": {
    "app_name": "VSCode"
  }
}

Advanced Features

Multi-App Orchestration

{
  "name": "switch_app",
  "arguments": {
    "app_name": "Chrome"
  }
}

{
  "name": "type",
  "arguments": {
    "text": "https://github.com"
  }
}

{
  "name": "key",
  "arguments": {
    "keys": ["Enter"]
  }
}

Safety Guardrails

{
  "name": "set_guardrails",
  "arguments": {
    "blocked_actions": ["delete", "rm -rf /"],
    "max_retries": 3,
    "require_approval": ["execute", "delete"]
  }
}

Long-Horizon Task Execution

{
  "name": "execute_task",
  "arguments": {
    "task": "Open Chrome, search for AI agents, read top 3 results, summarize findings",
    "max_steps": 50,
    "timeout": 600
  }
}

Integration with AI Agents

Claude Code Integration

# In Claude Code, use Computer Use tools
> Take a screenshot
> Click the login button
> Type my email in the input
> Press Enter to submit

Multi-App Workflow Example

# Example workflow using Computer Use MCP
async def automate_workflow():
    # Open browser
    await switch_app("Chrome")
    
    # Navigate to site
    await type("https://example.com")
    await key("Enter")
    
    # Fill form
    await mouse_move(100, 200)
    await left_click()
    await type("testuser@example.com")
    
    # Submit
    await key("Tab")
    await type("password123")
    await key("Enter")

Pros

• ✅ Official Anthropic Computer Use integration
• ✅ Pixel-perfect desktop automation
• ✅ Multi-app orchestration
• ✅ 128K reasoning tokens for long tasks
• ✅ Safety guardrails and approval workflows
• ✅ MCP 2.0 support
• ✅ Cross-platform support

Cons

• ❌ Requires Claude 4.6 Opus (expensive)
• ❌ Desktop automation is inherently risky
• ❌ No native mobile device support
• ❌ Latency due to screenshot-action loop
• ❌ Requires ANTHROPIC_API_KEY

When to Use

• Desktop automation workflows
• Multi-app task coordination
• Complex UI interactions
• Testing and QA automation
• Digital assistant applications
• Accessibility-focused automation

Resources

• Anthropic Computer Use Docs
• MCP Documentation
• GitHub Repository
• Claude 4.6 Opus Announcement