SmallClaw

Overview

SmallClaw is a local-first AI agent framework built by XposeMarket for running agents on small language models — typically 1B–8B parameters on consumer hardware.

The premise is contrarian: while the industry chases frontier models, most agent tasks work acceptably on small models if the framework is designed for them. SmallClaw's bet is that local, cheap, private agent execution beats cloud API calls for a large class of use cases — personal assistants, home automation, offline tooling, and privacy-sensitive workloads.

The project gained 5 points on Show HN and is one of the few agent frameworks explicitly optimized for sub-10B models. It pairs with oMLX, llama.cpp, or LM Studio as inference backends.

Features

• Small-model-first design: prompt patterns, tool schemas, and system prompts all tuned for 1B–8B models
• Local inference only: no cloud API calls by default
• Multi-backend: llama.cpp, oMLX, LM Studio, vLLM, Ollama
• Tool-calling on small models: schemas simplified to reduce model confusion
• Prompt compression: aggressively compact system prompts and tool descriptions
• Offline operation: full agent functionality without internet access
• Privacy-first: all processing happens locally

Installation

# pip
pip install smallclaw

# With llama.cpp backend
pip install "smallclaw[llama-cpp]"

# With oMLX backend (Apple Silicon)
pip install "smallclaw[omlx]"

# With Ollama backend
pip install "smallclaw[ollama]"

Quick Start

from smallclaw import Agent, Tool

# Local model, no API key
agent = Agent(
    model="qwen2.5-7b-instruct",
    backend="llama-cpp",
    system_prompt="You are a helpful assistant. Be concise."
)

@agent.tool(description="Get current weather for a city")
def get_weather(city: str) -> str:
    return weather_api(city)

response = agent.run("What's the weather in Tokyo?")
print(response)

Multi-Model Routing

SmallClaw routes tasks to different models based on complexity:

from smallclaw import Agent, Router

router = Router(
    rules=[
        ("simple_qa", "qwen2.5-3b", "For simple questions under 50 tokens"),
        ("tool_use", "qwen2.5-7b", "For tasks requiring tool calls"),
        ("reasoning", "deepseek-r1-8b", "For complex reasoning tasks")
    ]
)

agent = Agent(router=router, backend="ollama")

Tool Definitions for Small Models

SmallClaw includes schema helpers designed for smaller models:

from smallclaw import SimpleTool

tool = SimpleTool(
    name="search_files",
    description="Find files by name pattern. Use when user asks about file location.",
    params={
        "pattern": "glob pattern, e.g. '*.md'",
        "directory": "starting directory, default '.'"
    }
)

The SimpleTool format is intentionally flatter than JSON Schema — small models handle simple parameter descriptions more reliably than nested schemas.

Pros

• ✅ Truly local: no cloud API required, no tokens leaked
• ✅ Small-model-tuned: prompts and tool schemas designed for sub-10B models
• ✅ Multi-backend: llama.cpp, oMLX, LM Studio, vLLM, Ollama
• ✅ Multi-model routing: dispatch to different local models based on task
• ✅ Privacy-first: everything stays on your machine
• ✅ Free to run: no API costs after initial model download

Cons

• ❌ Small-model quality ceiling: small models will fail on complex reasoning regardless of framework
• ❌ Local compute requirement: you need a machine with 8GB+ RAM and ideally an M-series Mac or RTX GPU
• ❌ Tiny community: ~50 GitHub stars, single maintainer
• ❌ No cloud deployment story: not designed for server-side multi-tenant use
• ❌ Tool schemas are simpler than JSON Schema: not fully standards-compliant with MCP

When to Use

Use SmallClaw when:

• You need agents that work offline (air-gapped environments, travel, privacy-sensitive work)
• You want to avoid API costs for high-volume, low-complexity agent tasks
• You're running on Apple Silicon — oMLX backend has excellent M-series performance
• You're prototyping local-first agent products that target consumer hardware

Prefer alternatives when:

• You need frontier-model quality — use LangGraph or CrewAI with OpenAI/Anthropic
• You need production observability — SmallClaw's telemetry is minimal
• You need multi-user deployment — SmallClaw is designed for single-user local use

Resources

• GitHub Repository
• Show HN discussion
• oMLX backend
• llama.cpp