LangSmith Self-Hosted with Multi-Agent LangGraph System

A complete setup guide for running LangSmith self-hosted tracing platform with a multi-agent system built using LangGraph. This project demonstrates how to set up observability for complex parallel agent workflows using self-hosted infrastructure.

Multi-Agent Graph Diagram Representation

Chat Demo

Important Notes

Self-Hosted Limitations: LangSmith self-hosted is an add-on to the LangSmith Enterprise plan. This development deployment does not include all the full production features available in the hosted LangSmith platform. Some advanced features, integrations, and capabilities are only available in the fully managed enterprise hosted solution.

This setup is designed for:

Local development and testing
Understanding LangSmith architecture
Building and debugging multi-agent systems
Learning about observability patterns

For production use with full feature parity, consider using the hosted LangSmith Enterprise service.

Overview

This project provides a complete implementation of:

LangSmith Self-Hosted: A Docker-based deployment of LangSmith for local development and testing
Multi-Agent System: A parallel agent architecture using LangGraph with planner, workers, and reducer agents
Tracing Infrastructure: Complete observability with traces stored in ClickHouse
No-Auth Development Mode: Simplified setup for local development

Architecture

Multi-Agent Flow

User Input
    ↓
PLANNER Agent
   Creates 3 tasks:
   1. Gather background information
   2. Extract key entities
   3. Propose next steps
    ↓
┌───┬───┬───┐
│ W │ W │ W │  Workers execute in parallel
│ o │ o │ o │
│ r │ r │ r │
│ k │ k │ k │
│ e │ e │ e │
│ r │ r │ r │
└───┴───┴───┘
    ↓
REDUCER Agent
   Synthesizes final results
    ↓
Complete Response

System Components

LangSmith Frontend: Web UI for viewing traces (localhost:1980)
LangSmith Backend: REST API for trace ingestion (localhost:1984)
ClickHouse: Time-series database for trace storage (localhost:8124)
PostgreSQL: Metadata storage (localhost:5433)
Redis: Caching layer (localhost:63791)
LangGraph: Multi-agent orchestration framework

Prerequisites

Before starting, ensure you have:

Docker and Docker Compose installed
Python 3.11+ with virtual environment support
OpenAI API Key (Get one here)
Linux/WSL2 or macOS with Docker support
Basic familiarity with Docker and Python

Quick Start

1. Clone and Setup

# Navigate to your project directory
cd ~/lg-parallel

# Create Python virtual environment
python3 -m venv .venv
source .venv/bin/activate  # On Windows: .venv\Scripts\activate

# Install dependencies
pip install -r requirements.txt

2. Generate Security Secret

# Generate a secure JWT secret
openssl rand -base64 32
# Copy the output - you'll need it next

3. Configure Environment

Copy the example environment file and customize it:

cd langsmith-dev
cp .env.example .env

Then edit langsmith-dev/.env and replace YOUR_GENERATED_SECRET_HERE with the output from step 2.

Or create langsmith-dev/.env file manually:

# Core Authentication (No-Auth Mode for Development)
AUTH_TYPE=none
OAUTH_CLIENT_ID=
OAUTH_ISSUER_URL=
OAUTH_CLIENT_SECRET=
BASIC_AUTH_ENABLED=false
BASIC_AUTH_JWT_SECRET=

# Generated Secret (replace with your openssl rand output)
API_KEY_SALT=YOUR_GENERATED_SECRET_HERE

# License Key
LANGSMITH_LICENSE_KEY=dev-local

# Service URLs (Internal)
LANGSMITH_URL=http://langchain-frontend:1980
SMITH_BACKEND_ENDPOINT=http://langchain-backend:1984

# Database Connection Strings (Internal Service Names)
POSTGRES_DATABASE_URI=postgres:postgres@langchain-db:5432/postgres
REDIS_DATABASE_URI=redis://langchain-redis:6379

# ClickHouse Configuration
CLICKHOUSE_HOST=langchain-clickhouse
CLICKHOUSE_PORT=8123
CLICKHOUSE_NATIVE_PORT=9000
CLICKHOUSE_USER=default
CLICKHOUSE_PASSWORD=password
CLICKHOUSE_DB=default
CLICKHOUSE_TLS=false

# Trace TTL Configuration (Valid JSON)
TRACE_TIER_TTL_DURATION_SEC_MAP={"longlived":34560000,"shortlived":1209600}

# Blob Storage (Disabled for Local Dev)
FF_S3_STORAGE_ENABLED=false
BLOB_STORAGE_BUCKET_NAME=
BLOB_STORAGE_API_URL=
BLOB_STORAGE_ACCESS_KEY=
BLOB_STORAGE_ACCESS_KEY_SECRET=

# Feature Flags
FF_ORG_CREATION_DISABLED=false
FF_TRACE_TIERS_ENABLED=true
FF_UPGRADE_TRACE_TIER_ENABLED=true

# Logging
LOG_LEVEL=info

4. Start Services

cd langsmith-dev
docker compose up -d

# Verify all services are running
docker ps --format "table {{.Names}}\t{{.Status}}"

5. Configure Your Application

Create myapp/.env:

# OpenAI API Key (Required)
OPENAI_API_KEY=sk-your-key-here

# LangSmith Configuration (No-Auth Mode)
LANGCHAIN_TRACING_V2=true
LANGSMITH_ENDPOINT=http://localhost:1980/api/v1
LANGSMITH_API_KEY=
LANGCHAIN_PROJECT=lg-parallel-dev

6. Run Your Multi-Agent System

# Return to project root
cd ..

# Run the test
python simple_test.py

Detailed Setup

Step 1: Docker Compose Configuration

The langsmith-dev/docker-compose.yaml file contains the complete service definitions. Key components:

Frontend Service:

Requires extra_hosts for Linux compatibility
Depends on backend services
Serves UI on port 1980

Backend Services:

Must have AUTH_TYPE, API_KEY_SALT, and X_SERVICE_AUTH_JWT_SECRET explicitly set
All backend services (backend, platform-backend, queue) must share the same JWT secret
Network aliases ensure proper service name resolution

Step 2: ClickHouse Configuration

Create langsmith-dev/users.xml:

<clickhouse>
    <users>
        <default>
            <access_management>1</access_management>
            <named_collection_control>1</named_collection_control>
            <show_named_collections>1</show_named_collections>
            <show_named_collections_secrets>1</show_named_collections_secrets>
            <profile>default</profile>
        </default>
    </users>
    <profiles>
        <default>
            <async_insert>1</async_insert>
            <async_insert_max_data_size>2000000</async_insert_max_data_size>
            <wait_for_async_insert>0</wait_for_async_insert>
            <parallel_view_processing>1</parallel_view_processing>
            <materialize_ttl_after_modify>0</materialize_ttl_after_modify>
            <wait_for_async_insert_timeout>25</wait_for_async_insert_timeout>
            <allow_simdjson>0</allow_simdjson>
            <lightweight_deletes_sync>0</lightweight_deletes_sync>
        </default>
    </profiles>
</clickhouse>

Step 3: Network Configuration

Ensure your docker-compose.yaml includes:

# For platform-backend
langchain-platform-backend:
  networks:
    default:
      aliases:
        - langchain-platform-backend

# For frontend (Linux only)
langchain-frontend:
  extra_hosts:
    - "host.docker.internal:host-gateway"

Step 4: Environment Variable Injection

Critical environment variables must be explicitly passed to containers:

langchain-backend:
  environment:
    - AUTH_TYPE=${AUTH_TYPE:-none}
    - API_KEY_SALT=${API_KEY_SALT}
    - X_SERVICE_AUTH_JWT_SECRET=${API_KEY_SALT}
    - POSTGRES_DATABASE_URI=${POSTGRES_DATABASE_URI}
    - REDIS_DATABASE_URI=${REDIS_DATABASE_URI}
    # ... additional vars

langchain-platform-backend:
  environment:
    - AUTH_TYPE=${AUTH_TYPE:-none}
    - API_KEY_SALT=${API_KEY_SALT}
    - X_SERVICE_AUTH_JWT_SECRET=${API_KEY_SALT}
    # ... additional vars

langchain-queue:
  environment:
    - AUTH_TYPE=${AUTH_TYPE:-none}
    - API_KEY_SALT=${API_KEY_SALT}
    - X_SERVICE_AUTH_JWT_SECRET=${API_KEY_SALT}
    # ... additional vars

Testing Your Multi-Agent System

Basic Test

The simple_test.py script demonstrates a minimal multi-agent setup with parallel execution:

python simple_test.py

This test creates two agents that run in parallel and merge their results through a join node.

Advanced Test

The test_agent.py script uses your full application graph from myapp/src/app_graph.py:

python test_agent.py

This executes the complete planner-worker-reducer flow with OpenAI integration.

Expected Output

Configuration:
   Endpoint: http://localhost:1980/api/v1
   Project: lg-parallel-dev
   Tracing: true
   API Key: (empty - no-auth mode)

Running parallel multi-agent system...
Execution completed!

Result:
   user: Do the tasks in parallel
   agent_a: A: fetched docs
   agent_b: B: did a web lookup
   system: JOIN: Do the tasks in parallel | A: fetched docs | B: did a web lookup

To view traces:
   - Open http://localhost:1980 in your browser
   - Look for project 'lg-parallel-dev'
   - You should see a run with nodes: agent_a, agent_b, and join

Viewing Traces

Access Points

Service	URL	Description
LangSmith UI	`http://localhost:1980`	Main UI for viewing traces
Backend API	`http://localhost:1984/api/v1`	REST API
API Docs	`http://localhost:1984/api/docs`	OpenAPI documentation
Playground	`http://localhost:3001`	LangSmith Playground
ACE Backend	`http://localhost:1987`	ACE Backend
ClickHouse	`http://localhost:8124`	ClickHouse UI
PostgreSQL	`localhost:5433`	PostgreSQL database
Redis	`localhost:63791`	Redis cache

View Traces in UI

Open http://localhost:1980 in your browser
Navigate to your project (lg-parallel-dev)
Click on a run to see the execution graph
Examine agent steps, inputs, outputs, and timing

View Traces via API

from langsmith import Client

client = Client(
    api_url="http://localhost:1980/api/v1",
    api_key=None  # No-auth mode
)

# Get all runs in your project
runs = client.list_runs(project_name="lg-parallel-dev")
for run in runs:
    print(f"Run ID: {run.id}, Name: {run.name}")

Troubleshooting

Services Not Starting

Problem: Docker containers fail to start or restart continuously.

Solution:

# Check logs
docker logs langsmith-dev_langchain-backend_1

# Restart all services
cd langsmith-dev
docker compose down -v
docker compose up -d

Frontend Can't Connect to Backend

Problem: nginx: host not found in upstream "langchain-platform-backend"

Solution:

Ensure network alias is set in docker-compose.yaml

Verify all services are on the same network:

docker network inspect langsmith-dev_default

JWT Token Errors

Problem: Error parsing token err="token contains an invalid number of segments"

Solution:

Verify API_KEY_SALT is set in .env
Ensure X_SERVICE_AUTH_JWT_SECRET=${API_KEY_SALT} in all backend services

Restart services:

docker compose down -v
docker compose up -d

ClickHouse Restarting

Problem: ClickHouse container exits with error

Solution:

Check users.xml for unsupported settings
Ensure no allow_materialized_view_with_bad_select setting exists
Remove the setting if present

Traces Not Appearing

Problem: UI loads but no traces visible

Solution:

Verify tracing is enabled: LANGCHAIN_TRACING_V2=true
Check backend logs: docker logs langsmith-dev_langchain-backend_1
Verify endpoint: LANGSMITH_ENDPOINT=http://localhost:1980/api/v1

Query ClickHouse directly:

docker exec -it langsmith-dev_langchain-clickhouse_1 clickhouse-client
SHOW TABLES;
SELECT * FROM traces LIMIT 10;

OpenAI API Errors

Problem: OPENAI_API_KEY not set

Solution:

# Set in environment
export OPENAI_API_KEY='sk-your-key'

# Or in myapp/.env
echo "OPENAI_API_KEY=sk-your-key" >> myapp/.env

Import Errors

Problem: ModuleNotFoundError

Solution:

# Activate virtual environment
source .venv/bin/activate

# Install dependencies
pip install -r requirements.txt

Project Structure

lg-parallel/
├── README.md                      # This file
├── langsmith-dev/                 # LangSmith Docker setup
│   ├── docker-compose.yaml        # Docker Compose configuration
│   ├── users.xml                  # ClickHouse configuration
│   └── .env                       # Environment variables (create this)
├── myapp/                         # Your multi-agent application
│   ├── src/
│   │   └── app_graph.py          # Main agent graph
│   ├── .env                       # Application environment
│   └── requirements.txt          # Python dependencies
├── simple_test.py                # Minimal parallel agent test
├── test_agent.py                 # Full application test
├── .gitignore                    # Git ignore rules
└── requirements.txt              # Root dependencies

Additional Resources

Official Documentation

Community Support

Common Commands

# Start all services
cd langsmith-dev && docker compose up -d

# Stop all services
cd langsmith-dev && docker compose down

# Stop and remove volumes (deletes data)
cd langsmith-dev && docker compose down -v

# View logs
docker logs -f langsmith-dev_langchain-backend_1

# Check service status
docker ps --format "table {{.Names}}\t{{.Status}}"

# Restart a specific service
docker compose restart langchain-backend

# Run your tests
python simple_test.py
python test_agent.py

# Check API health
curl http://localhost:1980/info
curl http://localhost:1984/api/v1/info

Production Considerations

For production deployments, you should:

Enable Authentication: Configure OIDC or Basic Auth
Use HTTPS: Set up SSL certificates
Secure Secrets: Use proper secrets management (not plain text .env files)
Database Backups: Configure regular backups for PostgreSQL and ClickHouse
Monitoring: Set up monitoring and alerting
Network Security: Configure proper firewall rules
Resource Limits: Set appropriate CPU and memory limits for containers
Logging: Configure centralized logging
High Availability: Set up redundant services

License

This project is for development and testing purposes. LangSmith self-hosted requires appropriate licensing for production use.

Contributing

Contributions are welcome. Please follow these steps:

Fork the repository
Create a feature branch
Make your changes
Submit a pull request

Support

For issues and questions:

Check the troubleshooting section
Review the logs: docker logs <service-name>
Consult the official documentation
Ask in the community Discord

Success Checklist

Your LangSmith self-hosted environment with multi-agent system is ready.

Name		Name	Last commit message	Last commit date
.venv		.venv
images		images
langsmith-dev		langsmith-dev
myapp		myapp
.gitignore		.gitignore
README.md		README.md
README_LANGSMITH_SETUP.md		README_LANGSMITH_SETUP.md
SETUP.md		SETUP.md
debug.log		debug.log
diagnose_langsmith.sh		diagnose_langsmith.sh
docker-compose.yml		docker-compose.yml
docker_endpoints_check.sh		docker_endpoints_check.sh
manual_trace_viewer.py		manual_trace_viewer.py
setup_env.sh		setup_env.sh
simple_test.py		simple_test.py
test_agent.py		test_agent.py
view_traces.py		view_traces.py

Latest commit

History

Folders and files

Repository files navigation

LangSmith Self-Hosted with Multi-Agent LangGraph System

Important Notes

Table of Contents

Overview

Architecture

Multi-Agent Flow

System Components

Prerequisites

Quick Start

1. Clone and Setup

2. Generate Security Secret

3. Configure Environment

4. Start Services

5. Configure Your Application

6. Run Your Multi-Agent System

Detailed Setup

Step 1: Docker Compose Configuration

Step 2: ClickHouse Configuration

Step 3: Network Configuration

Step 4: Environment Variable Injection

Testing Your Multi-Agent System

Basic Test

Advanced Test

Expected Output

Viewing Traces

Access Points

View Traces in UI

View Traces via API

Troubleshooting

Services Not Starting

Frontend Can't Connect to Backend

JWT Token Errors

ClickHouse Restarting

Traces Not Appearing

OpenAI API Errors

Import Errors

Project Structure

Additional Resources

Official Documentation

Community Support

Common Commands

Production Considerations

License

Contributing

Support

Success Checklist

About

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages