Add LangSmith Cloud observability for all AI API calls

[pyrex41]

Summary

Add comprehensive LangSmith Cloud observability for monitoring and debugging all AI API calls in production.

What's Traced

• ✅ OpenAI (GPT-4o, GPT-4o-mini) - scene generation, semantic augmentation, prompt parsing
• ✅ Anthropic (Claude) - prompt parsing, creative direction
• ✅ XAI (Grok-4) - AI-powered image pair selection, property scene analysis
• ✅ Replicate API - image/video/audio generation (Flux, SkyReels, Veo3, Hailuo)
• ✅ HTTP requests - all FastAPI endpoints with hierarchical trace nesting

Key Features

• 📊 Real-time traces in LangSmith dashboard at smith.langchain.com
• 💰 Automatic cost tracking and token usage per model
• 🐛 Full prompt/response inspection for debugging
• ⚡ Latency monitoring and error tracking
• 🏷️ Filter by tags: openai, anthropic, xai, grok, replicate, http_request
• 🌲 Hierarchical traces showing full request → LLM call flow

Implementation

Dependencies Added

• langsmith - Core tracing SDK
• langchain-openai - OpenAI integration
• langchain-anthropic - Anthropic integration

Code Changes (14 files, 364 additions)

• Added @traceable decorators to all LLM provider methods
• Implemented LangSmithMiddleware for both FastAPI apps
• Updated config files with LangSmith environment variables
• Updated docker-compose.yml with tracing env vars

Documentation

• Created LANGSMITH_SETUP.md with complete setup guide
• Includes quick start, configuration, troubleshooting

Files Changed

• LANGSMITH_SETUP.md - Complete setup documentation
• pyproject.toml - Dependencies
• backend/config.py - LangSmith settings
• backend/main.py - HTTP request middleware
• backend/llm_interpreter.py - OpenAI tracing
• backend/services/scene_generator.py - OpenAI tracing
• backend/services/xai_client.py - XAI/Grok tracing
• backend/services/replicate_client.py - Replicate tracing
• promptparser/.env.example - Example env vars
• promptparser/app/core/config.py - LangSmith settings
• promptparser/app/main.py - HTTP request middleware
• promptparser/app/services/llm/openai_provider.py - OpenAI tracing
• promptparser/app/services/llm/claude_provider.py - Anthropic tracing
• docker-compose.yml - Environment variables

Setup

1. Sign up at smith.langchain.com (free)
2. Add to .env:

   LANGCHAIN_TRACING_V2=true
   LANGCHAIN_API_KEY=<your-key>

3. Run: uv pip install -e .
4. Start app → traces appear automatically ✨

Notes

• Tracing is disabled by default (LANGCHAIN_TRACING_V2=false)
• No performance impact when disabled (~0ms overhead)
• When enabled: ~5-20ms per trace (acceptable for production)
• Health checks and static files are excluded from tracing

🤖 Generated with Claude Code