🔦 Elastic Lens — Production-Aware AI Coding

Your AI coding assistant can read your code. It can't see your production. Elastic Lens fixes that.

---

🚨 The Problem

Every AI coding tool — Cursor, GitHub Copilot, Amp — operates in a vacuum. They can read your code, suggest refactors, and write tests. But they're completely blind to production:

• They don't know your function handles 18,000 requests/day
• They don't know it has a 2.3% error rate since last week's deployment
• They don't know the last Stripe SDK change caused a P1 outage
• They don't know there's a NullPointerException on line 139

The data already exists in Elasticsearch — APM transactions, error logs, deployment records, incident history. It's just not connected to where developers work.

---

✨ The Solution

Elastic Lens bridges production data in Elasticsearch with your coding environment through two interfaces, powered by a single Elasticsearch Agent Builder agent.

VS Code Extension

• CodeLens annotations — inline production metrics (🟢🟡🔴) above every function
• Hover tooltips — deployment history, incident records, SLA status
• Diagnostics — warning/error squiggles on high error-rate functions
• Sidebar chat — conversational AI with streaming responses for deep-dive investigation

MCP Server (Agent Builder built-in)

Agent Builder has a built-in MCP server — zero custom server code. Any MCP-compatible tool (Cursor, Claude Desktop, Amp, Codex) connects instantly:

• "Is it safe to refactor processPayment?" → data-backed risk profile
• "What's causing the errors?" → semantic search across error logs
• "Show me the deployment that broke this" → ES|QL with LOOKUP JOINs

---

🏗️ Architecture

One data layer. One agent. Two surfaces.

Agent Builder Tools

Tool	Type	Description
lens.get_function_metrics	ES|QL	Call volume, error rate, latency with service metadata JOIN
lens.get_risk_profile	ES|QL	Full risk assessment with 3 LOOKUP JOINs (services, deployments, incidents)
lens.search_errors	Index Search	Semantic search (ELSER) across error logs
lens.get_deployment_history	ES|QL	Recent deployments for a service
lens.get_service_overview	ES|QL	Health overview of all functions in a service
lens.monitor_after_deploy	Workflow	Post-deployment monitoring with rollback recommendations

The Showcase: 3-Way LOOKUP JOIN

The get_risk_profile tool chains three LOOKUP JOINs in a single ES|QL query — correlating APM metrics with service metadata, deployment history, and incident records:

FROM apm_transactions
| WHERE transaction.name == ?function_name
| STATS total_calls = COUNT(*), errors = SUM(is_error),
        avg_latency_ms = AVG(transaction.duration.us) / 1000,
        p99_latency_ms = PERCENTILE(transaction.duration.us, 99) / 1000
  BY service.name
| LOOKUP JOIN services_lookup ON service.name
| LOOKUP JOIN deployments_lookup ON service.name
| LOOKUP JOIN incidents_lookup ON service.name

---

🚀 Getting Started

Prerequisites

• Elastic Cloud account (free trial works)
• Node.js 18+
• uv (Python package manager)
• VS Code 1.85+

Step 1: Clone and Configure Credentials

git clone https://github.com/SinghCoder/elastic-lens.git
cd elastic-lens
cp .env.sample .env

Edit .env with your Elastic Cloud Serverless credentials:

ES_URL=https://your-project-id.es.us-central1.gcp.elastic.cloud
KIBANA_URL=https://your-project-id.kb.us-central1.gcp.elastic.cloud
ES_API_KEY=your-api-key
API_KEY=your-api-key

Variable	Where to get it
ES_URL	Elastic Cloud console → your project → Endpoints → Elasticsearch URL
KIBANA_URL	Same page → Endpoints → Kibana URL
ES_API_KEY / API_KEY	Kibana → Project settings → Management → API Keys → Create API key

Tip: One API key works for both ES_API_KEY and API_KEY. All scripts auto-load from .env.

Step 2: Create Indices and Load Lookup Data

cd elasticsearch
bash setup.sh

Creates 6 indices (2 standard + 4 lookup mode) and bulk-loads services, deployments, incidents, and endpoints.

Step 3: Generate Mock APM Data

cd ../scripts
uv run generate_data.py

Generates ~100K APM transactions and ~5K correlated error logs across 30 days. processPayment has an elevated error rate since deploy v2.4.1 — the demo's "hero bug."

uv run verify_data.py   # verify data looks correct

Step 4: Set Up Agent Builder

cd ../agent-builder
bash setup.sh

Registers all tools and the orchestrator agent via Kibana API.

Step 5: Connect MCP to Your AI Tool

Codex

~/.codex/config.toml:

[mcp_servers.elastic-lens]
url = "https://YOUR_KIBANA_URL/api/agent_builder/mcp"
http_headers = { "Authorization" = "ApiKey YOUR_API_KEY", "kbn-xsrf" = "true" }

Cursor

~/.cursor/mcp.json:

{
  "mcpServers": {
    "elastic-lens": {
      "command": "npx",
      "args": ["mcp-remote", "https://YOUR_KIBANA_URL/api/agent_builder/mcp", "--header", "Authorization:ApiKey YOUR_API_KEY"]
    }
  }
}

Claude Desktop

~/Library/Application Support/Claude/claude_desktop_config.json:

{
  "mcpServers": {
    "elastic-lens": {
      "command": "npx",
      "args": ["mcp-remote", "https://YOUR_KIBANA_URL/api/agent_builder/mcp", "--header", "Authorization:ApiKey YOUR_API_KEY"]
    }
  }
}

Amp

.ampmcp in project root:

{
  "mcpServers": {
    "elastic-lens": {
      "command": "npx",
      "args": ["mcp-remote", "https://YOUR_KIBANA_URL/api/agent_builder/mcp", "--header", "Authorization:ApiKey YOUR_API_KEY"]
    }
  }
}

Step 6: Install VS Code Extension

cd vscode-extension
npm install
npm run compile

Press F5 in VS Code to launch the Extension Development Host.

Step 7: Try It

Open sample-project/ in the dev window:

cp sample-project/.elastic-lens.sample.json sample-project/.elastic-lens.json
# Edit with your Kibana URL and API key (or set KIBANA_URL / API_KEY env vars)

You should see:

• 🔴 Red CodeLens on processPayment — high error rate
• 🟡 Yellow CodeLens on calculateTax — approaching SLA
• 🟢 Green CodeLens on getUser, validateCart, etc.
• Hover any function for full risk profile with incidents and deployments
• Warning squiggles on processPayment

In your AI tool (Cursor/Claude/Amp), try:

Is it safe to refactor processPayment in checkout-service?

---

📁 Project Structure

elastic-lens/
├── elasticsearch/          # Index mappings, lookup data, setup script
├── scripts/               # Python data generation and verification
├── agent-builder/         # Agent + tool definitions, setup script
├── vscode-extension/      # VS Code extension (CodeLens, Hover, Diagnostics, Sidebar)
├── sample-project/        # Demo e-commerce API with .elastic-lens.json
├── assets/                # Architecture diagrams and images
└── mcp-config.json        # MCP client config template

📝 License

MIT — see LICENSE