INSIGHTS

A plug-and-play AI observability adaptor for any project. Track attribution, metrics, and quality across Copilot, Claude Code, Cursor, Gemini, and any AI agent.

---

Three Pillars

Pillar	What It Tracks	Source
🔍 Attribution	AI vs human code lines per commit (git hooks)	attribution/
📊 Metrics	Token usage, time saved, cost efficiency	metrics/
🎯 Quality	LLM output quality vs April 2026 benchmarks	quality/

All three are available from a single unified dashboard at http://localhost:9090: a small hub with tabs; each tab embeds the full-page UIs in dashboard/views/ (served as /views/*.html).

---

Quick Start (5 Minutes)

1. Install

# From your project root:
bash INSIGHTS/scripts/setup.sh

This installs git hooks, creates .env.insights, and sets up directories.

2. Configure

cp INSIGHTS/.env.example .env.insights
# Edit .env.insights: set INSIGHTS_PROJECT_NAME and API keys

3. Start Dashboard

python INSIGHTS/dashboard/serve.py --open
# Opens http://localhost:9090

4. Set Your Agent

export AI_AGENT=claude-code    # or: copilot, cursor, gemini
export AI_MODEL="Claude Sonnet 4.6"

---

Project Structure

INSIGHTS/
├── attribution/                    # AI Code Attribution
│   ├── analyzer/
│   │   ├── commit_analyzer.py      # Pre-push + CI analyzer
│   │   └── git_utils.py            # Git utilities
│   ├── interceptor/
│   │   └── log_ai_changes.py       # Pre-commit diff capture
│   ├── hooks/
│   │   ├── install_hooks.sh        # One-command hook installer
│   │   ├── pre-commit              # Captures staged diffs
│   │   └── pre-push                # Classifies AI vs human
│   ├── server/
│   │   └── serve_attribution.py    # Standalone server (port 9092)
│   └── data/                       # Attribution JSONs (sample data included)
│
├── metrics/                        # AI Usage Metrics
│   ├── server/
│   │   └── serve_metrics.py        # Standalone server (port 9091)
│   ├── schema/
│   │   └── metrics_schema.json     # JSON schema v1.1.0
│   └── data/                       # Metrics JSONs (sample data included)
│
├── quality/                        # LLM Quality Evaluator (DeepEval)
│   ├── evaluator.py                # Main CLI runner
│   ├── metrics/
│   │   ├── llm_benchmark_quality.py   # MMLU-Pro, IFEval, MT-Bench
│   │   ├── code_generation_quality.py # HumanEval+, SWE-bench, LiveCodeBench
│   │   ├── reasoning_quality.py       # GPQA Diamond, MATH-500, BIG-Bench Hard
│   │   └── agent_task_quality.py      # SWE-bench, AgentBench, GAIA
│   ├── requirements.txt
│   └── results/                    # Eval results (sample data included)
│
├── dashboard/
│   ├── serve.py                    # Unified server (port 9090) — use this
│   ├── index.html                  # Hub: tab bar + iframes into views/
│   └── views/                      # Full-page UIs (served as /views/*.html)
│       ├── ai-metrics-collector.html
│       ├── ai-quality-evaluator.html
│       └── ai-code-attribution.html
│
├── integrations/                   # Agent adaptor configs
│   ├── README.md                   # Integration guide
│   ├── github-copilot/
│   │   ├── copilot-instructions.md
│   │   └── agents/
│   │       ├── insights-metrics.agent.md
│   │       └── insights-quality.agent.md
│   ├── claude-code/
│   │   └── CLAUDE.md
│   ├── cursor/
│   │   └── rules/insights.mdc
│   ├── gemini/
│   │   └── AGENTS.md
│   └── workflows/
│       └── ai-attribution.yml      # GitHub Actions CI workflow
│
├── scripts/
│   └── setup.sh                    # One-command installer
├── .env.example                    # Environment variables template
└── .gitignore

---

Key Commands

# Unified dashboard (all 3 pillars)
python INSIGHTS/dashboard/serve.py --open

# Attribution only
python INSIGHTS/attribution/analyzer/commit_analyzer.py --report

# Ingest metrics
curl -X POST http://localhost:9090/api/metrics/ingest \
  -H "Content-Type: application/json" \
  -d '{"action":"coding","ai_actual_mins":5,"time_saved_mins":115,"complexity_tier":"large"}'

# Quality evaluation
python INSIGHTS/quality/evaluator.py --metric code_generation --input-file my_code.py
python INSIGHTS/quality/evaluator.py --all --output "LLM output text"

---

API Endpoints

Method	Path	Description
GET	/api/metrics	Aggregated metrics data
GET	/api/evals	Latest quality evaluation
GET	/api/attribution	Attribution data (all commits)
GET	/api/attribution/latest	Most recent commit attribution
GET	/api/health	Server health check
POST	/api/metrics/ingest	Add a metrics entry
POST	/api/attribution/ingest	Add an attribution record

---

Quality Benchmarks (April 2026)

Metric	Key Benchmarks	Frontier Baseline
llm_benchmark	MMLU-Pro, IFEval, MT-Bench	>90% MMLU-Pro
code_generation	HumanEval+, SWE-bench Verified, LiveCodeBench	>92% HumanEval+, ~55% SWE-bench
reasoning	GPQA Diamond, MATH-500, BIG-Bench Hard	>75% GPQA, >95% MATH-500
agent_task	AgentBench, GAIA, Berkeley Function Calling	~75% GAIA

Threshold: 0.7 (configurable via EVAL_THRESHOLD).

---

Agent Integration

Agent	Config File	How to Use
GitHub Copilot	integrations/github-copilot/copilot-instructions.md	Copy to .github/copilot-instructions.md
Claude Code	integrations/claude-code/CLAUDE.md	Copy to project root as CLAUDE.md
Cursor	integrations/cursor/rules/insights.mdc	Copy to .cursor/rules/
Gemini/Antigravity	integrations/gemini/AGENTS.md	Reference in AGENTS.md
GitHub Actions	integrations/workflows/ai-attribution.yml	Copy to .github/workflows/

See integrations/README.md for the full integration guide.

---

Environment Variables

INSIGHTS_BASE_DIR=./INSIGHTS          # INSIGHTS root (auto-detected)
INSIGHTS_PROJECT_NAME=MyProject       # Shown in quality eval criteria
INSIGHTS_AI_LOGS_DIR=.ai-logs         # Pre-commit log directory
AI_AGENT=copilot                      # Active agent name
AI_MODEL="Claude Sonnet 4.6"         # Active model
INSIGHTS_PORT=9090                    # Dashboard port
EVAL_THRESHOLD=0.7                    # Quality passing threshold
ANTHROPIC_API_KEY=sk-ant-...          # For LLM judge

---

Data Formats

• Attribution: attribution/data/attribution_<sha>.json
• Metrics: metrics/data/ai_usage_metrics_<YYYY-MM-DD>.json (schema v1.1.0)
• Quality: quality/results/eval_results_<YYYY-MM-DD>.json

Sample data from ANYMUS project is included for immediate dashboard testing.