PeakInfer

Run AI inference at peak performance.

PeakInfer scans your code. Finds every LLM call. Shows you exactly what's holding back your latency, throughput, and reliability.

30 seconds. Zero config. Real numbers.

npm install -g @kalmantic/peakinfer
peakinfer analyze .

---

The Problem

Your code says streaming: true. Your runtime shows 0% streams.

That's drift—and it's killing your latency.

What You Think	What's Actually Happening
Streaming enabled	Blocking calls
Fast responses	p95 latency 5x slower than benchmarks
Retry logic works	Never triggered
Fallbacks ready	Never tested

Static analysis sees code. Monitoring sees requests. Neither sees the gap.

PeakInfer sees both.

---

What Is Peak Inference Performance?

Peak Inference Performance means: Improving latency, throughput, reliability, and cost without changing evaluated behavior.

No one else correlates what PeakInfer sees together:

CODE                 RUNTIME              BENCHMARKS           EVALS
────                 ───────              ──────────           ─────

What you             What actually        The upper bound      Your quality
declared             happened             of possible          gate

streaming: true      0% streaming         InferenceMAX:        "extraction" 94%
model: gpt-4o        p95: 2400ms          gpt-4o p95: 1200ms   accuracy

        └───────────────────┴────────────────────┴───────────────────┘
                                     │
                                     ▼
                                PEAKINFER
                               (correlation)

---

The Four Dimensions

PeakInfer analyzes every inference point across four dimensions:

Dimension	What We Find	Typical Improvement
Latency	Missing streaming, blocking calls, p95 gaps	50-80% faster
Throughput	Sequential loops, no batching	10-50x improvement
Reliability	No retry, no fallback, no timeout	99%+ uptime
Cost	Wrong model for the job	60-90% reduction

---

How It Works

1. Scan Your Code

peakinfer analyze ./src

Finds every inference point. OpenAI, Anthropic, Azure, Bedrock, self-hosted. All of them.

2. See What's Holding You Back

7 inference points found
39 issues detected

LATENCY:
- Streaming configured but not consumed (p95: 2400ms, should be 400ms)
- Blocking calls in hot path (6x latency penalty)

THROUGHPUT:
- Sequential batch processing (50x throughput opportunity)

RELIABILITY:
- Zero error handling across all LLM calls
- No fallback on critical inference path

QUICK WINS:
- Enable streaming consumption: -80% latency
- Add retry logic: +99% reliability
- Parallelize batch: 50x throughput

3. Catch Drift Before Production

Add to every PR:

- uses: kalmantic/peakinfer-action@v1
  with:
    path: ./src
    token: ${{ secrets.PEAKINFER_TOKEN }}

---

Installation

npm install -g @kalmantic/peakinfer

Requires Node.js 18+.

---

First-Time Setup

PeakInfer uses Claude for semantic analysis. You provide your own Anthropic API key (BYOK mode).

Step 1: Get an Anthropic API Key

1. Go to console.anthropic.com
2. Create an account or sign in
3. Navigate to API Keys and create a new key
4. Copy the key (starts with sk-ant-)

Step 2: Configure Your API Key

Option A: Environment File (Recommended)

# .env
ANTHROPIC_API_KEY=sk-ant-your-key-here

Option B: Shell Export

export ANTHROPIC_API_KEY=sk-ant-your-key-here

Step 3: Verify Setup

peakinfer analyze . --verbose

BYOK Mode: Your API key, your costs, full transparency. Analysis runs locally. No data sent to PeakInfer servers.

---

Commands

# Basic scan
peakinfer analyze .

# With code fix suggestions
peakinfer analyze . --fixes

# With HTML report
peakinfer analyze . --html --open

# Compare to InferenceMAX benchmarks
peakinfer analyze . --benchmark

# With runtime correlation (drift detection)
peakinfer analyze . --events production.jsonl

# Fetch runtime from observability platforms
peakinfer analyze . --runtime helicone --runtime-key $HELICONE_KEY

# Full analysis
peakinfer analyze . --fixes --benchmark --html --open

CLI Options

Flag	Description
Output
--fixes	Show code fix suggestions for each issue
--html	Generate HTML report
--pdf	Generate PDF report
--open	Auto-open report in browser/viewer
--output <format>	Output format: text, json, or inference-map
--verbose	Show detailed analysis logs
Runtime Data
--events <file>	Path to runtime events file (JSONL)
--events-url <url>	URL to fetch runtime events
--runtime <source>	Fetch from: helicone, langsmith
--runtime-key <key>	API key for runtime source
--runtime-days <n>	Days of runtime data (default: 7)
Comparison
--compare [runId]	Compare with previous analysis run
--benchmark	Compare to InferenceMAX benchmarks
--predict	Generate deploy-time latency predictions
--target-p95 <ms>	Target p95 latency for budget calculation
Cost Control
--estimate	Show cost estimate before analysis
--yes	Auto-proceed without confirmation
--max-cost <dollars>	Skip if estimated cost exceeds threshold
--cached	View previous analysis (offline)

---

GitHub Action

Every PR. Every merge. Automatic.

name: PeakInfer
on: [pull_request]

permissions:
  contents: read
  pull-requests: write

jobs:
  analyze:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: kalmantic/peakinfer-action@v1
        with:
          token: ${{ secrets.PEAKINFER_TOKEN }}
          github-token: ${{ github.token }}

See peakinfer-action for full documentation.

---

Runtime Drift Detection

PeakInfer's real power: correlating code with runtime behavior.

# From file
peakinfer analyze ./src --events events.jsonl

# From Helicone
peakinfer analyze ./src --runtime helicone --runtime-key $HELICONE_KEY

# From LangSmith
peakinfer analyze ./src --runtime langsmith --runtime-key $LANGSMITH_KEY

Supported formats: JSONL, JSON, CSV, OpenTelemetry, Jaeger, Zipkin, LangSmith, LiteLLM, Helicone.

---

Supported Providers

Provider	Status
OpenAI	Full support
Anthropic	Full support
Azure OpenAI	Full support
AWS Bedrock	Full support
Google Vertex	Full support
vLLM / TensorRT-LLM	HTTP detection
LangChain / LlamaIndex	Framework support

---

Community Templates

43 templates across two categories:

Insight Templates (12)

Detect issues: streaming drift, overpowered model, context accumulation, token underutilization, retry explosion, untested fallback, dead code, and more.

Optimization Templates (31)

Actionable fixes: model routing, batch utilization, prompt caching, vLLM high-throughput, GPTQ quantization, TensorRT-LLM, multi-provider fallback, auto-scaling, and more.

---

Pricing

CLI: Free forever. BYOK — you provide your Anthropic API key.

GitHub Action:

• Free: 50 credits one-time (6-month expiry)
• Starter: $19 for 200 credits
• Growth: $49 for 600 credits
• Scale: $149 for 2,000 credits
• Mega: $499 for 10,000 credits

No subscriptions. No per-seat pricing. Team pooling.

View pricing →

---

What's Included

Feature	Status
Unified Prompt-Based Analysis	✅
GitHub Action with PR Comments	✅
Code Fix Suggestions	✅
Runtime Drift Detection	✅
InferenceMAX Benchmark Comparison	✅
43 Optimization Templates	✅
Run History & Comparison	✅
BYOK Mode (CLI)	✅

---

Links

• VS Code Extension
• MCP Server (Claude Desktop)
• GitHub Action
• Report Issues

---

Built by Kalmantic. Apache-2.0 license.