RarePath

RarePath is a small SaaS-style implementation of the central idea in Microsoft Research's STEAM: Observability-Preserving Trace Sampling: keep traces that preserve useful behavioral diversity instead of retaining a uniform random sample.

This is a learning implementation, not a reproduction of STEAM's trained graph neural network.

Architecture

customer application
      │ OTLP/HTTP JSON + API key
      ▼
Go ingestion API ── bounded queue ── batch workers
      │                                  │
      │                                  ▼
      │                         Python graph analyzer
      │                         • graph fingerprint
      │                         • rarity / error / latency score
      │                         • keep or sample out
      │                                  │
      └──────── query API ◀── selected trace store

• Go data plane: tenant authentication, an OTLP/HTTP JSON subset, bounded buffering, backpressure with HTTP 429, batching, concurrent workers, in-memory storage, query endpoints, counters, and graceful shutdown.
• Python analysis plane: converts spans into graph nodes and parent-child edges, fingerprints the graph, and retains rare shapes, errors, and latency outliers. Its HTTP boundary can later host STEAM's GNN-based representation.
• SaaS boundary: every trace is assigned from its API key to a tenant; analysis history and query results are isolated per tenant.

Run locally

Requirements: Go 1.22+ and Python 3.10+. No third-party packages are needed.

Terminal 1:

make run-analyzer

Terminal 2:

make run-api

Terminal 3:

make demo

Or run both services with:

docker compose up --build

API

The demo key is demo-key, mapped to tenant demo.

curl -X POST http://localhost:8080/v1/traces \
  -H 'Content-Type: application/json' \
  -H 'X-API-Key: demo-key' \
  --data-binary @examples/checkout-trace.json

curl http://localhost:8080/api/traces \
  -H 'X-API-Key: demo-key'

curl http://localhost:8080/api/stats \
  -H 'X-API-Key: demo-key'

POST /v1/traces accepts the standard OTLP JSON nesting resourceSpans → scopeSpans → spans and the common span fields used in the example. It is intentionally a subset, not a complete OTLP implementation.

Configure multiple tenants with:

RAREPATH_API_KEYS='key-one:tenant-a,key-two:tenant-b'

Sampling behavior

For each tenant, the analyzer keeps:

1. the first few examples of each trace graph;
2. every trace containing an error;
3. traces whose root latency is more than twice the recent tenant baseline.

Repeated healthy traces with an already-known graph are sampled out. Each kept trace includes a score and human-readable reasons.

Run the tests with:

make test

Production evolution

The intentionally simple pieces have clear production replacements:

• in-memory queue → Kafka, NATS, or a managed stream;
• in-memory store → ClickHouse or another trace-optimized analytical store;
• API-key environment variable → hashed keys in a tenant database;
• graph fingerprint → STEAM-style learned graph representation with logical constraints and diversity-aware selection;
• process counters → Prometheus/OpenTelemetry metrics and alerting.