Skip to content

tier 3 | receiver — per-source (service.name) rate limit + Sources panel #71

@zfogg

Description

@zfogg

Overview

spaniel currently has API + WS authentication via --bearer-token
(#52, pending). Two adjacent gaps for the "spaniel is the OTLP proxy our
team runs" story:

  1. OTLP rate limiting per source — when one badly-behaved service
    floods the receiver, it shouldn't starve the rest. Group by
    service.name (resource attribute) and apply a token bucket per group.
  2. Per-source bookkeeping — show in the BottomBar (or a new "Sources"
    panel) which services are producing how many spans/sec, error rate,
    and bytes/sec. Mirror Cloudflare's "Top hosts" panel.

This builds on the dropped-span counters from #56 (tail sampling) but
operates per service, not per ingestion budget.

Backend

internal/ingestion/limiter.go (new):

type SourceLimiter struct {
  RPS      float64       // per-source token bucket fill rate
  Burst    int           // bucket capacity
  Period   time.Duration // counter window for the stats panel
}

Per-source counters (rolling 60s): accepted, rejected, bytes,
error_count, last_seen.

internal/api/router.go: new endpoint

GET /api/sources
→ { sources: [{ service, accepted_per_sec, rejected_per_sec, error_rate, last_seen_ns, bytes_per_sec }] }

Frontend

frontend/src/components/SourcesPanel.tsx (new): a slide-in panel
reachable from the BottomBar (click the active session chip).

  • Table: service · accepted/s · rejected/s · errors · last seen · bytes/s
  • Top-10 by accepted/s, sort options.
  • A red badge on a row whose rejected_per_sec > 0 (rate-limited).

Wire frontend/src/lib/api.ts with the new types + endpoint.

CLI

--source-rps N      # default 0 = unlimited
--source-burst N    # default = RPS * 5

Config keys mirror.

Tests

Go:

  • internal/ingestion/limiter_test.go: token-bucket math + per-source
    isolation (service A flood doesn't reduce B's bucket).
  • internal/api/sources_test.go: ingest spans from 2 services; GET
    /api/sources; assert each row's accepted_per_sec is positive.

Frontend:

  • Playwright (frontend/e2e/sources.spec.ts, new): stub the endpoint
    with 3 sources, one rate-limited; assert the rate-limit badge appears
    on that row only.

References

  • Existing forwarder rate / counters: internal/forwarder/forwarder.go
  • Existing per-session counters: internal/storage/db.go (Stats)
  • BottomBar mount point: frontend/src/components/BottomBar.tsx

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions