Skip to content

Architecture: generic EventDedup<K> — one TTL set used by all native channels #24

@theognis1002

Description

@theognis1002

Problem Statement

Each native channel in crates/gateway/src/{slack,discord,telegram,teams,signal}/ keeps its own seen-event set with TTL expiry — Slack's EventDeduplicator, Discord's InteractionDeduplicator, etc. The interfaces differ in name only. A race condition fixed in one channel is still latent in the others, and switching the dedup store (in-memory → sqlite/redis) for scale-out would require touching every channel.

Solution

Introduce a generic EventDedup<K: Hash + Eq + Clone> with a documented TTL/eviction policy and an injectable Clock for testability. Replace each per-channel dedup implementation with EventDedup<ChannelEventId>.

User Stories

  1. As a Borg contributor, I want all native channels to share one event-deduplication implementation, so that a race condition fixed once also gets fixed everywhere.
  2. As a Borg contributor, I want to swap the dedup store (in-memory → sqlite/redis) by changing one type, so that horizontal scale-out doesn't require touching every channel.
  3. As a Borg contributor, I want to unit-test dedup behavior (TTL expiry, eviction order, concurrent inserts) once and have the guarantees apply to every channel.
  4. As a Borg user, I want duplicate webhook deliveries to continue being suppressed exactly as today, so that no message gets processed twice.

Implementation Decisions

  • Add EventDedup<K: Hash + Eq + Clone> to gateway (or core if reused outside gateway later) with: configurable TTL, configurable max size, an explicit Clock trait so tests can advance time deterministically.
  • The contract (TTL semantics, eviction policy, thread-safety guarantees) is documented on the type — part of the interface, not folklore.
  • Each channel's existing dedup struct is replaced with EventDedup<NativeId> where NativeId is the platform's event/message ID type.
  • Same TTLs and capacities as today; no behavior change unless a channel is provably wrong (document deltas in the PR).

Testing Decisions

A good test asserts on observable dedup outcomes (insert returns "new" vs "duplicate", entries expire after TTL, capacity-bounded eviction).

  • Unit tests against a deterministic Clock: insert/dedup happy path, TTL expiry, eviction order under capacity pressure, idempotency under repeated inserts.
  • One concurrent-insert test using tokio::join! to assert thread-safety guarantees match the documented contract.
  • Channels stop testing dedup themselves — those tests either move to EventDedup or are deleted as fluff.
  • Prior art: existing per-channel dedup tests in crates/gateway/src/{slack,discord,telegram,teams,signal}/.

Out of Scope

  • Migrating dedup off in-memory storage. The new type makes that swap a one-file change later, but no swap happens in this PRD.
  • Cross-channel dedup or shared state between channels.
  • Persisting dedup state across process restarts.

Further Notes

  • Smallest of the architecture refactors and a good warm-up for whoever picks up the larger ones.
  • Consider naming the trait Clock rather than something Borg-specific, so the same testing pattern can be reused elsewhere (heartbeat scheduler, rate limiter).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions