Skip to content

feat: add IBigSegmentStore interface + Redis and DynamoDB stores#536

Merged
beekld merged 11 commits into
mainfrom
beeklimt/SDK-2363
May 28, 2026
Merged

feat: add IBigSegmentStore interface + Redis and DynamoDB stores#536
beekld merged 11 commits into
mainfrom
beeklimt/SDK-2363

Conversation

@beekld
Copy link
Copy Markdown
Contributor

@beekld beekld commented May 19, 2026

Ticket: SDK-2363 · Follows #534

Summary

Adds the public IBigSegmentStore interface, the Membership / StoreMetadata value types, and concrete DynamoDB + Redis implementations. Schema strings match what the Relay Proxy writes.

auto redis_store = RedisBigSegmentStore::Create("redis://localhost:6379", "prefix");
auto dynamo_store = DynamoDBBigSegmentStore::Create("my-table", "prefix", options);

Design notes

  • synchronizedOn parsing. Stored as DynamoDB N / Redis string. Both stores reject malformed values (non-numeric strings, wrong DynamoDB attribute type) rather than silently returning 0, matching the existing dynamodb_source.cpp row validation.
  • No hashing here. The interface contract says callers pass the already-hashed base64 SHA-256 context key; the SDK will hash in the wrapper, not the store implementations.

Not in scope

  • BigSegmentsBuilder config plumbing.
  • BigSegmentStoreWrapper (LRU cache + staleness polling) and the hashing path.
  • Evaluator wiring / replacing the rules.cpp big-segments TODO.

Test plan

  • 7 Membership unit tests pass.
  • 9 RedisBigSegmentStore integration tests pass against Redis 7 (docker).
  • 8 DynamoDBBigSegmentStore integration tests pass against DynamoDB Local.
  • Full server-sdk test suite (468 tests) still green.
  • Schema constants match what Relay writes (verified against the LocalStack/Redis fixture data).

Note

Medium Risk
New external-store read path for segment targeting with strict schema validation; not yet wired into evaluation, so production flag behavior is unchanged until follow-up PRs land.

Overview
Introduces a Big Segments persistent store surface for the server SDK: public IBigSegmentStore, inline Membership / StoreMetadata, and read-only Redis and DynamoDB backends that follow Relay’s key/schema layout (prefix scoping, include/exclude refs, sync metadata).

Stores perform point lookups by opaque context hash, build membership via Membership::FromSegmentRefs (inclusion wins on overlap), and return errors on malformed DynamoDB attribute types or non-numeric sync timestamps instead of silent empty/zero results. Config, caching wrapper, hashing, and flag evaluation wiring are not included in this change.

Integration tests cover membership/metadata behavior, prefix isolation, and error paths against Redis and DynamoDB Local.

Reviewed by Cursor Bugbot for commit d2622d9. Bugbot is set up for automated code reviews on this repo. Configure here.

@beekld beekld marked this pull request as ready for review May 20, 2026 18:50
@beekld beekld requested a review from a team as a code owner May 20, 2026 18:50
Comment thread libs/server-sdk-dynamodb-source/src/dynamodb_big_segment_store.cpp
Comment thread libs/server-sdk-dynamodb-source/src/dynamodb_big_segment_store.cpp
Copy link
Copy Markdown

@cursor cursor Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 0082f6e. Configure here.

Comment thread libs/server-sdk-redis-source/src/redis_big_segment_store.cpp
Comment thread libs/server-sdk-redis-source/src/redis_big_segment_store.cpp
Comment thread libs/server-sdk-redis-source/src/redis_big_segment_store.cpp
Comment thread libs/server-sdk-redis-source/src/redis_big_segment_store.cpp Outdated
Comment thread libs/server-sdk-redis-source/src/redis_big_segment_store.cpp Outdated
Comment thread libs/server-sdk-dynamodb-source/src/dynamodb_big_segment_store.cpp Outdated
@beekld beekld requested a review from kinyoklion May 27, 2026 23:53
@beekld beekld merged commit 136aca7 into main May 28, 2026
47 checks passed
@beekld beekld deleted the beeklimt/SDK-2363 branch May 28, 2026 22:57
@github-actions github-actions Bot mentioned this pull request May 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants