feat: Make snapshots optional with cursor-based filtering (HYP-148)#70
feat: Make snapshots optional with cursor-based filtering (HYP-148)#70
Conversation
…Socket protocol Implement HYP-148 to make snapshots optional and support cursor-based filtering: - Add with_snapshot, after, and snapshot_limit fields to Subscription struct - Add seq field to Frame for live update cursor tracking - Implement EntityCache::get_after() for filtering entities by cursor - Update WebSocket server to respect with_snapshot flag - Update projector to include seq in Frame from SlotContext Clients can now: - Skip initial snapshot with with_snapshot: false - Resume from specific point with after cursor (_seq value) - Limit snapshot size with snapshot_limit - Track position via seq field in live updates
Update Rust SDK for HYP-148 protocol changes:
- Add with_snapshot, after, and snapshot_limit fields to Subscription
- Add builder methods: with_snapshot(), after(), with_snapshot_limit()
- Add seq field to Frame struct for cursor tracking
- Update ConnectionManager to send new subscription fields
- Maintain backward compatibility with defaults
Usage:
let sub = Subscription::new("Game/list")
.with_snapshot(false)
.after("123:000000000042")
.with_snapshot_limit(100);
Update TypeScript and React SDKs for HYP-148 protocol changes:
- Add withSnapshot, after, and snapshotLimit to Subscription and WatchOptions
- Add seq field to EntityFrame for cursor tracking
- Add new fields to ViewHookOptions and ListParamsBase for React hooks
- Connection automatically includes new fields via spread operator
Usage:
// Skip snapshot
view.watch({ withSnapshot: false })
// Resume from cursor
view.watch({ after: "123:000000000042", snapshotLimit: 100 })
// React hooks
useListView(Game.list, { after: lastSeq, withSnapshot: false })
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
…ptions Update SDKs to fully expose HYP-148 subscription options: Rust SDK: - Add with_snapshot, after, and snapshot_limit fields to UseBuilder - Add builder methods: with_snapshot(), after(), with_snapshot_limit() - Similar updates to WatchBuilder and RichWatchBuilder React SDK: - Update useListView to pass withSnapshot, after, snapshotLimit through - Update useStateView to pass withSnapshot, after, snapshotLimit through - Ensure all subscription options are included in useEffect and refresh dependencies
Ensure with_snapshot, after, and snapshot_limit are properly passed from builders through to actual WebSocket subscriptions: - Update UseStream, EntityStream, and RichEntityStream to accept and store new fields - Update all builder poll_next implementations to pass new fields when creating streams - Update connection.ensure_subscription_with_opts calls to use actual values instead of None - Ensure UseBuilder, WatchBuilder, and RichWatchBuilder all properly wire up the fields
Greptile SummaryThis PR adds cursor-based filtering and optional snapshot delivery to the WebSocket subscription system across the Rust server, Rust SDK, and TypeScript/React SDKs. The implementation is generally well-structured: the server-side Key issues found:
Confidence Score: 3/5
Important Files Changed
Sequence DiagramsequenceDiagram
participant C as Client (TS/Rust SDK)
participant WS as WebSocket Server
participant Cache as EntityCache
participant Bus as Bus Manager
C->>WS: subscribe { view, withSnapshot, after, snapshotLimit }
alt withSnapshot === false
WS->>Bus: get_or_create_bus()
WS->>Bus: rx.borrow_and_update() (mark seen)
Note over WS: No snapshot sent
else withSnapshot === true (default)
WS->>Bus: get_or_create_bus()
alt after cursor provided
WS->>Cache: get_after(view, cursor, snapshotLimit)
Cache-->>WS: Vec<(key, entity)> sorted ascending by _seq
else no cursor
WS->>Cache: get_all(view)
Cache-->>WS: Vec<(key, entity)> (unordered)
alt snapshotLimit set
Note over WS: sort descending, truncate to limit ⚠️ stays descending
end
end
WS->>WS: filter by matches_key
WS->>C: SnapshotFrame (batched)
WS->>Bus: rx.borrow_and_update()
end
loop Live Updates
Bus-->>WS: entity update (with seq from slot_context)
WS->>C: Frame { op, key, data, seq }
end
|
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
Group 5 optional subscription parameters into SubscriptionOptions struct to reduce ensure_subscription_with_opts from 8 args to 3.
The snapshot_limit subscription parameter was only being honored when an after cursor was set. When after was None, get_all() was called without any limit, returning the full unlimited snapshot. This fix adds the limit parameter to get_all() and passes snapshot_limit in both otel and non-otel code paths for List/Append mode.
Fixes field name mismatch between TypeScript client and Rust server. The TypeScript Subscription interface uses camelCase (withSnapshot, snapshotLimit), but the Rust struct used snake_case. Added #[serde(rename_all = "camelCase")] attribute so the server properly recognizes these fields. Updated tests to use camelCase JSON keys.
The snapshot_limit was being applied to the unfiltered cache before the matches_key filter, causing clients to receive fewer entities than the limit when filtering was in use. Now the limit is applied after filtering so clients get up to snapshot_limit matching entities.
When only snapshot_limit is set (no cursor), entities were being taken from DashMap in non-deterministic order. Now we sort by _seq descending to ensure deterministic results, matching the behavior when an after cursor is provided. This applies to both the otel and non-otel code paths for List/Append mode subscriptions.
When both `after` (cursor) and `snapshot_limit` are set, the descending sort was overwriting the ascending order from `get_after`, causing the wrong subset to be returned (most recent N instead of next N). - Only apply descending sort when there's no cursor - Pass snapshot_limit to get_after to avoid loading unnecessary data - Remove redundant .take() call since limiting now happens in get_after
…he and WebSocket handlers
1 participant
Summary
Implements HYP-148: Make snapshots optional with cursor-based filtering for WebSocket connections.
Changes
Server (
rust/hyperstack-server)Subscriptionstruct with:with_snapshot: Option<bool>- Skip initial snapshot when false (defaults to true for backward compatibility)after: Option<String>- Cursor to resume from specific_seqpointsnapshot_limit: Option<usize>- Limit number of entities in snapshotseq: Option<String>field toFramestruct for live update cursor trackingEntityCache::get_after()method for efficient cursor-based filteringwith_snapshotflag across all modes (State, List, Append)Rust SDK (
rust/hyperstack-sdk)Subscription:.with_snapshot(bool)- Control snapshot inclusion.after(str)- Set resume cursor.with_snapshot_limit(usize)- Set snapshot limitseqfield toFramestructTypeScript/React SDKs
SubscriptionandWatchOptions:withSnapshot?: booleanafter?: stringsnapshotLimit?: numberseq?: stringtoEntityFrameViewHookOptions,ListParamsBase)useListViewanduseStateViewhooks to pass new fields throughUsage Examples
Rust SDK
Using Subscription directly (for advanced use cases):
Using view builders (standard API):
TypeScript/React
Standard view API:
React hooks:
Backward Compatibility
✅ Fully backward compatible
with_snapshotdefaults totrue, existing clients receive snapshots as before