Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Separate service for serving ATProto's subscribeRepos #630

Closed
Tracked by #381
snarfed opened this issue Sep 1, 2023 · 1 comment
Closed
Tracked by #381

Separate service for serving ATProto's subscribeRepos #630

snarfed opened this issue Sep 1, 2023 · 1 comment

Comments

@snarfed
Copy link
Owner

snarfed commented Sep 1, 2023

ATProto BGSes get all of our activity via websocket and the com.atproto.sync.subscribeRepos XRPC subscription. We need to send all new ATProto commits and blocks over that single websocket connection, for all users.

Our current architecture can't do that as is because it autoscales, so we have multiple instances, so not all writes go through a single instance that we can route to BGS subscribers in memory. GAE Standard's manual scaling has specific subdomains for each instance, but not Flex. 😐

Options:

  1. Run a separate Flex service just to serve subscribeRepos.
  2. Fan writes out to all instances to serve over subscribeRepos. How? Eventarc, Cloud Tasks etc are great, but I'm not sure how I'd make sure every instance consumes them and then also that they're marked as completed.
@snarfed snarfed changed the title Separate service for serving Separate service for serving ATProto's subscribeRepos Sep 1, 2023
@snarfed
Copy link
Owner Author

snarfed commented Sep 8, 2023

Progress here! New hub service, I'm enqueuing Cloud Tasks when I write new ATProto commits, and hub is receiving them and enqueuing them for subscribers. Testing is underway.

@snarfed snarfed closed this as completed Sep 14, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant