Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(pubsublite): improve handling of backend unavailability #3846

merged 5 commits into from Mar 25, 2021


Copy link

@tmdiep tmdiep commented Mar 22, 2021

  • Define ErrBackendUnavailable to allow users to detect and handle backend unavailability during publish and subscribe.
  • Increase settings to very high values:
    • PublishSettings.Timeout and ReceiveSettings.Timeout to 1 week.
    • PublishSettings.BufferedByteLimit to 10 GB.
  • Add examples for handling publish and subscribe errors.
- define ErrBackendUnavailable
- mitigate stream initialization delays
@tmdiep tmdiep force-pushed the unavailable_error branch from 51f23f8 to d554104 Mar 24, 2021
@@ -82,7 +82,7 @@ var DefaultPublishSettings = PublishSettings{
DelayThreshold: 10 * time.Millisecond,
CountThreshold: 100,
ByteThreshold: 1e6,
Timeout: 60 * time.Minute,
Timeout: 72 * time.Hour,

This feels a little arbitrary. Is there an InfiniteFuture() equivalent on Go? Same comment on the ReceiveSettings.

Copy link
Contributor Author

@tmdiep tmdiep Mar 25, 2021

There isn't a defined infinite duration. It's an int64, so we can compute the max value, however I wanted to avoid depending on implementation details.

I changed this to 1 week, as it would be concerning if a client wasn't able to connect within that time. The original 3 days was to allow clients to cover a weekend.

pubsublite/internal/wire/rpc.go Show resolved Hide resolved
pubsublite/internal/wire/streams.go Outdated Show resolved Hide resolved
pubsublite/internal/wire/streams.go Outdated Show resolved Hide resolved
@tmdiep tmdiep merged commit db31457 into googleapis:master Mar 25, 2021
3 checks passed
@tmdiep tmdiep deleted the unavailable_error branch Mar 25, 2021
@tmdiep tmdiep restored the unavailable_error branch Jun 26, 2021
@tmdiep tmdiep deleted the unavailable_error branch Jun 28, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

2 participants