fix: readyz never true of multiple gRPC syncs are used - #1985
Merged
toddbaert merged 1 commit intoJul 24, 2026
Conversation
The gRPC sync guarded its one-shot readiness with a package-level sync.Once. Because that Once is shared across every grpc.Sync in the process, only the first instance whose stream connects ran the closure and set its own ready flag; all other instances stayed not-ready forever, and IsReady() kept returning false for them. flagd builds a distinct grpc.Sync per configured grpc source and the runtime only reports ready once every sync is ready, so any flagd with two or more grpc sources never passed /readyz even after all streams were healthy. Set g.ready directly in handleFlagSync (the assignment is idempotent), which makes readiness per instance, matching the other sync implementations. Add a regression test that starts two independent grpc.Sync instances and asserts both become ready. Signed-off-by: Anas Khan <83116240+anxkhn@users.noreply.github.com>
✅ Deploy Preview for polite-licorice-3db33c canceled.
|
📝 WalkthroughWalkthroughThe ChangesReadiness Flag Fix and Test
Estimated code review effort: 2 (Simple) | ~10 minutes Sequence Diagram(s)sequenceDiagram
participant Test as Test_MultipleSyncsBecomeReady
participant Sync1 as Sync Instance 1
participant Sync2 as Sync Instance 2
participant Server as bufconn gRPC Server
Test->>Sync1: Sync(ctx1, dataSync)
Test->>Sync2: Sync(ctx2, dataSync)
Sync1->>Server: stream request (handleFlagSync)
Server-->>Sync1: SyncFlags payload
Sync1->>Sync1: set ready = true
Sync2->>Server: stream request (handleFlagSync)
Server-->>Sync2: SyncFlags payload
Sync2->>Sync2: set ready = true
Test->>Sync1: IsReady()
Test->>Sync2: IsReady()
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
toddbaert
approved these changes
Jul 24, 2026
3 tasks
Merged
toddbaert
pushed a commit
that referenced
this pull request
Jul 27, 2026
🤖 I have created a release *beep* *boop* --- <details><summary>flagd: 0.16.1</summary> ## [0.16.1](flagd/v0.16.0...flagd/v0.16.1) (2026-07-27) ### 🐛 Bug Fixes * add server ReadTimeouts, update security deps ([#1980](#1980)) ([0e1e8b3](0e1e8b3)) * handle invalid selector errors ([#1976](#1976)) ([409a62e](409a62e)) * **security:** update module google.golang.org/grpc to v1.82.1 [security] ([#2005](#2005)) ([4281adf](4281adf)) * **security:** update vulnerability-updates [security] ([#1984](#1984)) ([2a5a6b6](2a5a6b6)) ### ✨ New Features * **sync:** configurable gRPC keepalive enforcement on sync server ([#1999](#1999)) ([19e57b9](19e57b9)) </details> <details><summary>flagd-proxy: 0.9.7</summary> ## [0.9.7](flagd-proxy/v0.9.6...flagd-proxy/v0.9.7) (2026-07-27) ### 🐛 Bug Fixes * add server ReadTimeouts, update security deps ([#1980](#1980)) ([0e1e8b3](0e1e8b3)) * **security:** update module google.golang.org/grpc to v1.82.1 [security] ([#2005](#2005)) ([4281adf](4281adf)) * **security:** update vulnerability-updates [security] ([#1984](#1984)) ([2a5a6b6](2a5a6b6)) </details> <details><summary>core: 0.16.1</summary> ## [0.16.1](core/v0.16.0...core/v0.16.1) (2026-07-27) ### 🐛 Bug Fixes * **certreloader:** re-check reload condition under write lock ([#1994](#1994)) ([86489da](86489da)) * handle invalid selector errors ([#1976](#1976)) ([409a62e](409a62e)) * readyz never true of multiple gRPC syncs are used ([#1985](#1985)) ([29833e8](29833e8)) * **security:** update module google.golang.org/grpc to v1.82.1 [security] ([#2005](#2005)) ([4281adf](4281adf)) * **security:** update vulnerability-updates [security] ([#1984](#1984)) ([2a5a6b6](2a5a6b6)) * **sync:** avoid send on closed channel in fileinfo watcher Close ([#1992](#1992)) ([a7339ea](a7339ea)) * **sync:** handle non-string YAML map keys in config parsing ([#1990](#1990)) ([3c72e6f](3c72e6f)) * **sync:** re-establish file watch after delete and restore ([#2001](#2001)) ([d434adf](d434adf)) * **sync:** synchronize readiness state ([#2006](#2006)) ([317a7e0](317a7e0)) ### ✨ New Features * improve ETag and body hash support to blob/http sync ([#1991](#1991)) ([1e210d4](1e210d4)) ### 🧹 Chore * fix flaky test ([7ff137d](7ff137d)) </details> --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). Signed-off-by: OpenFeature Bot <109696520+openfeaturebot@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.



Because that
Onceis shared across everygrpc.Syncin the process, only thefirst instance whose stream connects runs the closure and sets its own
readyflag. Every other
grpc.Syncinstance stays not-ready forever, so itsIsReady()keeps returning false.flagd builds a distinct
grpc.Syncper configured grpc source(
SyncBuilder.newGRPCincore/pkg/sync/builder/syncbuilder.go), and theruntime only reports ready once every sync is ready (
Runtime.isReadyinflagd/pkg/runtime/runtime.goreturns false if anySyncs[].IsReady()isfalse). The net effect is that any flagd configured with two or more
grpc://sync sources never passes
/readyz, even after all of its gRPC streams arehealthy, so the pod stays NotReady.
The fix
Set
g.ready = truedirectly inhandleFlagSyncand drop the package-levelonce(and its now-unusedsyncimport alias). The assignment is idempotent,so this is a no-op for the single-source case and makes readiness per instance
for the multi-source case. This also matches the other sync implementations:
the blob, file, and http syncs already set a plain
ready booldirectly, andkubernetes tracks it per instance with an
atomic.Bool. The package-levelOncewas the outlier.The write still happens-before the update pushed onto the
dataSyncchannel,so no new concurrent read path is introduced and the change stays race-clean.
Tests
Adds
Test_MultipleSyncsBecomeReady, which starts two independentgrpc.Syncinstances (each backed by its own bufconn server), drives each one, and asserts
both report
IsReady() == true. The test synchronizes by receiving a payloadfrom the sync channel before reading
IsReady(), so it establishes ahappens-before edge and is race-free by construction rather than by timing.
Before this change the test fails on the second instance
("sync 1 should be ready after its stream connects"); after it, both pass.
Readiness was previously untested in this file, so this also closes that
coverage gap.
Verified with:
go build ./...go test -race -covermode=atomic -cover -short ./pkg/sync/grpc/(fromcore/)golangci-lint run ./pkg/sync/grpc/...