Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

incrementalXDS does not cancel existing watch #446

Closed
jparklab opened this issue Jun 9, 2021 · 8 comments
Closed

incrementalXDS does not cancel existing watch #446

jparklab opened this issue Jun 9, 2021 · 8 comments

Comments

@jparklab
Copy link

jparklab commented Jun 9, 2021

Cancel on watch object is called when watch does not exist in the map (https://github.com/envoyproxy/go-control-plane/blob/main/pkg/server/delta/v3/server.go#L168). (It is currently no-op since when 'ok' is false, watch will have the zero value. )
Cancel should be called when 'ok' is true since we expect cancel to be called when we find watch in the map.

// cancel existing watch to (re-)request a newer version
watch, ok := watches.deltaWatches[typeURL]
if !ok {
   ...
   // We must signal goroutine termination to prevent a race between the cancel closing the watch
   // and the producer closing the watch.
   watch.Cancel()
@alecholmez
Copy link
Contributor

alecholmez commented Jun 10, 2021

@snowp I think this is actually a problem that we might have to think about with regards to the nonce handling too.

SOTW will cancel if the nonces are equal:

responseNonce, seen := values.nonces[typeUrl]
if !seen || responseNonce == nonce {
	if cancel, seen := values.cancellations[typeUrl]; seen && cancel != nil {
		cancel()
	}
	values.cancellations[typeUrl] = s.cache.CreateWatch(req, values.responses)
}

But delta does the above and only looks for something that doesn't exist.

@snowp
Copy link
Contributor

snowp commented Jun 10, 2021

Wouldn't we just cancel the watch whenever we get a response? I'm not sure how nonce handling comes into play since its essentially not used in delta atm?

@alecholmez
Copy link
Contributor

alecholmez commented Jun 10, 2021

Ah yea we could do that, I was trying to rationalize around handling ACKs, etc... but that's a good point if the nonce really isn't used in delta atm then we can just cancel. I'll test this change

@jparklab
Copy link
Author

@alecholmez thanks for looking into the issue, and making a patch quickly. I have been waiting for the incremental XDS and it's great to have it.
While I was testing our controller with the latest code, I found that there are a few issues and summarized it in #448. It would be nice if you can take a look at it as well while making a patch for this issue.

@github-actions
Copy link

This issue has been automatically marked as stale because it has not had activity in the last 30 days. It will be closed in the next 7 days unless it is tagged "help wanted" or "no stalebot" or other activity occurs. Thank you for your contributions.

@github-actions github-actions bot added the stale label Jul 14, 2021
@alecholmez
Copy link
Contributor

@jparklab This is now patched in the server code. Let me know if this issue is resolved!

@jparklab
Copy link
Author

@alecholmez Thanks for the patch. I'll test it with out tests and post an update.

@alecholmez
Copy link
Contributor

alecholmez commented Aug 2, 2021

I'm going to close this with regards to the patch, but if the issue pops up again we can reopen

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants