Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: avoid scheduling an instance refresh if the context is done #491

Merged
merged 1 commit into from
Jan 24, 2024

Conversation

jomaresch
Copy link
Contributor

Hello,

I discovered that closing the instance will cause an infinite loop in the scheduleRefresh function.
Closing the Instance will cancel the context, which causes an error in the Limiter:

ctx, cancel := context.WithTimeout(i.ctx, i.refreshTimeout)
defer cancel()
err := i.l.Wait(ctx)
if err != nil {

Every error here will trigger a new refresh with 0 sec delay:
// if failed, scheduled the next refresh immediately
if r.err != nil {
i.next = i.scheduleRefresh(0)
// If the latest result is bad, avoid replacing the

This refresh will also fail because of the canceled context.
The result is high resource consumption, once the instance has been closed, because of this infinite loop.

As a fix I propose to check the context earlier and don't schedule a new refresh once the context is closed.

I didn't write a test, because i did't find a way to properly test the absence of the loop. But I'm open for suggestions.

@jomaresch jomaresch requested review from cthumuluru, nancynh and a team as code owners January 20, 2024 14:43
Copy link

google-cla bot commented Jan 20, 2024

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

Copy link

🤖 I detect that the PR title and the commit message differ and there's only one commit. To use the PR title for the commit history, you can use Github's automerge feature with squashing, or use automerge label. Good luck human!

-- conventional-commit-lint bot
https://conventionalcommits.org/

@enocom
Copy link
Member

enocom commented Jan 22, 2024

Nice find @jomaresch. If you can sign the CLA, we're happy to look at this. Otherwise, we can fix it in a separate PR.

@jomaresch
Copy link
Contributor Author

@enocom I already signed the CLA. The failing check, which indicated the missing CLA, are also gone.
Am I missing anything?

@enocom enocom added the tests: run Label to trigger Github Action tests. label Jan 22, 2024
@github-actions github-actions bot removed the tests: run Label to trigger Github Action tests. label Jan 22, 2024
@enocom enocom self-requested a review January 23, 2024 04:33
@enocom
Copy link
Member

enocom commented Jan 23, 2024

Looking at this closely, I see there's a deeper problem. Yes, we need to move the context check up as you've done here, but we also need to stop the timers in Close. We probably also need to add a context check to ForceRefresh.

Would you be interested in landing this change, or prefer that I take it over?

See #493

@enocom enocom force-pushed the fix-instance-close branch 4 times, most recently from 8429586 to a21c87f Compare January 24, 2024 18:55
@enocom enocom added the tests: run Label to trigger Github Action tests. label Jan 24, 2024
@github-actions github-actions bot removed the tests: run Label to trigger Github Action tests. label Jan 24, 2024
@enocom enocom force-pushed the fix-instance-close branch 2 times, most recently from 9f0d8be to acf8f83 Compare January 24, 2024 19:03
Copy link
Member

@enocom enocom left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added the timer change and moved the context check up to the top of the function.

Thanks for sending this! We'll get it into the next release.

enocom added a commit to GoogleCloudPlatform/cloud-sql-go-connector that referenced this pull request Jan 24, 2024
@enocom enocom added the tests: run Label to trigger Github Action tests. label Jan 24, 2024
@github-actions github-actions bot removed the tests: run Label to trigger Github Action tests. label Jan 24, 2024
@enocom enocom changed the title Fix infinite loop on shutdown fix: avoid scheduling an instance refresh if the context is done Jan 24, 2024
enocom added a commit to GoogleCloudPlatform/cloud-sql-go-connector that referenced this pull request Jan 24, 2024
@enocom enocom added the tests: run Label to trigger Github Action tests. label Jan 24, 2024
@github-actions github-actions bot removed the tests: run Label to trigger Github Action tests. label Jan 24, 2024
enocom added a commit to GoogleCloudPlatform/cloud-sql-go-connector that referenced this pull request Jan 24, 2024
@enocom enocom merged commit 42c8ae3 into GoogleCloudPlatform:main Jan 24, 2024
22 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants