-
Notifications
You must be signed in to change notification settings - Fork 2.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
refactor: Remove time.After
from any Loops
#14265
Conversation
49f394d
to
d751950
Compare
Could we add a link to the article in the commit msg about time.After not being GC'd? It will help the future's readers. |
d751950
to
ac98143
Compare
test-me-please |
pkg/inctimer/inctimer.go
Outdated
func New() (*IncTimer, func() bool) { | ||
t := time.NewTimer(time.Nanosecond) | ||
return &IncTimer{ | ||
t: t, | ||
}, t.Stop | ||
} | ||
|
||
// After returns a channel that will fire after | ||
// the specified duration. | ||
func (it *IncTimer) After(d time.Duration) <-chan time.Time { | ||
// We cannot call reset on an expired timer, | ||
// so we need to stop it and drain it first. | ||
// See https://golang.org/pkg/time/#Timer.Reset for more details. | ||
if !it.t.Stop() { | ||
// It could be that the channel was read already | ||
select { | ||
case <-it.t.C: | ||
default: | ||
} | ||
} | ||
it.t.Reset(d) | ||
return it.t.C | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm not sure why we need a wrapper. Why isn't the in-place fix mentioned in the medium article sufficient?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
+1
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The solution suggested in the article is actually partially incorrect. In order to reset a timer safely (as suggested in the article), you have to also stop the timer and clear the channel.
I think this wrapper is useful because it basically boils the following code:
t := time.NewTimer(time.Nanosecond)
defer t.Stop()
for {
...
if !t.Stop() {
// It could be that the channel was read already
select {
case <-t.C:
default:
}
}
t.Reset(someInterval)
select {
...
case <- t.C:
}
}
to:
t, stop := inctimer.New()
defer stop()
for {
...
select {
...
case <- t.After(someInterval):
}
}
In one case I believe the After
function is called even more than once, so that would have either required the creation of two separate timers or the same hard reset logic repeated twice above the select.
I find this wrapper approach to be a lot more elegant and easier to use.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The code looks good, but I'm also wondering the same as Aditi. Do we need a wrapper? If so, that would be helpful context in the commit msg and in the godoc of the inctimer
package itself, for why it's necessary.
pkg/inctimer/inctimer.go
Outdated
func New() (*IncTimer, func() bool) { | ||
t := time.NewTimer(time.Nanosecond) | ||
return &IncTimer{ | ||
t: t, | ||
}, t.Stop | ||
} | ||
|
||
// After returns a channel that will fire after | ||
// the specified duration. | ||
func (it *IncTimer) After(d time.Duration) <-chan time.Time { | ||
// We cannot call reset on an expired timer, | ||
// so we need to stop it and drain it first. | ||
// See https://golang.org/pkg/time/#Timer.Reset for more details. | ||
if !it.t.Stop() { | ||
// It could be that the channel was read already | ||
select { | ||
case <-it.t.C: | ||
default: | ||
} | ||
} | ||
it.t.Reset(d) | ||
return it.t.C | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
+1
ac98143
to
f2b0cc7
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Awesome, looks great! Thanks for adding the extra context. I think it makes sense to have the wrapper.
Thinking a bit into the future, I wonder how we can prevent uses of time.After
without requiring code reviewers to remember. Help me understand, should we be replacing all uses of time.After
with IncTimer
going forward? If so, then we could do similar to what we have for lock.Mutex
where we don't allow the use of the upstream sync.Mutex
via a script (contrib/scripts/lock-check.sh
) and this is enforced via our Golang GH Actions job.
fc9d0db
to
ed93de6
Compare
53eb47a
to
afccab3
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, thanks 🚀 Just one question/suggestion, does not need a second review.
Really cool work on the customvet
check 🎉
32b4dc2
to
af80505
Compare
`time.After` is not garbage collected until after its channel fires once. In situations where it is called repeatedely this *can* lead to significant memory build up. Instead of creating a new timer object for every iteration, a single timer, with a hard-reset mechanism, is used in situations where `time.After` is being called repeatedely. See the following article for details: https://medium.com/@oboturov/golang-time-after-is-not-garbage-collected-4cbc94740082 pkg: Add inctimer The new inctimer package is useful for boiling down the monotonous taks of "hard" resetting the reused timer. In order to reuse a timer to solve the `time.After` garbage collection problem the timer must be stopped, have its channel cleared (or not, if it was already read), and then reset. Duplicating this code all over the cilium code base seems superfluous, a small package that mimics the more terse `time.After` method seems more desireable. Signed-off-by: Nate Sweet <nathanjsweet@pm.me>
af80505
to
16d810e
Compare
test-me-please |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🚀
This seems to have broken CI, see e.g. https://jenkins.cilium.io/job/cilium-ginkgo/job/cilium/job/master/6273
Looks like the CI runs failed on this PR already 😞 |
This reverts commit 7d39243 / PR #14265 Reason for revert: This broke the build in the master pipeline, e.g. https://jenkins.cilium.io/job/cilium-ginkgo/job/cilium/job/master/6273/ 10:58:36 runtime: contrib/scripts/custom-vet-check.sh 10:58:36 runtime: # cd .; git clone -- https://github.com/cilium/customvet /home/vagrant/go/src/github.com/cilium/customvet 10:58:36 runtime: fatal: could not create work tree dir '/home/vagrant/go/src/github.com/cilium/customvet': Permission denied 10:58:36 runtime: package github.com/cilium/customvet: exit status 128 10:58:41 runtime: flag provided but not defined: -timeafter.ignore 10:58:41 runtime: usage: go vet [-n] [-x] [-vettool prog] [build flags] [vet flags] [packages] 10:58:41 runtime: Run 'go help vet' for details. 10:58:41 runtime: Run 'go tool vet -help' for the vet tool's flags. 10:58:41 runtime: Makefile:506: recipe for target 'precheck' failed 10:58:41 runtime: make: *** [precheck] Error 2 Apparently this failure was already present on the CI runs on PR #14265 itself. Signed-off-by: Tobias Klauser <tklauser@distanz.ch>
I've sent a revert for this to unbreak CI: #14371 |
This reverts commit 7d39243 / PR #14265 Reason for revert: This broke the build in the master pipeline, e.g. https://jenkins.cilium.io/job/cilium-ginkgo/job/cilium/job/master/6273/ 10:58:36 runtime: contrib/scripts/custom-vet-check.sh 10:58:36 runtime: # cd .; git clone -- https://github.com/cilium/customvet /home/vagrant/go/src/github.com/cilium/customvet 10:58:36 runtime: fatal: could not create work tree dir '/home/vagrant/go/src/github.com/cilium/customvet': Permission denied 10:58:36 runtime: package github.com/cilium/customvet: exit status 128 10:58:41 runtime: flag provided but not defined: -timeafter.ignore 10:58:41 runtime: usage: go vet [-n] [-x] [-vettool prog] [build flags] [vet flags] [packages] 10:58:41 runtime: Run 'go help vet' for details. 10:58:41 runtime: Run 'go tool vet -help' for the vet tool's flags. 10:58:41 runtime: Makefile:506: recipe for target 'precheck' failed 10:58:41 runtime: make: *** [precheck] Error 2 Apparently this failure was already present on the CI runs on PR #14265 itself. Signed-off-by: Tobias Klauser <tklauser@distanz.ch>
time.After
is not garbage collected until afterits channel fires once. In situations where it is called
repeatedely this can lead to significant memory build up.
Instead of creating a new timer object for every iteration,
a single timer, with a hard-reset mechanism, is used in
situations where
time.After
is being called repeatedely.Signed-off-by: Nate Sweet nathanjsweet@pm.me
Fixes: #14219