Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[receiver/k8sobjects] Fix memory leak caused by pull mode's interval ticker #31919

Merged

Conversation

crobert-1
Copy link
Member

@crobert-1 crobert-1 commented Mar 22, 2024

Description:

The k8sobjects receiver was starting a ticker when in pull mode, that would tick every set interval. This ticker needs to be stopped during shutdown, so I added functionality to cancel the context during shutdown, properly stopping the ticker.

The original intention here was to add goleak, but due to a bug in internal/k8stest it's failing on the end to end test. This change still fixes a bug, so I believe it's worth merging even without goleak at this point. Note that this change does reduce the number and frequency of goleak failures.

Testing:
All existing tests are passing.

@crobert-1 crobert-1 marked this pull request as draft March 22, 2024 23:43
@crobert-1
Copy link
Member Author

I'll investigate failing goleak test and mark as Ready to Review when I've resolved it.

@TylerHelmuth
Copy link
Member

I wonder how related this is to #31644

@crobert-1
Copy link
Member Author

I wonder how related this is to #31644

I suspect this is actually frequency of #31047, since the only test that's failing is using k8stest, and the leaked goroutines have the same stack.

@crobert-1 crobert-1 marked this pull request as ready for review March 28, 2024 23:21
@crobert-1
Copy link
Member Author

I've updated the issue description. goleak is still failing on the end to end test due to a bug in internal/k8stest. I believe this is still worth merging to fix the memory leak detected, even if goleak isn't included.

@TylerHelmuth TylerHelmuth merged commit 9d8b55c into open-telemetry:main Apr 2, 2024
144 checks passed
@github-actions github-actions bot added this to the next release milestone Apr 2, 2024
@crobert-1 crobert-1 deleted the goleak_k8sobjectsreceiver branch April 2, 2024 21:12
ycombinator pushed a commit to ycombinator/opentelemetry-collector-contrib that referenced this pull request Apr 9, 2024
…ticker (open-telemetry#31919)

**Description:** <Describe what has changed.>
<!--Ex. Fixing a bug - Describe the bug and how this fixes the issue.
Ex. Adding a feature - Explain what this achieves.-->
The k8sobjects receiver was starting a ticker when in pull mode, that
would tick every set interval. This ticker needs to be stopped during
shutdown, so I added functionality to cancel the context during
shutdown, properly stopping the ticker.

The original intention here was to add `goleak`, but due to a bug in
`internal/k8stest` it's failing on the end to end test. This change
still fixes a bug, so I believe it's worth merging even without `goleak`
at this point. Note that this change does reduce the number and
frequency of goleak failures.

**Testing:** <Describe what testing was performed and which tests were
added.>
All existing tests are passing.
rimitchell pushed a commit to rimitchell/opentelemetry-collector-contrib that referenced this pull request May 8, 2024
…ticker (open-telemetry#31919)

**Description:** <Describe what has changed.>
<!--Ex. Fixing a bug - Describe the bug and how this fixes the issue.
Ex. Adding a feature - Explain what this achieves.-->
The k8sobjects receiver was starting a ticker when in pull mode, that
would tick every set interval. This ticker needs to be stopped during
shutdown, so I added functionality to cancel the context during
shutdown, properly stopping the ticker.

The original intention here was to add `goleak`, but due to a bug in
`internal/k8stest` it's failing on the end to end test. This change
still fixes a bug, so I believe it's worth merging even without `goleak`
at this point. Note that this change does reduce the number and
frequency of goleak failures.

**Testing:** <Describe what testing was performed and which tests were
added.>
All existing tests are passing.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants