Skip to content

Commit

Permalink
[receiver/jmx] Fix memory leak on shutdown (open-telemetry#32289)
Browse files Browse the repository at this point in the history
**Description:** <Describe what has changed.>
<!--Ex. Fixing a bug - Describe the bug and how this fixes the issue.
Ex. Adding a feature - Explain what this achieves.-->
The receiver was starting a goroutine that would run without being
stopped during shutdown. This changes the goroutine to be stopped during
shutdown.

`goleak` is also added as a part of this change.

**Link to tracking Issue:** <Issue number if applicable>
open-telemetry#30438

**Testing:** <Describe what testing was performed and which tests were
added.>
All existing tests are passing, as well as added `goleak` check.
  • Loading branch information
crobert-1 authored and rimitchell committed May 8, 2024
1 parent d8f102b commit 6b1e5b1
Show file tree
Hide file tree
Showing 4 changed files with 60 additions and 4 deletions.
27 changes: 27 additions & 0 deletions .chloggen/goleak_jmxrec.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
# Use this changelog template to create an entry for release notes.

# One of 'breaking', 'deprecation', 'new_component', 'enhancement', 'bug_fix'
change_type: bug_fix

# The name of the component, or a single word describing the area of concern, (e.g. filelogreceiver)
component: jmxreceiver

# A brief description of the change. Surround your text with quotes ("") if it needs to start with a backtick (`).
note: Fix memory leak during component shutdown

# Mandatory: One or more tracking issues related to the change. You can use the PR number here if no issue exists.
issues: [32289]

# (Optional) One or more lines of additional information to render under the primary note.
# These lines will be padded with 2 spaces and then inserted directly into the document.
# Use pipe (|) for multiline entries.
subtext:

# If your change doesn't affect end users or the exported elements of any package,
# you should instead start your pull request title with [chore] or use the "Skip Changelog" label.
# Optional: The change log or logs in which this entry should be included.
# e.g. '[user]' or '[user, api]'
# Include 'user' if the change is relevant to end users.
# Include 'api' if there is a change to a library API.
# Default: '[user]'
change_logs: []
1 change: 1 addition & 0 deletions receiver/jmxreceiver/go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ require (
go.opentelemetry.io/collector/receiver/otlpreceiver v0.97.1-0.20240409140257-792fac1b62d4
go.opentelemetry.io/otel/metric v1.24.0
go.opentelemetry.io/otel/trace v1.24.0
go.uber.org/goleak v1.3.0
go.uber.org/zap v1.27.0
)

Expand Down
14 changes: 14 additions & 0 deletions receiver/jmxreceiver/package_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
// Copyright The OpenTelemetry Authors
// SPDX-License-Identifier: Apache-2.0

package jmxreceiver

import (
"testing"

"go.uber.org/goleak"
)

func TestMain(m *testing.M) {
goleak.VerifyTestMain(m)
}
22 changes: 18 additions & 4 deletions receiver/jmxreceiver/receiver.go
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,7 @@ type jmxMetricReceiver struct {
otlpReceiver receiver.Metrics
nextConsumer consumer.Metrics
configFile string
cancel context.CancelFunc
}

func newJMXMetricReceiver(
Expand All @@ -54,6 +55,8 @@ func newJMXMetricReceiver(
func (jmx *jmxMetricReceiver) Start(ctx context.Context, host component.Host) error {
jmx.logger.Debug("starting JMX Receiver")

ctx, jmx.cancel = context.WithCancel(ctx)

var err error
jmx.otlpReceiver, err = jmx.buildOTLPReceiver()
if err != nil {
Expand Down Expand Up @@ -98,13 +101,19 @@ func (jmx *jmxMetricReceiver) Start(ctx context.Context, host component.Host) er
return err
}
go func() {
for range jmx.subprocess.Stdout { // nolint
// ensure stdout/stderr buffer is read from.
// these messages are already debug logged when captured.
for {
select {
case <-ctx.Done():
return
case <-jmx.subprocess.Stdout:
// ensure stdout/stderr buffer is read from.
// these messages are already debug logged when captured.
continue
}
}
}()

return jmx.subprocess.Start(context.Background())
return jmx.subprocess.Start(ctx)
}

func (jmx *jmxMetricReceiver) Shutdown(ctx context.Context) error {
Expand All @@ -114,6 +123,11 @@ func (jmx *jmxMetricReceiver) Shutdown(ctx context.Context) error {
jmx.logger.Debug("Shutting down JMX Receiver")
subprocessErr := jmx.subprocess.Shutdown(ctx)
otlpErr := jmx.otlpReceiver.Shutdown(ctx)

if jmx.cancel != nil {
jmx.cancel()
}

removeErr := os.Remove(jmx.configFile)
if subprocessErr != nil {
return subprocessErr
Expand Down

0 comments on commit 6b1e5b1

Please sign in to comment.