Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pprof modularization #24114

Merged
merged 5 commits into from Mar 13, 2023

Conversation

pippolo84
Copy link
Member

Modularize the pprof related code into a cell. The cell is used by the agent, the operator and the clustermesh-apiserver to register the pprof HTTP handlers and export runtime profiling data.

The already present pprof.Enable function is marked as deprecated, but not removed, since hubble-relay still depends on it and that is not yet modularized to use hive and cells.

Differently from the agent and the clustermesh-apiserver, the operator prefixes the pprof related flags with operator-. To reuse the same cell, we use an operator specific config type, in order to map the equivalent fields to the operator-specific flag names. Also, when importing the cell, an additional call to cell.Config is requested to register the default config type (for agent and clustermesh-apiserver) or the operator-specific one.

The PR also adds an integration-style unit test to verify the correct registration of the pprof HTTP handlers when the cell is used.

Fixes: #24048

@pippolo84 pippolo84 requested review from a team as code owners March 1, 2023 13:04
@pippolo84 pippolo84 requested review from thorn3r and squeed March 1, 2023 13:04
@maintainer-s-little-helper maintainer-s-little-helper bot added the dont-merge/needs-release-note-label The author needs to describe the release impact of these changes. label Mar 1, 2023
@pippolo84 pippolo84 added release-note/misc This PR makes changes that have no direct user impact. area/modularization labels Mar 1, 2023
@maintainer-s-little-helper maintainer-s-little-helper bot removed the dont-merge/needs-release-note-label The author needs to describe the release impact of these changes. label Mar 1, 2023
@pippolo84 pippolo84 force-pushed the pr/pippolo84/pprof-modularization branch from 4702a80 to eaf0612 Compare March 1, 2023 13:27
@pippolo84
Copy link
Member Author

Integration tests are failing due to a mismatch between the Go version used by the workflow and the one set in go.mod following this commit.
I'll rebase onto master after this PR will be merged, it should fix this.

pkg/pprof/cell_test.go Outdated Show resolved Hide resolved
pkg/pprof/cell.go Outdated Show resolved Hide resolved
operator/cmd/flags.go Show resolved Hide resolved
operator/option/config.go Show resolved Hide resolved
// PprofAddressAPIServer is the default value for pprof in the clustermesh-apiserver
PprofAddressAPIServer = "localhost"

// PprofPortAgent is the default value for pprof in the agent
PprofPortAgent = 6060

// PprofPortAgent is the default value for pprof in the operator
// PprofPortOperator is the default value for pprof in the operator
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove and just do the defaults inline where we define the default config? Makes more sense to me that way, but happy to hear if someone has an argument for keeping pkg/defaults.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately those defaults are used also by bugtool. Considered that, it probably makes sense to leave them in pkg/defaults.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As a trade-off, I ended up moving the default values from pkg/defaults to the binary specific option package (pkg/option for the agent, operator/option for the operator and so on).

pkg/pprof/cell_test.go Outdated Show resolved Hide resolved
@pippolo84 pippolo84 force-pushed the pr/pippolo84/pprof-modularization branch 2 times, most recently from 89c6ca1 to 8b39491 Compare March 1, 2023 19:07
Copy link
Member

@YutaroHayakawa YutaroHayakawa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ClusterMesh part looks good. Thanks!

@pippolo84 pippolo84 force-pushed the pr/pippolo84/pprof-modularization branch from 8b39491 to 605778c Compare March 2, 2023 08:50
@pippolo84
Copy link
Member Author

Uploaded a minor change (and related test) to avoid starting a net.Listener in case of pprof disabled by config.

@pippolo84 pippolo84 force-pushed the pr/pippolo84/pprof-modularization branch from 605778c to 003c692 Compare March 2, 2023 09:17
@pippolo84
Copy link
Member Author

Updated cmdref documentation.

@pippolo84 pippolo84 force-pushed the pr/pippolo84/pprof-modularization branch from 003c692 to c1d9dbd Compare March 2, 2023 14:15
@pippolo84 pippolo84 force-pushed the pr/pippolo84/pprof-modularization branch 2 times, most recently from f88383a to 940927e Compare March 3, 2023 12:12
@pippolo84 pippolo84 requested a review from joamaki March 3, 2023 13:48
pkg/pprof/cell.go Outdated Show resolved Hide resolved
pkg/pprof/cell.go Outdated Show resolved Hide resolved
pkg/pprof/cell.go Outdated Show resolved Hide resolved
@@ -17,6 +17,8 @@ import (
var log = logging.DefaultLogger.WithField(logfields.LogSubsys, "pprof")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove and use injected logrus.FieldLogger

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is used by pprof.Enable, that we still need for hubble-relay. Being outside of the cell lifecycle, we won't have any injected logrus.FieldLogger here.
But this should be gone once hubble-relay will be refactored to use the pprof cell.

pkg/pprof/cell.go Outdated Show resolved Hide resolved
@pippolo84 pippolo84 force-pushed the pr/pippolo84/pprof-modularization branch from 940927e to fa2edbd Compare March 8, 2023 15:31
@pippolo84
Copy link
Member Author

/test

Modularize pprof related code into a cell. The cell is used by the
agent, the operator and the clustermesh-apiserver.

The already present pprof.Enable function is marked as deprecated.

Differently from the agent and the clustermesh-apiserver, the operator
prefixes the pprof related flags with `operator-`.  To reuse the same
cell, we use an operator specific config type, in order to map the
equivalent fields to the operator-specific flag names.  Also, when
importing the cell, an additional call to cell.Config is requested to
register the default config type (for agent and clustermesh-apiserver)
or the operator-specific one.

Signed-off-by: Fabio Falzoi <fabio.falzoi@isovalent.com>
Add two unit-style integration tests to verify the correct registration
of the pprof HTTP handlers. Specifically, the first one check that no
listeners is started when pprof is disabled and the other one exercises
the `debug/pprof/heap` endpoint when pprof is enabled.  Both tests
verify that the cell is not leaking any goroutines after the hive stops.

Signed-off-by: Fabio Falzoi <fabio.falzoi@isovalent.com>
This commit regenerates the cmdref documentation files following the
modularization of pprof.

Signed-off-by: Fabio Falzoi <fabio.falzoi@isovalent.com>
Fix a typo (`Private` -> `Provide`) in ProvidePrivate function inline
documentation.

Signed-off-by: Fabio Falzoi <fabio.falzoi@isovalent.com>
Move pprof related options default values (address and port) from
pkg/defaults to binary specific packages, like pkg/option for the agent,
operator/option for the operator and clustermesh-apiserver/option for
the clustermesh server.
This is done to avoid polluting the global defaults package with
defaults that are not really global but binary specific.

Signed-off-by: Fabio Falzoi <fabio.falzoi@isovalent.com>
@pippolo84 pippolo84 force-pushed the pr/pippolo84/pprof-modularization branch from fa2edbd to 342998a Compare March 8, 2023 16:41
@pippolo84
Copy link
Member Author

/test

@pippolo84
Copy link
Member Author

test-runtime hit #23495

@pippolo84
Copy link
Member Author

CI is happy (minus an unrelated flake as reported in previous comment), reviews are in, marking ready to merge.

@pippolo84 pippolo84 added the ready-to-merge This PR has passed all tests and received consensus from code owners to merge. label Mar 9, 2023
@jrajahalme jrajahalme merged commit ca1cfb5 into cilium:master Mar 13, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/modularization ready-to-merge This PR has passed all tests and received consensus from code owners to merge. release-note/misc This PR makes changes that have no direct user impact.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Modularization: pprof
5 participants