-
Notifications
You must be signed in to change notification settings - Fork 2.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Assign specific, unique ports for pprof (Agent, Operator, Hubble Relay) #15441
Assign specific, unique ports for pprof (Agent, Operator, Hubble Relay) #15441
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's worth mentioning that both the cilium-agent and cilium-operator
share the Cilium configmap so if a port was provided, then it will be
used for both, and will cause the same port collision issue. The user
must edit the cilium-operator deployment or the cilium-agent daemonset
directly in order to pass different ports to each component to properly
workaround the port collision.
Why not use different default ports?
That could work too. The reason I didn't change the defaults is because pprof is well-known to be at 6060. What do others think? |
For gops we also use different default ports, see #14329 and last commit documented them in Documentation/operations/system_requirements.rst. |
5461b0c
to
85a8030
Compare
@christarazi I don't think this is needed forf Hubble Relay as it run in its own netns. EDIT: I'm fine adding the option to Relay, just wondering if that would be OK to keep the default 6060. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
👍
@kaworu Please see this thread - https://cilium.slack.com/archives/C2B917YHE/p1616610511066100 |
Closed and reopened as the GitHub actions were stuck. test-me-please |
This allows users to enable pprof for both cilium-agent and cilium-operator on the same machine without running into: ``` level=warning msg="Unable to serve pprof API" error="listen tcp 127.0.0.1:6060: bind: address already in use" subsys=pprof ``` It also provides users with the ability to specify the port via a flag. Note that this flag is not exposed via Helm because it is a flag mainly used for debugging, and shouldn't need to be exposed fully to the user. A developer / advanced user can just as easily edit the daemonset or the deployment object to append an extra arg. Also, given the fact that the ports are unique across the components, that reduces the need to provide a knob via Helm. To avoid port collisions, the Agent, Operator, and Hubble Relay are all assigned the following specific ports respectively: 6060, 6061, 6062. Signed-off-by: Chris Tarazi <chris@isovalent.com>
Following the commit to (re)assign the pprof default port, update the documentation with the new information. Signed-off-by: Chris Tarazi <chris@isovalent.com>
85a8030
to
dc5521f
Compare
test-me-please |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @aditighag!
This allows users to enable pprof for both cilium-agent and
cilium-operator on the same machine without running into:
It also provides users with the ability to specify the port via a flag.
Note that this flag is not exposed via Helm because it is a flag mainly
used for debugging, and shouldn't need to be exposed fully to the user.
A developer / advanced user can just as easily edit the daemonset or the
deployment object to append an extra arg. Also, given the fact that the
ports are unique across the components, that reduces the need to
provide a knob via Helm.
To avoid port collisions, the Agent, Operator, and Hubble Relay are all
assigned the following specific ports respectively: 6060, 6061, 6062.
Signed-off-by: Chris Tarazi chris@isovalent.com