[backport v2.11] webhook tls error and bump dynamiclistener v0.6.4-rc.2 #1153
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Issue: rancher/rancher#48484
Problem
The Rancher webhook can generate a large volume of http: TLS handshake error logs when a client attempts to connect with a missing or invalid certificate. While these are often non-critical, they can overwhelm the logs, making it difficult to debug genuine issues. The current implementation doesn't provide a way to suppress these logs without impacting other important error messages. The community has requested a solution to reduce log noise caused by these specific errors, as detailed in issue #48484.
Solution
http: TLS handshake errormessages are downgraded from error to debug level. This means the messages will only appear in the logs if the CATTLE_DEBUG environment variable is set to trueTest Scenarios
Scenario 1:
CATTLE_DEBUG = falseResult: No TLS handshake error messages are logged. The messages are logged at the debug level but are not displayed because CATTLE_DEBUG is false.
Scenario 2:
CATTLE_DEBUG = trueResult: TLS handshake error messages are logged at the debug level, as expected.
Log Example: level=debug msg="2025/08/21 18:03:25 http: TLS handshake error from 172.31.7.103:40900: EOF"
How to reproduce and force the error:
NOTE: The error happens when using rke2, not k3s.
Set the environment variables on rancher-webhook deployment (kubectl -n cattle-system edit deploy rancher-webhook)
CATTLE_DEBUG = "true"/"false"
Login on Rancher UI (or create/delete a new user)
Check the logs from rancher-webhook (kubectl logs -l app=rancher-webhook -n cattle-system)