Skip to content

Conversation

@marytlf
Copy link
Contributor

@marytlf marytlf commented Oct 30, 2025

Issue: rancher/rancher#48484

Problem

The Rancher webhook can generate a large volume of http: TLS handshake error logs when a client attempts to connect with a missing or invalid certificate. While these are often non-critical, they can overwhelm the logs, making it difficult to debug genuine issues. The current implementation doesn't provide a way to suppress these logs without impacting other important error messages. The community has requested a solution to reduce log noise caused by these specific errors, as detailed in issue #48484.

Solution

http: TLS handshake error messages are downgraded from error to debug level. This means the messages will only appear in the logs if the CATTLE_DEBUG environment variable is set to true

Test Scenarios

Scenario 1: CATTLE_DEBUG = false
Result: No TLS handshake error messages are logged. The messages are logged at the debug level but are not displayed because CATTLE_DEBUG is false.

Scenario 2: CATTLE_DEBUG = true

Result: TLS handshake error messages are logged at the debug level, as expected.

Log Example: level=debug msg="2025/08/21 18:03:25 http: TLS handshake error from 172.31.7.103:40900: EOF"

How to reproduce and force the error:
NOTE: The error happens when using rke2, not k3s.

Set the environment variables on rancher-webhook deployment (kubectl -n cattle-system edit deploy rancher-webhook)
CATTLE_DEBUG = "true"/"false"
Login on Rancher UI (or create/delete a new user)
Check the logs from rancher-webhook (kubectl logs -l app=rancher-webhook -n cattle-system)

@marytlf marytlf requested a review from a team as a code owner October 30, 2025 18:33
@marytlf marytlf merged commit b307a3d into rancher:release/v0.7 Oct 30, 2025
2 checks passed
@marytlf
Copy link
Contributor Author

marytlf commented Nov 3, 2025

backport v2.11: rancher/rancher#52537

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants