Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RabbitMQ Scaler exposes sensible information (HTTP Host) in logs in case of failure #2040

Closed
JorTurFer opened this issue Aug 19, 2021 · 5 comments · Fixed by #2041
Closed
Labels
bug Something isn't working

Comments

@JorTurFer
Copy link
Member

JorTurFer commented Aug 19, 2021

Report

I was configuring KEDA with RabbitMQ Scaler in new workloads and configured the auth wrong. By chance I have checked the metrics server logs and I found this:

E0819 17:36:36.435338       1 provider.go:100] keda_metrics_adapter/provider "msg"="error getting metric for scaler" "error"="error inspecting rabbitMQ: error requesting rabbitMQ API status: 401 Unauthorized, response: {\"error\":\"not_authorised\",\"reason\":\"Login failed\"}, from: https://USER_DELETED_BY_ME:PASSWORD_DELETED_BY_ME@HOST_DELETED_BY_ME/api/queues/zltapp_shared/saas"  "scaledObject.Name"="one-rabbit-sidecar" "scaledObject.Namespace"="zltapp-one" "scaler"={}
E0819 17:36:36.435388       1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"No matching metrics found for rabbitmq-saas"}
E0819 17:36:51.553647       1 provider.go:100] keda_metrics_adapter/provider "msg"="error getting metric for scaler" "error"="error inspecting rabbitMQ: error requesting rabbitMQ API status: 401 Unauthorized, response: {\"error\":\"not_authorised\",\"reason\":\"Login failed\"}, from: https://USER_DELETED_BY_ME:PASSWORD_DELETED_BY_ME@HOST_DELETED_BY_ME/api/queues/zltapp_shared/saas"  "scaledObject.Name"="one-rabbit-sidecar" "scaledObject.Namespace"="zltapp-one" "scaler"={}
E0819 17:36:51.553687       1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"No matching metrics found for rabbitmq-saas"}
E0819 17:37:06.653856       1 provider.go:100] keda_metrics_adapter/provider "msg"="error getting metric for scaler" "error"="error inspecting rabbitMQ: error requesting rabbitMQ API status: 401 Unauthorized, response: {\"error\":\"not_authorised\",\"reason\":\"Login failed\"}, from: https://USER_DELETED_BY_ME:PASSWORD_DELETED_BY_ME@HOST_DELETED_BY_ME/api/queues/zltapp_shared/saas"  "scaledObject.Name"="one-rabbit-sidecar" "scaledObject.Namespace"="zltapp-one" "scaler"={}
E0819 17:37:06.653903       1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"No matching metrics found for rabbitmq-saas"}

As you can see, the HOST is totally exposed

Expected Behavior

KEDA should protect confitential data

Actual Behavior

RabbitMQ Scaler exposes the HOST as part of the error message and it is sent to the log

Steps to Reproduce the Problem

  1. Deploy a ScaledObject that uses RabbitMQ and introduce a mistake in the HOST
  2. Check the logs

Logs from KEDA metrics server

E0819 17:36:36.435338       1 provider.go:100] keda_metrics_adapter/provider "msg"="error getting metric for scaler" "error"="error inspecting rabbitMQ: error requesting rabbitMQ API status: 401 Unauthorized, response: {\"error\":\"not_authorised\",\"reason\":\"Login failed\"}, from: https://USER_DELETED_BY_ME:PASSWORD_DELETED_BY_ME@HOST_DELETED_BY_ME/api/queues/zltapp_shared/saas"  "scaledObject.Name"="one-rabbit-sidecar" "scaledObject.Namespace"="zltapp-one" "scaler"={}
E0819 17:36:36.435388       1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"No matching metrics found for rabbitmq-saas"}
E0819 17:36:51.553647       1 provider.go:100] keda_metrics_adapter/provider "msg"="error getting metric for scaler" "error"="error inspecting rabbitMQ: error requesting rabbitMQ API status: 401 Unauthorized, response: {\"error\":\"not_authorised\",\"reason\":\"Login failed\"}, from: https://USER_DELETED_BY_ME:PASSWORD_DELETED_BY_ME@HOST_DELETED_BY_ME/api/queues/zltapp_shared/saas"  "scaledObject.Name"="one-rabbit-sidecar" "scaledObject.Namespace"="zltapp-one" "scaler"={}
E0819 17:36:51.553687       1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"No matching metrics found for rabbitmq-saas"}
E0819 17:37:06.653856       1 provider.go:100] keda_metrics_adapter/provider "msg"="error getting metric for scaler" "error"="error inspecting rabbitMQ: error requesting rabbitMQ API status: 401 Unauthorized, response: {\"error\":\"not_authorised\",\"reason\":\"Login failed\"}, from: https://USER_DELETED_BY_ME:PASSWORD_DELETED_BY_ME@HOST_DELETED_BY_ME/api/queues/zltapp_shared/saas"  "scaledObject.Name"="one-rabbit-sidecar" "scaledObject.Namespace"="zltapp-one" "scaler"={}
E0819 17:37:06.653903       1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"No matching metrics found for rabbitmq-saas"}

KEDA Version

2.4.0

Kubernetes Version

1.21

Platform

No response

Scaler Details

RabbitMQ

Anything else?

I have tried and this error only happens with HTTP protocol. With AMQP the ScaledObject isn't able to create the HPA and the metric server doesn't expose any info

@JorTurFer JorTurFer added the bug Something isn't working label Aug 19, 2021
@JorTurFer
Copy link
Member Author

I think that this has an easy solution, replacing the password in the errors to ensure that we protect that kind of data.
I will create a PR with the fix ASAP

@JorTurFer JorTurFer changed the title RabbitMQ Scaler exposes sensible information in logs in case of failure RabbitMQ Scaler exposes sensible information (HTTP Host) in logs in case of failure Aug 19, 2021
@zroubalik
Copy link
Member

@JorTurFer good catch, thx :)

@JorTurFer
Copy link
Member Author

JorTurFer commented Aug 20, 2021

@zroubalik I was lucky. A typo in my release pipeline has discovered this 🤣 🤣

@tomkerkhove
Copy link
Member

Thanks!

@JorTurFer
Copy link
Member Author

you're welcome :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants