Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Expose K8S API response for failure #6551

Open
patrick-stephens opened this issue Dec 14, 2022 · 0 comments
Open

Expose K8S API response for failure #6551

patrick-stephens opened this issue Dec 14, 2022 · 0 comments
Assignees

Comments

@patrick-stephens
Copy link
Contributor

patrick-stephens commented Dec 14, 2022

Is your feature request related to a problem? Please describe.

The response from the K8S API when we attempt to query metadata is only logged at debug level.
This means if you have a misconfiguration (e.g. RBAC does not allow us to query things with the service account) then you just get a generic "not ok" message logged.
Stepping up to debug level shows the actual response saying RBAC is not configured in this case so is incredibly helpful: in the example below you can see the response message explaining the failure immediately at debug level - by default you just get the warn message with no details.

[2022/12/08 15:53:38] [ info] [filter:kubernetes:kubernetes.0] testing connectivity with API server...
[2022/12/08 15:53:38] [debug] [filter:kubernetes:kubernetes.0] Send out request to API Server for pods information
[2022/12/08 15:53:38] [debug] [http_client] not using http_proxy for header
[2022/12/08 15:53:38] [debug] [http_client] server kubernetes.default.svc:443 will close connection #23
[2022/12/08 15:53:38] [debug] [filter:kubernetes:kubernetes.0] Request (ns=default, pod=calyptia-cluster-logging-316c-dcr7d) http_do=0, HTTP Status: 403
[2022/12/08 15:53:38] [debug] [filter:kubernetes:kubernetes.0] HTTP response
{"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"pods \"calyptia-cluster-logging-316c-dcr7d\" is forbidden: User \"system:serviceaccount:default:default\" cannot get resource \"pods\" in API group \"\" in the namespace \"default\"","reason":"Forbidden","details":{"name":"calyptia-cluster-logging-316c-dcr7d","kind":"pods"},"code":403}

[2022/12/08 15:53:38] [ warn] [filter:kubernetes:kubernetes.0] could not get meta for POD calyptia-cluster-logging-316c-dcr7d

Describe the solution you'd like
Log the response, particularly for failure cases, at info or above level.

Describe alternatives you've considered

Additional context

Deploying FB into a cluster without setting up the relevant permissions for K8S API querying took a while to figure out.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant