-
Notifications
You must be signed in to change notification settings - Fork 52
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
grpc.code=Unknown err="error making mount request: failed to login: 403 errors #112
Comments
Can you paste your audit logs from vault server. Same timeline? |
Hi @adrafiq , The error I got from vault server is the following. I am only testing with one pod and there are many errors repeating like below.
I am not sure what I missed, I am still pretty new in kubernetes. Appreciate your help a lot.
|
Hi @tirvan, by chance was your vault pod deleted and recreated at some point? With K8s 1.21, the vault pod will probably be using an ephemeral projected service token, which only lasts the lifetime of the pod. So that might be why the vault error indicates that the token in the k8s auth config is no longer valid. You could try re-applying that config to see if that helps. Otherwise you can try using the service account's default token when setting up the k8s auth method. Its issuer will be the default of VAULT_SECRET_NAME=$(kubectl get secrets -n development --output=json | jq -r '.items[].metadata | select(.name|startswith("hashicorp-vault-token-")).name')
kubectl get secret -n development $VAULT_SECRET_NAME --output='go-template={{ .data.token }}' | base64 --decode |
Hi @tvoran , Thank you very much. You are right about vault pod got recreated at some point, after checking the age, the vault pod is indeed got killed/restarted as the csi provider is older in age. After re-applying the config, everything works great and I am able to see my secrets now. Again, appreciate so much for your help. Regards, |
hello, i am getting the same 403 error .. here is my audit log
and from my csi driver pods i am getting the following error
FYI : my vault cluster is in another EKS cluster i have followed the documentation here https://developer.hashicorp.com/vault/tutorials/kubernetes/kubernetes-external-vault#configure-kubernetes-authentication |
Hi,
I am using Azure managed kubernetes with version 1.21.2, followed the instruction given here step by steps.
https://www.hashicorp.com/blog/retrieve-hashicorp-vault-secrets-with-kubernetes-csi
I double check many times the steps but don't seems to miss anything. I have enabled debug mode for csi and here are the errors.
Service account "internal-app" is already created. The error might seems obvious but I totally no clue what did I miss, been googling for few days but could not find the solution to my problem.
Appreciate your help.
My secret store
The text was updated successfully, but these errors were encountered: