Not a compact JWS error

[baptiste-bonnaudet]

Hey I have an issue delegating authentication to k8s, I configured my backend as follow:

K8s CA cert

/tmp/example.crt contains the K8s cert to authenticate to the API, from my ~/.kube/config

tokenreview JWT

ACCOUNT_TOKEN=$(kubectl -n default get secret kubectl -n default get serviceaccount vault-tokenreview -o jsonpath='{.secrets[0].name}' -o jsonpath='{.data.token}' | base64 --decode)

vault write -f auth/kube-example/config kubernetes_host=$GKE_URL kubernetes_ca_cert=@/tmp/example.crt token_reviewer_jwt=$ACCOUNT_TOKEN

vault write auth/kube-example/role/myapplication \
    bound_service_account_names=myapplication \
    bound_service_account_namespaces=myapplication \
    policies=myapplication \
    ttl=48h \
    max_ttl=48h

Created before with the config:

apiVersion: v1
kind: ServiceAccount
metadata:
  name: vault-tokenreview
  namespace: default
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
  name: vault-tokenreview-binding
  namespace: default
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: system:auth-delegator
subjects:
- kind: ServiceAccount
  name: vault-tokenreview
  namespace: default

Pod SA

curl --request POST --data '{"jwt": "$(cat /var/run/secrets/kubernetes.io/serviceaccount/token)", "role": "myapplication"}' https://vault.example.com/v1/auth/kube-example/login
{"errors":["not a compact JWS"]}

I tried playing with both jwts and base64 encoding but no luck and this is driving me crazy. Am I missing something?

[baptiste-bonnaudet]

Also reading the config does not print the reviewer token.

vault read -format=json auth/kube-example/config
{
  "request_id": "c33f0370-e735-41bd-a4df-1bfcecc039fc",
  "lease_id": "",
  "lease_duration": 0,
  "renewable": false,
  "data": {
    "kubernetes_ca_cert": "-----BEGIN CERTIFICATE-----
...
-----END CERTIFICATE-----\n",
    "kubernetes_host": "https://99.99.99.99",
    "pem_keys": []
  },
  "warnings": null
}

Also on the pod:

cat /var/run/secrets/kubernetes.io/serviceaccount/token | base64 -d
{"alg":"RS256","typ":"JWT"}{"iss":"kubernetes/serviceaccount","kubernetes.io/serviceaccount/namespace":"myapplication","kubernetes.io/serviceaccount/secret.name":"myapplication-token-22zbk","kubernetes.io/serviceaccount/service-account.name":"myapplication","kubernetes.io/serviceaccount/service-account.uid":"6f431400-4342-11e8-9c86-41010a8e01a5","sub":"system:serviceaccount:myapplication:myapplication"}!Ɩvk�}hSG����:�;��k����^)�)�b�ݍ!Μ�%d�_�&�;�0��S�ǔ���u�b�'l&C^���ʌ��r�U��!�42�i��>׼0�y�Ê6{���"�6t����~/m�K����0�qPZ����f�
                 ��Z�?q��7���Xo
                               ��օ�L8�\Z�ne�0��闿lkϴ*�
����#Y�6Tu4f��p��ɢ��dX'x?�&�-L�ߪ"��7m)base64: truncated base64 input

[briankassouf]

Single quotes won't interpolate the cat command, instead try

"{\"jwt\": \"$(cat /var/run/secrets/kubernetes.io/serviceaccount/token)\", \"role\": \"myapplication\"}"

[kakshay7]

Hi did you found a solution to this problem?

[kakshay7]

If I'm pasting the jwt token in the curl command then it is working, but when I'm substituting the value of jwt then its giving me same error.

Please let me know how you solved it.
Thanks

[nitishm]

@briankassouf This problem still persists. The solution recommended doesn't seem to resolve the issue.
I have not been able to get past https://learn.hashicorp.com/vault/identity-access-management/vault-agent-k8s#step-3-verify-the-kubernetes-auth-method-configuration

~ # curl --request POST --data '{"jwt": "$(cat /var/run/secrets/kubernetes.io/service
account/token", "role": "example"}' $VAULT_ADDR/v1/auth/kubernetes/login | jq
{
 "errors": [
   "not a compact JWS"
 ]
}

I have tried this with the vault-auth serviceaccount (as per the example), ensuring that the kubernetes/auth is configured with the same JWT token, and yet it fails.

Can I get some assistance with this ?

[calvn]

@nitishm You're using single quotes in your data payload, which won't interpolate the cat command execution as Brian mentioned above. You'll need to use double quotes and escape the JSON strings.

Give this a try:

curl --request POST --data "{\"jwt\": \"$(cat \/var\/run\/secrets\/kubernetes.io\/service\r\naccount\/token\", \"role\": \"example\"}" $VAULT_ADDR/v1/auth/kubernetes/login | jq

A more concrete example of the behavior:

› echo '"$(pwd)"'
"$(pwd)"

vs

› echo "\"$(pwd)\""
"/tmp"

[nitishm]

Thanks @calvn . It seems to definitely be a problem with the json string format and the cat interpolations. I moved the body into a payload.json file and changed the request to ,
curl --request POST --data @payload.json $VAULT_ADDR/v1/auth/kubernetes/login | jq
, and that worked.

[abdennour]

Thank you @calvn for the hint. It saves my day.
However, you just need to surrounnd $(cat ) by single quotes again , instead of escaping the double quotes everywhere.

So "$(cat ...)" must be "'$(cat ...)'"

curl --request POST \
  --data '{"jwt": "'$(cat /var/run/secrets/kubernetes.io/serviceaccount/token)'", "role": "myapplication"}'  \
    https://vault.example.com/v1/auth/kube-example/login

[ListentoNews]

@calvn can you tell me how can achieve this from within terraform?

i have tried this

resource "vault_kubernetes_auth_backend_config" "kubernetes_auth_config" {
  backend=vault_auth_backend.kubernetes.path
  kubernetes_host="https://$${KUBERNETES_PORT_443_TCP_ADDR}:443"
  kubernetes_ca_cert="@/var/run/secrets/kubernetes.io/serviceaccount/ca.crt"
  token_reviewer_jwt="\"$(cat /var/run/secrets/kubernetes.io/serviceaccount/token)\""
}

but it gives the very same error not a compact JWS

[Jasstkn]

@calvn can you tell me how can achieve this from within terraform?

i have tried this

resource "vault_kubernetes_auth_backend_config" "kubernetes_auth_config" {
  backend=vault_auth_backend.kubernetes.path
  kubernetes_host="https://$${KUBERNETES_PORT_443_TCP_ADDR}:443"
  kubernetes_ca_cert="@/var/run/secrets/kubernetes.io/serviceaccount/ca.crt"
  token_reviewer_jwt="\"$(cat /var/run/secrets/kubernetes.io/serviceaccount/token)\""
}

but it gives the very same error not a compact JWS

Do you use decoded value? It works for me once I decoded value via base64 -d

[gabrielrinaldi]

@ListentoNews did you figure this out? I am having the exact same issue

[Jasstkn]

Maybe this will help hashicorp/terraform-provider-vault#793