Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SELF_HOSTED_SETUP: hack/self-hosted omits "kid" field. STS Fails validation #12

Closed
Jacobious52 opened this issue Sep 19, 2019 · 3 comments · Fixed by #13

Comments

@Jacobious52
Copy link

@Jacobious52 Jacobious52 commented Sep 19, 2019

What happened:
For https://github.com/aws/amazon-eks-pod-identity-webhook/blob/master/SELF_HOSTED_SETUP.md

./hack/self-hosted/main.go outputs a JWK that STS can't validate. It is missing the empty "kid": "", field in keys.json as the serialisation of the Go program omits empty fields. STS then fails to validate the JWT (see below). I needed to manually add the empty "kid" field to the keys.json for validation to work.

Error output:

~ # aws sts assume-role-with-web-identity \
>  --role-arn $AWS_ROLE_ARN \
>  --role-session-name mh9test \
>  --web-identity-token file://$AWS_WEB_IDENTITY_TOKEN_FILE \
>  --duration-seconds 1000 > /tmp/irp-cred.txt
An error occurred (InvalidIdentityToken) when calling the AssumeRoleWithWebIdentity operation: Couldn't retrieve verification key from your identity provider,  please reference AssumeRoleWithWebIdentity documentation for requirements

keys.json:

{
    "keys": [
        {
            "use": "sig",
            "kty": "RSA",
            "alg": "RS256",
            "n": "(redacted)",
            "e": "AQAB"
        }
    ]
}

What you expected to happen:

./hack/self-hosted/main.go should explicitly create an empty "kid" field.

keys.json:

{
    "keys": [
        {
            "use": "sig",
            "kty": "RSA",
            "kid": "",
            "alg": "RS256",
            "n": "(redacted)",
            "e": "AQAB"
        }
    ]
}

How to reproduce it (as minimally and precisely as possible):

Anything else we need to know?:

Environment:

  • AWS Region: us-west-2
  • EKS Platform version (if using EKS, run aws eks describe-cluster --name <name> --query cluster.platformVersion): N/A
  • Kubernetes version (if using EKS, run aws eks describe-cluster --name <name> --query cluster.version): v1.13.10
  • Webhook Version: N/A
@siwyd

This comment has been minimized.

Copy link

@siwyd siwyd commented Sep 19, 2019

I noticed this as well yesterday... I'm using jq for now to insert the empty kid when generating the keys.json:

go run ./hack/self-hosted/main.go -key /etc/kubernetes/pki/sa.pub | jq '.keys[0].kid = ""' > jwks.json
@siwyd

This comment has been minimized.

Copy link

@siwyd siwyd commented Sep 19, 2019

And actually, in order to be compatible for when k8s starts to fill in the kid, I think this is the safest keys.json for now:

go run ./hack/self-hosted/main.go -key /etc/kubernetes/pki/sa.pub -kid "$(sha1sum /etc/kubernetes/pki/sa.pub | awk '{print $1}')" | jq '.keys += [.keys[0]] | .keys[1].kid = ""'

This will generate a keys.json like the following:

{
  "keys": [
    {
      "use": "sig",
      "kty": "RSA",
      "kid": "aefcb29bcefe60d5cc97e89a90c671d92a5412b9",
      "alg": "RS256",
      "n": "uJk_nzshm7ujO50hniR8kM2lhi0nkWLw2sHzKN1rBXTuhS4-KKSfUhLRFaLTektN1QGmR2f1QGOc8reM6fzlDX7t0GeSTmE1SHFd0klbxVOQoa-LbNOhM5yO4nHrYe3QAMETLz_bptCCgA2i4jjK4cA4xcE8cJkBjs1rw6xHW_vIAJ0aJpd5Uu695hPDiUb2DHaeFbogA0iEk9678nz0GeVLInOvI6vuBNjal_KYdI-_NNBSj3kuTXxSSOrYOljkkI-wcn9Ai19-cQyDkUiZPV18p9T3Qe79MgEHLqUDfcIQ5BSZ_mmG1ffM01YDMD-T57LXwn-P6sKXjrasOxi8Cw",
      "e": "AQAB"
    },
    {
      "use": "sig",
      "kty": "RSA",
      "kid": "",
      "alg": "RS256",
      "n": "uJk_nzshm7ujO50hniR8kM2lhi0nkWLw2sHzKN1rBXTuhS4-KKSfUhLRFaLTektN1QGmR2f1QGOc8reM6fzlDX7t0GeSTmE1SHFd0klbxVOQoa-LbNOhM5yO4nHrYe3QAMETLz_bptCCgA2i4jjK4cA4xcE8cJkBjs1rw6xHW_vIAJ0aJpd5Uu695hPDiUb2DHaeFbogA0iEk9678nz0GeVLInOvI6vuBNjal_KYdI-_NNBSj3kuTXxSSOrYOljkkI-wcn9Ai19-cQyDkUiZPV18p9T3Qe79MgEHLqUDfcIQ5BSZ_mmG1ffM01YDMD-T57LXwn-P6sKXjrasOxi8Cw",
      "e": "AQAB"
    }
  ]
}

A PR was merged into k8s recently which will cause k8s to fill in the kid with that value, allowing rotation of keys. I can't find the PR anymore though, @micahhausler knows perhaps? Anyway, that keys.json should allow your keys to still be validated when suddenly tokens are issued with the non-empty kid.

@micahhausler

This comment has been minimized.

Copy link
Member

@micahhausler micahhausler commented Sep 19, 2019

Ah this is annoying, square/go-jose omits kid from the json if it is empty. There are changes upstream to add KID for 1.16+, but I think the solution from @siwyd is the safest/best for now

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.