Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
Troubleshooting docs for k8s default token expiring #2274
Rough notes below.
Basically ... we'd see an error like 'get nodes' not authorized. To fix it, I delete the default token that the pipelien pod is using, delete the pipeline pod, and it gets restarted w the updated token. Not sure how/why k8s default token would get out of sync like this.
when spinning up a local cluster
i get an error causing CLB on pipeline pods
stating they dont have creds to do getnodes
i suspect my DNS mod to kube launch
just now checked on it
relaunching and making sure I use default DNS setting
so validated w default DNS:
digging into hte k8s sys apiserver pod
see some errors:
E0705 20:11:15.141701 1 authentication.go:58] Unable to authenticate the request due to an error: [invalid bearer token, [invalid bearer token, crypto/rsa: verification error]]
and i can't quite get to the apiserver start command to see the flags
maybe i could check syslog on that pod
but i do see the known_tokens.csv
i17-07-06[13:57:14]:pachyderm:130$sudo find / -name known_tokens.csv > knowntokensfind.log
but getting the default token ... i don't see anything that matches up ...
17-07-06[14:01:30]:pachyderm:0$kc get secret default-token-7qfx3 -o yaml
this thread suggests deleting the token helps:
unclear how it would've changed // how my signing auth would've changed
yup! that worked