[kong] Don't recreate service account #31
Conversation
|
Bother, that was a convenient hack. I think we should still provide a path for users to still have a managed ServiceAccount if they enable the controller or PSP on an existing installation, but those conversions are hopefully rare enough that we should not need to handle them automatically in the standard chart. We can excise the existing hook ServiceAccount from In the FAQ entry, we can include the existing temp user block and instruct users how to add it temporarily. They'll need to pull down a local copy of the chart, append the block to Revisiting some of my prior comments, I think that's still better than either of the other alternatives. Using a temporary user name is possible, but users presumably want their PSP to apply to migrations Jobs going forward after they enable it, and a temporary SA complicates that. We could also add an additional option in values.yaml for creating a temporary SA and instruct users to only use it in the one case, but I think that'd only be warranted if we expected this to happen often. |
yasn77
force-pushed
the
fix_serviceaccount_deletion
branch
from
9cb453a
to
5cb7b02
Compare
@rainest I agree. It is probably better to leave this as documentation rather than get the code to do it for the user. In most cases, if a user is aware of the issue, it isn't a big issue to create a service account and use that |
yasn77
force-pushed
the
fix_serviceaccount_deletion
branch
from
5cb7b02
to
e531f53
Compare
There was a problem hiding this comment.
Diff to move instructions to FAQ with example error and instructions for temporary modifications (along with some other minor fixes) at https://gist.github.com/rainest/0c2c918decc08d785e19467e5ca465a5
@yasn77 could you apply that?
Helm upgrade recreates service account, which in turn recreates the secret token
for the service account. This breaks current deployments. Below are symptoms:
Before Upgrade:
--------------
```
kubectl -n kong get serviceaccount -l app.kubernetes.io/name=kong-app -o jsonpath='{.items[].secrets}'
[map[name:kong-tst-kong-app-token-k7qv2]]
```
```
kubectl -n kong get pod kong-tst-kong-app-cc9787b65-fwzrd -o json | jq '.spec.containers[] | select(.name == "ingress-controller").volumeMounts'
[
{
"mountPath": "/var/run/secrets/kubernetes.io/serviceaccount",
"name": "kong-tst-kong-app-token-k7qv2",
"readOnly": true
}
]
```
(Note: Token is the same)
After Upgrade:
-------------
```
kubectl -n kong get serviceaccount -l app.kubernetes.io/name=kong-app -o jsonpath='{.items[].secrets}'
[map[name:kong-tst-kong-app-token-rrzfh]]
```
```
kubectl -n kong get pod kong-tst-kong-app-cc9787b65-fwzrd -o json | jq '.spec.containers[] | select(.name == "ingress-controller").volumeMounts'
[
{
"mountPath": "/var/run/secrets/kubernetes.io/serviceaccount",
"name": "kong-tst-kong-app-token-k7qv2",
"readOnly": true
}
]
```
(Note: Tokens are different)
From the logs (after upgrade):
Tiller:
------
```
[tiller] 2020/01/31 17:01:46 deleting pre-upgrade hook kong-tst-kong-app for release kong-tst due to "before-hook-creation" policy
[kube] 2020/01/31 17:01:46 Starting delete for "kong-tst-kong-app" ServiceAccount
[kube] 2020/01/31 17:01:46 Waiting for 60 seconds for delete to be completed
[kube] 2020/01/31 17:01:48 building resources from manifest
[kube] 2020/01/31 17:01:48 creating 1 resource(s)
[kube] 2020/01/31 17:01:48 Watching for changes to ServiceAccount kong-tst-kong-app with timeout of 5m0s
[kube] 2020/01/31 17:01:48 Add/Modify event for kong-tst-kong-app: ADDED
```
From IngresController:
----------------------
```
E0131 18:09:30.368424 1 reflector.go:125] pkg/mod/k8s.io/client-go@v0.0.0-20190819141724-e14f31a72a77/tools/cache/reflector.go:98: Failed to list *v1beta1.Ingress: Unauthorized
E0131 18:09:31.206518 1 reflector.go:125] pkg/mod/k8s.io/client-go@v0.0.0-20190819141724-e14f31a72a77/tools/cache/reflector.go:98: Failed to list *v1.KongCredential: Unauthorized
E0131 18:09:31.371432 1 reflector.go:125] pkg/mod/k8s.io/client-go@v0.0.0-20190819141724-e14f31a72a77/tools/cache/reflector.go:98: Failed to list *v1beta1.Ingress: Unauthorized
E0131 18:09:32.214002 1 reflector.go:125] pkg/mod/k8s.io/client-go@v0.0.0-20190819141724-e14f31a72a77/tools/cache/reflector.go:98: Failed to list *v1.KongCredential: Unauthorized
E0131 18:09:32.379662 1 reflector.go:125] pkg/mod/k8s.io/client-go@v0.0.0-20190819141724-e14f31a72a77/tools/cache/reflector.go:98: Failed to list *v1beta1.Ingress: Unauthorized
```
This is due to a pre hook that creates the service account and then deletes
(then is created again).
Stop this behaviour and document manual upgrades.
yasn77
force-pushed
the
fix_serviceaccount_deletion
branch
from
e531f53
to
1a70365
Compare
Done 👍 |
Remove the temporary ServiceAccount created via hook during pre-upgrade migrations. This was previously in place to automatically handle users upgrading from a configuration that did not use a ServiceAccount to one that did. However, this temporary account overwrites credentials for the real account if left in the template. Add instructions for temporarily modifying the template to re-add this hook to the FAQ, along with examples of the error that will appear.
|
@yasn77 Thank you for your contribution. Please fill in the contributor form to claim your Kong swag: https://github.com/Kong/kong/blob/master/CONTRIBUTING.md#contributor-t-shirt. |
See issue #30