Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Canary uninstall removes Istio service account. #30441

Closed
mike-oakley opened this issue Jan 27, 2021 · 2 comments
Closed

Canary uninstall removes Istio service account. #30441

mike-oakley opened this issue Jan 27, 2021 · 2 comments
Assignees
Labels
area/upgrade Issues related to upgrades lifecycle/automatically-closed Indicates a PR or issue that has been closed automatically. lifecycle/stale Indicates a PR or issue hasn't been manipulated by an Istio team member for a while
Milestone

Comments

@mike-oakley
Copy link

mike-oakley commented Jan 27, 2021

Bug description
I was trying out the canary install method. With two installs on a cluster, one 1.7.6 version installed with istioctl without a revision and one with a revision specified (version 1.8.2), deleting the revisioned install seems to remove the service account associated globally with Istio. The relevant lines in the istioctl output when running the uninstall are:

  Removed ServiceAccount:istio-system:istio-reader-service-account.
  Removed ServiceAccount:istio-system:istiod-service-account.
  Removed RoleBinding:istio-system:istiod-istio-system.
  Removed Role:istio-system:istiod-istio-system.

This caused the existing 1.7.6 istiod deployment to become unauthenticated with the k8s api, breaking new sidecar deployments which couldn't talk to istiod.

Re-running the in-place upgrade for 1.7.6 after the service account was deleted seemed to fix the problem (after a restart of istiod), as it re-created the service account. The relevant service account objects and their lifetime as reported by kubectl also indicate that the service account was erroneously deleted (from kubectl get serviceaccounts):

istio-system          default                              1         245d
istio-system          istio-reader-service-account         1         18m
istio-system          istiod-service-account               1         18m

Am I doing something wrong with the canary install? From the documentation, I was under the impression that running istioctl install --set revision=<rev> and istioctl x uninstall --revision <rev> for a specific revision should not impact an existing install in any way?

Thanks!

[ ] Docs
[ ] Installation
[ ] Networking
[ ] Performance and Scalability
[ ] Extensions and Telemetry
[ ] Security
[ ] Test and Release
[ ] User Experience
[ ] Developer Infrastructure
[x] Upgrade

Expected behavior

Installing/Uninstalling canary installs has no affect on existing installs.

Steps to reproduce the bug

Version (include the output of istioctl version --remote and kubectl version --short and helm version --short if you used Helm)
Istio: 1.7.6/1.8.2
kubectl:

Client Version: v1.19.3
Server Version: v1.17.14-gke.1200

How was Istio installed?
istioctl (with and without canary revision)

Environment where the bug was observed (cloud vendor, OS, etc)
GKE

@istio-policy-bot istio-policy-bot added the area/upgrade Issues related to upgrades label Jan 27, 2021
@richardwxn richardwxn added this to the 1.10 milestone Jan 31, 2021
@richardwxn
Copy link
Contributor

I think the issue is because the SAs also installed with revision label, though some of them are not actually revisioned. I would create a PR to fix it.

@istio-policy-bot istio-policy-bot added the lifecycle/stale Indicates a PR or issue hasn't been manipulated by an Istio team member for a while label May 2, 2021
@istio-policy-bot
Copy link

🚧 This issue or pull request has been closed due to not having had activity from an Istio team member since 2021-01-31. If you feel this issue or pull request deserves attention, please reopen the issue. Please see this wiki page for more information. Thank you for your contributions.

Created by the issue and PR lifecycle manager.

@istio-policy-bot istio-policy-bot added the lifecycle/automatically-closed Indicates a PR or issue that has been closed automatically. label May 17, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/upgrade Issues related to upgrades lifecycle/automatically-closed Indicates a PR or issue that has been closed automatically. lifecycle/stale Indicates a PR or issue hasn't been manipulated by an Istio team member for a while
Projects
None yet
Development

No branches or pull requests

3 participants