Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Upgrading Amazon EKS-D cluster from v1.18 to v1.19 fails due to kubeadm CoreDNS validations failing #1284

Closed
xmudrii opened this issue Mar 19, 2021 · 3 comments
Closed

Upgrading Amazon EKS-D cluster from v1.18 to v1.19 fails due to kubeadm CoreDNS validations failing #1284

xmudrii opened this issue Mar 19, 2021 · 3 comments
Labels
Epic kind/bug Categorizes issue or PR as related to a bug. sig/cluster-management Denotes a PR or issue as being assigned to SIG Cluster Management.

Comments

@xmudrii
Copy link
Member

xmudrii commented Mar 19, 2021

What happened:

Trying to upgrade an Amazon EKS-D cluster from v1.18 to v1.19 fails due to kubeadm CoreDNS validations failing. Two preflight checks are failing:

  • CoreDNSUnsupportedPlugins
  • CoreDNSMigration

Ignoring those preflight checks doesn't help because kubeadm fails to upgrade CoreDNS with the following error:

[172.31.25.181] [upgrade/postupgrade] FATAL post-upgrade error: unable to get list of changes to the configuration.: start version 'v1.7.0-eks-1-18-1' not supported
[172.31.25.181] To see the stack trace of this error execute with --v=5 or higher

What is the expected behavior:

It's possible to upgrade Amazon EKS-D clusters from v1.18 to v1.19.

How to reproduce the issue:

  1. Create Amazon EKS-D cluster running v1.18
  2. Upgrade the cluster to Amazon EKS-D v1.19

Anything else we need to know?

It seems like there are two issues:

  1. EKS-D v1.18 uses CoreDNS v1.7.0, while EKS-D v1.19 uses CoreDNS v1.8.0. It seems like kubeadm doesn't support upgrading CoreDNS directly from v1.7.0 to v1.8.0. Instead, it should be upgraded to v1.7.1 and then to v1.8.0.
  2. CoreDNS images provided by EKS-D have the EKS-D suffix (e.g. v1.7.0-eks-1-18-1). kubeadm drops the v prefix, but it doesn't drop the -eks-1-18-1 suffix. Since kubeadm strictly compares the version, the suffix causes the validation to fail.

Information about the environment:
KubeOne version (kubeone version): master and v1.2.0
Provider you're deploying cluster on: AWS
Operating system you're deploying on: Amazon Linux 2
@xmudrii xmudrii added kind/bug Categorizes issue or PR as related to a bug. sig/cluster-management Denotes a PR or issue as being assigned to SIG Cluster Management. Epic labels Mar 19, 2021
@xmudrii xmudrii mentioned this issue Mar 23, 2021
Merged
@kubermatic-bot
Copy link
Contributor

Issues go stale after 90d of inactivity.
After a furter 30 days, they will turn rotten.
Mark the issue as fresh with /remove-lifecycle stale.

If this issue is safe to close now please do so with /close.

/lifecycle stale

@kubermatic-bot kubermatic-bot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Jul 19, 2021
@xmudrii
Copy link
Member Author

xmudrii commented Jul 20, 2021

/remove-lifecycle stale

@kubermatic-bot kubermatic-bot removed the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Jul 20, 2021
@xmudrii
Copy link
Member Author

xmudrii commented Dec 29, 2021

Kubernetes 1.18 and 1.19 clusters are not supported any longer. Also, Amazon EKS-D support is removed starting with KubeOne 1.4.

@xmudrii xmudrii closed this as completed Dec 29, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Epic kind/bug Categorizes issue or PR as related to a bug. sig/cluster-management Denotes a PR or issue as being assigned to SIG Cluster Management.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@xmudrii @kubermatic-bot