Skip to content
This repository has been archived by the owner on May 30, 2023. It is now read-only.

Flatcar support for EKS on versions above 1.21 #2110

Merged
merged 4 commits into from
Aug 23, 2022

Conversation

EhsanLunar
Copy link
Contributor

@EhsanLunar EhsanLunar commented Aug 22, 2022

Description

When an EC2 instance boots up with a flatcar image (even the latest) the kubelet fails.
The userdata defines (and should do so) that the /etc/eks/bootstrap.sh should run, which it does.

This seems to add a ExecStartPre to the kubelet.service:
ExecStartPre=/usr/share/oem/eks/download-kubelet.sh

Both the bootstrap.sh and the download-kubelet.sh are consistent with:
https://github.com/flatcar-linux/coreos-overlay/blob/main/coreos-base/flatcar-eks/files/bootstrap.sh
https://github.com/flatcar-linux/coreos-overlay/blob/main/coreos-base/flatcar-eks/files/download-kubelet.sh

The download-kubelet.sh fails with Unsupported Kubernetes version because in the case statement on line 24->50 (https://github.com/flatcar-linux/coreos-overlay/blob/main/coreos-base/flatcar-eks/files/download-kubelet.sh#L25) only has values for kubernetes version 1.15 -> 1.21

If I manually alter the file and add 1.22 (when I test this on 1.22.9 kubernetes version deployment) and re-run the bootsrap.sh it works fine as far as I can see, the node than joins the cluster and shows up as Ready and pods starting running on the node.

The last PR I can see on this particular thing was done about a year ago f0da7f8

Impact

No EKS support for kubernetes versions higher than 1.21

Environment and steps to reproduce

  1. Set-up: Create an EKS cluster with the latest flatcar AMI in the worker nodes
  2. Task: SSH into the node (probably through a Bastion)
  3. Action(s): No actions needed
  4. Error: kubelet.service fails because the download-kubelet.sh doesn't have download locations for kubernetes version above 1.21

Expected behavior

Download locations for kubernetes versions 1.22 and 1.23 (EKS doesn't have support for 1.24 yet it seems) should be located inside the download-kubelet.sh

Additional information

By running aws s3 ls s3://amazon-eks/ you can list the available locations of the other versions, so for it should result in this:

case $CLUSTER_VERSION in
  1.23)
    S3_PATH="1.23.9/2022-07-27"
    ;;
  1.22)
    S3_PATH="1.22.12/2022-07-27"
    ;;
  1.21)
    S3_PATH="1.21.2/2021-07-05"
    ;;
  1.20)
    S3_PATH="1.20.4/2021-04-12"
    ;;
  1.19)
    S3_PATH="1.19.6/2021-01-05"
    ;;
  1.18)
    S3_PATH="1.18.9/2020-11-02"
    ;;
  1.17)
    S3_PATH="1.17.12/2020-11-02"
    ;;
  1.16)
    S3_PATH="1.16.15/2020-11-02"
    ;;
  1.15)
    S3_PATH="1.15.12/2020-11-02"
    ;;
  *)
    echo "Unsupported Kubernetes version"
    exit 1
    ;;
esac

[Title: describe the change in one sentence]

[ describe the change in 1 - 3 paragraphs ]

How to use

[ describe what reviewers need to do in order to validate this PR ]

Testing done

[Describe the testing you have done before submitting this PR. Please include both the commands you issued as well as the output you got.]

Closes: flatcar/Flatcar#829

  • Changelog entries added in the respective changelog/ directory (user-facing change, bug fix, security fix, update)

## Description

When an EC2 instance boots up with a flatcar image (even the latest) the kubelet fails.
The userdata defines (and should do so) that the `/etc/eks/bootstrap.sh` should run, which it does.

This seems to add a ExecStartPre to the kubelet.service:
`ExecStartPre=/usr/share/oem/eks/download-kubelet.sh`

Both the `bootstrap.sh` and the `download-kubelet.sh` are consistent with:
https://github.com/flatcar-linux/coreos-overlay/blob/main/coreos-base/flatcar-eks/files/bootstrap.sh
https://github.com/flatcar-linux/coreos-overlay/blob/main/coreos-base/flatcar-eks/files/download-kubelet.sh

The `download-kubelet.sh` fails with `Unsupported Kubernetes version` because in the case statement on line 24->50 (https://github.com/flatcar-linux/coreos-overlay/blob/main/coreos-base/flatcar-eks/files/download-kubelet.sh#L25) only has values for kubernetes version 1.15 -> 1.21

If I manually alter the file and add 1.22 (when I test this on 1.22.9 kubernetes version deployment) and re-run the `bootsrap.sh` it works fine as far as I can see, the node than joins the cluster and shows up as `Ready` and pods starting running on the node.

The last PR I can see on this particular thing was done about a year ago flatcar-archive@f0da7f8

## Impact

No EKS support for kubernetes versions higher than 1.21

## Environment and steps to reproduce

1. **Set-up**: Create an EKS cluster with the latest flatcar AMI in the worker nodes
2. **Task**: SSH into the node (probably through a Bastion)
3. **Action(s)**:  No actions needed
4. **Error**: kubelet.service fails because the download-kubelet.sh doesn't have download locations for kubernetes version above 1.21

## Expected behavior

Download locations for kubernetes versions 1.22 and 1.23 (EKS doesn't have support for 1.24 yet it seems) should be located inside the download-kubelet.sh

## Additional information

By running `aws s3 ls s3://amazon-eks/` you can list the available locations of the other versions, so for it should result in this:

``` sh
case $CLUSTER_VERSION in
  1.23)
    S3_PATH="1.23.9/2022-07-27/"
    ;;
  1.22)
    S3_PATH="1.22.12/2022-07-27/"
    ;;
  1.21)
    S3_PATH="1.21.2/2021-07-05"
    ;;
  1.20)
    S3_PATH="1.20.4/2021-04-12"
    ;;
  1.19)
    S3_PATH="1.19.6/2021-01-05"
    ;;
  1.18)
    S3_PATH="1.18.9/2020-11-02"
    ;;
  1.17)
    S3_PATH="1.17.12/2020-11-02"
    ;;
  1.16)
    S3_PATH="1.16.15/2020-11-02"
    ;;
  1.15)
    S3_PATH="1.15.12/2020-11-02"
    ;;
  *)
    echo "Unsupported Kubernetes version"
    exit 1
    ;;
esac
```
Copy link
Contributor

@tormath1 tormath1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot for your contribution! It looks good to me. There's only one missing thing: a changelog entry (as it will be included in the next set of releases).
You might find changelog entry examples in the following directory: https://github.com/flatcar-linux/coreos-overlay/tree/main/changelog/bugfixes

@tormath1 tormath1 added the main label Aug 22, 2022
Co-authored-by: Mathieu Tortuyaux <mathieu.tortuyaux@gmail.com>
@EhsanLunar
Copy link
Contributor Author

@tormath1 Something like this? :)

….21.md

Co-authored-by: Mathieu Tortuyaux <mathieu.tortuyaux@gmail.com>
@EhsanLunar
Copy link
Contributor Author

@tormath1 Thanks! :)

@EhsanLunar
Copy link
Contributor Author

@tormath1
If this is merged now, when will it be available?
Im guessing it will be available in the next alpha, beta and stable?

Reason I'm asking is because, running flatcar on EKS using Cluster API doesn't work at all atm because:

  1. Flatcar doesn't support versions above 1.21
  2. Bootstrapping aws cluster (which is necessary in ClusterAPI) isn't supported in EKS versions below 1.22

@tormath1
Copy link
Contributor

@EhsanLunar that is correct. Once merged, this will be backported to current channels and available in the next releases.

@tormath1 tormath1 merged commit ba31150 into flatcar-archive:main Aug 23, 2022
@tormath1
Copy link
Contributor

Thanks @EhsanLunar for this contribution, this is great! 🚀

Cherry-picked to:

  • flatcar-3305
  • flatcar-3277
  • flatcar-3227

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Flatcar not supporting EKS on versions above 1.21
2 participants