Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add k0s support #1123

Merged
merged 29 commits into from
Jan 11, 2024
Merged

add k0s support #1123

merged 29 commits into from
Jan 11, 2024

Conversation

brunnels
Copy link
Contributor

@brunnels brunnels commented Jan 8, 2024

#1120

  • Support for k0s
  • Replace democratic-csi with openebs-localpath
  • Add support for templating into a private Github repo
  • Rename bootstrap vars bootstrap_kube_vip_addr -> bootstrap_kubeapi_addr and bootstrap_local_path_provisioner_path -> bootstrap_local_storage_path
  • Update Cilium to be 1.15.0 compatible

@github-actions github-actions bot added area/kubernetes Changes made in the kubernetes directory area/addons area/bootstrap Changes made in the bootstrap directory area/taskfile labels Jan 8, 2024
@onedr0p onedr0p marked this pull request as draft January 8, 2024 17:20
@onedr0p
Copy link
Owner

onedr0p commented Jan 8, 2024

Thanks for taking the time on adding this, great work so far! Few comments on this PR:

  1. Completed I think we should have this variable at the top of the config file and renamed from

    # set true to modify templates to work in k0s environment
    k0s:
      enabled: false

    to (hints at talos support someday 😆 )

    # Distribution can either be 'k3s' or 'k0s'
    # Note that changing this to k0s will ignore `system_upgrade_controller` in the addons.
    bootstrap_distribution: k3s
  2. Completed I would like to have either openebs-local-storage OR democratic-csi. The latter I've tested or know works on k3s (and talos). I can do testing on my end if you really want to move forward with openebs. It doesn't matter too much to me which solution we choose as long as it has the option to specify the data path on the host in the helm values.

  3. Completed system-upgrade-controller will need to be disabled even if the user tries to enable it in the addons when they choose k0s

  4. Completed bootstrap_private_github_repo only seems to have checks that verifies the private repo exists, however Flux needs to be modified and a secret needs to be added to support this. I originally left this out because I wanted people to keep it public to get better support. After they had it working and running they could read the README to make it private.

  5. k3s needs to have certain metrics dropped in a few places in kube-prometheus-stack, I don't think that should matter too much for k0s. My main question is does k0s work well with monitoring the kubelet, kubeApiServer, kubeControllerManager and kubeEtcd components out of the box or is there something that needs to be enabled or changed in the kube-prometheus-stack Helm values.

  6. Completed Lastly, you mentioned the nodes password field in the config. If you want to remove this option and clean up the places it is referenced I am fine with that since SSH done with a private SSH key and the nodes should have passwordless sudo. If not I'll clean it up later.

A couple other areas of improvement would be updating the README and adding support for k0s with renovate if needed.

*  Replaced democratic-csi with openebs
*  Added notes about system-upgrade-controller being ignored with k0s
*  Removed password for nodes
.taskfiles/K0sTasks.yaml Outdated Show resolved Hide resolved
*  Removed democratic-csi helm repo
*  Added missing openebs helm repo
*  Fixed template formatting
*  Made bootstrap_private_github_repo hidden/undocumented
*  Updated flux cluster config to accommodate bootstrap_private_github_repo
*  set chart wait false
*  fixed openebs path
*  properly added openebs helm repo
*  renamed bootstrap_kube_vip_addr to bootstrap_kubeapi_addr
*  renamed bootstrap_kube_vip_addr derived variables accordingly
Taskfile.yaml Outdated Show resolved Hide resolved
.taskfiles/K0sTasks.yaml Outdated Show resolved Hide resolved
*  moved coredns under addons rather than optional
*  fixed typo in coredns task
*  normalized formatting in k0s-config
*  disabled telemetry in k0s-config
*  added renovate to k0s-config
*  removed tunnel and added routing mode in k0s-config cilium chart values
*  updated docs in config example
*  fixed issue with k0s-config controller role so it didn't default to worker+controller if more than one master
@github-actions github-actions bot added the area/github Changes made in the github directory label Jan 11, 2024
@onedr0p onedr0p marked this pull request as ready for review January 11, 2024 19:58
@onedr0p onedr0p merged commit d093d0a into onedr0p:main Jan 11, 2024
1 check passed
@brunnels brunnels deleted the k0s-main branch January 11, 2024 20:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/bootstrap Changes made in the bootstrap directory area/github Changes made in the github directory area/kubernetes Changes made in the kubernetes directory area/taskfile
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants