Feature: coredns config editing

[SebSa]

I have run into an issue with CoreDNS having a random loadbalancing policy applied to all /etc/resolve.conf entries by default.
For reasons, we need to set the coreDNS policy to sequential, so it steps through the internal upstream servers first, then goes on to the upstream external servers. Without this change the cluster is unusable for us.
I could patch this manually, but I think it would be nice to have a way to specify coredns config in k0sctl config syntax.

[twz123]

See #4459 (comment):

Currently, I think the only option to reliably change the CoreDNS configuration is to roll your own CoreDNS manifests. You can start the controller nodes using --disable-components=coredns and take over ownership of the manifests in <data-dir>/manifests/coredns.

K0s currently lacks a general-purpose way of customizing builtin Kubernetes resources, unfortunately.

[till]

@twz123 what would be the suggestion when the cluster already exists? I am in a similar situation where I need to apply changes to the coredns config, but not sure if adding my own manifests somewhere now would help, as maybe they get overwritten when I run k0sctl apply again?

[zakkg3]

for the CAPI clusters, we have some infra nodes to run the rest of the control plane (coredns, cilium operator, and other operatos). so +1 here and bumping this up :)

Thanks for this k0s project, is great!!