Doc: Update k3s cilium installation to match k3s default podCIDR

[vincentmli]

Current k3s installation instruction allow k3s to
use the default podCIDR 10.42.0.0/16 for pod, which can be seen from node.Spec:

spec:
podCIDR: 10.42.0.0/24
podCIDRs:

• 10.42.0.0/24

but "cilium install" result in pod IP to be allocated from 10.0.0.0/8 range, this cause confusion to users, the cilium installation should allow allocated pod IP match default k3s podCIDR.

Please ensure your pull request adheres to the following guidelines:

• For first time contributors, read Submitting a pull request
• All code is covered by unit and/or runtime tests where feasible.
• All commits contain a well written commit description including a title,
  description and a Fixes: #XXX line if the commit addresses a particular
  GitHub issue.
• If your commit description contains a Fixes: <commit-id> tag, then
  please add the commit author[s] as reviewer[s] to this issue.
• All commits are signed off. See the section Developer’s Certificate of Origin
• Provide a title or release-note blurb suitable for the release notes.
• Are you a user of Cilium? Please add yourself to the Users doc
• Thanks for contributing!

Fixes: #issue-number

Update k3s cilium installation to match k3s default podCIDR

[maintainer-s-little-helper]

Commit 0ef08ef83877c9410561cb60cee1fc52c6ca4b93 does not contain "Signed-off-by".

Please follow instructions provided in https://docs.cilium.io/en/stable/contributing/development/contributing_guide/#developer-s-certificate-of-origin

[maintainer-s-little-helper]

Commit 676055067d381c044fe89838e2aa4b3f7e7d9359 does not contain "Signed-off-by".

Please follow instructions provided in https://docs.cilium.io/en/stable/contributing/development/contributing_guide/#developer-s-certificate-of-origin

[joestringer]

🤔 do you think it's possible for cilium install to autodetect this range somehow by querying the cluster?

[vincentmli]

🤔 do you think it's possible for cilium install to autodetect this range somehow by querying the cluster?

it might be possible, not familiar with cilium cli and how easy to implement that, not much in autodetect for k3s https://github.com/cilium/cilium-cli/blob/main/install/autodetect.go, assume cilium cli found the podCIR, cilium cli would still do something with ipam.operator.clusterPoolIPv4PodCIDR, no? still prefer simple document update :)

[joestringer]

Yeah, that's the key question - If the Pod CIDR is somehow exposed through some resource for example then we could potentially implement this in an automatic way for every user of the Cilium-CLI. K3s isn't the only platform that may encounter this problem, so if we can come up with a reasonable solution for other platforms as well then we should.

At the same time, I appreciate that this docs change is more immediate and practical, so I'm not against simply documenting this as an alternative. 👍

[vincentmli]

I agree cilium cli could be more smart to detect the cluster and setup the correct podCIDR.

[zacharysarah]

One nit, otherwise LGTM. Approving preemptively with the understanding that the change must be made prior to merge.