BMO won't start on K3s server

[s0uky]

What steps did you take and what happened:
Clean build of BMO through deploy.sh script deployed into the K3s cluster never start, manager container ends with the error below in log:

{"level":"error","ts":1707749352.3224826,"logger":"setup","msg":"unable to start manager","error":"failed to determine if *v1.Secret is namespaced: failed to get restmapping: failed to get server groups: Get \"https://10.43.0.1:443/api\": context deadline exceeded","stacktrace":"main.main\n\t/workspace/main.go:227\nruntime.main\n\t/usr/local/go/src/runtime/proc.go:250"}

and changing with this error on every restart:

{"level":"error","ts":1707747096.004748,"logger":"setup","msg":"unable to start manager","error":"failed to determine if *v1.Secret is namespaced: failed to get restmapping: failed to get server groups: Get \"https://10.43.0.1:443/api\": Service Unavailable","stacktrace":"main.main\n\t/workspace/main.go:227\nruntime.main\n\t/usr/local/go/src/runtime/proc.go:250"}

Other deployments on the same K3s cluster working normally, e.g. ironic.

What did you expect to happen:
BMO in running state.

Anything else you would like to add:
IP 10.43.0.1 is default kubernetes service, other deployments communicate normally with API server.
CNI is the Cilium in version 1.14.3 and in the Cilium log is nothing suspiciously.

Environment:

• Baremetal Operator version: 0.5.0
• k3s version v1.29.1+k3s2 (57482a1c)
• go version go1.21.6

/kind bug

[metal3-io-bot]

This issue is currently awaiting triage.
If Metal3.io contributors determine this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.
The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[Rozzii]

/help
Well I have never used BMO with K3s controlplane cluster, the Metal3-io community does not test K3s as a official controlplane.
We run tests with kind and minikube and on regular K8s setups , I hope there is someone in the community who could help you.
I would suggest joining the official slack channel, the mailing list or the community meeting and asking this question there.

[metal3-io-bot]

@Rozzii:
This request has been marked as needing help from a contributor.

Please ensure the request meets the requirements listed here.

If this request no longer meets these requirements, the label can be removed
by commenting with the /remove-help command.

In response to this:

/help
Well I have never used BMO with K3s controlplane cluster, the Metal3-io community does not test K3s as a official controlplane.
We run tests with kind and minikube and on regular K8s setups , I hope there is someone in the community who could help you.
I would suggest joining the official slack channel, the mailing list or the community meeting and asking this question there.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[goshlanguage]

I've run BMO on k3s. I use standard CIDR for k8s services, but other than that in a similar config. Seems like an implementation specific issue. Have you tried using kustomize to deploy instead of the deploy script perhaps?

v1.Secret is namespaced: failed to get restmapping makes me wonder if there isn't something going wrong with service account token or some such thing

[metal3-io-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues will close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

/lifecycle stale

[metal3-io-bot]

Stale issues close after 30d of inactivity. Reopen the issue with /reopen. Mark the issue as fresh with /remove-lifecycle stale.

/close

[metal3-io-bot]

@metal3-io-bot: Closing this issue.

In response to this:

Stale issues close after 30d of inactivity. Reopen the issue with /reopen. Mark the issue as fresh with /remove-lifecycle stale.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.