Got errors on kubebuilder v3 quick start #3016

astraw99 · 2022-10-13T14:44:19Z

What broke? What's expected?

Using the latest kubebuilder cli, quick start a new project based on the official document, got these errors:

Enable the leader-election, then make run, got error:

go fmt ./...
go vet ./...
go run ./main.go
1.665666448465308e+09   ERROR   setup   unable to start manager {"error": "unable to find leader election namespace: not running in-cluster, please specify LeaderElectionNamespace"}
main.main
...
exit status 1
make: *** [run] Error 1

After make deploy IMG=xxx/xxx:tag, the controller-pod got error:

1.6656687258729894e+09  ERROR   controller-runtime.client.config        unable to get kubeconfig        {"error": "open /var/run/secrets/kubernetes.io/serviceaccount/token: permission denied"}
sigs.k8s.io/controller-runtime/pkg/client/config.GetConfigOrDie
        /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.13.0/pkg/client/config/config.go:153
main.main
        /workspace/main.go:68
runtime.main
        /usr/local/go/src/runtime/proc.go:250

As the doc said Important: Run "make" to regenerate code after modifying this file, but make did not generate latest CRD fields.

Reproducing this issue

Just quick start a new project based on the official document, then will reproduce this issue.

KubeBuilder (CLI) Version

Version: main.version{KubeBuilderVersion:"3.7.0", KubernetesVendor:"1.24.1", GitCommit:"3bfc84ec8767fa760d1771ce7a0cb05a9a8f6286", BuildDate:"2022-09-20T17:21:57Z", GoOs:"darwin", GoArch:"amd64"}

PROJECT version

No response

Plugin versions

No response

Other versions

No response

Extra Labels

No response

The text was updated successfully, but these errors were encountered:

laxmikantbpandhare · 2022-10-13T15:18:10Z

make run is working for me. But I haven't updated any code for the leader election.

laxmikantbhaskarpandhare@lpandhar-mac kb % make run
test -s /Users/laxmikantbhaskarpandhare/comparison-memcached-operator/remove/kb/bin/controller-gen || GOBIN=/Users/laxmikantbhaskarpandhare/comparison-memcached-operator/remove/kb/bin go install sigs.k8s.io/controller-tools/cmd/controller-gen@v0.10.0
/Users/laxmikantbhaskarpandhare/comparison-memcached-operator/remove/kb/bin/controller-gen rbac:roleName=manager-role crd webhook paths="./..." output:crd:artifacts:config=config/crd/bases
/Users/laxmikantbhaskarpandhare/comparison-memcached-operator/remove/kb/bin/controller-gen object:headerFile="hack/boilerplate.go.txt" paths="./..."
go fmt ./...
go vet ./...
go run ./main.go
1.665674102135634e+09	INFO	controller-runtime.metrics	Metrics server is starting to listen	{"addr": ":8080"}
1.665674102135971e+09	INFO	setup	starting manager
1.665674102136213e+09	INFO	Starting server	{"kind": "health probe", "addr": "[::]:8081"}
1.665674102136213e+09	INFO	Starting server	{"path": "/metrics", "kind": "metrics", "addr": "[::]:8080"}
1.6656741021363149e+09	INFO	Starting EventSource	{"controller": "guestbook", "controllerGroup": "webapp.my.domain", "controllerKind": "Guestbook", "source": "kind source: *v1.Guestbook"}
1.665674102136364e+09	INFO	Starting Controller	{"controller": "guestbook", "controllerGroup": "webapp.my.domain", "controllerKind": "Guestbook"}
1.665674102236641e+09	INFO	Starting workers	{"controller": "guestbook", "controllerGroup": "webapp.my.domain", "controllerKind": "Guestbook", "worker count": 1}

laxmikantbpandhare · 2022-10-13T15:19:29Z

May I know what changes you did to the leader election? For this - Enable the leader-election

astraw99 · 2022-10-13T15:22:27Z

May I know what changes you did to the leader election? For this - Enable the leader-election

On generated main.go L57, just set leader-election true.

flag.BoolVar(&enableLeaderElection, "leader-elect", true, ...

everettraven · 2022-10-13T15:33:50Z

1.665666448465308e+09 ERROR setup unable to start manager {"error": "unable to find leader election namespace: not running in-cluster, please specify LeaderElectionNamespace"}

Using make run runs the controller as a process external to the cluster (on your local machine). Encountering this error when setting leader election and then running outside of the cluster makes sense to me.

1.6656687258729894e+09 ERROR controller-runtime.client.config unable to get kubeconfig {"error": "open /var/run/secrets/kubernetes.io/serviceaccount/token: permission denied"}

This error is a bit different. I'm not sure why it would be getting a permission denied when attempting to load the kubeconfig.

It seems possible it could be related to ServiceAccounts no longer auto-generating tokens by default as of kubernetes 1.24 (see changelog here: https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.24.md#urgent-upgrade-notes).

If you update the config/rbac/service_account.yaml file and add this to the end of it:

automountServiceAccountToken: true

does running make deploy IMG=... end up working?

everettraven · 2022-10-13T16:18:23Z

After running through the tutorial locally and using make deploy IMG=... I was unable to replicate the error:

1.6656687258729894e+09 ERROR controller-runtime.client.config unable to get kubeconfig {"error": "open /var/run/secrets/kubernetes.io/serviceaccount/token: permission denied"}

I even attempted it after forcing the --leader-elect flag to default to true as mentioned:

May I know what changes you did to the leader election? For this - Enable the leader-election

On generated main.go L57, just set leader-election true.
flag.BoolVar(&enableLeaderElection, "leader-elect", true, ...

I did not need to do:

If you update the config/rbac/service_account.yaml file and add this to the end of it:
automountServiceAccountToken: true
does running make deploy IMG=... end up working?

liggitt · 2022-10-13T19:58:13Z

It seems possible it could be related to ServiceAccounts no longer auto-generating tokens by default as of kubernetes 1.24 (see changelog here: https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.24.md#urgent-upgrade-notes).

If you update the config/rbac/service_account.yaml file and add this to the end of it:
automountServiceAccountToken: true

That should not be necessary. Service accounts still mount tokens by default in 1.24, they just do so using ephemerally generated tokens, not tokens stored in Secret API objects

astraw99 · 2022-10-14T03:53:57Z

After running through the tutorial locally and using make deploy IMG=... I was unable to replicate the error:

My cluster K8s version is:

Client Version: version.Info{Major:"1", Minor:"21", GitVersion:"v1.21.0", GitCommit:"cb303e613a121a29364f75cc67d3d580833a7479", GitTreeState:"clean", BuildDate:"2021-04-08T16:31:21Z", GoVersion:"go1.16.1", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"21", GitVersion:"v1.21.0", GitCommit:"cb303e613a121a29364f75cc67d3d580833a7479", GitTreeState:"clean", BuildDate:"2021-04-08T16:25:06Z", GoVersion:"go1.16.1", Compiler:"gc", Platform:"linux/amd64"}

Kubebuilder version is:

Version: main.version{KubeBuilderVersion:"3.7.0", KubernetesVendor:"1.24.1", GitCommit:"3bfc84ec8767fa760d1771ce7a0cb05a9a8f6286", BuildDate:"2022-09-20T17:21:57Z", GoOs:"darwin", GoArch:"amd64"}

With these context, will replicate the error. @everettraven

camilamacedo86 · 2022-10-17T10:48:43Z

Hi @liggitt,

The following steps are not accurate.

go fmt ./...
go vet ./...
go run ./main.go
1.665666448465308e+09   ERROR   setup   unable to start manager {"error": "unable to find leader election namespace: not running in-cluster, please specify LeaderElectionNamespace"}
main.main

Could you please perform the suggested steps to check if the error is still faced?

astraw99 · 2022-10-17T13:01:50Z

May I know what changes you did to the leader election? For this - Enable the leader-election

On generated main.go L57, just set leader-election true.
flag.BoolVar(&enableLeaderElection, "leader-elect", true, ...

@camilamacedo86 I modified this first, and then make install, make run, then will encounter the LeaderElectionNamespace error.
PTAL thanks.

camilamacedo86 · 2022-10-17T16:07:23Z

Hi @astraw99,

I see. If you enable the leader election you are facing the problem {"error": "unable to find leader election namespace: not running in-cluster, please specify LeaderElectionNamespace"}. Then:

a) Is the issue faced when/if you update the config/manager.yaml to pass the flag == true and deploy the project on the cluster as well? OR the namespace is only required if/when you run it outside of the cluster for developing/test purposes with make run?

PS.: Note that in the quick start we do not have any step saying for someone to enable the leader election. Therefore, the title/description here does not seem accurate. It is more like so far "default scaffold does not work if/when we enable leader election and we use make run to test the project outside of the cluster"

astraw99 · 2022-10-18T07:06:55Z

Hi @astraw99,

I see. If you enable the leader election you are facing the problem {"error": "unable to find leader election namespace: not running in-cluster, please specify LeaderElectionNamespace"}. Then:

a) Is the issue faced when/if you update the config/manager.yaml to pass the flag == true and deploy the project on the cluster as well? OR the namespace is only required if/when you run it outside of the cluster for developing/test purposes with make run?

PS.: Note that in the quick start we do not have any step saying for someone to enable the leader election. Therefore, the title/description here does not seem accurate. It is more like so far "default scaffold does not work if/when we enable leader election and we use make run to test the project outside of the cluster"

Yes, it is the make run to test the project outside of the cluster got the leader-election namespace error.
Raised a PR #3021 to fix it.

camilamacedo86 · 2022-10-18T15:11:01Z

HI @astraw99,

Is the same error faced using make deploy IMG or not?

astraw99 · 2022-10-19T02:44:13Z

HI @astraw99,

Is the same error faced using make deploy IMG or not?

No, make deploy into the cluster works well.
Only faced the sa permission error, opened a new issue #3028 to clarify it.

jmrodri · 2022-10-20T16:12:52Z

/label triage/accepted

camilamacedo86 · 2022-10-22T07:54:11Z

Hi @astraw99,

Only faced the sa permission error, opened a new issue #3028 to clarify it.

The default scaffold does have not SA. So, how can that can be faced by default?

However, this issue was used to just add the manifest target to the make build target (#3020). Let's discuss the other points in the #3028

astraw99 added the kind/bug Categorizes issue or PR as related to a bug. label Oct 13, 2022

astraw99 mentioned this issue Oct 13, 2022

🐛: Fix controller-pod SA permission denied error #3017

Closed

This was referenced Oct 16, 2022

✨ Update make build cmd with manifests generation #3020

Merged

🐛 Add leader election namespace to fix manager start error #3021

Closed

k8s-ci-robot closed this as completed in #3020 Oct 22, 2022

astraw99 mentioned this issue Oct 25, 2022

📖 Add kubebuilder FAQ section #3044

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Got errors on kubebuilder v3 quick start #3016

Got errors on kubebuilder v3 quick start #3016

astraw99 commented Oct 13, 2022 •

edited

laxmikantbpandhare commented Oct 13, 2022

laxmikantbpandhare commented Oct 13, 2022 •

edited

astraw99 commented Oct 13, 2022

everettraven commented Oct 13, 2022

everettraven commented Oct 13, 2022

liggitt commented Oct 13, 2022

astraw99 commented Oct 14, 2022 •

edited

camilamacedo86 commented Oct 17, 2022

astraw99 commented Oct 17, 2022

camilamacedo86 commented Oct 17, 2022 •

edited

astraw99 commented Oct 18, 2022

camilamacedo86 commented Oct 18, 2022

astraw99 commented Oct 19, 2022 •

edited

jmrodri commented Oct 20, 2022

camilamacedo86 commented Oct 22, 2022 •

edited

Got errors on kubebuilder v3 quick start #3016

Got errors on kubebuilder v3 quick start #3016

Comments

astraw99 commented Oct 13, 2022 • edited

What broke? What's expected?

Reproducing this issue

KubeBuilder (CLI) Version

PROJECT version

Plugin versions

Other versions

Extra Labels

laxmikantbpandhare commented Oct 13, 2022

laxmikantbpandhare commented Oct 13, 2022 • edited

astraw99 commented Oct 13, 2022

everettraven commented Oct 13, 2022

everettraven commented Oct 13, 2022

liggitt commented Oct 13, 2022

astraw99 commented Oct 14, 2022 • edited

camilamacedo86 commented Oct 17, 2022

astraw99 commented Oct 17, 2022

camilamacedo86 commented Oct 17, 2022 • edited

astraw99 commented Oct 18, 2022

camilamacedo86 commented Oct 18, 2022

astraw99 commented Oct 19, 2022 • edited

jmrodri commented Oct 20, 2022

camilamacedo86 commented Oct 22, 2022 • edited

astraw99 commented Oct 13, 2022 •

edited

laxmikantbpandhare commented Oct 13, 2022 •

edited

astraw99 commented Oct 14, 2022 •

edited

camilamacedo86 commented Oct 17, 2022 •

edited

astraw99 commented Oct 19, 2022 •

edited

camilamacedo86 commented Oct 22, 2022 •

edited