Kustomize v3 does not provide bin for mac (arm64)

[camilamacedo86]

What broke? What's expected?

In order to support go 1.18 (#2486) we change the Makefile:

KUSTOMIZE = $(shell pwd)/bin/kustomize
kustomize: ## Download kustomize locally if necessary.
	$(call go-get-tool,$(KUSTOMIZE),sigs.k8s.io/kustomize/kustomize/v3@v3.8.7)

To use the script:

KUSTOMIZE_INSTALL_SCRIPT ?= "https://raw.githubusercontent.com/kubernetes-sigs/kustomize/master/hack/install_kustomize.sh"
.PHONY: kustomize
kustomize: $(KUSTOMIZE) ## Download kustomize locally if necessary.
$(KUSTOMIZE):
	curl -s $(KUSTOMIZE_INSTALL_SCRIPT) | bash -s -- $(subst v,,$(KUSTOMIZE_VERSION)) $(LOCALBIN)

However, the v3 versions do not provide darwin arm64 binaries:

See https://github.com/kubernetes-sigs/kustomize/releases/tag/kustomize%2Fv3.10.0. (latest)
But it is provided for v4: https://github.com/kubernetes-sigs/kustomize/releases/tag/kustomize%2Fv4.5.3

Then, we need to:
a) verify if we have not another option to install kustomize v3
OR
b) revisit the option to update to kustomize v4 (it has breaking changes, see: #2498 (comment). Also, see #2498 (comment))

Reproducing this issue

No response

KubeBuilder (CLI) Version

master

PROJECT version

3

Plugin versions

Default version 3

Other versions

No response

Extra Labels

/kind regression

[camilamacedo86]

@ryantking we need to check this one.
It blocks the release.

[camilamacedo86]

arm64 is still not supported (we might able to begin to provide its bins for the next KB release, see #2505). So we could say that is a feature request.

To add the convey of the discussions made so far:

if we bump kustomize v4 on the plugin go/v3

• then we provide support for arm64
• however, we also might face the risk of introducing a breaking change (we cannot introduce breaking changes for stable plugins)
• theoretically, if we do not change the scaffolds to use the new feature provided in kustomize v4 and do not remove what is deprecated then it could still be compatible (we cannot for example vars and replacements then which means that the scaffold will not work with v3)
• but that probably also means adding a test where we ensure that the default scaffold still working with v3

PS: Actually to introduce the changes scaffold changes of kustomize we will need a new kustomize plugin and not go/v4: See: https://github.com/kubernetes-sigs/kubebuilder/pull/2498/files. However, we also bump the version on the plugin which is used to generate scaffold the Makefile in the target to download the bin. More info: #2503 (comment)

Alternative Options:
a) Check if the kustomize project would be able to begin to provide the bins for os arm64
OR/AND
b) Just use kustomize v4 in a new plugin go/v4-alpha so that we can ensure that we will not break anything for the stable plugins and allow this option for the users.
c) Another idea was to check if we could not provide kustomize v4 as optional via flag/makefile target, e.g:

kustomize-v4  EXPERIMENTAL: Scaffold Makefiles with kustomize v4 instead of v3. Required for Apple silicon.

Then we do a check like the following for this arch:

if runtime.GOOS == "Darwin" && runtime.GOARDH == "aarm6" {
    return errors.New("Apple Silicon users must run with `--kustomize-v4` enabled. Read more about this here: https://a-fantastic-document/")
}

So that we would provide the option for the users but we would not be introducing it by default.

[camilamacedo86]

From the master branch is possible to know we use init --plugins=base.go/v3,kustomize/v4-alpha so that we would scaffold the golang based plugin used by default with the new kustomize.

[SkYNewZ]

Hi. I cannot use the kustomize alpha plugin.

❯ kubebuilder init --plugins=base.go/v3,kustomize/v4-alpha
Error: no plugin could be resolved with key "kustomize/v4-alpha" (plugin version is unstable, there may be an upgrade available: https://kubebuilder.io/migration/plugin/plugins.html)
❯ kubebuilder version
Version: main.version{KubeBuilderVersion:"HEAD-8ec681a", KubernetesVendor:"unknown", GitCommit:"8ec681ae5a7bf3751e2539a7b7b4677c255f6a0f", BuildDate:"2022-06-20T11:56:04Z", GoOs:"darwin", GoArch:"arm64"}

[camilamacedo86]

Hi @SkYNewZ,

The plugin is only available on master, to call it you --plugins="kustomize/v2-alpha,base.go.kubebuilder.io/v3" (not v4-alpha)

Also, see; #2758
After we merge this one the stable plugin (default scaffold done with kubebuilder init) will have the bump. However, the syntax done for it will still use for example vars and not replacements which works from 4x only.

[SkYNewZ]

Thanks you @camilamacedo86 !