Create linux-arm64 builds of kapp-controller

[joshrosso]

This PR requires:

• Create linux-arm64 binaries imgpkg#352
• Create linux-arm64 binaries vendir#143

to be resolved. For now, I am referencing arm64 binaries of vendir and imgpkg I have built.

Using this Dockerfile, multi-arch images can be built using:

docker buildx build --platform linux/arm64,linux/amd64 --push -t "joshrosso/kapp-controller-multi:v0.x.1" .

change the value of -t.

I've pushed a projects.registry.vmware.com/tce/kapp-controller-multi:v0.30.1 to test. It works for me, on a kind cluster running on arm64.

Resolves: #573

[benmoss]

Marking as a draft since it's not ready to be merged yet. These ? lines are I think just merge mistakes.

What about using docker buildx for multi-platform images? I wish we could use ko, but our heavy dependency on binaries makes it difficult.

[joshrosso]

Marking as a draft since it's not ready to be merged yet.

These ? lines are I think just merge mistakes.

I'll address the merge issues.

What about using docker buildx for multi-platform images? I wish we could use ko, but our heavy dependency on binaries makes it difficult.

This PR makes changes required to use docker buildx:

[benmoss]

My bad, early morning reading comprehension is not at peak performance

[joshrosso]

My bad, early morning reading comprehension is not at peak performance

All good, I feel your pain.

Let me know if you have suggested approaches for checksum resolution.

These changes make the Dockerfile a bit ugly, i'm happy to add alternative approaches. On the flip-side, this approach is pretty "clear" (ie not clever) in what it's doing IMO.

[joe-kimmel-vmw]

i think this will also break our dev deploy script. it might even be an invitation to revisit the assumptions made by that script.

The valuable part of that script is that it builds the executable on the developer's local system which has a ton of cached build artifacts, which saves us a few minutes during that "inner loop" iteration process, but obviously isn't hermetic the way you'd want for proper testing & releasing. The rest of that script is just a particular implementation of how to get there. I can take a look at it later in the week.

[joshrosso]

i think this will also break our dev deploy script. it might even be an invitation to revisit the assumptions made by that script.

The valuable part of that script is that it builds the executable on the developer's local system which has a ton of cached build artifacts, which saves us a few minutes during that "inner loop" iteration process, but obviously isn't hermetic the way you'd want for proper testing & releasing. The rest of that script is just a particular implementation of how to get there. I can take a look at it later in the week.

I see that.

I can add architecture detection to that script, as right now it's failing outside of the context of buildx since $TARGETARCH won't be propagated.

I'm happy to do any work, just want guidance from this team as I don't know how "smart" you want something like this script to become.

[benmoss]

• vendir arm64 build
• imgpkg arm64 build (PR has been merged, just waiting on a new release)
• update release process to release a manifest list

I don't think we need to update hack/deploy.sh or hack/dev-deploy.sh, we can continue just using single arch builds, if you set defaults in the Dockerfile for TARGETOS and TARGETARCH it should get the tests passing again. I don't know if kbld supports multi-arch, so some investigation will be needed there since I think that's what we use for releasing right now.

I'm also happy to take this PR over if you want

[benmoss]

Closing in favor of #680, apologies for the delay on this