Improve docker image build time through caches

[sansmoraxz]

Add build cache for incremental builds.

As per this guide we get significant performance boost for incremental builds.

Since the docker files are already in one place, I suspect that the cache is being shared. Anyway we will get a huge performance boost while building locally. Personal observations show around 5x improvements for minute changes although this may vary.

Checklist

• When introducing a new scaler, I agree with the scaling governance policy
• I have verified that my change is according to the deprecations & breaking changes policy
• Changelog has been updated and is aligned with our changelog requirements
• Commits are signed with Developer Certificate of Origin (DCO - learn more)

Relates to #5313

[github-actions]

Thank you for your contribution! 🙏 We will review your PR as soon as possible.

While you are waiting, make sure to:

• Add an entry in our changelog in alphabetical order and link related issue
• Update the documentation, if needed
• Add unit & e2e tests for your changes
• GitHub checks are passing
• Is the DCO check failing? Here is how you can fix DCO issues

Learn more about:

• Our contribution guide

[JorTurFer]

Nice improvement! This will help for sure with local builds. Have you checked if we can integrate this with GH Caches? I mean, workflows executed on GH Hosted runners won't have anything cached. We added tasks for exporting the cache:

keda/.github/workflows/main-build.yml

Lines 31 to 41 in da70147

	- name: Go modules cache
	uses: actions/cache@704facf57e6136b1bc63b828d79edcd491f0ee84 # v3.3.2
	with:
	path: ${{ steps.go-paths.outputs.mod_cache }}
	key: ${{ runner.os }}-go-mod-${{ hashFiles('**/go.sum') }}
	- name: Go build cache
	uses: actions/cache@704facf57e6136b1bc63b828d79edcd491f0ee84 # v3.3.2
	with:
	path: ${{ steps.go-paths.outputs.build_cache }}
	key: ${{ runner.os }}-go-build-cache-${{ hashFiles('**/go.sum') }}

Is there any known path that we can include to cache across CI executions?

[sansmoraxz]

As I understand, when we mount it using the cache mounts the cache volume is maintained separately by docker. The entire feature seems to be poorly documented and while following the breadcrumbs ended up to this issue.

I think the GHA compatibility is not yet there. Buildkit doesn't seem to export cache directory, and ability to control storage location requires some weird hacks.

keda/.github/workflows/main-build.yml

Lines 31 to 41 in da70147

	- name: Go modules cache
	uses: actions/cache@704facf57e6136b1bc63b828d79edcd491f0ee84 # v3.3.2
	with:
	path: ${{ steps.go-paths.outputs.mod_cache }}
	key: ${{ runner.os }}-go-mod-${{ hashFiles('**/go.sum') }}
	- name: Go build cache
	uses: actions/cache@704facf57e6136b1bc63b828d79edcd491f0ee84 # v3.3.2
	with:
	path: ${{ steps.go-paths.outputs.build_cache }}
	key: ${{ runner.os }}-go-build-cache-${{ hashFiles('**/go.sum') }}

I suspect bind mounts while building would be easier to pull off for these.

[JorTurFer]

Honestly, improving developer experience is mire than enough for me, as generating images is done by a CI check, where 10 minutes more or less doesn't make the difference at all. If the local experience has been improved, I think we can merge it as it is.
WDYT @kedacore/keda-core-contributors ?

[JorTurFer]

/run-e2e internal
Update: You can check the progress here

[sansmoraxz]

I might have figured something out for the pipeline.

Swapping the mount type before building the image might work.

Something like:

sed -i '/ARG GOCACHE/a ARG LOCALCACHE=/root/.cache/go-build' Dockerfile*
sed -i 's|--mount=type=cache,target=${GOCACHE}|--mount=type=bind,source=${LOCALCACHE},target=${GOCACHE}|' Dockerfile*

WDYT?

[zroubalik]

LGTM, nice improvement

[JorTurFer]

nah, don't worry @sansmoraxz .
Improving developer experience is always nice, but for CI I think that we aren't earning so much because a few minutes more or less don't make the difference during a CI check.

[JorTurFer]

Cloud you resolve the Changelog conflict? then we can merge the PR 😄

[sansmoraxz]

Pretty much the reason I delayed adding to the changelog.

Ahh well, rebased.

[JorTurFer]

/skip-e2e