Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HelmChart: enable dependency manager to use the in memory cache #667

Merged
merged 1 commit into from
Apr 19, 2022

Conversation

souleb
Copy link
Member

@souleb souleb commented Apr 12, 2022

This PR:

  • enables a chartRepository to strategically load an Index from a memory cache
  • provide cache events metrics

Tests

With cache enabled

for the case we used the following flags

     spec:
       containers:
       - args:
         - --helm-cache-purge-interval=10m
         - --helm-cache-ttl=1h
         - --helm-cache-max-size=10
         - --events-addr=http://notification-controller.flux-system.svc.cluster.local./
         - --watch-all-namespaces=true
         - --log-level=info
         - --log-encoding=json
         - --enable-leader-election
         - --storage-path=/data
         - --storage-adv-addr=source-controller.$(RUNTIME_NAMESPACE).svc.cluster.local.

We have 2 HelmRepositories:

  • Bitnami (interval 10m), 12.8 MB, with 10 running Helmrelease (interval 1m)
  • Prometheus-Stack(Interval 10M), with 1 running HelmRelease (interval 1m)

After 1h, we have gathered the followiing information from grafana

Screenshot 2022-04-12 at 15 08 22

Screenshot 2022-04-12 at 15 09 42

Screenshot 2022-04-12 at 15 12 37

the metric query for HitRate is sum(rate(gotk_cache_events_total{event_type="cache_hit"}[1m]))/sum(rate(gotk_cache_events_total[1m]))

With cache disabled

Screenshot 2022-04-19 at 11 44 23

Screenshot 2022-04-19 at 11 44 43

Signed-off-by: Soule BA soule@weave.works

@souleb souleb marked this pull request as draft April 12, 2022 13:15
@souleb souleb force-pushed the use-cache-when-building-deps branch from cbcc067 to ab417cd Compare April 12, 2022 15:46
@souleb souleb marked this pull request as ready for review April 12, 2022 15:48
@stefanprodan stefanprodan changed the title HelmController: enable dependency manager to use the in memory cache HelmChart: enable dependency manager to use the in memory cache Apr 12, 2022
@stefanprodan stefanprodan added enhancement New feature or request area/helm Helm related issues and pull requests labels Apr 12, 2022
@souleb souleb force-pushed the use-cache-when-building-deps branch 2 times, most recently from 472902e to 149ebb7 Compare April 13, 2022 14:01
go.mod Outdated Show resolved Hide resolved
@souleb souleb force-pushed the use-cache-when-building-deps branch from 149ebb7 to 53850dd Compare April 15, 2022 13:57
@souleb souleb force-pushed the use-cache-when-building-deps branch from a9fb8e6 to 97ba45d Compare April 19, 2022 08:59
If implemented this will:
- enable the helmCharts dependency manager to use the helm in memry
  cache to retrieve reconciled HelmRepositories indexes.
- record cache events.

Signed-off-by: Soule BA <soule@weave.works>
@souleb souleb force-pushed the use-cache-when-building-deps branch from 97ba45d to 0df8dcc Compare April 19, 2022 09:46
Copy link
Member

@hiddeco hiddeco left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you very much for following up on this @souleb. 💯

@hiddeco hiddeco merged commit b9b123c into fluxcd:main Apr 19, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/helm Helm related issues and pull requests enhancement New feature or request
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

None yet

3 participants