Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Chart repo index fetcher is moved to a background job #156

Merged
merged 1 commit into from Aug 13, 2019

Conversation

@icanhazbroccoli
Copy link
Contributor

commented Aug 12, 2019

Due to a massive overhead that happened as a consequence of a heavy
chart repo index request pattern (introduced in 0.5.0), application and
installation controller latencies spiked up.

This change is aiming to get rid of ad-hoc repo index fetch approach and
move this heavy operation to a background job.

From now on, every repo instance start polling chart repo index every 10
seconds. Once the data is successfully fetched, it is preserved as a
repo attribute unmarshalled. The first fetch is blocking: assuming
Shipper starts cold, there is no previous cache we can rely upon (it
starts in a new container). On top of it, index data is never cached on
the disk as there is no use for it any longer: in-memory only.

If repo fails to fetch repo index, it behaves quite naively: simply
spins next iteration with the same delay.

Signed-off-by: Oleg Sidorov oleg.sidorov@booking.com

Chart repo index fetcher is moved to a background job
Due to a massive overhead that happened as a consequence of a heavy
chart repo index request pattern (introduced in 0.5.0), application and
installation controller latencies spiked up.

This change is aiming to get rid of ad-hoc repo index fetch approach and
move this heavy operation to a background job.

From now on, every repo instance start polling chart repo index every 10
seconds. Once the data is successfully fetched, it is preserved as a
repo attribute unmarshalled. The first fetch is blocking: assuming
Shipper starts cold, there is no previous cache we can rely upon (it
starts in a new container). On top of it, index data is never cached on
the disk as there is no use for it any longer: in-memory only.

If repo fails to fetch repo index, it behaves quite naively: simply
spins next iteration with the same delay.

Signed-off-by: Oleg Sidorov <oleg.sidorov@booking.com>

@parhamdoustdar parhamdoustdar merged commit c38d450 into master Aug 13, 2019

2 checks passed

Travis CI - Branch Build Passed
Details
Travis CI - Pull Request Build Passed
Details

@parhamdoustdar parhamdoustdar deleted the olegs/async-chart-repo-refresh branch Aug 13, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.