Repository Cache on S3 not giving good improvements

[robertkowalski]

How are you running Renovate?

Self-hosted

If you're self-hosting Renovate, tell us what version of Renovate you run.

34.115.1

If you're self-hosting Renovate, select which platform you are using.

GitHub Enterprise Server

If you're self-hosting Renovate, tell us what version of the platform you run.

GitHub Enterprise Server 3.6.7

Was this something which used to work for you, and then stopped?

I never saw this working

Wanted end result.

The S3 based repository cache brings a significant improvement in repositories

What you tried so far.

Based on the suggestion of @rarkins in #19906 we did enable the S3 based repository cache.

We see that cache files are written and also in our logs that the cache is read with logging: RepoCacheS3.read() - success which originates from here: https://github.com/renovatebot/renovate/blob/8a08c8916bfc30ffed63f0a1ff81a968af81f10e/lib/util/cache/repository/impl/s3.ts#LL38

We have a large monorepo where renovate runtime is about 20 minutes. After enabling the cache we only did see a 1 minute improvement in runtime for the repo.

For our whole infrastructure with ~360 repos big and small its only a 10min improvement in overall runtime.

Why is this? Shouldn't the improvement be much bigger?

Relevant debug logs

Logs

Copy/paste the relevant log(s) here, between the starting and ending backticks

[rarkins]

The repository cache should present the following benefits:

• If the repo base branch commit is unchanged since the last run, the old extract result is used
• If a branch already exists and has same config as last run and is not conflicted/out of date/etc, then it will be skipped

If both of the above are true, then git cloning is skipped altogether for the run. In the hosted app we see cloning skipped 50-70% of the time, resulting in a ~30% performance improvement compared to when repo cache is off.

First thing you should do is check the Repository finished log messages to see if cloned is true or false.

[robertkowalski]

cloned is false for us, but we see 9%ish improvement for us

[viceice]

then you're probably a lot commits per day on main branches, so less cache hits.