Include disabled repos in the cache

[jamietanna]

What would you like Renovate to be able to do?

When performing an --autodiscover on a particularly large organisation, with ~2000 repos, caching is being used to improve the performance.

However, this leads to a cache miss for these repos consistently, leading to a lot of wasted downloads.

If you have any ideas on how this should be implemented, please tell us here.

Store the disabled repos' .git folder (#18739) or the full repository, so repeated git clones can be replaced with a git fetch.

Is this a feature you are interested in implementing yourself?

Maybe

[rarkins]

There is no "the cache" - Renovate has multiple. Also, what type of disabling do you mean, e.g. disable in config, or disabled due to being archived, etc?

[jamietanna]

Sorry, by "the cache", I am referring to persistRepoData.

By "disabled", I mean a repo that is not onboarded. I agree that archived repos should be excluded.

[rarkins]

We don't usually use "cache" to describe the persistRepoData feature.

Regarding this feature, are you sure it saves you a lot of time? In the hosted app we saw the average repo take ~3 seconds for cloning, and the issues with persisting repo data and copying it back and forth didn't seem worth it.

Also a repo which is not onboarded isn't considered "disabled" in Renovate terminology. You're referring to repos which have a "Configure Renovate" PR open, not yet merged?

[jamietanna]

Ah that's fair. We were hoping that it would make things a little bit quicker, but if you've also not had great success with it, maybe we'll leave it off. It was aiming to improve speed to counteract renovatebot/github-action#648 (also see renovatebot/github-action#646)

I mentioned "disabled", as I saw it in this output:

 INFO: Repository started (repository={org}/{repo})
       "renovateVersion": "34.2.0"
 INFO: Repository is disabled - skipping (repository={org}/{repo})

This is based on the following in our config.js:

  requireConfig: "required",
  onboarding: false,

And the repos that show this message do not have an onboarding PR, nor are onboarded.

[rarkins]

If you have requireConfig=required then yes I'd also consider that "disabled".

I think if you're interested in optimizations then you should take a look at what we're doing with repositoryCache. It's still classified as experimental but we try not to make breaking changes. In the hosted app now 80% of the time we don't even clone a repo. However it's not optimized for disabled as you have it, and could be a little tricky to know if a config file has been added.