Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make periodic cache cleanup retention times configurable #7018

Open
wszeboreq opened this issue Oct 4, 2018 · 10 comments
Open

Make periodic cache cleanup retention times configurable #7018

wszeboreq opened this issue Oct 4, 2018 · 10 comments

Comments

@wszeboreq
Copy link

@wszeboreq wszeboreq commented Oct 4, 2018

Expected Behavior

Periodic Gradle cache cleanup was implemented as #1085. Comment #1085 (comment) summarizes the implemented strategy. It would be great to have various hardcoded cleanup algorithm parameters configurable, at least the 7/30 days retention times. This would give the additional flexibility and allow tuning the periodic cache cleanup to fit various possible scenarios and environments.

Current Behavior

Currently the 7/30 days retention times described in the mentioned comment are not configurable.

Context

Our projects use some dependency artifacts having significant disk sizes (e.g. 1 GB). Practically each day we have a new version of majority of these dependencies. This leads to '.gradle/caches/modules-2/files-2.1' grow bigger and bigger each day, leading to free disk space problems on the build servers. 30 days of cache retention time in this scenario may lead to requirement of 30 GB free cache disk space for each bigger dependency.

Your Environment

------------------------------------------------------------
Gradle 4.10.1
------------------------------------------------------------

Build time:   2018-09-12 11:33:27 UTC
Revision:     76c9179

Kotlin DSL:   1.0-rc-6
Kotlin:       1.2.61
Groovy:       2.4.15
Ant:          Apache Ant(TM) version 1.9.11 compiled on March 23 2018
JVM:          1.8.0_151 (Oracle Corporation 25.151-b12)
OS:           Windows 10 10.0 amd64
@Barteks2x

This comment has been minimized.

Copy link

@Barteks2x Barteks2x commented Nov 5, 2018

Would it be also possible to give an option to completely disable it? On a development machine where space usage is not an issue, I don't want to wait for gradle to redownload things just because I didn't actually use gradle in a week.

@marcphilipp

This comment has been minimized.

Copy link
Member

@marcphilipp marcphilipp commented Nov 5, 2018

@Barteks2x That's part of #6084.

@marcphilipp marcphilipp modified the milestones: 5.1 RC1, 5.2 RC1 Dec 4, 2018
@lukeu

This comment has been minimized.

Copy link
Contributor

@lukeu lukeu commented Dec 22, 2018

I praise the auto-cleanup feature, but I work on various projects (like open source, or in-house utils) intermittently. I might dip in only after 3, 6 or 12 months to tweak something. I might be offline / travelling when I do so, and suddenly I find I can't tweak something simple [Update: or even just launch the existing app from my IDE] because my day-to-day work has cleaned out required dependencies.

A thing that has hits me is that I sometimes customise Eclipse's classpath (as a pragmatic workaround or knocking up small utilities quickly) adding .jars directly from gradle's cache. I don't mind updating these occasionally (as new versions come in, old versions go away), but it would be good if I could extend the timeout so that it doesn't happen too often.

Finally, I frequently git-bisect way back in history: multiple years is not uncommon.

Disc space is a non-issue for me. I would use settings like 90/365.

@lukeu

This comment has been minimized.

Copy link
Contributor

@lukeu lukeu commented Dec 22, 2018

Just curious about a workaround. (My main projects will likely stay on 4.10 for some time longer. And hmm.... if any side-project happens to run 4.10 I guess it would zap the cache too.)

Maybe a cron-job? Would it be sufficient to say run gradlew tasks on all the projects I want to ensure aren't "cache flushed"? Or I would actually have to build them?

@marcphilipp

This comment has been minimized.

Copy link
Member

@marcphilipp marcphilipp commented Jan 2, 2019

gradlew tasks would not be sufficient, you'd have to build them in order to keep the dependencies in the cache.

@Clintonio

This comment has been minimized.

Copy link

@Clintonio Clintonio commented Jan 10, 2019

Just throwing in support for this issue. It will be very useful for CI agents to be able to decrease the time period for a company that has a wide range of different builds with different dependency sets but limited disc space on CI agents.

@big-guy big-guy modified the milestones: 5.2 RC1, 5.3 RC1 Jan 22, 2019
@big-guy big-guy added the @core label Jan 31, 2019
@big-guy big-guy modified the milestones: 5.3 RC1, 5.4 RC1 Jan 31, 2019
@big-guy big-guy modified the milestones: 5.4 RC1, 6.0 RC1 Mar 4, 2019
@Mahoney

This comment has been minimized.

Copy link

@Mahoney Mahoney commented Mar 18, 2019

Wouldn’t it make sense to be able to set a maximum cache size, and only use LRU ejection until the cache was below that limit?

@lukeu

This comment has been minimized.

Copy link
Contributor

@lukeu lukeu commented Mar 18, 2019

For the benefit of others watching / finding this, it looks like we can disable it completely with :

org.gradle.cache.cleanup=false

in the gradle.properties. See a7257ab for #6371 (Oct 2018, since v5.0)

Would be great to document this. (I'd been keeping an eye out for such a fix, but I missed it because the discussion here indicated it wasn't done & it appeared under "external contributions" in the 5.0 release notes, rather than closed-issues.)

@marcphilipp

This comment has been minimized.

Copy link
Member

@marcphilipp marcphilipp commented Mar 18, 2019

@lukeu It's not yet documented because it only works when specified in ~/.gradle/gradle.properties.

@wszeboreq

This comment has been minimized.

Copy link
Author

@wszeboreq wszeboreq commented Sep 11, 2019

Any chances it will be implemented in some upcoming release? We are still very interested in possibility of configuring the retention times - even via some unofficial properties.

@big-guy big-guy modified the milestones: 6.0 RC1, 6.2 RC1 Sep 18, 2019
@big-guy big-guy removed this from the 6.2 RC1 milestone Jan 8, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
8 participants
You can’t perform that action at this time.