Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
Separate the resolution cache and repository cache in Ivy #5844
Ivy separates it's notion of repository cache (where the jars are stored) from the resolution cache (where it's xml files are stored). See http://ant.apache.org/ivy/history/2.4.0/settings/caches.html for more information. In pants you cant separate the notion of the two cache directories so if you want to share a cache with multiple workers they will overwrite and delete the others resolution xml files.
Separate the notion of repository cache from resolution cache so you can share jars between builds and they won't overwrite or delete the intermediate xml files.
ivy-cache can still be specified and will be the default for the resolution and repository cache if they are not specified.
With this change we can share a repository cache across our CI workers which reduces the test start up time.
In order for it to work you need to specify the settings in pants.ini e.g.
And update the ivysettings.xml with the repository cache information
left a comment
For the record: Twitter has switched completely off of Ivy to coursier, which has a concurrency safe cache, and much, much better resolution throughput. See https://groups.google.com/d/topic/pants-devel/fsNOo_iaE40/discussion . cc @wisechengyi
4 times, most recently
May 22, 2018
Oh sorry, I accidentally made the PR off a branch from master.
Is the only blocking feedback to deprecate the cache-dir option? I was thinking we should keep that around so it works similar to the ivy config which has
Works for me, assuming there is sufficient validation in place. Will take another look.