infra: added quarterly cache clean up for travis #931

rnveach · 2022-11-04T03:54:37Z

Similar to checkstyle/checkstyle#12365 (comment)

rnveach · 2022-11-04T03:59:27Z

Will change to quarterly after run and stats.

romani

90 might be better period to minimize downloads and keep removing old artifacts.

coveralls · 2022-11-04T04:10:30Z

Coverage remained the same at 98.798% when pulling a2c8961 on rnveach:monthly-cache-cleanup into c74de7f on sevntu-checkstyle:master.

rnveach · 2022-11-04T04:13:04Z

Is interesting here. Master is 2600 megs but this PR only reduced it to 1,000 megs (38% reduction).

It is a similar savings, but I am surprised we have 1 gig of dependencies.

rnveach · 2022-11-04T04:14:18Z

90 might be better

Done.

rnveach · 2022-11-04T04:33:56Z

https://app.travis-ci.com/github/sevntu-checkstyle/sevntu.checkstyle/jobs/587535354

[WARNING] Error initializing: org.eclipse.tycho.p2.resolver.P2DependencyResolver@10db6131
java.lang.RuntimeException: java.io.IOException: File /home/travis/.m2/repository/org/eclipse/tycho/tycho-bundles-external/1.6.0/eclipse/configuration/config.ini does not exist

Rerunning to see if it is stable.

rnveach · 2022-11-04T04:46:19Z

From my local:

/.m2/repository/org/eclipse/tycho/tycho-bundles-external/1.6.0$ find -follow
.
./_remote.repositories
./tycho-bundles-external-1.6.0.zip
./tycho-bundles-external-1.6.0.zip.sha1
./eclipse
./eclipse/configuration
./eclipse/configuration/config.ini
./eclipse/.eclipseproduct
./eclipse/plugins
./eclipse/plugins/org.sat4j.core_2.3.5.v201308161310.jar
./eclipse/plugins/org.eclipse.equinox.p2.jarprocessor_1.1.500.v20191014-1732.jar
./eclipse/plugins/org.eclipse.equinox.p2.updatesite_1.1.300.v20191016-0515.jar
./eclipse/plugins/org.tukaani.xz_1.8.0.v20180207-1613.jar
./eclipse/plugins/org.eclipse.equinox.p2.artifact.repository_1.3.300.v20191015-1504.jar
./eclipse/plugins/org.eclipse.equinox.p2.garbagecollector_1.1.300.v20191015-1207.jar
./eclipse/plugins/org.eclipse.core.contenttype_3.7.500.v20190916-2125.jar
./eclipse/plugins/org.eclipse.equinox.p2.metadata_2.4.600.v20191002-0702.jar
./eclipse/plugins/org.eclipse.core.jobs_3.10.600.v20191122-2104.jar
./eclipse/plugins/org.eclipse.equinox.concurrent_1.1.400.v20190621-0852.jar
./eclipse/plugins/org.eclipse.tycho.noopsecurity_1.6.0.jar
./eclipse/plugins/org.apache.commons.codec_1.10.0.v20180409-1845.jar
./eclipse/plugins/org.apache.httpcomponents.httpclient_4.5.6.v20190503-0009.jar
./eclipse/plugins/org.eclipse.equinox.simpleconfigurator.manipulator_2.1.400.v20191015-1836.jar
./eclipse/plugins/org.eclipse.ecf.provider.filetransfer.httpclient45_1.0.100.v20191012-1656.jar
./eclipse/plugins/org.eclipse.equinox.security_1.3.400.v20191008-0645.jar
./eclipse/plugins/org.eclipse.equinox.registry_3.8.600.v20191017-2055.jar
./eclipse/plugins/org.eclipse.equinox.p2.engine_2.6.500.v20191014-1645.jar
./eclipse/plugins/org.eclipse.core.runtime_3.17.0.v20191122-2104.jar
./eclipse/plugins/org.eclipse.osgi.services_3.8.0.v20190206-2147.jar
./eclipse/plugins/org.sat4j.pb_2.3.5.v201404071733.jar
./eclipse/plugins/org.eclipse.core.net_1.3.700.v20191122-2107.jar
./eclipse/plugins/org.eclipse.equinox.p2.publisher.eclipse_1.3.400.v20191018-2256.jar
./eclipse/plugins/org.eclipse.equinox.p2.transport.ecf_1.2.300.v20191001-0955.jar
./eclipse/plugins/org.eclipse.osgi_3.15.100.v20191114-1701.jar
./eclipse/plugins/org.eclipse.equinox.p2.repository.tools_2.2.400.v20191016-0510.jar
./eclipse/plugins/org.eclipse.ecf.ssl_1.2.300.v20191012-1926.jar
./eclipse/plugins/org.eclipse.equinox.ds_1.6.100.v20191001-1333.jar
./eclipse/plugins/org.eclipse.equinox.app_1.4.300.v20190815-1535.jar
./eclipse/plugins/org.eclipse.ecf.filetransfer_5.0.300.v20191020-1846.jar
./eclipse/plugins/org.eclipse.ecf_3.9.4.v20191020-1846.jar
./eclipse/plugins/org.eclipse.equinox.common_3.10.600.v20191004-1420.jar
./eclipse/plugins/org.apache.commons.logging_1.2.0.v20180409-1502.jar
./eclipse/plugins/org.eclipse.equinox.simpleconfigurator_1.3.400.v20191015-1836.jar
./eclipse/plugins/org.eclipse.equinox.frameworkadmin_2.1.400.v20191002-0702.jar
./eclipse/plugins/org.eclipse.equinox.p2.publisher_1.5.300.v20191016-0507.jar
./eclipse/plugins/org.eclipse.equinox.p2.core_2.6.200.v20191014-1220.jar
./eclipse/plugins/org.eclipse.equinox.frameworkadmin.equinox_1.1.200.v20191015-1207.jar
./eclipse/plugins/org.eclipse.equinox.p2.director.app_1.1.400.v20191015-1945.jar
./eclipse/plugins/org.eclipse.equinox.preferences_3.7.600.v20191017-2055.jar
./eclipse/plugins/org.apache.felix.scr_2.1.14.v20190123-1619.jar
./eclipse/plugins/org.eclipse.equinox.launcher_1.5.600.v20191014-2022.jar
./eclipse/plugins/org.eclipse.osgi.util_3.5.300.v20190708-1141.jar
./eclipse/plugins/org.eclipse.equinox.p2.director_2.4.500.v20191014-1220.jar
./eclipse/plugins/org.eclipse.equinox.p2.touchpoint.natives_1.3.400.v20191017-1917.jar
./eclipse/plugins/org.eclipse.equinox.p2.metadata.repository_1.3.300.v20191002-0702.jar
./eclipse/plugins/org.eclipse.ecf.provider.filetransfer.ssl_1.0.101.v20191012-1926.jar
./eclipse/plugins/org.eclipse.equinox.p2.touchpoint.eclipse_2.2.500.v20191114-1206.jar
./eclipse/plugins/org.eclipse.osgi.compatibility.state_1.1.600.v20190814-1451.jar
./eclipse/plugins/org.eclipse.equinox.p2.repository_2.4.600.v20191016-0510.jar
./eclipse/plugins/org.eclipse.ecf.identity_3.9.300.v20191020-1846.jar
./eclipse/plugins/org.apache.httpcomponents.httpcore_4.4.10.v20190123-2214.jar
./eclipse/plugins/org.eclipse.ecf.provider.filetransfer_3.2.500.v20191017-1905.jar

Haven't checked travis, but it seems like tycho is using this folder for its own purposes and doesn't realize when things are partially removed.

romani · 2022-11-04T04:50:52Z

Maybe it s better to just remove whole cache ones in several months( Travis cron execution) or skip eclipse from removal.

rnveach · 2022-11-04T04:59:32Z

Confirmed on my local that tycho will reset when the whole directory is deleted and not fail.

It must unzip the zip file it stores there as the dates on the newly created files are 2020, so that explains why they were slated for deletion. The zip file has todays date and that is why it wasn't deleted.

rnveach · 2022-11-04T05:03:55Z

Adding -type d will ensure we delete the whole directory but we would still require to ignore this directory and sub-folders.

https://stackoverflow.com/questions/13868821/shell-script-to-delete-directories-older-than-n-days

rnveach · 2022-11-09T18:48:43Z

.ci/travis.sh

@@ -37,6 +37,10 @@ run-command)
  fi
  ;;

+quarterly-cache-cleanup)
+  find ~/.m2 -maxdepth 4 -type d -mtime +90 -exec rm -rf {} \;


maxdepth is to not go too far into tycho. We can exclude a specific directory, but it doesn't seem possible to exclude it's children. So this was needed for that.

-type d is to only target full directory deletions.

rnveach · 2022-11-09T19:35:18Z

CI failure is only from same eclipse-cs test. I believe this to be the same cache issue from the first deletion attempt and it will not be fixed until the cache is manually cleared just this one time. It is not an issue from the new changes, IMO.

romani

Ok to merge if it works and reduses something

rnveach · 2022-11-15T22:23:11Z

Just blocking again until issues settle down in main repo.

rnveach self-assigned this Nov 4, 2022

romani approved these changes Nov 4, 2022

View reviewed changes

rnveach force-pushed the monthly-cache-cleanup branch from f9700ed to fda8c3f Compare November 4, 2022 04:13

romani approved these changes Nov 4, 2022

View reviewed changes

rnveach added the blocked label Nov 4, 2022

infra: added quarterly cache clean up for travis

a2c8961

rnveach force-pushed the monthly-cache-cleanup branch from fda8c3f to a2c8961 Compare November 9, 2022 18:46

rnveach commented Nov 9, 2022

View reviewed changes

rnveach removed the blocked label Nov 9, 2022

rnveach assigned romani and unassigned rnveach Nov 9, 2022

rnveach requested a review from romani November 9, 2022 18:49

rnveach changed the title ~~infra: added monthly cache clean up for travis~~ infra: added quarterly cache clean up for travis Nov 9, 2022

romani approved these changes Nov 12, 2022

View reviewed changes

romani assigned rnveach and unassigned romani Nov 12, 2022

rnveach added the blocked label Nov 15, 2022

rnveach mentioned this pull request Nov 28, 2022

config: upgrade eclipsecs-sevntu-plugin to 10.4 #941

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

infra: added quarterly cache clean up for travis #931

infra: added quarterly cache clean up for travis #931

rnveach commented Nov 4, 2022

rnveach commented Nov 4, 2022

romani left a comment

coveralls commented Nov 4, 2022 •

edited

rnveach commented Nov 4, 2022 •

edited

rnveach commented Nov 4, 2022

rnveach commented Nov 4, 2022

rnveach commented Nov 4, 2022

romani commented Nov 4, 2022 •

edited

rnveach commented Nov 4, 2022 •

edited

rnveach commented Nov 4, 2022

rnveach Nov 9, 2022

rnveach commented Nov 9, 2022 •

edited

romani left a comment

rnveach commented Nov 15, 2022

infra: added quarterly cache clean up for travis #931

Are you sure you want to change the base?

infra: added quarterly cache clean up for travis #931

Conversation

rnveach commented Nov 4, 2022

rnveach commented Nov 4, 2022

romani left a comment

Choose a reason for hiding this comment

coveralls commented Nov 4, 2022 • edited

rnveach commented Nov 4, 2022 • edited

rnveach commented Nov 4, 2022

rnveach commented Nov 4, 2022

rnveach commented Nov 4, 2022

romani commented Nov 4, 2022 • edited

rnveach commented Nov 4, 2022 • edited

rnveach commented Nov 4, 2022

rnveach Nov 9, 2022

Choose a reason for hiding this comment

rnveach commented Nov 9, 2022 • edited

romani left a comment

Choose a reason for hiding this comment

rnveach commented Nov 15, 2022

coveralls commented Nov 4, 2022 •

edited

rnveach commented Nov 4, 2022 •

edited

romani commented Nov 4, 2022 •

edited

rnveach commented Nov 4, 2022 •

edited

rnveach commented Nov 9, 2022 •

edited