Improve woodpecker build times #1139
Conversation
|
Hmm it is still downloading gradle, even though we are using the exact version as the one defined in the dockerfile |
|
12m36s new record, but uses gradle instead of the gradlew, I rather have the cache working |
|
@dessalines could those secrets be set |
|
Once the person who runs that gets back to me, and I update those creds, I'll trigger this a few times to see if it works. |
|
I don't think it works yet, couldn't test it really locally. But I think this has the potential to have huge time savings due to configuration-cache. It will be able to completely reuse steps from previous iterations. |
|
I added the secrets to woodpecker, I'll see if it works. |
|
It did seem to correctly save the cache, so we'll have to see if it works after we merge, and tweak it as necessary. |
|
The cache isn't doing anything though. I don't think its being correctly connected to the cimg/android containers |
|
Well now it works, took 8m38s, which 2m30s were spent on the cache itself. I wonder how much it is storing and if it storing stuff we don't need. @dessalines can you see the contents of that cache? |
|
lmao it does 6min now without the cache |
|
The caching system does not seem to be very reliable, if it deems to be too unreliable I'll remove it. If the cache would have been hosted within the same subnetwork as these agents. Then it would been considerably faster but as of now it does take up some significant time. A better solution would ve been to use a volume to store the cache, much faster and no network failures |
|
The build on that last one still took 30m, so it seems the cache isn't even doing anything. |
|
Oh but it does, you can tell because it didn't download the wrapper. I made it so that it only saves the cache if the gradle.kts changes. Sometimes the CI is pretty slow |
Set the same versions used as the one in the "new" woodpecker config. This reduces the build time as it does not have to download gradle on each task as it is not cached. And has to install build tools. This also removes a failure point of woodpecker as it occasionally fails at doing these tasks.
Removes .depcache, did not work anyway and if you did want to make it work it requires a lot of manual maintenance to keep it working. Essential you have to collect the cache manually and upload to it each agent (or retrieve it from a other system) and have to redo that each time gradle/ dependency changes https://docs.gradle.org/current/userguide/dependency_resolution.html#sub:shared-readonly-cache
Set a fixed buildtools version the same as the pipeline prevents having to reinstall it each time.
Setups caching mechanism that should cache the gradle wrapper and the .gradle caches