This is a hands-on exercise to go along with the Maintaining Optimal Gradle Build Cache Performance training module. In this exercise you will go over the following:
- Using the Develocity Build Validation Scripts to identify build caching issues
- Finished going through the relevant sections in the training course
This repository contains the project with build caching issues. We will use the Develocity Build Validation Scripts to identify these issues and address them.
- Clone this repository in a new directory location on your machine
In the rest of this README the location will be referred to as /path/to/dpeuni-gradle-maintain-cache-perf.
Whenever you see this, replace it with the location you are using.
We will use the DPE University Develocity instance as the remote cache. If you haven't already done so, you can authenticate with the Develocity service by running:
./gradlew provisionGradleEnterpriseAccessKeyThe output of the task will indicate a browser window will come up from which you can complete the authentication:
Once the browser window comes up you can enter a title for the access key that will be created or go with the suggested title:
Once confirmed you will see the following message and you can close the browser window and return to the editor:
-
Open the Develocity Build Validation Scripts installation instructions
-
Copy the installation command
-
Open a terminal, create a new directory for the installation location and run the command there. The installation location should be outside the project directory.
-
Go to the directory containing the Develocity Build Validation Scripts, you will see contents similar to:
$ ls -ltr
total 192
-rw-r--r-- 1 adayal wheel 862 Oct 25 2023 network.settings
-rw-r--r-- 1 adayal wheel 438 Oct 25 2023 mapping.example
drwxr-xr-x 27 adayal wheel 864 Oct 25 2023 lib
-rw-r--r-- 1 adayal wheel 6 Oct 25 2023 VERSION
-rw-r--r-- 1 adayal wheel 404 Oct 25 2023 README.md
-rw-r--r-- 1 adayal wheel 11356 Oct 25 2023 LICENSE
-rwxr-xr-x 1 adayal wheel 16628 Oct 25 2023 05-validate-remote-build-caching-ci-local.sh
-rwxr-xr-x 1 adayal wheel 10954 Oct 25 2023 04-validate-remote-build-caching-ci-ci.sh
-rwxr-xr-x 1 adayal wheel 11958 Oct 25 2023 03-validate-local-build-caching-different-locations.sh
-rwxr-xr-x 1 adayal wheel 10874 Oct 25 2023 02-validate-local-build-caching-same-location.sh
-rwxr-xr-x 1 adayal wheel 11336 Oct 25 2023 01-validate-incremental-building.shLet's run experiment 2 to identify local caching issues when the build is run from the same location.
-
Go to the directory where the Develocity Build Validation Scripts are installed
-
Run the second script in interactive mode:
./02-validate-local-build-caching-same-location.sh -i -
Press the Enter key to start the experiment as the following prompt indicates:
Press <Enter> to get started with the experiment.4. The project does not use the Common Custom User Data Gradle plugin, and we will proceed without it. Press the Enter key when you see the following prompt:
Press <Enter> once you have (optionally) configured your build with the Common Custom User Data Gradle plugin and pushed the changes.- You should have already authenticated with Develocity, so press the Enter key when you see the following prompt:
Press <Enter> once you have (optionally) adjusted your access permissions and configured the API credentials on your machine.- Enter the file path to the project using the
file://URL. Replace/path/to/dpeuni-gradle-maintain-cache-perfwith the actual location:
What is the URL for the Git repository that contains the project to validate? file:///path/to/dpeuni-gradle-maintain-cache-perf- For the next 4 questions use the default values (as denoted by the values in parentheses) by pressing the Enter key without typing anything else:
What is the branch for the Git repository that contains the project to validate? <the repository's default branch>
What is the commit id for the Git repository that contains the project to validate? <the branch's head>
What are additional options to use when cloning the Git repository? --depth=1
What is the directory to invoke the build from? <the repository's root directory>- When the script asks what task to invoke, specify the
buildtask:
What are the Gradle tasks to invoke? build- For the last question, press Enter without typing anything else:
What are additional cmd line arguments to pass to the build invocation? <none>- When you see the following prompt, press Enter:
Press <Enter> to check out the project from Git.It will create a separate clone of the repository for the experiment.
NOTE: If you get an error here, check the file URL of the repository.
- When it prompts to run the first build press Enter:
Press <Enter> to run the first build of the experiment.It will run the build task on the project.
- When it prompts to run the second build press Enter:
Press <Enter> to run the second build of the experiment.It will run the clean task followed by the build task on the project.
- After it has finished, it will prompt you to continue and look at the results. Press the Enter key:
Press <Enter> to measure the build results.In the Performance Characteristics section, it will inform you there is one
task with a potential build caching issue.
- Open the
Executed cacheable taskslink. It will inform you the:app:testtask was executed even though it should have gotten it's output from the cache.
- Open the
Task inputs comparisonlink. It will inform you that themodel-1.0.jarfile from themodelsubproject changed. This is an input to the:app:testtask.
- Clear the filters on the top right of the page
- Expand the
jartask on themodelsubproject. You will see themodel/build/tmp/jar/MANIFEST.MFfile which is an input to this task changed.
- In the terminal, notice the
Experiment artifact dirin theSummarysection:
All the files for the two builds are in this directory.
- Run a
diffto see how themodel/build/tmp/jar/MANIFEST.MFfile changed between the builds:
NOTE: For each experiment there exists a latest symlink that always points to the most recent experiment's artifact directory.
Otherwise, adjust the path for the diff as per your Experiment artifact dir.
$ diff .data/02-validate-local-build-caching-same-location/latest/first*/model/build/tmp/jar/MANIFEST.MF .data/02-validate-local-build-caching-same-location/latest/second*/model/build/tmp/jar/MANIFEST.MF
2c2
< Implementation-Timestamp: 2024-05-17T13:49:25.219892Z
---
> Implementation-Timestamp: 2024-05-17T13:50:52.688382ZWe can see a timestamp is changing in the file. You may recognize this issue from a previous training.
- We can use runtime classpath normalization to ignore the timestamp attribute. Open
app/build.gradle.ktsand add the following:
normalization {
runtimeClasspath {
metaInf {
ignoreAttribute("Implementation-Timestamp")
}
}
}- Commit the change to the local repository:
$ git commit -am "Use runtime classpath normalization to ignore timestamp attribute in manifest file"- Run the experiment again. At the end of the output you will see a command you can run to do this:
You should see no issues found this time. If you do, be sure your changes are committed. This is required because the experiments always use a fresh clone of the repository.
Each experiment identifies different kinds of build caching issues. Using experiment 2 we identified and fixed one issue. Now let's run the next experiment to see if any other issues are identified.
-
Go to the directory where the Develocity Build Validation Scripts are installed
-
You can run the third script in interactive mode, however since it is similar to the second script, we can run it using the command as follows:
NOTE: Replace /path/to/dpeuni-gradle-maintain-cache-perf with the location of the project.
./03-validate-local-build-caching-different-locations.sh -r file:///path/to/dpeuni-gradle-maintain-cache-perf -t buildThe experiment will run and find 1 task with a potential build caching issue:
- Open the
Executed cacheable taskslink. It will inform you the:app:countSrctask was executed even though it should have gotten it's output from the cache.
- Open the
Task inputs comparisonlink. It will inform you that theapp/srcinput for the task has different absolute paths.
That tells us that instead of using relative paths, this input has it's
path sensitivity set to absolute. This experiment runs the two builds in
different directories, so the absolute paths would have been different for
this input.
Let us confirm this is the issue.
5. Open app/build.gradle.kts and go to the countSrc task configuration.
You will see the srcDir input has been set to use absolute path sensitivity
When you encounter this, you should consider if the input really requires the full absolute path, or if using the relative path is good enough.
-
Change the
path sensitivityto relative. -
Commit the change to the local repository:
$ git commit -am "countSrc task's srcDir input should have relative path sensitivity"- Run the experiment again:
NOTE: Replace /path/to/dpeuni-gradle-maintain-cache-perf with the location of the project.
./03-validate-local-build-caching-different-locations.sh -r file:///path/to/dpeuni-gradle-maintain-cache-perf -t buildYou should no longer see any issues. If you do, be sure your changes are committed. This is required because the experiments always use a fresh clone of the repository.















