Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Make UI test results properly cacheable in the CI #950

Closed
wants to merge 5 commits into from
Closed

Make UI test results properly cacheable in the CI #950

wants to merge 5 commits into from

Conversation

PawelLipski
Copy link
Collaborator

@PawelLipski PawelLipski commented Jun 28, 2022 •
edited

Now if there's no change in production code, or UI tests (so e.g. just tests or CI config or build config changed but with no change to the published plugin itself), then UI tests are not being rerun in the CI.

Here are the build times (with binary compatibility check aka ./gradlew runPluginVerifier skipped, coz it's uncacheable as for now):

image

☝🏻 16 minutes is to build from scratch, 2:30 to rerun when nothing changed in the plugin.
Used to be like 8-10 minutes to rerun coz UI tests were always re-executed, caching didn't work :/

This PR includes both a fix for #945 and a completely unrelated workaround for JetBrains/intellij-platform-gradle-plugin#1039 (which won't be needed soon once a fix to gradle-intellij-plugin is released).

@PawelLipski PawelLipski self-assigned this Jun 28, 2022
@PawelLipski PawelLipski linked an issue Jun 28, 2022 that may be closed by this pull request
uiTest_<intellij-version> Gradle tasks are effectively uncachable #945
Closed
@PawelLipski PawelLipski added this to the v3.0.0 milestone Jun 28, 2022
@PawelLipski PawelLipski added testing Relates to tests and testability code quality Code quality build ci performance and removed code quality Code quality labels Jun 28, 2022
PawelLipski
PawelLipski commented Jun 28, 2022
View reviewed changes
.circleci/config.yml
# language=sh
command: |
./gradlew test
# TODO (JetBrains/gradle-intellij-plugin#1039): during tests, classpath.index file is created by com.intellij.util.lang.PathClassLoader
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good Tzar Gradle, bad boyars plugins

PawelLipski
PawelLipski commented Jun 28, 2022
View reviewed changes
.circleci/config.yml Outdated
@@ -28,12 +28,12 @@ jobs:
keys:
# First, try finding a cache entry with the same set of libraries
# that also comes from the same branch (so as to make the best use of Gradle compilation cache).
- gradle-deps-v5-{{ checksum "gradle/libs.versions.toml" }}-{{ .Branch }}
- gradle-deps-v22-{{ checksum "gradle/libs.versions.toml" }}-{{ .Branch }}
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:ups:

PawelLipski
PawelLipski commented Jun 28, 2022
View reviewed changes
uiTests/build.gradle
@@ -0,0 +1,58 @@
apply plugin: 'org.gradle.scala'
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is adapted with some changes from the deleted fragment of build.gradle

PawelLipski
PawelLipski commented Jun 28, 2022
View reviewed changes
uiTests/build.gradle

dependencies {
// Needed so that the main classpath of the plugin is considered by Gradle as an input to `uiTest` task.
// Otherwise, changes in the plugin would NOT cause the UI tests to be re-executed when caching is enabled.
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the new addition. Previously, UI tests resided in root project, so they depended on src/main code of root project anyway. Now they're in a subproject... so I needed to add this atypical construct (subproject depending on root project, not the other way round)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This entire extraction to a separate subproject was needed as fix (workaround, kinda, again...) for #945

@PawelLipski PawelLipski marked this pull request as ready for review June 28, 2022 13:50
@PawelLipski PawelLipski marked this pull request as draft June 28, 2022 14:39
@PawelLipski
Copy link
Collaborator Author

Meh this is hopeless, too much complication in Gradle config for too little benefit, closing :trollface:

This solution is far from complete, it turned out that the current setup has a side effect of messing up with :jar task... specifically, jar manifests are getting overwritten from the complex form written by gradle-intellij-plugin:

==> ./frontend/icons/build/tmp/jar/MANIFEST.MF <==
Manifest-Version: 1.0
Created-By: Gradle 7.4.2
Build-JVM: 11.0.15 (Red Hat, Inc. 11.0.15+10)
Version: unspecified
Build-Plugin: Gradle IntelliJ Plugin
Build-Plugin-Version: 1.6.0
Build-OS: Linux 5.17.5-300.fc36.x86_64 amd64
Build-SDK: IC-2022.1.1

into a simple default:

==> ./frontend/icons/build/tmp/jar/MANIFEST.MF <==
Manifest-Version: 1.0

when uiTest is executed. Thus, caching gets screwed up again ☹️

@PawelLipski PawelLipski modified the milestones: v3.0.0, v2.0.2 Jun 30, 2022
@mkondratek mkondratek deleted the bugfix/945-ui-test-cache branch October 20, 2022 16:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@azdrojowa123 azdrojowa123 Awaiting requested review from azdrojowa123

@mkondratek mkondratek Awaiting requested review from mkondratek

Assignees

@PawelLipski PawelLipski

Labels
build ci performance testing Relates to tests and testability
Projects
None yet
Milestone
v2.1.0
Development

Successfully merging this pull request may close these issues.

uiTest_<intellij-version> Gradle tasks are effectively uncachable
1 participant
@PawelLipski