[#412] Basic regression tests with different Gradle versions #414
Conversation
Due to incompatibility between Gradle 2 and 3.
Caused by: java.lang.NoSuchMethodError: org.gradle.api.tasks.TaskOutputs.file(Ljava/lang/Object;)Lorg/gradle/api/tasks/TaskOutputFilePropertyBuilder;
at org.shipkit.internal.gradle.notes.ReleaseNotesPlugin$2.execute(ReleaseNotesPlugin.java:83)
MethodHandles has been added in Java 7.
| @@ -1,10 +1,15 @@ | |||
| package org.shipkit.gradle | |||
|
|
|||
| import org.gradle.testkit.runner.BuildResult | |||
| import testutil.GradleSpecification | |||
|
|
|||
| class ShipkitGradlePluginIntegTest extends GradleSpecification { | |||
There was a problem hiding this comment.
Do those tests work cleanly from IDEA? It's super important that's easy to run them in IDEA.
There was a problem hiding this comment.
I would keep them in a separate sourceset otherwise:
Anyway, by default, just the current version is tested from IDE. It can be tweaked in Idea test runner to have more version tested if anyone needs to do that
| @@ -86,7 +85,9 @@ class ShipkitBintrayPluginTest extends PluginSpecification { | |||
| } | |||
|
|
|||
| def "prints informative message before upload"() { | |||
| BintrayUploadTask u = project.tasks[ShipkitBintrayPlugin.BINTRAY_UPLOAD_TASK] | |||
| //BintrayUploadTask cannot be referenced directly due to runtime error related to changes between Gradle 2 and 3: | |||
There was a problem hiding this comment.
Does it mean that Shipkit does not work with Gradle 2.x? :)
There was a problem hiding this comment.
No, it's a strange issue related to the fact that Bintray plugin is built with Gradle 2.4 and it fails in integration tests run with newer Gradle versions by TestKit (it can, as there were incompatible changes in Gradle). I wanted to reproduce it in a separate Gradle project with just the Bintray plugin applied (and newer Gradle version) to report the issue to them, but it worked fine in that configuration.
If you have some space time, you can try to trace the issue and help Bintray team solve it (nevertheless, it hasn't been reported before, therefore, it should be rare).
| .buildAndFail() | ||
| } catch (Exception e) { | ||
| println " ---- build.gradle ---- \n" + buildFile.text + "\n ------------------------" | ||
| throw e | ||
| } | ||
| } | ||
|
|
||
| protected List<String> skippedTaskPathsGradleBugWorkaround(String output) { |
There was a problem hiding this comment.
Nice catch and workaround. Thank you!
|
|
||
| @CompileStatic | ||
| @Slf4j | ||
| trait GradleVersionsDeterminer { |
There was a problem hiding this comment.
This is pretty cool, thank you! It's nice to have basic coverage for different Gradle versions!
.travis.yml
Outdated
| matrix: | ||
| include: | ||
| - jdk: oraclejdk8 | ||
| env: SHIPKIT_REGRESSION_TESTS=quick |
There was a problem hiding this comment.
Can you document the env variables, how they are supposed to be used by Shipkit engineers. Perhaps in the dev guide readme file / contributing file. What is tested when ran locally, what when ran by CI, how to debug when CI fails. We love knowledge :)
subprojects/shipkit/shipkit.gradle
Outdated
| @@ -25,7 +25,7 @@ dependencies { | |||
| testCompile("org.spockframework:spock-core:1.1-groovy-2.4") { | |||
| exclude module: "groovy-all" | |||
| } | |||
|
|
|||
| testCompile 'info.solidsoft.spock:spock-global-unroll:0.5.1' | |||
There was a problem hiding this comment.
I like software components from solidsoft domain :) What goodness this dependency brings in? Perhaps short comment in build.gradle.
There was a problem hiding this comment.
From https://github.com/szpak/spock-global-unroll :
[The] Spock extension automatically enabling parametrized tests unrolling in the whole project.
It is enough to put the spock-gradle-unroll JAR on a classpath. No additional configuration is required. All parametrized tests (features) will unrolled automatically as the @unroll annotation would be put on it.
I put a short comment in shipkit.gradle.
People should be able to upgrade from 3.0 to 3.5.1. In addition having tests for 2.14.1 it's likely to compatible with 3.0 if it is with 3.5.1.
Implementation of #412. Currently, the integration tests with TestKit in the project covers only very basic elements, however, it detected an issue with Gradle 2 compatibility after an upgrade to Gradle 3+ (#413). The mechanism will be possible to reuse in new tests.
By default only current Gradle version is used for regression testing. Travis is configured to test with a few more versions.
Some other minor changes related to integration tests have been made.
In fact, I wanted to upgrade ShipKit to Gradle 4 to follow the Gradle development, however, until #413 is resolved or it is decided to drop Gradle 2 support (probably it's good for the projects to migrate anyway) it is problematic.
Btw, related TestKit bug in Gradle 4.0+ - gradle/gradle#2732
Closes #412.