Support composite builds of multimodule gradle projects #29233

tomas1885 · 2022-11-13T12:24:13Z

Support composite builds of multimodule gradle projects.
Support hot reloading and source watching composite builds.
Support using local extensions as a composite build.

quarkus-bot · 2022-11-13T12:24:15Z

Thanks for your pull request!

The title of your pull request does not follow our editorial rules. Could you have a look?

title should not end up with dot

This message is automatically generated by a bot.

tomas1885 · 2022-11-13T12:27:39Z

fixes #29234

glefloch · 2022-11-14T14:40:05Z

Thanks for this contribution @tomas1885, this looks good to me.

Could you also add an integration test here : https://github.com/quarkusio/quarkus/tree/main/integration-tests/gradle

tomas1885 · 2022-11-14T16:37:47Z

Hi @glefloch, can you give me an hint on how to run a single integration test locally without running the whole gradle plugin test suite?
I also noticed the contribution guide mentions that some extra steps are needed when adding integration tests (not sure if that applies to integration test of an existing module).

tomas1885 · 2022-11-14T18:54:51Z

Hi @glefloch, I added 2 integration tests for this fix, one for dev mode and another for build.
Currently no integration tests for extensions project, I might be able to add these in the next couple of days if necessary.

aloubyansky · 2022-11-14T19:29:18Z

...-model/src/main/java/io/quarkus/gradle/dependency/ApplicationDeploymentClasspathBuilder.java

@@ -213,7 +214,9 @@ private void setUpDeploymentConfiguration() {
                    final DependencyHandler dependencies = project.getDependencies();
                    final Set<Dependency> deploymentDependencies = new HashSet<>();
                    for (ExtensionDependency extension : extensions) {
-                        if (extension instanceof LocalExtensionDependency) {
+                        if (extension instanceof IncludedBuildExtensionDependency) {
+                            deploymentDependencies.add(((IncludedBuildExtensionDependency) extension).getDeployment());


Could we do it the way it's done below? i.e.

deploymentDependencies.add(dependencies.create( extension.getDeploymentModule().getGroupId() + ":" + extension.getDeploymentModule().getArtifactId() + ":" + extension.getDeploymentModule().getVersion()));

It might work, as it creates a similar object (the one returned from the extension object is an ExternalModuleDependecy while the one returned from the dependencies.create is a decorated object), in any case we would need to differentiate between a locally built extension and a composite built one. I would rather consolidate these 3 type into a self describing one, but didn't want to change the current code to that extent. I also dislike the addition of the boolean parameter to the loadExtensionInfo in the ToolingUtils class, but again, didn't want to change the code to that extent.

aloubyansky · 2022-11-14T19:44:05Z

devtools/gradle/gradle-model/src/main/java/io/quarkus/gradle/tooling/ToolingUtils.java

+    public static Project includedBuildProject(IncludedBuildInternal includedBuild,
+            final ProjectComponentIdentifier componentIdentifier) {
+        return includedBuild.getTarget().getMutableModel().getRootProject().findProject(
+                componentIdentifier.getProjectPath());


Is there a way to get hold of a project using non-internal API?

I was also kind of hesitant to use this, as relaying on Internal implementation details usually has ‏consequences, but haven't found a different way. I believe there's no other way, as the current Gradle project (under build) is by definition unrelated to the included build, so getting a reference to the other Gradle project can't be done through the current Gradle project object graphs.

aloubyansky · 2022-11-17T16:00:47Z

...s/quarkus-example-extension/runtime/src/main/resources/META-INF/quarkus-extension.properties

@@ -0,0 +1 @@
+deployment-artifact=org.acme.extensions\:quarkus-example-extension\:1.0


Suggested change

deployment-artifact=org.acme.extensions\:quarkus-example-extension\:1.0

deployment-artifact=org.acme.extensions\:quarkus-example-extension-deployment\:1.0

aloubyansky · 2022-11-17T16:02:30Z

...ensions/quarkus-example-extension/runtime/src/main/resources/META-INF/quarkus-extension.yaml

+#  extension-dependencies:
+#    - "io.quarkus:quarkus-arc::jar"
+#    - "io.quarkus:quarkus-core::jar"
+#artifact: "com.neema:quarkus-logzio::jar:@version@"


The elements commented above are supposed to be filled in by the generator, aren't they? The comments should be removed either way.

tomas1885 · 2022-11-21T05:56:18Z

@aloubyansky Any hint on how to resolve the CI issue with the missing dependency version for the composite build integration test with the local extension? The actual integration test works, but it seems to fail due to missing version in the whole gradle integration tests POM.

aloubyansky · 2022-11-21T22:12:58Z

Good question. @famod do you have any suggestion?
@tomas1885 you could try looking into the update-dependencies.sh why it's picking it up. There are other extensions in the testsuite that are not "noticed" by the script. It looks like it's collecting artifacts with quarkus- suffix. Perhaps, you could try renaming quarkus-example-extension to simply example-extension or something?

famod · 2022-11-22T00:17:21Z

Yeah, either rename it so that it's not picked up or remove it from the list of artifact ids to be processed: https://github.com/quarkusio/quarkus/blob/main/integration-tests/gradle/update-dependencies.sh#L40
E.g. add -e '/quarkus-example-extension/d' (if you take this route, please also extend the comment here: https://github.com/quarkusio/quarkus/blob/main/integration-tests/gradle/update-dependencies.sh#L37)

aloubyansky · 2022-11-22T21:10:41Z

@tomas1885 thanks for your dedication so far :) we'll need your commits to be squashed before we merge it, btw.

tomas1885 · 2022-11-23T18:12:28Z

@aloubyansky I squashed my commits but I see this PR carries extra commits from a main branch sync I did after the first commit for this branch, do I need to rebase everything on top? Should I resync with the latest main again? What's your preferred practice?

aloubyansky · 2022-11-23T18:49:04Z

We want only your changes in the PR. Rebasing and squashing is usually all that's necessary.

tomas1885 · 2022-11-28T10:48:28Z

Hi @aloubyansky, I squashed the commit a few days ago, forgot to ping you, is there anything else that should be done here?

aloubyansky · 2022-11-28T11:29:46Z

...s/gradle/src/main/resources/basic-composite-build-extension-project/application/build.gradle

+dependencies {
+    implementation enforcedPlatform("${quarkusPlatformGroupId}:${quarkusPlatformArtifactId}:${quarkusPlatformVersion}")
+    testImplementation 'org.junit.jupiter:junit-jupiter-api:5.8.1'
+    testRuntimeOnly 'org.junit.jupiter:junit-jupiter-engine:5.8.1'


Looks like I missed these versions. The junit dependencies are included in the quarkus-bom and they are 5.9.1 now. Could you simply remove the versions?

aloubyansky · 2022-11-28T11:30:14Z

...e/src/main/resources/basic-composite-build-extension-project/libraries/libraryA/build.gradle

+    implementation platform("${quarkusPlatformGroupId}:${quarkusPlatformArtifactId}:${quarkusPlatformVersion}")
+    implementation ("${quarkusPlatformGroupId}:quarkus-arc:${quarkusPlatformVersion}")
+    testImplementation 'org.junit.jupiter:junit-jupiter-api:5.8.1'
+    testRuntimeOnly 'org.junit.jupiter:junit-jupiter-engine:5.8.1'


aloubyansky · 2022-11-28T11:30:26Z

...e/src/main/resources/basic-composite-build-extension-project/libraries/libraryB/build.gradle

+    implementation platform("${quarkusPlatformGroupId}:${quarkusPlatformArtifactId}:${quarkusPlatformVersion}")
+    implementation ("${quarkusPlatformGroupId}:quarkus-arc:${quarkusPlatformVersion}")
+    testImplementation 'org.junit.jupiter:junit-jupiter-api:5.8.1'
+    testRuntimeOnly 'org.junit.jupiter:junit-jupiter-engine:5.8.1'


aloubyansky · 2022-11-28T11:30:37Z

...ation-tests/gradle/src/main/resources/basic-composite-build-project/application/build.gradle

+dependencies {
+    implementation enforcedPlatform("${quarkusPlatformGroupId}:${quarkusPlatformArtifactId}:${quarkusPlatformVersion}")
+    testImplementation 'org.junit.jupiter:junit-jupiter-api:5.8.1'
+    testRuntimeOnly 'org.junit.jupiter:junit-jupiter-engine:5.8.1'


aloubyansky · 2022-11-28T11:30:53Z

...ests/gradle/src/main/resources/basic-composite-build-project/libraries/libraryA/build.gradle

+    implementation platform("${quarkusPlatformGroupId}:${quarkusPlatformArtifactId}:${quarkusPlatformVersion}")
+    implementation ("${quarkusPlatformGroupId}:quarkus-arc:${quarkusPlatformVersion}")
+    testImplementation 'org.junit.jupiter:junit-jupiter-api:5.8.1'
+    testRuntimeOnly 'org.junit.jupiter:junit-jupiter-engine:5.8.1'


aloubyansky · 2022-11-28T11:31:06Z

...ests/gradle/src/main/resources/basic-composite-build-project/libraries/libraryB/build.gradle

+    implementation platform("${quarkusPlatformGroupId}:${quarkusPlatformArtifactId}:${quarkusPlatformVersion}")
+    implementation ("${quarkusPlatformGroupId}:quarkus-arc:${quarkusPlatformVersion}")
+    testImplementation 'org.junit.jupiter:junit-jupiter-api:5.8.1'
+    testRuntimeOnly 'org.junit.jupiter:junit-jupiter-engine:5.8.1'


…n, to avoid being picked up by update-dependencies.sh Change composite build test project extension name deployment-artifact property to the correct one Remove comments from composite build test project extension manifest Fix composite build test project build integration test Add composite build with extension integration tests Fix basic composite build integration test Add integration tests for gradle composite builds (dev mode and build) Support composite builds of multimodule gradle projects. Support hot reloading and source watching composite builds. Support using local extensions as a composite build. Remove explicit jupiter version

aloubyansky · 2022-11-28T16:39:17Z

Thanks a lot @tomas1885

quarkus-bot bot added area/devtools Issues/PR related to maven, gradle, platform and cli tooling/plugins area/gradle Gradle labels Nov 13, 2022

tomas1885 mentioned this pull request Nov 13, 2022

Better gradle composite build support #29234

Closed

tomas1885 changed the title ~~Support composite builds of multimodule gradle projects.~~ Support composite builds of multimodule gradle projects Nov 13, 2022

glefloch requested review from glefloch and aloubyansky November 14, 2022 14:35

glefloch force-pushed the composite_build_support branch from 7ffdf1c to 82cd8a5 Compare November 14, 2022 14:39

tomas1885 force-pushed the composite_build_support branch from 82cd8a5 to 83e8f74 Compare November 14, 2022 15:29

aloubyansky reviewed Nov 14, 2022

View reviewed changes

This comment has been minimized.

Sign in to view

aloubyansky reviewed Nov 17, 2022

View reviewed changes

This comment has been minimized.

Sign in to view

tomas1885 force-pushed the composite_build_support branch from c91f5da to da58dab Compare November 23, 2022 15:29

quarkus-bot bot added area/documentation area/kafka area/oidc area/redis area/resteasy area/security labels Nov 23, 2022

quarkus-bot bot added this to To do in Quarkus Documentation Nov 23, 2022

tomas1885 force-pushed the composite_build_support branch from da58dab to f871021 Compare November 23, 2022 19:25

cescoffier removed area/kafka area/redis labels Nov 28, 2022

aloubyansky reviewed Nov 28, 2022

View reviewed changes

tomas1885 force-pushed the composite_build_support branch from f871021 to bd18833 Compare November 28, 2022 13:19

aloubyansky approved these changes Nov 28, 2022

View reviewed changes

Quarkus Documentation automation moved this from To do to Reviewer approved Nov 28, 2022

aloubyansky merged commit 631d433 into quarkusio:main Nov 28, 2022

Quarkus Documentation automation moved this from Reviewer approved to Done Nov 28, 2022

quarkus-bot bot added this to the 2.15 - main milestone Nov 28, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support composite builds of multimodule gradle projects #29233

Support composite builds of multimodule gradle projects #29233

tomas1885 commented Nov 13, 2022

quarkus-bot bot commented Nov 13, 2022

tomas1885 commented Nov 13, 2022

glefloch commented Nov 14, 2022

tomas1885 commented Nov 14, 2022

tomas1885 commented Nov 14, 2022

aloubyansky Nov 14, 2022

tomas1885 Nov 14, 2022

aloubyansky Nov 14, 2022

tomas1885 Nov 14, 2022

This comment has been minimized.

aloubyansky Nov 17, 2022

aloubyansky Nov 17, 2022

This comment has been minimized.

tomas1885 commented Nov 21, 2022

aloubyansky commented Nov 21, 2022

famod commented Nov 22, 2022

aloubyansky commented Nov 22, 2022

tomas1885 commented Nov 23, 2022 •

edited

aloubyansky commented Nov 23, 2022

tomas1885 commented Nov 28, 2022

aloubyansky Nov 28, 2022

aloubyansky Nov 28, 2022

aloubyansky Nov 28, 2022

aloubyansky Nov 28, 2022

aloubyansky Nov 28, 2022

aloubyansky Nov 28, 2022

aloubyansky commented Nov 28, 2022

		@@ -0,0 +1 @@
		deployment-artifact=org.acme.extensions\:quarkus-example-extension\:1.0

Support composite builds of multimodule gradle projects #29233

Support composite builds of multimodule gradle projects #29233

Conversation

tomas1885 commented Nov 13, 2022

quarkus-bot bot commented Nov 13, 2022

tomas1885 commented Nov 13, 2022

glefloch commented Nov 14, 2022

tomas1885 commented Nov 14, 2022

tomas1885 commented Nov 14, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

This comment has been minimized.

Choose a reason for hiding this comment

Choose a reason for hiding this comment

This comment has been minimized.

tomas1885 commented Nov 21, 2022

aloubyansky commented Nov 21, 2022

famod commented Nov 22, 2022

aloubyansky commented Nov 22, 2022

tomas1885 commented Nov 23, 2022 • edited

aloubyansky commented Nov 23, 2022

tomas1885 commented Nov 28, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

aloubyansky commented Nov 28, 2022

tomas1885 commented Nov 23, 2022 •

edited