Split signature-generator and Gradle plugin into separate projects #124

aSemy · 2023-03-01T13:50:00Z

As part of preparatory refactoring for #88, this PR splits up the signature-generator project and Gradle-plugin into two separate projects.

Description

Because Gradle plugins must use Gradle's embedded version of Kotlin, splitting up the projects means the BCV Gradle plugin can be built using/targetting Gradle's embedded Kotlin, while the signature-generator can be built using another version of Kotlin.

It also means that the BCV Gradle plugin can use the kotlin-dsl plugin, which adds some Gradle specific configuration to help plugin development, but is not necessary in the signature-generator.

Splitting up the projects also helps prepare for using the Gradle Worker API, so when the BCV Gradle plugin runs the signature-generator it can specifically control the dependencies used, and potentially default to kotlinx-metadata dependency provided by KGP (if present).

Summary of changes

signatures-generator contains the JVM code, and is built using the standard KGP
gradle-plugin contains only the Gradle Plugin code, and is built using Gradle's embedded-Kotlin
Configuration between the two projects is shared using convention-plugins
refactored Maven publication build config to use more Gradle idiomatic functionality, and because the signing plugin is flaky.
I put them together in ./modules dir. It's a personal convention that I use a lot of my projects, because it's nice that the built libraries are together under one directory and not mixed up with other build/gradle/IDE/git directories.

Breaking changes

There should be no functional changes from this PR - the plugin and library should work exactly as before, however

there are now two new artifacts instead of one
The Gradle plugin ID is the same, so users who use the plugins {} DSL just need to bump the version.
Because the signature-generator can only be published to Maven Central, and the default repository for plugins is gradlePluginPortal(), the BCV Gradle plugin won't be able to fetch the signature-generator by default

TODO

UPDATE: not an issue - GPP proxies Maven Central
Because the signature-generator can only be published to Maven Central, and the default repository for plugins is gradlePluginPortal(), the BCV Gradle plugin won't be able to fetch the signature-generator by default. There are a few ways around this:
- update the docs, tell users to add mavenCentral() as a plugin repo
- shadow signature-generator into BCV-Gradle https://plugins.gradle.org/docs/publish-plugin#shadowing
- use the Worker API (maybe? I think this will fetch dependencies using the build-script repos)
There are now two new artifacts, and the old one will continue to exist. Do you want to publish Maven relocation information?

Dependencies

This PR depends on

- bump Gradle Plugin Publish Plugin version - build script updates & improvements - use Java Toolchains to set the Java version - use jvm-test-fixtures plugin for functionalTest sources - update build config, remove redundant Maven publication

- bump Gradle Plugin Publish Plugin version - build script updates & improvements - use Java Toolchains to set Java version - use jvm-test-fixtures plugin for functionalTest sources

- create convention plugins for common and java-specific config - add Gradle version catalog - simplify some build config

…ects - create new convention plugins - kotlin-gradle-plugin, for building Gradle plugins written in Kotlin - kotlin-jvm, for building Kotlin/JVM projects - maven-publishing, for sharing common maven-publish settings - update gitignore, because `build/` dirs might be nested, and to include common IDE/OS property/language files

# Conflicts: # build.gradle.kts

aSemy · 2023-05-01T18:27:08Z

build.gradle.kts

The configuration in this file was moved to buildSrc convention plugins, so that it could be shared between the two new subprojects as appropriate.

aSemy · 2023-05-01T18:48:53Z

buildSrc/build.gradle.kts

-        groovy.setSrcDirs(emptyList<String>())
+    implementation("org.gradle.kotlin:gradle-kotlin-dsl-plugins:$expectedKotlinDslPluginsVersion") {
+        because("allows applying the `kotlin-dsl` / `embedded-kotlin` plugins in buildSrc convention plugins")
    }


Sorry, I know you like removing the unnecessary src/main dirs, but in this case I thought it was extra configuration that didn't really bring much. Given how Gradle can be quite fragile, and how IntelliJ has tenuous support of buildSrc as it is, I thought it was better remove anything not strictly necessary.

It brings a flat folder structure that we would like to embrace everywhere and get rid of the legacy nested format. Dogfooding and reporting all the problems in the process in an important part here.

I'm okay with this change and can tweak it later after all the merges though

- create convention plugins for common and java-specific config - add Gradle version catalog - simplify some build config

* The ABI shape will be reduced in the future, this is a technical change

qwwdfsad · 2023-05-25T15:21:21Z

Sorry for the delay here, it's a planning time in Kotlin, so I fail to spare a decent chunk of continuous time to get back on it.
Next week hopefully!

aSemy · 2023-05-26T14:55:18Z

No probs. This is a big change so take your time 👍

qwwdfsad · 2023-06-29T14:16:32Z

I've asked my colleague to help with the Gradle part, he mentioned the following potential caveats:

Compilation with embedded Kotlin might lead to LV/AV issues: e.g. with Gradle 8, embedded Kotlin immediately sets LV 1.8, rendering the plugin unusable with Gradle 6.9+. Probably can be fixed with explicit LV/AV for compilation
"the BCV Gradle plugin won’t be able to fetch the signature-generator by default". If I got this right, it's not an issue, because GPP proxies maven central as well
It's an open question (and it's unclear whether it's an issue at all) how to compile Gradle plugin against signatures-generator built with too new Kotlin version

aSemy · 2023-07-01T07:50:58Z

I've asked my colleague to help with the Gradle part, he mentioned the following potential caveats:

Compilation with embedded Kotlin might lead to LV/AV issues: e.g. with Gradle 8, embedded Kotlin immediately sets LV 1.8, rendering the plugin unusable with Gradle 6.9+. Probably can be fixed with explicit LV/AV for compilation

Gradle Test Kit has been set to use Gradle 7.4.2 and the tests seems to work fine (although I don't really understand how or why...).

binary-compatibility-validator/src/functionalTest/kotlin/kotlinx/validation/api/TestDsl.kt

Line 34 in 14bd77c

.withGradleVersion("7.4.2")

"the BCV Gradle plugin won’t be able to fetch the signature-generator by default". If I got this right, it's not an issues, because GPP proxies maven central as well

Ah okay, interesting, thanks! And also, now that I think about it, it has to be the case. Lots of Gradle plugins depend on non-plugin dependencies, and they work fine. BCV depends on the Kotlin stdlib already, and that must come from Maven Central.

It's an open question (and it's unclear whether it's an issue at all) how to compile Gradle plugin against signatures-generator built with too new Kotlin version

Yes, good point. I expect that any problems would be picked up by the Gradle Test Kit test. Also, any further incompatibility problems would be reduced in the future by my plan to use a Gradle Worker to run signatures-generator in an isolated classpath.

qwwdfsad

Thank you, and sorry for the delay!

I gave it one more look, and it seems good in general
I have a few questions and an issue with non-conventional Gradle DSL and Maven Coordinates issue and I think it will be good to go

qwwdfsad · 2023-09-18T13:29:22Z

buildSrc/build.gradle.kts

-        groovy.setSrcDirs(emptyList<String>())
+    implementation("org.gradle.kotlin:gradle-kotlin-dsl-plugins:$expectedKotlinDslPluginsVersion") {
+        because("allows applying the `kotlin-dsl` / `embedded-kotlin` plugins in buildSrc convention plugins")
    }


It brings a flat folder structure that we would like to embrace everywhere and get rid of the legacy nested format. Dogfooding and reporting all the problems in the process in an important part here.

I'm okay with this change and can tweak it later after all the merges though

qwwdfsad · 2023-09-18T13:31:20Z

buildSrc/src/main/kotlin/conventions/kotlin-gradle-plugin.gradle.kts

+}
+
+tasks.validatePlugins {
+    //enableStricterValidation.set(true) // TODO enable validation


Could you please elaborate on when it's okay to enable it to what the blockers are?

qwwdfsad · 2023-09-18T13:34:20Z

buildSrc/src/main/kotlin/conventions/maven-publishing.gradle.kts

+
+
+// the signing plugin must be configured *after* publications are created
+afterEvaluate {


Could you please elaborate on why we have to rely on afterEvaluate even though previously it was not the case?

I'm a bit concerned here as the afterEvaluate block always causes troubles in the long run, as more and more workarounds start piling there (esp. with AGP present)

qwwdfsad · 2023-09-18T14:00:29Z

modules/gradle-plugin/build.gradle.kts

+
+
+@Suppress("UnstableApiUsage")
+gradlePlugin {


I gave local publication a go:

On conventionally-configured Gradle projects with Kotlin DSL and plugins everything is really smooth

On groovy DSL old-way (kotlinx.coroutines 😄 ) I failed to depend on the plugin:

The plugin is published to ~/.m2/repository/org/jetbrains/kotlinx/binary-compatibility-validator/org.jetbrains.kotlinx.binary-compatibility-validator.gradle.plugin/ which cannot be dependent on as is

I've checked out the resulting POM, and its coordinates are the following:

<groupId>org.jetbrains.kotlinx</groupId> <artifactId>gradle-plugin</artifactId> <version>0.14.0-qtest</version>

Two points here:

gradle-plugin is not really a good name in kotlinxnamespace, I suggest renaming it to something more specific, e.g.bcv-gradle-pluginorbinary-compatibility-validator-gradle-plugin`

Figuring this out is hard and (we already had this in Kover) will lead to a lot of questions "how to upgrade with classpath dependency"; let's publish maven relocation information

qwwdfsad · 2023-09-18T14:42:53Z

Also, if it works for you, could you please update Kotlin to 1.9.0 alongside this change?

aSemy force-pushed the feat/split_projects branch from 30b7e24 to 4366602 Compare March 1, 2023 13:51

aSemy mentioned this pull request Mar 1, 2023

Apply kotlin-dsl plugin #125

Closed

aSemy changed the base branch from master to bcv-gradle-rework March 3, 2023 13:49

aSemy marked this pull request as ready for review April 27, 2023 08:30

aSemy added 2 commits May 1, 2023 17:50

Annotate file paths with @Language("file-reference") (Kotlin#126)

8abc330

Update to Gradle 8 (Kotlin#122)

3bec20a

- bump Gradle Plugin Publish Plugin version - build script updates & improvements - use Java Toolchains to set the Java version - use jvm-test-fixtures plugin for functionalTest sources - update build config, remove redundant Maven publication

qwwdfsad force-pushed the bcv-gradle-rework branch from b5e0510 to 3bec20a Compare May 1, 2023 15:50

aSemy added 12 commits May 1, 2023 18:53

Update to Gradle 8

8803529

- bump Gradle Plugin Publish Plugin version - build script updates & improvements - use Java Toolchains to set Java version - use jvm-test-fixtures plugin for functionalTest sources

initial convention plugins

070b323

- create convention plugins for common and java-specific config - add Gradle version catalog - simplify some build config

comment out AGP

5a18844

update libs.versions.toml

a3c33d5

fixes after rebase

84857fb

apply BCV, create API dump

ee05a88

commit API dump

223b410

Merge branch 'feat/apply_bcv' into feat/split_projects

e11432a

# Conflicts: # build.gradle.kts

split the .api file

d934ebe

move api dumps

d99463e

update api dump files

3c61170

aSemy force-pushed the feat/split_projects branch from 6fb9d87 to 3c61170 Compare May 1, 2023 18:25

aSemy commented May 1, 2023

View reviewed changes

aSemy added 3 commits May 1, 2023 20:28

update comments

3d15e8c

use typesafe project-accessor

201b2d0

update comment for gradle-kotlin-dsl-plugins dependency

2ba179a

aSemy commented May 1, 2023

View reviewed changes

aSemy added 5 commits May 2, 2023 17:50

Initial Gradle convention plugins (Kotlin#123)

bef42e0

- create convention plugins for common and java-specific config - add Gradle version catalog - simplify some build config

Merge branch 'bcv-gradle-rework' into feat/apply_bcv

20c5f21

Merge branch 'feat/apply_bcv' into feat/split_projects

6e44e7b

Apply BCV, commit API dump (Kotlin#131)

14bd77c

* The ABI shape will be reduced in the future, this is a technical change

Merge branch 'bcv-gradle-rework' into feat/split_projects

62abbcf

qwwdfsad self-requested a review May 4, 2023 14:51

qwwdfsad force-pushed the bcv-gradle-rework branch from 14bd77c to 3d46759 Compare September 18, 2023 11:51

qwwdfsad requested changes Sep 18, 2023

View reviewed changes

qwwdfsad deleted the branch Kotlin:bcv-gradle-rework January 12, 2024 14:22

qwwdfsad closed this Jan 12, 2024



		// the signing plugin must be configured after publications are created
		afterEvaluate {



		@Suppress("UnstableApiUsage")
		gradlePlugin {

Split signature-generator and Gradle plugin into separate projects #124

Split signature-generator and Gradle plugin into separate projects #124

Uh oh!

Conversation

aSemy commented Mar 1, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Summary of changes

Breaking changes

TODO

Dependencies

Uh oh!

aSemy May 1, 2023

Choose a reason for hiding this comment

Uh oh!

aSemy May 1, 2023

Choose a reason for hiding this comment

Uh oh!

qwwdfsad Sep 18, 2023

Choose a reason for hiding this comment

Uh oh!

qwwdfsad commented May 25, 2023

Uh oh!

aSemy commented May 26, 2023

Uh oh!

qwwdfsad commented Jun 29, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

aSemy commented Jul 1, 2023

Uh oh!

qwwdfsad left a comment

Choose a reason for hiding this comment

Uh oh!

qwwdfsad Sep 18, 2023

Choose a reason for hiding this comment

Uh oh!

qwwdfsad Sep 18, 2023

Choose a reason for hiding this comment

Uh oh!

qwwdfsad Sep 18, 2023

Choose a reason for hiding this comment

Uh oh!

qwwdfsad Sep 18, 2023

Choose a reason for hiding this comment

Uh oh!

qwwdfsad commented Sep 18, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

aSemy commented Mar 1, 2023 •

edited

Loading

qwwdfsad commented Jun 29, 2023 •

edited

Loading

qwwdfsad commented Sep 18, 2023 •

edited

Loading