Modularize `:configuration-cache` #29207

bamboo · 2024-05-17T22:44:59Z

Initially by extracting three modules:

:configuration-problems-base: base utilities for reporting configuration problems with structured messages and property traces
:object-serialization: core configuration cache serialization api and implementation but no codecs
:java-language-extensions-kt: Java/Kotlin stdlib extensions shared by all modules

This is just the first step, subsequent PRs will detangle and move more code from :configuration-cache into :object-serialization and rename all moved packages to match the new structure.

Context

:configuration-cache is currently one of the largest Gradle modules. It is time to modularize it to reduce compilation times and cognitive overhead.

Reviewing cheatsheet

Before merging the PR, comments starting with

❌ ❓must be fixed
🤔 💅 should be fixed
💭 may be fixed
🎉 celebrate happy things

abstratt

Good step in the right direction.

Only actual changes I could see were the introcdution of a base ProblemsListener implementation (AbstractProblemsListener), and introduction of a sealed class hierarchy for IsolateOwner implementations.

abstratt · 2024-05-20T13:22:41Z

platforms/core-configuration/kotlin-language-extensions/build.gradle.kts

-    suspend fun WriteContext.writeStateOf(bean: Any)
+dependencies {
+    api(libs.futureKotlin("stdlib"))
+    implementation(projects.javaLanguageExtensions)


Now that I see the parallel between this new module and the existing java-language-extensions, I have no issue with the new name. Actually, I think this module is a much better described as "stdlib-extensions" than the one for Java, as the latter actually introduces things that go way beyond providing general purpose extensions to the Java library.

It seems we are consuming java-language-extensions just to get access to Cast. That is fine, more of a problem with that module's purpose than with this one (and that module's scope is already an improvement over recent past).

abstratt · 2024-05-20T13:27:13Z

platforms/core-configuration/configuration-cache/build.gradle.kts

    api(projects.serviceProvider)
+    api(projects.configurationProblemsBase)
    api(project(":base-services"))
    api(project(":build-operations"))
    // TODO - it might be good to allow projects to contribute state to save and restore, rather than have this project know about everything


Ah, this refers to the design you mentioned we want to pursue, which would de-invert the dependencies.

abstratt · 2024-05-20T13:32:32Z

.../configuration-cache/src/main/kotlin/org/gradle/configurationcache/serialization/Contexts.kt

@@ -98,56 +89,10 @@ class DefaultWriteContext(
 }


-internal
-class LoggingTracer(


"deleted" classes here were moved to their own files

abstratt · 2024-05-20T13:34:16Z

...aph-serialization/src/main/kotlin/org/gradle/configurationcache/serialization/Combinators.kt

 import kotlin.coroutines.Continuation
 import kotlin.coroutines.CoroutineContext
 import kotlin.coroutines.coroutineContext
 import kotlin.coroutines.startCoroutine
 import kotlin.coroutines.suspendCoroutine


-internal
 fun <T> singleton(value: T): Codec<T> =
    SingletonCodec(value)




functions were moved to platforms/core-configuration/configuration-cache/src/main/kotlin/org/gradle/configurationcache/serialization/Unsupported.kt

abstratt · 2024-05-20T13:34:52Z

...aph-serialization/src/main/kotlin/org/gradle/configurationcache/serialization/Combinators.kt

@@ -275,104 +219,34 @@ suspend fun <K, V, T : MutableMap<K, V>> ReadContext.readMapEntriesInto(items: T
 }


-internal


Extension functions moved to platforms/core-configuration/configuration-cache/src/main/kotlin/org/gradle/configurationcache/serialization/ClassPathEncodingExtensions.kt

abstratt · 2024-05-20T13:37:43Z

...ect-graph-serialization/src/main/kotlin/org/gradle/configurationcache/serialization/Codec.kt

@@ -110,39 +111,12 @@ interface IsolateContext {
 }


-sealed class IsolateOwner {


Moved to own file and made part of a new sealed class hierachy

bamboo · 2024-05-20T14:55:42Z

...configuration-cache/src/main/kotlin/org/gradle/configurationcache/ConfigurationCacheState.kt

@@ -469,7 +474,7 @@ class ConfigurationCacheState(

            build.createProjects()

-            initProjectProvider(build::getProject)
+            setSingletonProperty<ProjectProvider>(build::getProject)


In order to decouple WriteContext from ProjectInternal

bot-gradle · 2024-05-20T19:00:04Z

The merge queue build has failed. Click here to see all failures.

bot-gradle · 2024-05-20T22:21:55Z

The merge queue build has failed. Click here to see all failures.

bamboo · 2024-05-21T17:39:56Z

@bot-gradle test AST

bot-gradle · 2024-05-21T17:40:51Z

I've triggered the following builds for you. Click here to see all build failures.

AllSmokeTestsPullRequestFeedback build

Initially by extracting three modules: - configuration-problems-base: base utilities for reporting configuration problems with structured messages and property traces - object-graph-serialization: aimed at holding the core configuration cache serialization implementation but currently only containing the detangled code - kotlin-language-extensions: holding the Kotlin extensions shared by all modules This is just the first step, subsequent commits will detangle and move more code from `:configuration-cache` into `:object-graph-serialization` and rename all moved packages to match the new structure.

Because it is used by `DefaultProblemsFactory` from `:configuration-problems-base`.

…-serialization`

To `:object-graph-serialization`.

`graph` was redundant.

With a potential impact to `gcc2speedscope` since generated type names are no longer unpacked to avoid a dependency on `:base-services`.

…ugin

…ects

Since the `kotlin` word in module names is reserved for Kotlin DSL modules.

bamboo added a:chore Minor issue without significant impact re:platformization Tracks work related to the platformization effort labels May 17, 2024

bamboo added this to the 8.9 RC1 milestone May 17, 2024

bamboo self-assigned this May 17, 2024

gitstream-cm bot added 20 min review and removed 20 min review labels May 17, 2024

bamboo force-pushed the bamboo/cc/modularize/i branch 2 times, most recently from 5c67a69 to 93425a0 Compare May 20, 2024 12:43

gitstream-cm bot removed the a:chore Minor issue without significant impact label May 20, 2024

abstratt approved these changes May 20, 2024

View reviewed changes

bamboo commented May 20, 2024

View reviewed changes

bamboo marked this pull request as ready for review May 20, 2024 15:10

bamboo requested review from a team as code owners May 20, 2024 15:10

bamboo requested review from jvandort and mlopatkin and removed request for a team May 20, 2024 15:10

bamboo added this pull request to the merge queue May 20, 2024

github-merge-queue bot removed this pull request from the merge queue due to failed status checks May 20, 2024

bamboo force-pushed the bamboo/cc/modularize/i branch 2 times, most recently from 72ccc61 to 761e557 Compare May 20, 2024 20:55

bamboo added this pull request to the merge queue May 20, 2024

github-merge-queue bot removed this pull request from the merge queue due to failed status checks May 20, 2024

bamboo force-pushed the bamboo/cc/modularize/i branch 2 times, most recently from a8e067e to a5c6b2c Compare May 21, 2024 17:16

gradle deleted a comment from bamboo May 21, 2024

bamboo added 15 commits May 21, 2024 16:09

Move InvalidUserCodeException to :java-language-extensions

4744dde

Because it is used by `DefaultProblemsFactory` from `:configuration-problems-base`.

Update subprojects.json

969ea32

Detangle CC serialization API from :core and move it `:object-graph…

9f9c171

…-serialization`

Fix :java-platform:projectHealth

adc7994

Fix :kotlin-language-extensions:projectHealth

a9ee488

Fix :object-graph-serialization:projectHealth

aee3e7a

Keep Kotlin ExperimentalContracts limited to projects that use it

08671d1

Move LoggingTracer from :configuration-cache

c9fe960

To `:object-graph-serialization`.

Rename :object-graph-serialization to :object-serialization

9d3a835

`graph` was redundant.

Detangle Contexts.kt from :core and move to :object-serialization

227620b

Move BindingsBackedCodec.kt to :object-serialization

ce32ba2

With a potential impact to `gcc2speedscope` since generated type names are no longer unpacked to avoid a dependency on `:base-services`.

Fix :configuration-cache:projectHealth

4f11038

Extract common Kotlin ExperimentalContracts config to convention pl…

f829c19

…ugin

Adjust expected count of packaged plugin jars to account for new proj…

6f69bfd

…ects

bamboo force-pushed the bamboo/cc/modularize/i branch from a5c6b2c to dd1e518 Compare May 21, 2024 19:10

bamboo added this pull request to the merge queue May 21, 2024

bamboo removed this pull request from the merge queue due to a manual request May 21, 2024

Rename kotlin-language-extensions to java-language-extensions-kt

8a941fd

Since the `kotlin` word in module names is reserved for Kotlin DSL modules.

bamboo force-pushed the bamboo/cc/modularize/i branch from dd1e518 to 8a941fd Compare May 21, 2024 19:40

bamboo added this pull request to the merge queue May 21, 2024

Merged via the queue into master with commit f052e10 May 21, 2024
26 checks passed

bamboo deleted the bamboo/cc/modularize/i branch May 21, 2024 21:56

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Modularize `:configuration-cache` #29207

Modularize `:configuration-cache` #29207

bamboo commented May 17, 2024 •

edited

abstratt left a comment

abstratt May 20, 2024

abstratt May 20, 2024

abstratt May 20, 2024

abstratt May 20, 2024

abstratt May 20, 2024

abstratt May 20, 2024

bamboo May 20, 2024

bot-gradle commented May 20, 2024

bot-gradle commented May 20, 2024

bamboo commented May 21, 2024

bot-gradle commented May 21, 2024

		@@ -275,104 +219,34 @@ suspend fun <K, V, T : MutableMap<K, V>> ReadContext.readMapEntriesInto(items: T
		}


		internal

		@@ -110,39 +111,12 @@ interface IsolateContext {
		}


		sealed class IsolateOwner {

Modularize :configuration-cache #29207

Modularize :configuration-cache #29207

Conversation

bamboo commented May 17, 2024 • edited

Context

Reviewing cheatsheet

abstratt left a comment

Choose a reason for hiding this comment

abstratt May 20, 2024

Choose a reason for hiding this comment

abstratt May 20, 2024

Choose a reason for hiding this comment

abstratt May 20, 2024

Choose a reason for hiding this comment

abstratt May 20, 2024

Choose a reason for hiding this comment

abstratt May 20, 2024

Choose a reason for hiding this comment

abstratt May 20, 2024

Choose a reason for hiding this comment

bamboo May 20, 2024

Choose a reason for hiding this comment

bot-gradle commented May 20, 2024

bot-gradle commented May 20, 2024

bamboo commented May 21, 2024

bot-gradle commented May 21, 2024

Modularize `:configuration-cache` #29207

Modularize `:configuration-cache` #29207

bamboo commented May 17, 2024 •

edited