Task stuck in 100% CPU loop within FinalizerGroup/DefaultExecutionPlan.finalizePlan on Gradle 7.6+ #25361
Comments
Can't find a workaround to gradle/gradle#25361 right now.
eskatos
added
in:scheduler
|
Thank you for your interest in Gradle! This issue needs a decision from the team responsible for that area. They have been informed. Response time may vary. |
|
I've taken a look at this and, in fact, this it not a hang, but just some really inefficient code. It will actually finish calculating the task graph after about an hour and a half (depending on the horsepower of your machine). There are about 1400 tasks that get executed with In any event, we need to optimize this. In the meantime, the following workaround will get you around the problem (i.e. the issue is related to the fact that the test tasks are both dependencies of and finalized by the report task): |
|
Actually, a simpler way to remove the task dependency is to use: |
|
Thank you very much! Will give that a go. (edit: seems to work fine, thank you!) Do you think this might bes something to do with the changes for I don't quite understand why the same behavior isn't observed when running Perhaps I could consider removing I do note however that we have a
|
Workaround dependencies created by the multi-module test reports creating issues with Gradle 7.6+ task configuration/finalization based on suggestion at gradle/gradle#25361 (comment) Seems to cause no ill side effects. To review these dependencies later.
|
There was definitely a change in 7.6 that attempted to more properly handle finalizer tasks and introduces the problem you've found. In the task graph, mustRunAfter relationships and dependsOn relationships are extremely similar, so it's likely the same problem as discussed in the issue you linked. The reason it's not an issue with One option to think about with respect to test report aggregation might be to use the test report aggregation plugin introduced in Gradle 7.4. It's a little bit different than your current configuration in that it uses test suites and aggregates by test type (which can be any arbitrary string). You'd have to convert your tests to use test suites, but your use case is pretty much what test suites were designed to address. It would likely simplify the configuration a good bit, I think, but not without some effort during conversion. |
When a finalizer finalizes multiple tasks, and also has dependencies, performance for calculating the task graph can be exceedingly poor for large task graphs. This addresses the problem in two ways: - Only schedule the members of a finalizer group once. - Instead of iterating over every member and deciding if it is a dependency of a finalized node, iterate over the finalized nodes and calculate its dependencies that are also members. This dramatically reduces the time to calculate the task graph in large builds that exhibit this behavior. Fixes #25361
When a finalizer finalizes multiple tasks, and also has dependencies, performance for calculating the task graph can be exceedingly poor for large task graphs. This addresses the problem in two ways: - Only schedule the members of a finalizer group once. - Instead of iterating over every member and deciding if it is a dependency of a finalized node, iterate over the finalized nodes and calculate its dependencies that are also members. This dramatically reduces the time to calculate the task graph in large builds that exhibit this behavior. Fixes #25361
When a finalizer finalizes multiple tasks, and also has dependencies, performance for calculating the task graph can be exceedingly poor for large task graphs. This addresses the problem in two ways: - Only schedule the members of a finalizer group once. - Instead of iterating over every member and deciding if it is a dependency of a finalized node, iterate over the finalized nodes and calculate its dependencies that are also members. This dramatically reduces the time to calculate the task graph in large builds that exhibit this behavior. Fixes #25361
…set of finalized tasks When a finalizer finalizes multiple tasks, and also has dependencies, performance for calculating the task graph can be exceedingly poor for large task graphs when a subset of the finalized tasks are executed. For instance, if a task finalizes all of the test tasks in a build, and also sets them as dependencies, and then only one test task is requested on the command line, it will trigger this behavior (oddly, requesting all test tasks avoids the problem). This addresses the problem in two ways: - Only schedule the members of a finalizer group once. - Instead of iterating over every member and deciding if it is a dependency of a finalized node, iterate over the finalized nodes and calculate its dependencies that are also members. This dramatically reduces the time to calculate the task graph in large builds that exhibit this behavior. The performance is still not great, but it's at least manageable even for very large builds. We can improve this more over time. Fixes #25361 Co-authored-by: Gary Hale <gary@gradle.com>
ov7a
removed
to-triage
👋 team-triage
When a finalizer finalizes multiple tasks, and also has dependencies, performance for calculating the task graph can be exceedingly poor for large task graphs. This addresses the problem in two ways: - Only schedule the members of a finalizer group once. - Instead of iterating over every member and deciding if it is a dependency of a finalized node, iterate over the finalized nodes and calculate its dependencies that are also members. This dramatically reduces the time to calculate the task graph in large builds that exhibit this behavior. Fixes #25361
When a finalizer finalizes multiple tasks, and also has dependencies, performance for calculating the task graph can be exceedingly poor for large task graphs. This addresses the problem in two ways: - Only schedule the members of a finalizer group once. - Instead of iterating over every member and deciding if it is a dependency of a finalized node, iterate over the finalized nodes and calculate its dependencies that are also members. This dramatically reduces the time to calculate the task graph in large builds that exhibit this behavior. Fixes #25361
Expected Behavior
./gradlew server:fastUnitTestof https://github.com/gocd/gocd should execute fine forGradle 7.6+, as it does on Gradle7.5.1.This task is defined here, and I cannot identify anything special about it to guess at the root cause or find a workaround.
Current Behavior
On Gradle
7.6,7.6.1,8.1.1and possibly other versions running this task just gets stuckCONFIGURINGand never terminates.Debug logs end with
Profiler samples always look something like
Context (optional)
The intent here is to just run a custom test task against an alternate java source set defined in a custom
configurationearlier. I have not found a workaround for this.Steps to Reproduce
Requires JDK 17 on path or locatable via Gradle toolchain
git clone https://github.com/gocd/gocd git checkout 23d5610cee3682e8e6e3375b58a6f240416954ec ./gradlew server:fastUnitTest --tests GoConfigDaoBasicTest # See that it is stuckDowngrade and see that it gets past configuring
Gradle version
8.1.1
Gradle version that used to work
7.5.1
Build scan URL (optional)
No response
Your Environment (optional)
The text was updated successfully, but these errors were encountered: