Issues with subproject setup

[smiklos]

Hi,
First of all, thank you for the plugin.

I'm trying to use this plugin in a subproject and facing an issue where all my projects, even unrelated subprojects are built multiple times as the plugin always adds 1-2 more tasks (depending on how many scala versions we use).

If found the following configuration to work better as it adds the extra build tasks to one of the active tasks that the current scala (sub)project has, effectively making it only run for projects that has the plugin turned on for.

        project.afterEvaluate {
            def recurseScalaVersions = determineScalaVersions()
            if (!project.ext.has("recursed") && !project.gradle.ext.has("recursionTaskAdded")) {
                def buildVersionTasks = recurseScalaVersions.collect { ver ->
                    project.tasks.create("recurseWithScalaVersion_$ver", GradleBuild) {
                        startParameter = project.gradle.startParameter.newInstance()
                        startParameter.projectProperties["scalaVersion"] = ver
                        startParameter.projectProperties["recursed"] = true
                        startParameter.excludedTaskNames = getRunOnceTasks()
                        tasks = project.gradle.startParameter.taskNames
                    }

                }
                def activeSubProjectTasks = findActiveTasks()
                if (!activeSubProjectTasks.isEmpty()) {
                    buildVersionTasks.each { activeSubProjectTasks.head().finalizedBy(it.path) }
                }
                project.gradle.ext.recursionTaskAdded = true
            }
        }
    }
    private List<String> findActiveTasks() {
        def startingTaskNames = project.gradle.startParameter.taskNames
        def subProjectTasksByStartingNames = startingTaskNames.collect {
            project.getTasksByName(it, false)
        }.flatten()
        if (subProjectTasksByStartingNames.isEmpty()) {
            return project.getDefaultTasks()
        }
        subProjectTasksByStartingNames
    }

Let me know if you have a better solution or if I should open a pr with my changes.

[nathanalderson]

Sorry for the slow response. I've been covered over in other work. Thanks so much for using the project and for raising an issue. This behavior is definitely something I'd love to fix.

That said, I don't think this is working for me. It's also possible I'm not understanding. :)

I created a simple project with two subprojects (subproject and subproject2), only one of which (subproject) applied the plugin (including your suggested modifications above). I gave the root project and each subproject a task dummy that just prints its own task path. When run from root, it still runs the the task twice on each project/subproject. Here's my output (including some debug prints I threw in):

» gradle dummy    

> Configure project :subproject
~~~~~ startingTasks: [task ':subproject:dummy']
~~~~~ setting task ':subproject:dummy' to be finalized by :subproject:recurseWithScalaVersion_2.12.1

> Task :dummy
***** :dummy

> Task :subproject:dummy
***** :subproject:dummy

> Task :scratch:dummy
***** :dummy

> Task :scratch:subproject:dummy
***** :subproject:dummy

> Task :subproject:recurseWithScalaVersion_2.12.1
***** :subproject2:dummy

> Task :subproject2:dummy
***** :subproject2:dummy

Is this what you'd expect? If so, what's the use case you're trying to address?

edit: btw, I renamed subProjectTasksByStartingNames to startingTasks when I ran this, which you'll see in the print statements near the top.

[sinwe]

I'm not sure if this is related, but I have a project with 3 sub-projects. I set scalaVersions = 2.12.4,2.11.1. When I ran release task, it literally create 1 version with 2.12 and 3 versions with 2.11.
So if for example my current version is 0.0.1-SNAPSHOT, during release it will create the following versions:

• _2.12-0.0.1
• _2.11-0.0.1
• _2.11-0.0.2
• _2.11-0.0.3

And the latest in the branch will become 0.0.4-SNAPSHOT

[adamdougal]

@nathanalderson Hey, thanks for creating this plugin!

I'm also having the same issue as @smiklos. I've created a project to demonstrate the issue here https://github.com/adamdougal/scala-multiversion-issue you should be able to check it out and play around with it.

When running the same task for all projects:

>> ./gradlew dummy --dry-run

:subproject1:dummy SKIPPED
:subproject2:dummy SKIPPED
:subproject1:recurseWithScalaVersion_2.11.11 SKIPPED

BUILD SUCCESSFUL in 0s

This works fine and as expected.

When I just run the dummy task for subproject2 it will still recurse subproject1:

>> ./gradlew subproject2:dummy --dry-run

:subproject2:dummy SKIPPED
:subproject1:recurseWithScalaVersion_2.11.11 SKIPPED

BUILD SUCCESSFUL in 0s

No tasks in subproject1 should have been run.

[nathanalderson]

@adamdougal Thanks for the report and the test case repo!

I can see why you are confused, but this is actually behaving correctly. The task you are seeing from subproject1 is just the recursion task, and the --dry-run flag can't "see" inside the recursion task. If you change your dummy tasks in both subproject1 and subproject2 to the following:

task dummy() { doLast { println it } }

You can see that it is only run for the requested subproject:

» gw subproject2:dummy

> Task :subproject2:dummy 
task ':subproject2:dummy'

> Task :scala-multiversion-issue:subproject2:dummy 
task ':scala-multiversion-issue:subproject2:dummy'


BUILD SUCCESSFUL in 0s
2 actionable tasks: 2 executed

If you run it with --console=plain you can see that the recursion task it uses is attached to subproject1:

» gw subproject2:dummy --console=plain
:subproject2:dummy
task ':subproject2:dummy'
:subproject1:recurseWithScalaVersion_2.11.11
:scala-multiversion-issue:subproject2:dummy
task ':scala-multiversion-issue:subproject2:dummy'

BUILD SUCCESSFUL in 0s
2 actionable tasks: 2 executed

[adamdougal]

But should that recursion task be run at all given that subproject2 has nothing to do with subproject1 which declares the plugin?

It also looks like the dummy task is executed twice for subproject2.

Run without the plugin:

>> ./gradlew subproject2:dummy --console=plain

:subproject2:dummy
task ':subproject2:dummy'

BUILD SUCCESSFUL in 0s
1 actionable task: 1 executed

Run with the plugin:

>> ./gradlew subproject2:dummy --console=plain

:subproject2:dummy
task ':subproject2:dummy'
:subproject1:recurseWithScalaVersion_2.11.11
:scala-multiversion-issue:subproject2:dummy
task ':scala-multiversion-issue:subproject2:dummy'

BUILD SUCCESSFUL in 0s
2 actionable tasks: 2 executed

This does seem to be causing issues with some of my subprojects that do deployments as it means the deployment happens twice.

[nathanalderson]

Yes, this is acting as intended. The recursion task just exists to run the other tasks recursively and just gets sort of arbitrarily attached to subproject1. The dummy task is executed twice for subproject2, once for each scala version (try this: task dummy { doLast { println "$it $scalaVersion" } }).

If you have tasks that should only run once, you can add them to runOnceTasks. From the README:

For this reason, you may optionally set the runOnceTasks property to a comma-separated list of task names/paths which should only be run once. If unset, the default is clean. To specify, for example, that the uploadArtifacts tasks of the javaSubproject project should only be run once, you would add runOnceTasks = clean, :javaSubproject:uploadArtifacts to your gradle.properties file.

[sinwe]

@nathanalderson, if it is the intended behavior, then the default runOnceTasks should be clean, release and uploadArchives. I can't imagine if people would like to run multiple upload or releases per scalaVersion

[nathanalderson]

I disagree. Internally at Adtran, for example, we absolutely want to publish our libraries for both versions. That's kind of the point. What *can* become problematic is if you have a mix of Scala and not-Scala subprojects (or Scala projects that don't apply this plugin). Those other subprojects' tasks will unnecessarily run once for each Scala version. That can be problematic for tasks like uploadArchives since it will try to publish the *same* artifact twice (which will usually fail, depending on your artifact repo potentially). A PR to avoid this behavior would be welcome, which is where this issue initially started.

…

On Wed, Apr 11, 2018, 20:35 sinwe ***@***.***> wrote: @nathanalderson <https://github.com/nathanalderson>, if it is the intended behavior, then the default runOnceTasks should be clean, release and upload archives. I can't imagine if people would like to run multiple upload or releases per scalaVersion — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#5 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ACLksLe5r-I6qY8x46Jw02sGLUL4kaWuks5tnq9ZgaJpZM4PYorx> .

[sinwe]

Are you saying runOnceTasks means run once during the entire build or run once per scala version? I was assuming that it run once per scala version. My problem was that the tasks run as many as the number of subprojects. So for example if you run release task and you have 3 sub projects, on one of the subproject it will run 3 release tasks. So if your current version is 0.0.1, with release task being executed 3x, you'll end up generating version 0.0.2, 0.0.3, and 0.0.4. Which I believe is not the intended target here.

[nathanalderson]

@sinwe Thanks for clarifying. Yes, runOnceTasks means "runs these tasks once during the entire build". The expected behavior otherwise is for tasks to run once per scala version. What you are describing would indeed be an issue, but a separate one. However, I can't reproduce it. If you can create or describe a small project which demonstrates the issue, please open a new issue and I'll be happy to investigate.

[lombardo-chcg]

Hello @nathanalderson ! Thanks for the work on this plugin.
I was able to reproduce something similar to what is being discussed in this issue, where my release task is being run multiple times despite it being in the list of runOnceTasks in my gradle.properies file.

steps to reproduce:

• clone https://github.com/lombardo-chcg/scala-multiversion-example

• git remote rm origin to avoid errors with the release plugin pushing changes

• run the release task: ./gradlew release -Prelease.useAutomaticVersion=true

after it completes, if you check the commit log there will be 2 entries similar to this:

commit c823ddafa3f79e1011e3f08851784b056b739f5a
    Prepare next release using scala version 2.11.8 '0.0.3'.

commit b618a250b64501de9bc09bc75de10443b09c587d
    Prepare next release using scala version 2.12.7 '0.0.2'.

so the problem here is that the release task was run 1 time for each scala version, creating an extra and unwanted version tag, despite the runOnceTasks settings. can you please take a look and let me know your thoughts?

[nathanalderson]

@lombardo-chcg I think this plugin and the net.researchgate.release plugin you are using just aren't going to play nicely together. I think the problem is that that plugin uses multiple tasks of type GradleBuild which essentially starts an entirely new Gradle invocation. The flags that my plugin uses to try and determine whether it should recurse don't propagate to the new GradleBuild tasks, so my plugin just keeps recursing! Notice in your logs that you'll see three instances of recurseWithScalaVersion_2.11.8. You should only get one.

I'm really not sure how to fix or even work around this. Using GradleBuild is problematic for this very reason (of course, I use it too, so 😇...). If you figure something out, I'd be happy to take a pull request, but otherwise, I'm not planning on investigating further.

That said, it does appear that the release task itself is only getting run once. You could try adding more individual tasks to runOnceTasks, but I suspect you're going to be fighting an uphill battle by combining these two plugins.

[lombardo-chcg]

@nathanalderson thank you for the reply - taught me something new about Gradle!

[nathanalderson]

@lombardo-chcg np! Thanks for using.

This issue has become cluttered with several unrelated issues. I'm going to close this issue. If any of the previous commenters wish to continue discussion, please start a new issue.