Gradle: can't debug composite build

[errael]

Apache NetBeans version

Apache NetBeans 16 release candidate

What happened

This issue discussed in dev,
https://lists.apache.org/thread/372jso34sfp42fotvfkhp2vj5r8vvq0h
The thread includes the suggestion to use a composit build with includeBuild and various suggestions on how to get debug to work. The problem is that although the Run button works on a composite build, the Debug button does not and results in the following error.

FAILURE: Build failed with an exception.

* What went wrong:
Problem configuring task :run from command line.
> Unknown command-line option '--debug-jvm'.

How to reproduce

zip file has trivial projects, lib, app, compose that demo the issues:
https://github.com/apache/netbeans/files/9847401/gradle-projects-compose.zip

0. clean userdir/cachedir
1. jar -xf attached-zip-file; lib,app,compose; chmod +x */gradlew
2. Open app,lib projects - trust yes, resolve problem
   Only lib can resolve, app depends on maven local
3. lib:publishToMavenLocal
4. app prime build, set as MainProject, run and see === Lib === output
5. Modify Lib.java message (in lib project)
6. Run app, note change to message not seen
7. Open project compose, set compose as main project
8. Run app, note change to message seen
9. Debug app
   Observe:

FAILURE: Build failed with an exception.

* What went wrong:
Problem configuring task :run from command line.
> Unknown command-line option '--debug-jvm'.

Did this work correctly in an earlier version?

No / Don't know

Operating System

Linux harmony 5.19.16-76051916-generic #202210150742166605324422.04~cf07008 SMP PREEMPT_DYNAMIC Tue O x86_64 x86_64 x86_64 GNU/Linux

JDK

openjdk version "11.0.2" 2019-01-15 OpenJDK Runtime Environment 18.9 (build 11.0.2+9) OpenJDK 64-Bit Server VM 18.9 (build 11.0.2+9, mixed mode)

Apache NetBeans packaging

Own source build

Anything else

No response

Are you willing to submit a pull request?

No

Code of Conduct

Yes

[errael]

@neilcsmith-net asked

Not sure if it's an issue or not looking at the list discussion?!

Wasn't sure myself. What finally convinced me that there's a bug, is that, in the list discussion, @lkishalmi suggested several custom actions to try and I couldn't get it to work; suggested a few things about the build.gradle script that didn't pan out. Maybe it's just a feature request or something to document. Not a show stopper for me, but without it there's arcane steps dealing with consistency with the local maven repo which can take a while to get right; might be a big problem for working with large gradle projects.

It reproduces easily with the tiny attached project, if there's an easy workaround, it would be great to know. And a simple fix, would be stupendous.

The gradle docs talk about Importing into the IDE which starts

One of the most useful features of composite builds is IDE integration. By applying the idea or eclipse plugin to your build, it is possible to generate a single IDEA or Eclipse project that permits all builds in the composite to be developed together.

Maybe I just need to find the NetBeans plugin to add to my build ;-)

[lkishalmi]

I'm sorry I had no time to play with the attached project. I'm sure that this is not a bug. NetBeans does not need any plugin (and I think in these days neither Eclipse nor IDEA needs a specific plugin) to work with Gradle.

The problem is that that the run task in this case is a "proxy" task for the included build run task, not a JavaExec type task. While the JavaExec task understands the --debug-jvm option the proxy task does not.

[errael]

BTW, I've discovered that if I check ProjectProperties > Build > Compile > IncludeOpenProjects which is desribed as

Create a composite project from this project and the other open Gradle projects by generating '--include-build' parameters.

then the Debug Main Project button works.

The problem is the "and the other open...". I can use this in small situations, but with a lot of projects open, it's not what I want. I'm not stuck on this, but it's a hole in the Gradle project support and/or a hole in the Gradle documentation.

not a bug

Hmm, "Can't debug composite Gradle project" doesn't feel like a feature request if composite projects are supported.

[errael]

IntelliJ docs https://www.jetbrains.com/help/idea/work-with-gradle-projects.html#gradle_composite_build IntelliJ docs starts with

You can use the settings.gradle file to include Gradle builds for your Gradle composite build

or

You can also use the Gradle tool window to configure the composite build

This 2nd method says

select projects that you want to include in your Gradle composite build

Being able to list the included projects rather than "all open".

I searched around, couldn't find anything beyond the gradle docs that say how to add a run task for composite builds. But that sure doesn't mean it's not there.

[lkishalmi]

Just played around with the attached project a bit.
Both run and debug actions can be configured without a major hustle. Merely by adding the following two lines to the gradle.properties of the composing build:

action.debug.args=:app:run --debug-jvm
action.run.args=:app:run

The registration of a run task is not required.

The only drawback is that the IDE would register the JDPA listener on Java projects only, so the debugger has to be manually attached from the menu, but that's possible.

As this is not a standard use-case for compositing builds, I'd say that's fine.

[errael]

Starting with clean userdir/cachedir

Setting those in gradle properties, when it runs I see

./gradlew --configure-on-demand :app:run --debug-jvm

And then it hangs, needing to manually abort

> Task :app:run
Listening for transport dt_socket at address: 5005

BUILD ABORTED

Is #4943 related to the hang? The comment "then the registration of the JDPA listener would g through ..." seems to relate. (I'm not seeing an exception, but not doing a hint thing.)

[lkishalmi]

That's not a hang. The process waits for a debugger to be attached. Use the Debug menu and attach to localhost port 5005 with dt-socket

[errael]

Cool, it worked! Hit a breakpoint. Thanks.

That's not a hang.

Got it, so #4943 will let the attach happen?

This wait explains why I couldn't get this to work in September with the real project. Looking back at the mailing list, I saw a "hang" but didn't pursue it, thinking my setup was wrong.

[lkishalmi]

No you can manually attach the debugger right now, as I've mentioned, from the Debug menu.

#4943 fixes the exception, that prevent the IDE do the attach automatically.

[errael]

There were two issues.

1. Getting the right Gradle incantation, provided in Gradle: can't debug composite build #4865 (comment).
2. What I misinterpreted as a hang and fixed by Prevent CCE NbGradleProjectImpl to DataObject #4943

[errael]

Closed too soon. Using NetBeans 16-rc4, after DebugMainProject stuck at

> Task :app:run
Listening for transport dt_socket at address: 5005

Using Debug > AttachDebugger can manually get things going.

[errael]

@neilcsmith-net , @lkishalmi still needs triage (since there's a workaround, not urgent (at least for me))

[lkishalmi]

Well, I can't reproduce this with 16-rc4 using the attached sample project and the aforementioned properties file.
I just open the compose and the app projects. Place a breakpoint at line 11 of the App.java. Set the compose as main project. Press the debug button on the tool bar an the debugger attaches automatically then stops at line 11 of the App.java

[errael]

Thanks for checking. Hmm, I copied the 15 userdir. It's late here, I'll poke at it tomorrow.

[errael]

Still can't get it working.

With abundance of caution, attaching zip with aforementioned properties file.
gradle-projects-compose-fix1.zip

Extract netbeans-16-rc4-bin.zip, start NetBeans-16-rc4 with clean userdir/cachdir
no import prev settings
turn off gradle cache:
uncheck Tools > Options > Java > Gradle > Experimental > UseOnDiskCache
make sure .../maven_repository/play/{lib,app} empty

Open only compose project, resolve problem.
Click DebugProject
Stuck at:

> Task :app:run
Listening for transport dt_socket at address: 5005

Manually attach debugger to get job running.

[lkishalmi]

The updated project works for me as well out of the box win 16-rc4. @errael any hidden exception in the IDE log?

Also it would be good if someone else could test it.

[errael]

would be good if someone else could test it.

Yes. If there are no takers, I'll ask around in dev on Monday.

Haven't seen any exceptions. I tried putting a breakpoint at JDPAOutputProcessor.processLine, the

JDPA_LISTEN = Pattern.compile("Listening for transport dt_socket at address: (\\d+)")

looked tempting, and then at it's constructor. No breakpoint. (I tried putting a bkpt in core.windows just to be sure I was getting some).

Any suggestions on things to look at?

[errael]

Looks like one expression of the problem is that some things related to execution don't make it into the project's lookup.

Any idea where these missing items would be put into the lookup for the gradle project?

   GradlePlainEscapeProcessor
    prj.getLookup()
        :app - 69 items
        :compose - 49 items
    constructor - processorsFromLookup - factories
        :app
            GradleProcessorFactory
            JavaCompilerProcessorFactory
            JDPAProcessorFactory
        :compose
            GradleProcessorFactory

    GradlePlainEscapeProcessor.processText(...)
        running :app
                processors	LinkedHashSet	ObjectVariable 	"size = 9"	
                [0]	GradleProcessorFactory$URLOutputProcessor	
                [1]	GradleProcessorFactory$GroovyCompilerProcessor	
                [2]	GradleProcessorFactory$WarningModeAllProcessor	
                [3]	GradleProcessorFactory$JavaCompilerProcessor	
                [4]	GradleProcessorFactory$GradleOutputProcessor	
                [5]	JavaCompilerProcessorFactory$StackTraceProcessor	
                [6]	JDPAProcessorFactory$JDPAOutputProcessor
                [7]	GradlePlainEscapeProcessor$1	
                [8]	GradlePlainEscapeProcessor$2	

        running :compose
                processors	LinkedHashSet	ObjectVariable 	"size = 7"	
                [0]	GradleProcessorFactory$URLOutputProcessor	
                [1]	GradleProcessorFactory$GroovyCompilerProcessor	
                [2]	GradleProcessorFactory$JavaCompilerProcessor	
                [3]	GradleProcessorFactory$GradleOutputProcessor	
                [4]	GradleProcessorFactory$WarningModeAllProcessor	
                [5]	GradlePlainEscapeProcessor$1	
                [6]	GradlePlainEscapeProcessor$2	

[errael]

Poking around NbGradleProjectImpl, it looks like the issue is that :compose does not have a java-base plugin so it does not get JDPAProcessorFactory (which is where Listening for transport dt_socket ... is detected.

Tried adding to :compose's build.gradle plugins { id 'java' } but then got error

A problem occurred evaluating root project 'compose'.
> Cannot add task 'clean' as a task with that name already exists.

and similarly with build. Commented out the clean and build tasks. But then when clicking Debug got an exception

WARNING [org.netbeans.modules.gradle.loaders.LegacyProjectLoader]: 
        object is not an instance of declaring class
java.lang.IllegalArgumentException: object is not an instance of declaring class
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        ...

(Been a great opportunity to look a little deeper into the miracle that is Lookup)
The following trace shows what paths are added to the project's lookup

=== :app

NbGradleProjectImpl:  constructor: id 2b5d41c5 :app      lookup 15,  basic 15,  dependent -1,  added -1,  

NbGradleProjectImpl: enter watcher: :app
NbGradleProjectImpl:	path: Projects/org-netbeans-modules-gradle/Lookup
NbGradleProjectImpl:	path: Projects/org-netbeans-modules-gradle/Plugins/_any/Lookup
NbGradleProjectImpl:      watcher: id 2b5d41c5 :app      lookup 15,  basic 15,  dependent -1,  added 34,  

NbGradleProjectImpl: enter watcher: :app
NbGradleProjectImpl:	path: Projects/org-netbeans-modules-gradle/Plugins/java/Lookup
NbGradleProjectImpl:	path: Projects/org-netbeans-modules-gradle/Plugins/java-base/Lookup
NbGradleProjectImpl:	path: Projects/org-netbeans-modules-gradle/Plugins/base/Lookup
NbGradleProjectImpl:	path: Projects/org-netbeans-modules-gradle/Plugins/root/Lookup
NbGradleProjectImpl:      watcher: id 2b5d41c5 :app      lookup 42*, basic 15,  dependent 44,  added 44,  JDPA

=== :compose

NbGradleProjectImpl:  constructor: id 66d3b84d :compose  lookup 15,  basic 15,  dependent -1,  added -1,  

NbGradleProjectImpl: enter watcher: :compose
NbGradleProjectImpl:	path: Projects/org-netbeans-modules-gradle/Lookup
NbGradleProjectImpl:	path: Projects/org-netbeans-modules-gradle/Plugins/_any/Lookup
NbGradleProjectImpl:      watcher: id 66d3b84d :compose  lookup 15,  basic 15,  dependent -1,  added 34,  

NbGradleProjectImpl: enter watcher: :compose
### NOTE: only root, not java-base so no JDPAProcessorFactory
NbGradleProjectImpl:	path: Projects/org-netbeans-modules-gradle/Plugins/root/Lookup
NbGradleProjectImpl:      watcher: id 66d3b84d :compose  lookup 24,  basic 15,  dependent 34,  added 34,  

[lkishalmi]

Well, at this point, it is more interesting that how does it work for me. The registration can be easily fixed. Going to do that this week, if I happen to find some time for that.

[errael]

it is more interesting that how does it work for me.

Agree, (un?)fortunately I can't explore that.

[lkishalmi]

Yeah, that's my issue. I'm going to take care of that.

[errael]

Following instruction from #4865 (comment) no change in symptoms. Stuck like:

JAVA_HOME="/ref/openjdk/java_home"
cd /junk/play/gradle-stuff/gradle-projects-compose-fix1/compose; ./gradlew --configure-on-demand :app:run --debug-jvm
...
> Task :app:run

Manually attaching debugger gets it to proceed.

This is the one where it never worked for me and it didn't fail for you.

@sdedic , @mbien If someone else could try this, only takes a few minutes. Whether it works or not, probably no reason to hold up 16u1since all the really hot stuff works. And since manually attaching the debugger works (and now I know about that), assuming that's not a problem for most people.

Could someone re-open this issue?