add support for NetBeans projects with existing source code

[bitrunner]

This plugin doesn't currently work with NetBeans projects that are created from existing source code.

Steps to reproduce:

1. Create a new NetBeans C++ project selecting "C/C++ Project with Existing Sources".
2. Select a directory containing a C++ source tree with unit tests.
3. Existing unit tests aren't recognized as such and can't be run in the IDE because the "Test" button is grayed out.

It would be great if the plugin could be taught how to work with unit tests in these kinds of NetBeans projects through manual configuration of project settings.

[offa]

This is already on my todo-list, since it would be very useful eg. for CMake based projects. Unfortunately I have to figure out how this works first. I'll post an update if there's something new.

[offa]

I figured out how to enable it:

1. The created NB Project requires a test folder within the project settings (not at filesystem, just in the NB project settings file; can be done automatically)
2. The project requires two defined make targets:
   • build-tests - builds the tests, executed before running the tests (can be empty)
   • test - runs the tests

The make targets are difficult to add through code, since they may conflict with the user defined or generated; eg. test is reserved by CMake for executing CTest (my workaround to this: don't call enable_testing()).

I was able to get both working - Test button and result output - using a Cmake based project. The (modified) plugin ensured (1) while I had to manually fix (2). I'll provide a appropriate dev-version of the plugin for testing.

[offa]

Finally here's a updated build _for testing_, which addresses this issue:

NBCndUnit-0.7.3-SNAPSHOT.zip

_{Sha 256: 4599778004ab042c103a462b1621c8b73b1d3283e09eac7a0a3aceb934bddbc3}

You can also check the dev/issue7 branch.

---

Warning

This updated plugin is for testing only since it will modify the configuration of the NB project. Do NOT use it for projects which your live belongs to or are otherwise important.

Best make a Backup of nbproject folder.

---

What's new

The new file wizard (New File -> Unit Tests -> ...) will ensure there's a Test Folder in the project settings; if there's none yet it's added.

You should get a Test Files Folder after adding a new Unit Test File through the wizard. There's no new directory created at filelevel btw. - only the configurations.xml is updated.

Projects requirements

Existing projects are required to have these two Maketargets:

• build-tests - builds the tests, executed before running the tests (can be empty)
• test - executes the tests

Both (Maketargets and Test folder) together should enable the Test-Button and Result window.

Again, please keep in mind: The version linked above is for testing purposes only. Please report any issues.

[offa]

@bitrunner any news on this?

[offa]

I have merged this into the development branch. There's also an UI option in the wizards now.

[offa]

Btw. CMake may require an cmake_policy(SET CMP0037 OLD) to workaround the reserved test target (see cmake-0012960).

[offa]

Support for existing projects is added into v0.7.3. I have just released that version, so it's already available through the Plugins Portal. After verification (usually some days) it'll be available a through the updater too.

[bitrunner]

Offa,

Thank you for working on this feature. I finally had a chance to try it. Unfortunately, I've found it to still be completely useless.

The first reason is the build target requirements. I'm using the waf build system (http://waf.io) which is written in python. The requirement that there must be a "build-tests" target is impossible to accommodate as python, like many other languages, doesn't allow hyphens in function names and build targets map directly to function names in waf.

I figure this is a limitation of netbeans core or CND and as such I'm not sure if the NBCndUnit plugin can do anything about it, but it would be nice if I could configure the required build targets in my netbeans project properties similar to how the build and clean commands are currently.

Also, I find it very awkward to go to the "new file wizard" to add the unit test capability to my project especially because I've already got dozens of unit test files and I don't want to add another one, I just want the unit test feature in netbeans to work. This is a one time requirement. It could be button somewhere in project properties that gets hidden in the event that it's already done.

Another thing I noticed is that the "Test" right click context menu item for existing unit test files and new ones added through the wizard is still missing. Perhaps this is due to my project missing the build-tests target, I'm not sure. I don't know how it detects which files are actually unit tests. If that's based on what directory files are in on the file system, it would be nice if this was also configurable in the project properties somewhere.

Anyhow, thanks again for making this plugin and working on my bug report.

Cheers,
Tony

-----Original Message-----
From: offa notifications@github.com
To: offa/NBCndUnit NBCndUnit@noreply.github.com
Cc: bitrunner tonyciavarella@netscape.net; Mention mention@noreply.github.com
Sent: Sun, Jul 24, 2016 11:32 am
Subject: Re: [offa/NBCndUnit] add support for NetBeans projects with existing source code (#7)

Support for existing projects is added into v0.7.3. I have just released that version, so it's already available through the Plugins Portal. After verification (usually some days) it'll be available a through the updater too.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

[offa]

I know, the solution is far away from being perfect. Unfortunately the project requirements – like make targets and test folder – are coded deep into the cnd module. Changing this would require a rewrite of basic Cnd Testrunner parts.

I agree, more control of how testing is done, would be really nice. This could also make the ugly cmake workaround unnecessary. But that's far away from the current api's capabilities. Using the new file wizard is a workaround to this issue. Btw. you dont have to keep the newly created file. The best way to handle this is automatically by the new project wizard, but that's another thing.

Another thing I noticed is that the "Test" right click context menu item for existing unit test files and new ones added through the wizard is still missing. Perhaps this is due to my project missing the build-tests target, I'm not sure. I don't know how it detects which files are actually unit tests. If that's based on what directory files are in on the file system, it would be nice if this was also configurable in the project properties somewhere.

Have you created a "New Test Folder" within "Test Files"? Netbeans doesn't detect what's a unit test, it just takes the source files of each "test folder" and compiles it into an executable. These test folders aren't filesystem based, they are just created within the projects configuration (in a xml file). Compilation / execution is done through the two defined make targets.

At the moment the things this plugin can do are very limited. Basically it parses the output of the unit tests and displays the results into netbeans (plus some new file wizards). For default project the test handling works fine, since every aspect is handled by netbeans. Working with custom projects, the situation is more complex indeed …

[offa]

This is a one time requirement. It could be button somewhere in project properties that gets hidden in the event that it's already done.

Yeah, makes sense. I'll take a look if there's better way to do this then the current one.

[bitrunner]

I see what you're saying about NB native projects vs. not. I appreciate this plugin and thank you for the time you've taken address the existing projects feature request and responding to my dopey e-mail. This plugin is definitely a step in the right direction.

I was able to get my waf based build project to work by making a really basic makefile that NB uses to invoke my real build system. In so doing, the makefile provides the build-tests target and it works.

I noticed a few things now that it actually works:

• NB has no idea what's a unit test and what isn't. This is fine. I just have to run all the unit tests at once using ALT-F6. It's awkward compared to how well NB works for Java projects, but it's functional. I've never tried NB with a C++ project that uses NB's built-in build system, so I'm not sure what the limitations are. I tried your suggestion and made a new Test Folder and put some of my existing unit tests in there, but it still seems confused and doesn't offer a "Test" option on the right click context menu for those files.
• I'm using google test and I noticed that it's important for the output of gtest to not be colored. If it is colored, the plugin doesn't pick up any results and it doesn't give any hint as to why. It kind of looks like the unit test crashed based on the output shown, but that's not the case. You might want to mention that somewhere in your documentation.
• When there's a test failure, google test outputs the file and line number of the failure, but that information isn't picked up by the Test Results GUI in NB. It would really be great if that was included so the failure could be clicked on in the Test Results GUI and have NB jump to the file and line of the error.

I'd say that last item has significant value added so that may be the only thing worth addressing at this point, if possible.

It sounds like we need to petition the CND developers to improve the core C++ unit test support in NB.

Thanks again!

Cheers,
-Tony

-----Original Message-----
From: offa notifications@github.com
To: offa/NBCndUnit NBCndUnit@noreply.github.com
Cc: bitrunner tonyciavarella@netscape.net; Mention mention@noreply.github.com
Sent: Sun, Jul 31, 2016 11:10 am
Subject: Re: [offa/NBCndUnit] add support for NetBeans projects with existing source code (#7)

I know, the solution is far away from being perfect. Unfortunately the project requirements – like make targets and test folder – are coded deep into the cnd module. Changing this would require a rewrite of basic Cnd Testrunner parts.
I agree, more control of how testing is done, would be really nice. This could also make the ugly cmake workaround unnecessary. But that's far away from the current api's capabilities. Using the new file wizard is a workaround to this issue. Btw. you dont have to keep the newly created file. The best way to handle this is automatically by the new project wizard, but that's another thing.

Another thing I noticed is that the "Test" right click context menu item for existing unit test files and new ones added through the wizard is still missing. Perhaps this is due to my project missing the build-tests target, I'm not sure. I don't know how it detects which files are actually unit tests. If that's based on what directory files are in on the file system, it would be nice if this was also configurable in the project properties somewhere.

Have you created a "New Test Folder" within "Test Files"? Netbeans doesn't detect what's a unit test, it just takes the source files of each "test folder" and compiles it into an executable. These test folders aren't filesystem based, they are just created within the projects configuration (in a xml file). Compilation / execution is done through the two defined make targets.
At the moment the things this plugin can do are very limited. Basically it parses the output of the unit tests and displays the results into netbeans (plus some new file wizards). For default project the test handling works fine, since every aspect is handled by netbeans. Working with custom projects, the situation is more complex indeed …
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

[offa]

Nice to hear, thanks! 😄 Getting such helpful feedback is really great!

• NB has no idea what's a unit test and what isn't. This is fine. I just have to run all the unit tests at once using ALT-F6. It's awkward compared to how well NB works for Java projects, but it's functional. I've never tried NB with a C++ project that uses NB's built-in build system, so I'm not sure what the limitations are. I tried your suggestion and made a new Test Folder and put some of my existing unit tests in there, but it still seems confused and doesn't offer a "Test" option on the right click context menu for those files.

Test Folder (should) provide a way to run at least a group of tests (= all within) so you don't have to run all tests of the project. I'll have a look at this.

• I'm using google test and I noticed that it's important for the output of gtest to not be colored. If it is colored, the plugin doesn't pick up any results and it doesn't give any hint as to why. It kind of looks like the unit test crashed based on the output shown, but that's not the case. You might want to mention that somewhere in your documentation.

Hmm, this shouldn't happen …

I'm using coloured Gtest output too and that's working. But maybe the colour escape sequences differ between systems. Do you have any additional parameters added to gtest? What OS are you using?

• When there's a test failure, google test outputs the file and line number of the failure, but that information isn't picked up by the Test Results GUI in NB. It would really be great if that was included so the failure could be clicked on in the Test Results GUI and have NB jump to the file and line of the error.

I'll check this.

It sounds like we need to petition the CND developers to improve the core C++ unit test support in NB.

True! 👍

[offa]

I have opened issues for the colour (#10) and line (#11) issue.