Provide JUnit output for PIO Unit Testing

[tschissler]

I was able to create a set of Unit Tests with PlatformIO and run them as native within a docker container. I also managed to run those tests as port of my build pipeline on Azure DevOps. All looks pretty good.

Thank you so much for building this amazing tool!!!

Only thing which nags me is that failing unit tests make my build fail with no additional information available before I dig into the log files.
It would be amazing, if I could extract the build results into a test result file in a JUnit format or something alike which I then could use to provide my build system with additional information.
I'm wondering if something like this is build in right now or if I have to build something on my own.

Useful links

• https://llg.cubic.org/docs/junit/
• https://help.catchsoftware.com/display/ET/JUnit+Format

[ivankravets]

Do you mean pio test -v?

[tschissler]

Not really. That would only add more details, if I understand this option correctly.
I have solved the issue by writing the console output to a file and then parsing this to create the XML format used by NUnit and others.
I'll publish the python script I created and link it to this thread within the next couple of days and then close it.

[ivankravets]

Maybe pio test —json-output will help here?

[ivankravets]

Where is the specification for this format in XML?

[tschissler]

@ivankravets I do not know the pio test --json-output option and on my version it seems not to be supported nor can I find documentation about it. That would certainly be helpful as JSON would be easier to parse than the text output.

The XML format I am using is this here: https://github.com/nunit/docs/wiki/Test-Result-XML-Format. But there would be other options possible as my goal is to use this task in a Azure DevOps build pipeline: https://docs.microsoft.com/en-us/azure/devops/pipelines/tasks/test/publish-test-results?view=azure-devops&tabs=yaml to publish my test results and make them available to be seen on the build report. The result then looks like this:

I have shared the Python script I have build here: https://github.com/tschissler/sharing/tree/master/Scripts
Bare with me as I am a Python newbie :-) But maybe someone else might find this helpful.

I'm now working on an article describing how to integrate Platformio with Azure DevOps CI/CD. Maybe you want to consider adding this option to your documentation of CI/CD systems.

[ivankravets]

I renamed this issue to "Provide JSON output for PIO Unit Testing". So we will implement it soon.

[jpsfs]

Hi!
I'm also looking for the same kind of functionality. Is there any idea when will this be available?

@tschissler could you share your yml pipeline?

[ivankravets]

Please vote for this feature here #2891 (comment) 👍

See https://github.com/platformio/platformio-core/issues?q=is%3Aissue+is%3Aopen+sort%3Areactions-%2B1-desc