New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Support for local ("PC") unit tests #519

neuhalje opened this Issue Feb 15, 2016 · 12 comments


None yet
5 participants

neuhalje commented Feb 15, 2016

For CI and testing reasons I would really like to run unit tests. For performance reasons these should run on the local machine (PC), and not on the device (in contrast to #408).

I managed to hack together an external_script (see here) that compiles and runs a single cpp file as unit test.

Adding multiple test files does NOT work, bc the compilation of 'program' lumps together every testfile in one executable and the linker fails with duplicate symbols.

If someone with SCons skills can help me, we could extend platformio ci to actually run unit tests.

@neuhalje neuhalje changed the title from Local ("PC") unit tests to Add Support for local ("PC") unit tests Feb 16, 2016


This comment has been minimized.

malachib commented Mar 23, 2016

Although not exactly the same thing, have you looked into using travis.yml? Works well if your project is on github & open-source

@ivankravets ivankravets added this to the 3.0.0 milestone Mar 27, 2016

@ivankravets ivankravets added the feature label Mar 27, 2016


This comment has been minimized.

neuhalje commented Mar 30, 2016

@malachib you are right. In fact that is one of the use cases I am thinking of. The current platformio ci infrastructure only allows compilation, but no test execution. That last bit is missing.


This comment has been minimized.

malachib commented Mar 30, 2016

Test execution of an embedded device is a very situation-specific thing. Even if Ivan put in full support for something like simavr, it would fall short of a complete suite. Or are you thinking of just IO-less testing which could all run on say an x86?


This comment has been minimized.


ivankravets commented Mar 30, 2016

Guys, I hope that I will think something as for this issue.


This comment has been minimized.

malachib commented Mar 30, 2016

Reviewing this, you seem to be wanting an x86-only test run situation. That seems fully feasible.

I do it with a distinct directory (test, alongside src) and custom makefiles that travis picks up, then use Catch.hpp unit test framework because it's light and easy with C++

I bet the same approach can work with pio+scons, however my scons skills are too nub-ish to make further recommendations


note: there is also a test.pio folder, but that is not automated testing and not relevant to our discussion, so ignore it


This comment has been minimized.

neuhalje commented Mar 31, 2016

@malachib yes, the goal is to run tests on x86.

A dedicated makefile for tests is a good possibility. For starters I will borrow your Makefile :-)

Having worked with java build tools I came to enjoy automatic (test) dependency resolution.


This comment has been minimized.

malachib commented Mar 31, 2016

Glad the makefile can provide some interim help !

Which tools did you use for your Java testing? I did a fair bit of Java back in the day, but have been a C# guy for the last 10 years


This comment has been minimized.

neuhalje commented Apr 1, 2016

@malachib Mainly JUnit and Mockito. At times also Spock.

@ivankravets ivankravets referenced this issue Jun 13, 2016


PlatformIO 3.0 Roadmap #423

28 of 37 tasks complete

This comment has been minimized.

ohhorob commented Jul 22, 2016

Is there any indication wether the testing configuration can handle using or emulating the same types as the target platform? Or is that just common, and independent of the target??

i.e. some variables I want to be uint8_t, and other uint32_t. Will the unit test compilation be able to ensure correctly sized integers/floats?


This comment has been minimized.


ivankravets commented Jul 22, 2016

@ohhorob No emulation/cross-compilation. All tests will bu run on your machine using OS's GCC compiler.


This comment has been minimized.

eerimoq commented Jul 27, 2016

I can't resist to promote Simba in this thread since it has the host PC test execution functionality already in place. I'm not saying you should use Simba, but instead I'm giving an example of how I work developing for embedded devices.

When I started the development of Simba RTOS I compiled and tested everything on my Linux PC. After all, the big job was to set the interfaces, not to execute the binary on an embedded device. Executing the binary on my Linux PC gave the possibility to use Linux tools, for example gdb, gprof, code coverage, etc. Very useful indeed!

Even today I usually implement and test everything on my Linux PC. When I'm satisfied I cross-compile the code and upload the binary to the embedded deivce.

Of course, certain functionality requires the embedded device, but a surprisingly big part does not.

Here is an example of how to build and run for two boards, linux and arduino_due:

   # Linux test compilation and execution (linux is the default board).
   $ make -s run
   <test soutput from linux stdout/stderr>

   # Arduino Due compilation and execution.
   $ make -s BOARD=arduino_due run
   <upload output>
   <test output from serial port from the embedded device>

The output from the test suite today is rather big, but if you have a PC with lots of RAM you can open it here:

Code coverage (press "src" at the bottom of the page):

Simba project page on GitHub:

ivankravets added a commit that referenced this issue Aug 10, 2016

ivankravets added a commit that referenced this issue Sep 9, 2016

Merge branch 'develop' into feature/unicode-issue-771
* develop:
  Fix incorrect line order when converting from INO to CPP and pointer is used
  Fix unit test
  Notify about `version` field when creating library
  Add support for SparkFun Blynk Board
  Return valid exit code from ``plaformio test`` command
  Disable SSL Server-Name-Indication for Python < 2.7.9
  Version bump to 3.0.1 (issue #772)
  Disable temporary SSL for PlatformIO services // Resolve #772
  Version bump to 3.0.0 (issues #770, #766, #747, #730, #765, #640, #659, #742, #459, #542, #763, #759, #753, #757, #749, #748, #745, #519, #709, #743, #413, #498, #410, #740, #361, #414, #554, #732, #588, #475, #461, #101, #719, #721, #537, #415, #522, #289, #556, #570, #456, #617, #432, #408, #479, #667, #510)
  Fix menu height for  docs
  Fix issue with multiple archives when linking firmware
  Add migration guide for PIO2 to PIO3
  Search libraries by headers/includes with ``platformio lib search --header`` option
  Update pio run command examples
  Add Unit Testing Demo
  Update PIO Plus badge title and link
  Add PlatformIO Plus badge
  Add links to PlatformIO Plus
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment