New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unit Testing for embedded #408

Closed
ivankravets opened this Issue Dec 28, 2015 · 10 comments

Comments

Projects
None yet
3 participants
@ivankravets
Member

ivankravets commented Dec 28, 2015

The issue is an addition to #258. Requested by @Oaz, @kireevco.

How will it work? Nothing new here, general Unit testing design

  1. Developer writes tests (using assert function/macro TBD) and places them into test folder
  2. Run tests pio test

PlatformIO will use each test as source code for the new project and the source code of existing project will be used as input library. The developer writes tests for the specific part of code, PlatformIO uploads it on-the-fly, tests, gathers results, shows reports to the developer.

Comments are welcome.

@ivankravets ivankravets added the feature label Dec 28, 2015

@ivankravets ivankravets self-assigned this Dec 28, 2015

@ivankravets ivankravets added this to the 3.0.0 milestone Dec 28, 2015

@ivankravets

This comment has been minimized.

@eerimoq

This comment has been minimized.

eerimoq commented Dec 30, 2015

This is how I normally writes tests: https://github.com/eerimoq/simba/blob/master/tst/kernel/event/main.c. It is very basic and works very well for me. There are plenty of examples in the tst-folder in that repository.

Then I build, upload and run on on target with make test, and I get the result. For arduino, the test report is written to the serial port. On native linux, stdout.

This example run is on native linux. I don't have an Embedded device nearby. I can supply the output from that as well later.

erik@erik-VirtualBox ~/workspace/simba/tst/kernel/event 
> make -s test
Generating settings.bin from ../../../make/settings.ini
Generating settings.h from ../../../make/settings.ini
Compiling main.c
Compiling ../../../src/boards/linux/board.c
Compiling ../../../src/mcus/linux/mcu.c
Compiling ../../../src/kernel/chan.c
Compiling ../../../src/kernel/event.c
Compiling ../../../src/kernel/fs.c
Compiling ../../../src/kernel/log.c
Compiling ../../../src/kernel/queue.c
Compiling ../../../src/kernel/sem.c
Compiling ../../../src/kernel/setting.c
Compiling ../../../src/kernel/shell.c
Compiling ../../../src/kernel/std.c
Compiling ../../../src/kernel/sys.c
Compiling ../../../src/kernel/thrd.c
Compiling ../../../src/kernel/time.c
Compiling ../../../src/kernel/timer.c
Compiling ../../../src/drivers/pin.c
Compiling ../../../src/drivers/sd.c
Compiling ../../../src/drivers/spi.c
Compiling ../../../src/drivers/uart.c
Compiling ../../../src/slib/crc.c
Compiling ../../../src/slib/fat16.c
Compiling ../../../src/slib/harness.c
Compiling ../../../src/slib/hash_map.c
Compiling ../../../src/slib/midi.c
Compiling gen/simba_gen.c
Linking event_suite.out
Running event_suite.out
./event_suite.out
app:   event_suite-0.0.0 built 2015-12-30 13:37 CET by erik.
board: Linux
mcu:   Linux
enter: test_read_write
exit: test_read_write: PASSED
enter: test_poll
exit: test_poll: PASSED
enter: test_poll_timeout
exit: test_poll_timeout: PASSED
            NAME           PARENT        STATE  PRIO   CPU  LOGMASK
            main                     suspended     0    0%     0x3f
            idle             main        ready   127    0%     0x3f
         monitor             main    suspended   -80    0%     0x3f
harness report: total(3), passed(3), failed(0)
event_suite:
exit: test_read_write: PASSED
exit: test_poll: PASSED
exit: test_poll_timeout: PASSED
erik@erik-VirtualBox ~/workspace/simba/tst/kernel/event 
> 
@ivankravets

This comment has been minimized.

Member

ivankravets commented Dec 30, 2015

@eerimoq could you provide test-output from AVR?

@eerimoq

This comment has been minimized.

eerimoq commented Dec 30, 2015

Tonight, when I get home. I'll add output for Arduino Nano and Arduino Due. It's pretty much the same as the above though. Ideally, the output should be the same independently of what hardware it is executed on. Except cases where a device has too little memory and the whole test suite will not fit.

@ivankravets

This comment has been minimized.

Member

ivankravets commented Dec 30, 2015

@eerimoq I agree with you. Now need to decide how to implement it: write own UNIT Tests framework or use existing solutions, like @ThrowTheSwitch https://github.com/ThrowTheSwitch/Unity

@eerimoq

This comment has been minimized.

eerimoq commented Dec 30, 2015

It's probably useful for many programmers to get the test framework generated when the create a project, but, also, I think platformio should provide a high level API that makes it possible to communicate with platformio with any test framework.

@ivankravets

This comment has been minimized.

Member

ivankravets commented Dec 30, 2015

Good. I'll wait output from AVR platform

@eerimoq

This comment has been minimized.

eerimoq commented Dec 30, 2015

@ivankravets

Here we go.

Arduino Nano

erik@erik-PC ~/simba/simba/tst/kernel/event
$ make -s BOARD=arduino_nano test
Generating settings.bin from ../../../make/settings.ini
Generating settings.h from ../../../make/settings.ini
Compiling main.c
Compiling ../../../src/boards/arduino_nano/board.c
Compiling ../../../src/mcus/atmega328p/mcu.c
Compiling ../../../src/kernel/chan.c
Compiling ../../../src/kernel/event.c
Compiling ../../../src/kernel/fs.c
Compiling ../../../src/kernel/log.c
Compiling ../../../src/kernel/queue.c
Compiling ../../../src/kernel/sem.c
Compiling ../../../src/kernel/setting.c
Compiling ../../../src/kernel/shell.c
Compiling ../../../src/kernel/std.c
Compiling ../../../src/kernel/sys.c
Compiling ../../../src/kernel/thrd.c
Compiling ../../../src/kernel/time.c
Compiling ../../../src/kernel/timer.c
Compiling ../../../src/drivers/adc.c
Compiling ../../../src/drivers/ds18b20.c
Compiling ../../../src/drivers/ds3231.c
Compiling ../../../src/drivers/exti.c
Compiling ../../../src/drivers/mcp2515.c
Compiling ../../../src/drivers/nrf24l01.c
Compiling ../../../src/drivers/spi.c
Compiling ../../../src/drivers/owi.c
Compiling ../../../src/drivers/pin.c
Compiling ../../../src/drivers/pwm.c
Compiling ../../../src/drivers/sd.c
Compiling ../../../src/drivers/uart.c
Compiling ../../../src/drivers/uart_soft.c
Compiling ../../../src/slib/crc.c
Compiling ../../../src/slib/fat16.c
Compiling ../../../src/slib/harness.c
Compiling ../../../src/slib/hash_map.c
Compiling ../../../src/slib/midi.c
Compiling gen/simba_gen.c
Linking event_suite.out
Running event_suite.out
avrdude -p atmega328p -D -P COM3 -c arduino -V -b 57600 -U eeprom:w:settings.bin:r

avrdude.exe: AVR device initialized and ready to accept instructions

Reading | ################################################## | 100% 0.00s

avrdude.exe: Device signature = 0x1e950f
avrdude.exe: reading input file "settings.bin"
avrdude.exe: writing eeprom (1024 bytes):

Writing | ################################################## | 100% 8.19s

avrdude.exe: 1024 bytes of eeprom written

avrdude.exe done.  Thank you.

avrdude -p atmega328p -D -P COM3 -c arduino -V -b 57600 -U flash:w:event_suite.hex

avrdude.exe: AVR device initialized and ready to accept instructions

Reading | ################################################## | 100% 0.00s

avrdude.exe: Device signature = 0x1e950f
avrdude.exe: reading input file "event_suite.hex"
avrdude.exe: input file event_suite.hex auto detected as Intel Hex
avrdude.exe: writing flash (22476 bytes):

Writing | ################################################## | 100% 11.26s

avrdude.exe: 22476 bytes of flash written

avrdude.exe done.  Thank you.

../../../make/run.py harness report: total\(\d+\), passed\(\d+\), failed\(\d+\) harness report: total\(\d+\), passed\(\d+\), failed\(0\)
app:   event_suite-0.0.0 built 2015-12-30 17:52 CET by erik.
board: Arduino Nano
mcu:   Atmel ATMega328p AVR @ 16MHz, 2k sram, 32k flash
enter: test_read_write
exit: test_read_write: PASSED
enter: test_poll
exit: test_poll: PASSED
enter: test_poll_timeout
exit: test_poll_timeout: PASSED
            NAME           PARENT        STATE  PRIO   CPU  MAX-STACK-USAGE  LOGMASK
            main                       current     0    0%       315/   931     0x3f
            idle             main        ready   127    0%        55/   156     0x3f
         monitor             main    suspended   -80    0%       104/   256     0x3f
harness report: total(3), passed(3), failed(0)
event_suite:
exit: test_read_write: PASSED
exit: test_poll: PASSED
exit: test_poll_timeout: PASSED

erik@erik-PC ~/simba/simba/tst/kernel/event
$

Arduino Due

erik@erik-PC ~/simba/simba/tst/kernel/event
$ make -s BOARD=arduino_due test
Generating settings.bin from ../../../make/settings.ini
Generating settings.h from ../../../make/settings.ini
Compiling main.c
Compiling ../../../src/boards/arduino_due/board.c
Compiling ../../../src/mcus/sam/sam3.c
Compiling ../../../src/mcus/sam/mcu.c
Compiling settings.c
Compiling ../../../src/kernel/chan.c
Compiling ../../../src/kernel/event.c
Compiling ../../../src/kernel/fs.c
Compiling ../../../src/kernel/log.c
Compiling ../../../src/kernel/queue.c
Compiling ../../../src/kernel/sem.c
Compiling ../../../src/kernel/setting.c
Compiling ../../../src/kernel/shell.c
Compiling ../../../src/kernel/std.c
Compiling ../../../src/kernel/sys.c
Compiling ../../../src/kernel/thrd.c
Compiling ../../../src/kernel/time.c
Compiling ../../../src/kernel/timer.c
Compiling ../../../src/drivers/adc.c
Compiling ../../../src/drivers/can.c
Compiling ../../../src/drivers/chipid.c
Compiling ../../../src/drivers/dac.c
Compiling ../../../src/drivers/exti.c
Compiling ../../../src/drivers/flash.c
Compiling ../../../src/drivers/mcp2515.c
Compiling ../../../src/drivers/pin.c
Compiling ../../../src/drivers/sd.c
Compiling ../../../src/drivers/spi.c
Compiling ../../../src/drivers/uart.c
Compiling ../../../src/drivers/usb.c
Compiling ../../../src/drivers/usb_host.c
Compiling ../../../src/drivers/usb/host/class/usb_host_class_hid.c
Compiling ../../../src/drivers/usb/host/class/usb_host_class_mass_storage.c
Compiling ../../../src/slib/crc.c
Compiling ../../../src/slib/fat16.c
Compiling ../../../src/slib/harness.c
Compiling ../../../src/slib/hash_map.c
Compiling ../../../src/slib/midi.c
Compiling gen/simba_gen.c
Linking event_suite.out
Running event_suite.out
Setting /dev/arduino to 1200 baud and setting DTR to reset the board.
Erase flash
Write 40888 bytes to flash
[==============================] 100% (160/160 pages)
Set boot flash true
app:   event_suite-0.0.0 built 2015-12-30 17:55 CET by erik.
board: Arduino Due
mcu:   Atmel SAM3X8E Cortex-M3 @ 84MHz, 96k sram, 512k flash
enter: test_read_write
exit: test_read_write: PASSED
enter: test_poll
exit: test_poll: PASSED
enter: test_poll_timeout
exit: test_poll_timeout: PASSED
            NAME           PARENT        STATE  PRIO   CPU  MAX-STACK-USAGE  LOGMASK
            main                       current     0    0%       612/ 93070     0x3f
            idle             main        ready   127    0%       160/   258     0x3f
         monitor             main    suspended   -80    0%       140/   514     0x3f
harness report: total(3), passed(3), failed(0)
event_suite:
exit: test_read_write: PASSED
exit: test_poll: PASSED
exit: test_poll_timeout: PASSED

erik@erik-PC ~/simba/simba/tst/kernel/event
$
@ivankravets

This comment has been minimized.

Member

ivankravets commented May 6, 2016

Another unit test framework for Arduino https://github.com/mmurdoch/arduinounit

valeros added a commit that referenced this issue May 30, 2016

ivankravets added a commit that referenced this issue Jun 15, 2016

@ivankravets

This comment has been minimized.

ivankravets added a commit that referenced this issue Jun 15, 2016

@ivankravets ivankravets changed the title from Unit testing for embedded to Unit Testing for embedded Jun 16, 2016

ivankravets added a commit that referenced this issue Sep 9, 2016

Merge branch 'develop' into feature/unicode-issue-771
* develop:
  Fix incorrect line order when converting from INO to CPP and pointer is used
  Fix unit test
  Notify about `version` field when creating library
  Add support for SparkFun Blynk Board
  Return valid exit code from ``plaformio test`` command
  Disable SSL Server-Name-Indication for Python < 2.7.9
  Version bump to 3.0.1 (issue #772)
  Disable temporary SSL for PlatformIO services // Resolve #772
  Version bump to 3.0.0 (issues #770, #766, #747, #730, #765, #640, #659, #742, #459, #542, #763, #759, #753, #757, #749, #748, #745, #519, #709, #743, #413, #498, #410, #740, #361, #414, #554, #732, #588, #475, #461, #101, #719, #721, #537, #415, #522, #289, #556, #570, #456, #617, #432, #408, #479, #667, #510)
  Fix menu height for  docs
  Fix issue with multiple archives when linking firmware
  Add migration guide for PIO2 to PIO3
  Search libraries by headers/includes with ``platformio lib search --header`` option
  Update pio run command examples
  Add Unit Testing Demo
  Update PIO Plus badge title and link
  Add PlatformIO Plus badge
  Add links to PlatformIO Plus
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment