Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[devel_transport] Async protocol support for utest (rebased 02-03-2016) #6

Closed
wants to merge 16 commits into from
Closed

[devel_transport] Async protocol support for utest (rebased 02-03-2016) #6

wants to merge 16 commits into from

Conversation

PrzemekWirkus
Copy link
Contributor

Please do not merge !, work in progress.

Note: I've extracted mbed-drivers/test_env.h APIs outside of mbed-drivers to greentea-client!

Description

This change ports to Greentea with Async feature inside. It is a breaking change.

Changes

  • Extract greentea_handlers to separate file.
  • Add support for Greentea test case reporting related events:
    • __testcase_count
    • __testcase_start
    • __testcase_finish
    • __testcase_summary
  • Update test/ to support Greentea test case reporting.
    • Ran test cases on K64F.

@PrzemekWirkus PrzemekWirkus changed the title [Async] Async protocol support for utest (do not merge) [Devel] Async protocol support for utest (do not merge) Feb 10, 2016
@PrzemekWirkus
Copy link
Contributor Author

Tested with K64F on frdm-k64f-gcc 2.0.0 target:

$ mbedgt --skip-build
mbedgt: checking for yotta target in current directory
        reason: no --target switch set
mbedgt: checking yotta target in current directory
        calling yotta: yotta --plain target
mbedgt: assuming default target as 'frdm-k64f-gcc'
mbedgt: detecting connected mbed-enabled devices...
mbedgt: detected 1 device
        detected 'K64F' -> 'K64F[0]', console at 'COM188', mounted at 'E:', target id '02400226d94b0e770000000000000000000000002492f3cf'
mbedgt: local yotta target search in './yotta_targets' for compatible mbed-target 'k64f'
        inside './yotta_targets\frdm-k64f-gcc' found compatible target 'frdm-k64f-gcc'
mbedgt: processing 'frdm-k64f-gcc' yotta target compatible platforms...
mbedgt: processing 'K64F' platform...
mbedgt: using platform 'K64F' for test:
        target_id_mbed_htm = '02400226d94b0e770000000000000000000000002492f3cf'
        mount_point = 'E:'
        target_id = '02400226d94b0e770000000000000000000000002492f3cf'
        serial_port = 'COM188'
        target_id_usb_id = '02400226d94b0e770000000000000000000000002492f3cf'
        platform_name = 'K64F'
        platform_name_unique = 'K64F[0]'
mbedgt: skipping calling yotta (specified with --skip-build option)
mbedgt: running 10 tests for target 'frdm-k64f-gcc' and platform 'K64F'
        use 1 instance for testing
mbedgt: test on hardware with target id: 02400226d94b0e770000000000000000000000002492f3cf
mbedgt: test suite 'utest-test-test_assertion_failure_test_setup' .................................... OK in 10.87 sec
        test case: 'dummy test' ...................................................................... OK in 0.03 sec
mbedgt: test case summary: 1 pass, 0 failures
mbedgt: test on hardware with target id: 02400226d94b0e770000000000000000000000002492f3cf
mbedgt: test suite 'utest-test-case_control_repeat' .................................................. OK in 13.83 sec
        test case: 'Control: CaseNext' ............................................................... OK in 0.04 sec
        test case: 'Control: NoRepeat' ............................................................... OK in 0.04 sec
        test case: 'Control: RepeatAll' .............................................................. OK in 0.09 sec
        test case: 'Control: RepeatHandler' .......................................................... OK in 0.40 sec
mbedgt: test case summary: 4 passes, 0 failures
mbedgt: test on hardware with target id: 02400226d94b0e770000000000000000000000002492f3cf
mbedgt: test suite 'utest-test-control_type' ......................................................... OK in 12.07 sec
        test case: 'Testing combinations of different group' ......................................... OK in 0.07 sec
        test case: 'Testing combinations of same group' .............................................. OK in 0.06 sec
        test case: 'Testing constants' ............................................................... OK in 0.04 sec
        test case: 'Testing constructors' ............................................................ OK in 0.05 sec
mbedgt: test case summary: 4 passes, 0 failures
mbedgt: test on hardware with target id: 02400226d94b0e770000000000000000000000002492f3cf
mbedgt: test suite 'utest-test-case_async_validate' .................................................. OK in 12.64 sec
        test case: 'Validate: Attributed Validation: Cancel Repeat' .................................. OK in 0.08 sec
        test case: 'Validate: Attributed Validation: Enable Repeat Handler' .......................... OK in 0.18 sec
        test case: 'Validate: Multiple Premature Validation' ......................................... OK in 0.07 sec
        test case: 'Validate: Multiple Validation' ................................................... OK in 0.14 sec
        test case: 'Validate: Premature Validation' .................................................. OK in 0.06 sec
        test case: 'Validate: Simple Validation' ..................................................... OK in 0.06 sec
mbedgt: test case summary: 6 passes, 0 failures
mbedgt: test on hardware with target id: 02400226d94b0e770000000000000000000000002492f3cf
mbedgt: test suite 'utest-test-case_setup_failure' ................................................... OK in 11.57 sec
        test case: 'Setup handler returns ABORT' ..................................................... OK in 0.13 sec
        test case: 'Setup handler returns CONTINUE' .................................................. OK in 0.06 sec
        test case: 'Setup handler returns IGNORE' .................................................... OK in 0.13 sec
mbedgt: test case summary: 3 passes, 0 failures
mbedgt: test on hardware with target id: 02400226d94b0e770000000000000000000000002492f3cf
mbedgt: test suite 'utest-test-readme_lambdas' ....................................................... OK in 13.06 sec
        test case: 'Asynchronous Test (200ms timeout)' ............................................... OK in 0.06 sec
        test case: 'Asynchronous Timeout Repeat' ..................................................... OK in 1.38 sec
        test case: 'Repeating Test' .................................................................. OK in 0.11 sec
        test case: 'Simple Test' ..................................................................... OK in 0.06 sec
mbedgt: test case summary: 4 passes, 0 failures
mbedgt: test on hardware with target id: 02400226d94b0e770000000000000000000000002492f3cf
mbedgt: test suite 'utest-test-readme' ............................................................... OK in 13.00 sec
        test case: 'Asynchronous Test (200ms timeout)' ............................................... OK in 0.06 sec
        test case: 'Asynchronous Timeout Repeat' ..................................................... OK in 1.38 sec
        test case: 'Repeating Test' .................................................................. OK in 0.10 sec
        test case: 'Simple Test' ..................................................................... OK in 0.06 sec
mbedgt: test case summary: 4 passes, 0 failures
mbedgt: test on hardware with target id: 02400226d94b0e770000000000000000000000002492f3cf
mbedgt: test suite 'utest-test-case_teardown_failure' ................................................ OK in 11.63 sec
        test case: 'Teardown handler returns ABORT' .................................................. OK in 0.06 sec
        test case: 'Teardown handler returns ABORT but is IGNORED' ................................... OK in 0.08 sec
        test case: 'Teardown handler returns CONTINUE' ............................................... OK in 0.06 sec
mbedgt: test case summary: 3 passes, 0 failures
mbedgt: test on hardware with target id: 02400226d94b0e770000000000000000000000002492f3cf
mbedgt: test suite 'utest-test-test_setup_failure' ................................................... OK in 10.99 sec
        test case: 'dummy test' ...................................................................... OK in 0.04 sec
        test case: 'dummy test 2' .................................................................... OK in 0.04 sec
mbedgt: test case summary: 2 passes, 0 failures
mbedgt: test on hardware with target id: 02400226d94b0e770000000000000000000000002492f3cf
mbedgt: test suite 'utest-test-case_control_async' ................................................... OK in 19.15 sec
        test case: 'Control: Await' .................................................................. OK in 1.40 sec
        test case: 'Control: CaseNext' ............................................................... OK in 0.04 sec
        test case: 'Control: NoTimeout' .............................................................. OK in 0.04 sec
        test case: 'Control: RepeatAllOnTimeout' ..................................................... OK in 0.09 sec
        test case: 'Control: RepeatHandlerOnTimeout' ................................................. OK in 1.76 sec
        test case: 'Control: Timeout (Failure)' ...................................................... OK in 0.20 sec
        test case: 'Control: Timeout (Success)' ...................................................... OK in 0.06 sec
mbedgt: test case summary: 7 passes, 0 failures
mbedgt: shuffle seed: 0.3357014907
mbedgt: test suite report:
+---------------+---------------+----------------------------------------------+--------+--------------------+-------------+
| target        | platform_name | test suite                                   | result | elapsed_time (sec) | copy_method |
+---------------+---------------+----------------------------------------------+--------+--------------------+-------------+
| frdm-k64f-gcc | K64F          | utest-test-case_async_validate               | OK     | 12.64              | shell       |
| frdm-k64f-gcc | K64F          | utest-test-case_control_async                | OK     | 19.15              | shell       |
| frdm-k64f-gcc | K64F          | utest-test-case_control_repeat               | OK     | 13.83              | shell       |
| frdm-k64f-gcc | K64F          | utest-test-case_setup_failure                | OK     | 11.57              | shell       |
| frdm-k64f-gcc | K64F          | utest-test-case_teardown_failure             | OK     | 11.63              | shell       |
| frdm-k64f-gcc | K64F          | utest-test-control_type                      | OK     | 12.07              | shell       |
| frdm-k64f-gcc | K64F          | utest-test-readme                            | OK     | 13.0               | shell       |
| frdm-k64f-gcc | K64F          | utest-test-readme_lambdas                    | OK     | 13.06              | shell       |
| frdm-k64f-gcc | K64F          | utest-test-test_assertion_failure_test_setup | OK     | 10.87              | shell       |
| frdm-k64f-gcc | K64F          | utest-test-test_setup_failure                | OK     | 10.99              | shell       |
+---------------+---------------+----------------------------------------------+--------+--------------------+-------------+
mbedgt: test suite results: 10 OK
mbedgt: test case report:
+---------------+---------------+----------------------------------------------+--------------------------------------------------------+--------+--------+--------+--------------------+
| target        | platform_name | test suite                                   | test case                                              | passed | failed | result | elapsed_time (sec) |
+---------------+---------------+----------------------------------------------+--------------------------------------------------------+--------+--------+--------+--------------------+
| frdm-k64f-gcc | K64F          | utest-test-test_assertion_failure_test_setup | dummy test                                             | 1      | 0      | OK     | 0.03               |
| frdm-k64f-gcc | K64F          | utest-test-case_control_repeat               | Control: CaseNext                                      | 1      | 0      | OK     | 0.04               |
| frdm-k64f-gcc | K64F          | utest-test-case_control_repeat               | Control: NoRepeat                                      | 1      | 0      | OK     | 0.04               |
| frdm-k64f-gcc | K64F          | utest-test-case_control_repeat               | Control: RepeatAll                                     | 10     | 0      | OK     | 0.09               |
| frdm-k64f-gcc | K64F          | utest-test-case_control_repeat               | Control: RepeatHandler                                 | 10     | 0      | OK     | 0.4                |
| frdm-k64f-gcc | K64F          | utest-test-control_type                      | Testing combinations of different group                | 1      | 0      | OK     | 0.07               |
| frdm-k64f-gcc | K64F          | utest-test-control_type                      | Testing combinations of same group                     | 1      | 0      | OK     | 0.06               |
| frdm-k64f-gcc | K64F          | utest-test-control_type                      | Testing constants                                      | 1      | 0      | OK     | 0.04               |
| frdm-k64f-gcc | K64F          | utest-test-control_type                      | Testing constructors                                   | 1      | 0      | OK     | 0.05               |
| frdm-k64f-gcc | K64F          | utest-test-case_async_validate               | Validate: Attributed Validation: Cancel Repeat         | 1      | 0      | OK     | 0.08               |
| frdm-k64f-gcc | K64F          | utest-test-case_async_validate               | Validate: Attributed Validation: Enable Repeat Handler | 2      | 0      | OK     | 0.18               |
| frdm-k64f-gcc | K64F          | utest-test-case_async_validate               | Validate: Multiple Premature Validation                | 1      | 0      | OK     | 0.07               |
| frdm-k64f-gcc | K64F          | utest-test-case_async_validate               | Validate: Multiple Validation                          | 1      | 0      | OK     | 0.14               |
| frdm-k64f-gcc | K64F          | utest-test-case_async_validate               | Validate: Premature Validation                         | 1      | 0      | OK     | 0.06               |
| frdm-k64f-gcc | K64F          | utest-test-case_async_validate               | Validate: Simple Validation                            | 1      | 0      | OK     | 0.06               |
| frdm-k64f-gcc | K64F          | utest-test-case_setup_failure                | Setup handler returns ABORT                            | 1      | 0      | OK     | 0.13               |
| frdm-k64f-gcc | K64F          | utest-test-case_setup_failure                | Setup handler returns CONTINUE                         | 1      | 0      | OK     | 0.06               |
| frdm-k64f-gcc | K64F          | utest-test-case_setup_failure                | Setup handler returns IGNORE                           | 1      | 0      | OK     | 0.13               |
| frdm-k64f-gcc | K64F          | utest-test-readme_lambdas                    | Asynchronous Test (200ms timeout)                      | 1      | 0      | OK     | 0.06               |
| frdm-k64f-gcc | K64F          | utest-test-readme_lambdas                    | Asynchronous Timeout Repeat                            | 1      | 0      | OK     | 1.38               |
| frdm-k64f-gcc | K64F          | utest-test-readme_lambdas                    | Repeating Test                                         | 2      | 0      | OK     | 0.11               |
| frdm-k64f-gcc | K64F          | utest-test-readme_lambdas                    | Simple Test                                            | 1      | 0      | OK     | 0.06               |
| frdm-k64f-gcc | K64F          | utest-test-readme                            | Asynchronous Test (200ms timeout)                      | 1      | 0      | OK     | 0.06               |
| frdm-k64f-gcc | K64F          | utest-test-readme                            | Asynchronous Timeout Repeat                            | 1      | 0      | OK     | 1.38               |
| frdm-k64f-gcc | K64F          | utest-test-readme                            | Repeating Test                                         | 2      | 0      | OK     | 0.1                |
| frdm-k64f-gcc | K64F          | utest-test-readme                            | Simple Test                                            | 1      | 0      | OK     | 0.06               |
| frdm-k64f-gcc | K64F          | utest-test-case_teardown_failure             | Teardown handler returns ABORT                         | 1      | 0      | OK     | 0.06               |
| frdm-k64f-gcc | K64F          | utest-test-case_teardown_failure             | Teardown handler returns ABORT but is IGNORED          | 1      | 0      | OK     | 0.08               |
| frdm-k64f-gcc | K64F          | utest-test-case_teardown_failure             | Teardown handler returns CONTINUE                      | 1      | 0      | OK     | 0.06               |
| frdm-k64f-gcc | K64F          | utest-test-test_setup_failure                | dummy test                                             | 1      | 0      | OK     | 0.04               |
| frdm-k64f-gcc | K64F          | utest-test-test_setup_failure                | dummy test 2                                           | 1      | 0      | OK     | 0.04               |
| frdm-k64f-gcc | K64F          | utest-test-case_control_async                | Control: Await                                         | 1      | 0      | OK     | 1.4                |
| frdm-k64f-gcc | K64F          | utest-test-case_control_async                | Control: CaseNext                                      | 1      | 0      | OK     | 0.04               |
| frdm-k64f-gcc | K64F          | utest-test-case_control_async                | Control: NoTimeout                                     | 1      | 0      | OK     | 0.04               |
| frdm-k64f-gcc | K64F          | utest-test-case_control_async                | Control: RepeatAllOnTimeout                            | 1      | 0      | OK     | 0.09               |
| frdm-k64f-gcc | K64F          | utest-test-case_control_async                | Control: RepeatHandlerOnTimeout                        | 1      | 0      | OK     | 1.76               |
| frdm-k64f-gcc | K64F          | utest-test-case_control_async                | Control: Timeout (Failure)                             | 0      | 0      | OK     | 0.2                |
| frdm-k64f-gcc | K64F          | utest-test-case_control_async                | Control: Timeout (Success)                             | 1      | 0      | OK     | 0.06               |
+---------------+---------------+----------------------------------------------+--------------------------------------------------------+--------+--------+--------+--------------------+
mbedgt: test case results: 38 OK
mbedgt: completed in 130.05 sec

@PrzemekWirkus
Copy link
Contributor Author

@niklas-arm I've already merged your devel_transport branch here.

@niklarm
Copy link
Contributor

niklarm commented Feb 11, 2016

Ok, I've added new (breaking) features to utest.

  1. Returning a positive number from the test_setup_handler allows you to choose the starting test case.
    This can be used by the greentea handlers to request the starting case from the greentea host and relay it to the harness. This is a non-breaking change.
  2. The test setup handler now receives a pointer to the test cases allowing the default greentea handlers to report all test cases with their properties to the greentea host before executing any tests. This is a breaking change, since it changes the signature of the test setup handler.

I've updated all utest selftests, which was done with a simple find and replace.
I also added a note in the greentea handlers to implement these future features.

Since we have to modify existing unittest written in utest anyway to update the MBED_HOSTTEST_* with GREENTEA_SETUP I feel like this is an acceptable solution.

@@ -15,7 +15,7 @@
* limitations under the License.
*/
#include "mbed-drivers/mbed.h"
#include "mbed-drivers/test_env.h"
#include "greentea-client/test_env.h"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎉 🎆 👍

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎊 🍪 🍸 😍

@PrzemekWirkus
Copy link
Contributor Author

@niklas-arm Cool! :) Nice new features for utest!

@niklarm
Copy link
Contributor

niklarm commented Feb 12, 2016

Just as a clarification, the new features aren't on the master branch yet, as they are built on this PR.
Please merge them into your PR. utest will be updated to v2.0.0 once we merge this PR.

@PrzemekWirkus
Copy link
Contributor Author

@niklas-arm I will merge it but not right away, got other things to break...But will remember about this!

@PrzemekWirkus PrzemekWirkus changed the title [Devel] Async protocol support for utest (do not merge) [devel_transport] Async protocol support for utest (do not merge) Feb 26, 2016
@niklarm
Copy link
Contributor

niklarm commented Mar 2, 2016

These changes will be a minor version ie. v1.10.0.

Two things:

  • Can you throw out both "Initialize default and ignore handlers in C++03." and the reverted commit?
  • Can you update the Changelog please?

The changes from my branch won't be merged into here, since they are a breaking change.
I suppose together with utest-tiny this will be v2.0.0.

@niklarm
Copy link
Contributor

niklarm commented Mar 2, 2016

Also remember to close #4.

Current utest test results:

mbedgt: test suite report:
+---------------+---------------+----------------------------------------------+--------+--------------------+-------------+
| target        | platform_name | test suite                                   | result | elapsed_time (sec) | copy_method |
+---------------+---------------+----------------------------------------------+--------+--------------------+-------------+
| frdm-k64f-gcc | K64F          | utest-test-case_async_validate               | OK     | 12.66              | shell       |
| frdm-k64f-gcc | K64F          | utest-test-case_control_async                | FAIL   | 19.17              | shell       |
| frdm-k64f-gcc | K64F          | utest-test-case_control_repeat               | OK     | 13.69              | shell       |
| frdm-k64f-gcc | K64F          | utest-test-case_setup_failure                | OK     | 11.28              | shell       |
| frdm-k64f-gcc | K64F          | utest-test-case_teardown_failure             | FAIL   | 11.57              | shell       |
| frdm-k64f-gcc | K64F          | utest-test-control_type                      | OK     | 11.9               | shell       |
| frdm-k64f-gcc | K64F          | utest-test-readme                            | OK     | 13.01              | shell       |
| frdm-k64f-gcc | K64F          | utest-test-readme_lambdas                    | OK     | 12.99              | shell       |
| frdm-k64f-gcc | K64F          | utest-test-test_assertion_failure_test_setup | OK     | 10.58              | shell       |
| frdm-k64f-gcc | K64F          | utest-test-test_setup_failure                | OK     | 10.62              | shell       |
+---------------+---------------+----------------------------------------------+--------+--------------------+-------------+

mbedgt: test case report:
+---------------+---------------+----------------------------------+--------------------------------------------------------+--------+--------+--------+--------------------+
| target        | platform_name | test suite                       | test case                                              | passed | failed | result | elapsed_time (sec) |
+---------------+---------------+----------------------------------+--------------------------------------------------------+--------+--------+--------+--------------------+
| frdm-k64f-gcc | K64F          | utest-test-case_control_repeat   | Control: CaseNext                                      | 1      | 0      | OK     | 0.09               |
| frdm-k64f-gcc | K64F          | utest-test-case_control_repeat   | Control: NoRepeat                                      | 1      | 0      | OK     | 0.09               |
| frdm-k64f-gcc | K64F          | utest-test-case_control_repeat   | Control: RepeatAll                                     | 10     | 0      | OK     | 0.13               |
| frdm-k64f-gcc | K64F          | utest-test-case_control_repeat   | Control: RepeatHandler                                 | 10     | 0      | OK     | 0.44               |
| frdm-k64f-gcc | K64F          | utest-test-control_type          | Testing combinations of different group                | 1      | 0      | OK     | 0.13               |
| frdm-k64f-gcc | K64F          | utest-test-control_type          | Testing combinations of same group                     | 1      | 0      | OK     | 0.13               |
| frdm-k64f-gcc | K64F          | utest-test-control_type          | Testing constants                                      | 1      | 0      | OK     | 0.09               |
| frdm-k64f-gcc | K64F          | utest-test-control_type          | Testing constructors                                   | 1      | 0      | OK     | 0.1                |
| frdm-k64f-gcc | K64F          | utest-test-case_async_validate   | Validate: Attributed Validation: Cancel Repeat         | 1      | 0      | OK     | 0.25               |
| frdm-k64f-gcc | K64F          | utest-test-case_async_validate   | Validate: Attributed Validation: Enable Repeat Handler | 2      | 0      | OK     | 0.26               |
| frdm-k64f-gcc | K64F          | utest-test-case_async_validate   | Validate: Multiple Premature Validation                | 1      | 0      | OK     | 0.13               |
| frdm-k64f-gcc | K64F          | utest-test-case_async_validate   | Validate: Multiple Validation                          | 1      | 0      | OK     | 0.21               |
| frdm-k64f-gcc | K64F          | utest-test-case_async_validate   | Validate: Premature Validation                         | 1      | 0      | OK     | 0.12               |
| frdm-k64f-gcc | K64F          | utest-test-case_async_validate   | Validate: Simple Validation                            | 1      | 0      | OK     | 0.2                |
| frdm-k64f-gcc | K64F          | utest-test-case_setup_failure    | Setup handler returns ABORT                            | 0      | 1      | FAIL   | 0.0                |
| frdm-k64f-gcc | K64F          | utest-test-case_setup_failure    | Setup handler returns CONTINUE                         | 1      | 0      | OK     | 0.0                |
| frdm-k64f-gcc | K64F          | utest-test-case_setup_failure    | Setup handler returns IGNORE                           | 0      | 1      | FAIL   | 0.0                |
| frdm-k64f-gcc | K64F          | utest-test-readme_lambdas        | Asynchronous Test (200ms timeout)                      | 1      | 0      | OK     | 0.12               |
| frdm-k64f-gcc | K64F          | utest-test-readme_lambdas        | Asynchronous Timeout Repeat                            | 1      | 0      | OK     | 1.43               |
| frdm-k64f-gcc | K64F          | utest-test-readme_lambdas        | Repeating Test                                         | 2      | 0      | OK     | 0.14               |
| frdm-k64f-gcc | K64F          | utest-test-readme_lambdas        | Simple Test                                            | 1      | 0      | OK     | 0.1                |
| frdm-k64f-gcc | K64F          | utest-test-readme                | Asynchronous Test (200ms timeout)                      | 1      | 0      | OK     | 0.14               |
| frdm-k64f-gcc | K64F          | utest-test-readme                | Asynchronous Timeout Repeat                            | 1      | 0      | OK     | 1.44               |
| frdm-k64f-gcc | K64F          | utest-test-readme                | Repeating Test                                         | 2      | 0      | OK     | 0.15               |
| frdm-k64f-gcc | K64F          | utest-test-readme                | Simple Test                                            | 1      | 0      | OK     | 0.09               |
| frdm-k64f-gcc | K64F          | utest-test-case_teardown_failure | Teardown handler returns ABORT                         | 1      | 0      | OK     | 0.12               |
| frdm-k64f-gcc | K64F          | utest-test-case_teardown_failure | Teardown handler returns ABORT but is IGNORED          | 1      | 0      | OK     | 0.15               |
| frdm-k64f-gcc | K64F          | utest-test-case_teardown_failure | Teardown handler returns CONTINUE                      | 1      | 0      | OK     | 0.13               |
| frdm-k64f-gcc | K64F          | utest-test-case_control_async    | Control: Await                                         | 1      | 0      | OK     | 1.45               |
| frdm-k64f-gcc | K64F          | utest-test-case_control_async    | Control: CaseNext                                      | 1      | 0      | OK     | 0.09               |
| frdm-k64f-gcc | K64F          | utest-test-case_control_async    | Control: NoTimeout                                     | 1      | 0      | OK     | 0.1                |
| frdm-k64f-gcc | K64F          | utest-test-case_control_async    | Control: RepeatAllOnTimeout                            | 1      | 0      | OK     | 0.15               |
| frdm-k64f-gcc | K64F          | utest-test-case_control_async    | Control: RepeatHandlerOnTimeout                        | 1      | 0      | OK     | 1.82               |
| frdm-k64f-gcc | K64F          | utest-test-case_control_async    | Control: Timeout (Failure)                             | 0      | 1      | FAIL   | 0.27               |
| frdm-k64f-gcc | K64F          | utest-test-case_control_async    | Control: Timeout (Success)                             | 1      | 0      | OK     | 0.11               |
+---------------+---------------+----------------------------------+--------------------------------------------------------+--------+--------+--------+--------------------+
From @niklas-arm:
* Report success to greentea for expected failures.
* Use the greentea setup handlers for correct reporting.
* Remove GREENTEA_START() from all tests.
* Reorder greentea_send_kv to be more readable.
 "C:/jslv3_1/ws/ybt@2/test/readme_lambdas/main.cpp", line 77:
 Error:  #2958: the body of a value-returning lambda with no explicit return
                type must be a single return statement

   Specification specification([](const size_t number_of_cases) {
@PrzemekWirkus PrzemekWirkus changed the title [devel_transport] Async protocol support for utest (do not merge) [devel_transport] Async protocol support for utest (rebased 02-03-2016) Mar 2, 2016
@PrzemekWirkus
Copy link
Contributor Author

@niklas-arm I rebased this PR already, I need to remove commits and update change-log.

@PrzemekWirkus
Copy link
Contributor Author

@niklas-arm Niklas, please release this as utest v1.10.0 so I can release greentea-client with latest dependency to utest v1.10.0.

@PrzemekWirkus
Copy link
Contributor Author

@niklas-arm I'm ready to release with yotta registry greentea-client with dependency to utest v.1.10.0 with our devel_transport Async changes.

@niklarm
Copy link
Contributor

niklarm commented Mar 2, 2016

Merged manually, since I needed to correct the changelog format ;-)

Released as v1.10.0.
Thanks.

@niklarm niklarm closed this Mar 2, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants