Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Unittest equeue tests #11067

Merged
merged 1 commit into from
Aug 20, 2019
Merged

Add Unittest equeue tests #11067

merged 1 commit into from
Aug 20, 2019

Conversation

Tharazi97
Copy link
Contributor

Description

Ported events queue test to the Greentea standard:
https://github.com/ARMmbed/mbed-os/blob/master/events/equeue/tests/tests.c

Pull request type

[ ] Fix
[ ] Refactor
[ ] Target update
[ ] Functionality change
[ ] Docs update
[x] Test update
[ ] Breaking change

Reviewers

@jamesbeyond @maciejbocianski

Release Notes

@ciarmcom
Copy link
Member

@Tharazi97, thank you for your changes.
@maciejbocianski @jamesbeyond @ARMmbed/mbed-os-core @ARMmbed/mbed-os-test @ARMmbed/mbed-os-maintainers please review.

@SeppoTakalo
Copy link
Contributor

Sounds a lot like unit testing.
Should most of the functionality be tested on unit/module testing framework in host machine, instead of spending time on running on embedded devices?

@jamesbeyond
Copy link
Contributor

Sounds a lot like unit testing.
Should most of the functionality be tested on unit/module testing framework in host machine, instead of spending time on running on embedded devices?

It is a valid point that Greentea tests adding CI executing time. How much effort would it be to converting these test to the UNITTEST ? @Tharazi97 @maciejbocianski

Copy link
Member

@bulislaw bulislaw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's a good point, we are trying to push the testing time down so all the not hw specific tests should be offloaded to unittest framework.

@Tharazi97
Copy link
Contributor Author

Right I will convert them to gtest.

@Tharazi97
Copy link
Contributor Author

I have converted them to googletests, but they don't work properly on Windows. Does it have to pass all the test on Windows, or is Windows not supported already?

@SeppoTakalo
Copy link
Contributor

Not passing on Windows, and passing on Linux, means that you might have just bug that is affected by uninitialised variable.
It is just a difference of compiler and operating system, whether uninitialised are zero, or random values.

Try running your tests using Valgrind to see where the problem is.

@jamesbeyond
Copy link
Contributor

I have converted them to googletests, but they don't work properly on Windows. Does it have to pass all the test on Windows, or is Windows not supported already?

@Tharazi97
NOT PASS Do you mean the new equeue test? or do you mean the existing UNITTEST ?
all existing UNITTEST should be all passed on Windows, if you git some issue that could mean your environments would have some issues

@Tharazi97
Copy link
Contributor Author

Other tests pass. I think that it could be caused by using pthreads library on MinGW, or by wrong reads of ticker function. I'm working on it.

@Tharazi97
Copy link
Contributor Author

I've commited some changes. Unit tests should work on Windows and Linux. But I suggest to leave greentea test as they are, or make them optional to compile, because Equeue is a real time solution and I think it should be able to test it with a ticker and threads available on targets.

@SeppoTakalo @jamesbeyond @maciejbocianski @bulislaw

@@ -1,835 +0,0 @@
/*
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure if it's good idea to remove this test file as it's part of equeue library and still runs on CI (https://github.com/ARMmbed/mbed-os/blob/master/.travis.yml#L230). It should be left as is and only disable on travis script.

@@ -1,424 +0,0 @@
/*
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test file also should stay, as it's part of equeue library and still runs on CI (https://github.com/ARMmbed/mbed-os/blob/master/.travis.yml#L232).

@maciejbocianski
Copy link
Contributor

maciejbocianski commented Jul 31, 2019

@bulislaw @SeppoTakalo @jamesbeyond
I think this is good idea to to leave greentea test as is, and make it optional.
It adds valuable documentation.

What do you think about adding global mechanism/flag to make our test/example codes be aware of running on CI (e.g. MBED_CI_TEST_RUN). That could easily allow to disable specific test/example code when running on CI

Similar flag SKIP_TIME_DRIFT_TESTS was introduced to skip some time drift tests on CI(86dab2f)

#ifndef MBED_CI_TEST_RUN
// when running on CI this test is disabled due to ...

// test code here

#endif 

@jamesbeyond jamesbeyond changed the title added Greentea equeue tests added Unittest equeue tests Aug 1, 2019
@jamesbeyond
Copy link
Contributor

@maciejbocianski @Tharazi97
What is the benefits to keep the GreenTea test?

@OPpuolitaival
Copy link
Contributor

If we have some benefits of Greentea tests then we can leave those using MBED_EXTENDED_TESTS flag to be make easier to run all tests if wanted. But if there are no benefits of those then those can be removed. Good option is to leave one or two greentea tests as hardware smoke test and remove the rests.

@maciejbocianski
Copy link
Contributor

What is the benefits to keep the GreenTea test?

The benefit is that we can test whole queue API on target hardware, not only partially as tests-events-queue does

@jamesbeyond
Copy link
Contributor

What is the benefits to keep the GreenTea test?

The benefit is that we can test whole queue API on target hardware, not only partially as tests-events-queue does

OK, please add the macro as O-P suggested

@Tharazi97 Tharazi97 force-pushed the equeue_tests branch 3 times, most recently from 5c3dca1 to 60f4d9e Compare August 9, 2019 09:32
@0xc0170
Copy link
Contributor

0xc0170 commented Aug 9, 2019

@Tharazi97 Would it make sense to squash (clean the history) - to make a history release ready

@0xc0170 0xc0170 changed the title added Unittest equeue tests Add Unittest equeue tests Aug 9, 2019
@0xc0170
Copy link
Contributor

0xc0170 commented Aug 9, 2019

CI started

@mbed-ci
Copy link

mbed-ci commented Aug 10, 2019

Test run: FAILED

Summary: 1 of 4 test jobs failed
Build number : 1
Build artifacts

Failed test jobs:

  • jenkins-ci/mbed-os-ci_greentea-test

@Tharazi97
Copy link
Contributor Author

CI fails seems not related.
| LPC55S69_S-ARMC6 | LPC55S69_S | mbed-os-tests-psa-crypto_access_control | TIMEOUT | 1800.7 | default |

@0xc0170
Copy link
Contributor

0xc0170 commented Aug 12, 2019

Test was restarted and is all good. I had to restart merge jenkins error, should also be fine (internal CI issue fixed now).

@0xc0170
Copy link
Contributor

0xc0170 commented Aug 12, 2019

Waiting for pr-merge fix, @ARMmbed/mbed-os-test please review

@0xc0170 0xc0170 dismissed bulislaw’s stale review August 13, 2019 08:41

Converted to unittests

@0xc0170
Copy link
Contributor

0xc0170 commented Aug 13, 2019

The CI status fixed, this is ready for integration

@jamesbeyond
Copy link
Contributor

@ARMmbed/mbed-os-maintainers Can this be merged?

@adbridge
Copy link
Contributor

This is on top of 5.14 changes so moved

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

10 participants