test_utils_interactive_sync: add a helper for synchronizing tests #11875
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
miri64
added
Area: tests
fjmolinas
requested changes
Aug 8, 2019
There was a problem hiding this comment.
I was able to reproduce the issue with ccs2538dk, this PR also fixes the issue for me.
master: RIOT_CI_BUILD=1 make -C tests/cond_order/ BOARD=cc2538dk flash
make: Entering directory '/home/francisco/workspace/RIOT/tests/cond_order'
/home/francisco/workspace/RIOT/dist/tools/pyterm/pyterm -p "/dev/ttyUSB1" -b "115200"
Twisted not available, please install it if you want to use pyterm's JSON capabilities
2019-08-08 16:13:07,023 - INFO # Connect to serial port /dev/ttyUSB1
Welcome to pyterm!
Type '/exit' to exit.
Timeout in expect script at "child.expect(u"T%i \(prio %i\): waiting on condition variable now" % (k, thread_prio[k]))" (tests/cond_order/tests/01-run.py:25)
/home/francisco/workspace/RIOT/tests/cond_order/../../Makefile.include:597: recipe for target 'test' failed
make: *** [test] Error 1
make: Leaving directory '/home/francisco/workspace/RIOT/tests/cond_order'
This PR: RIOT_CI_BUILD=1 make -C tests/cond_order/ BOARD=cc2538dk flash
make: Entering directory '/home/francisco/workspace/RIOT/tests/cond_order'
r
/home/francisco/workspace/RIOT/dist/tools/pyterm/pyterm -p "/dev/ttyUSB1" -b "115200"
Twisted not available, please install it if you want to use pyterm's JSON capabilities
2019-08-08 16:14:21,601 - INFO # Connect to serial port /dev/ttyUSB1
Welcome to pyterm!
Type '/exit' to exit.
2019-08-08 16:14:25,809 - INFO # READY
s
2019-08-08 16:14:25,874 - INFO # START
2019-08-08 16:14:25,875 - INFO # T3 (prio 6): waiting on condition variable now
2019-08-08 16:14:25,877 - INFO # T4 (prio 4): waiting on condition variable now
2019-08-08 16:14:25,890 - INFO # T5 (prio 0): waiting on condition variable now
2019-08-08 16:14:25,890 - INFO # T6 (prio 2): waiting on condition variable now
2019-08-08 16:14:25,891 - INFO # T7 (prio 1): waiting on condition variable now
2019-08-08 16:14:25,891 - INFO # First batch was signaled
2019-08-08 16:14:25,892 - INFO # T5 (prio 0): condition variable was signaled now
2019-08-08 16:14:25,909 - INFO # T7 (prio 1): condition variable was signaled now
2019-08-08 16:14:25,912 - INFO # T6 (prio 2): condition variable was signaled now
2019-08-08 16:14:25,913 - INFO # First batch has woken up
2019-08-08 16:14:25,914 - INFO # Second batch was signaled
2019-08-08 16:14:25,922 - INFO # T4 (prio 4): condition variable was signaled now
2019-08-08 16:14:25,923 - INFO # T3 (prio 6): condition variable was signaled now
2019-08-08 16:14:25,923 - INFO # Second batch has woken up
2019-08-08 16:14:25,923 - INFO #
make: Leaving directory '/home/francisco/workspace/RIOT/tests/cond_order'
Can you adapt tests/xtimer_usleep and tests/posix_time tom make use of this?
Also I think having the terminal output in the commit message is overkill, the PR is referenced by the commit and the output can be found there.
Sure I will add them too and add a note for the commit message. |
|
The |
cladmi
force-pushed
the
pr/test_utils/sync
branch
from
e92d4ca
to
cc6187b
Compare
|
Looks good. There aren't any code changes since |
fjmolinas
added
CI: ready for build
Add an implementation that waits for 's' to print 'START' and return. If 'r' is given is prints 'READY' to allow querying for state. The help and answered string have to be different to not match the other. Using puts/getchar was smaller than using `stdio_read/stdio_write` on the example I tested with `esp32`.
Add helper to do the synchronisation.
Use test_utils_interactive_sync for synchronizing some case treat the output before `reset` as the start of the test, which fails for some boards/configurations.
Use test_utils_interactive_sync for synchronizing test instead of the custom 'getchar' handling.
Use test_utils_interactive_sync for synchronizing test instead of the custom 'getchar' handling.
cladmi
force-pushed
the
pr/test_utils/sync
branch
from
cc6187b
to
7b6356b
Compare
|
I was failing due to failing compilation in |
ARRAY_SIZE is implicitly imported by "xtimer.h" which will be removed of this file.
Replace the 'xtimer_sleep' hack to prevent flooding at startup to use test_utils_interactive_sync for synchronizing.
cladmi
force-pushed
the
pr/test_utils/sync
branch
from
67476c4
to
ac8a6cf
Compare
|
Fixed I added a commit that includes "kernel_defines.h" to have |
|
I also pushed-force to have the correct order already. |
fjmolinas
approved these changes
Aug 15, 2019
|
Thank you for the review :) Less failing tests. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Contribution description
Add an implementation that waits for 's' to print 'START' and return.
If 'r' is given is prints 'READY' to allow querying for state.
This also provides a helper to use it in
testrunnertests.It should allow fixing tests where the input before reset is matched and on boards that cannot reset.
I used the function for
tests/cond_orderas example as it was failing oncc2650-launchpadduring 2019.07-RC1 testing.Testing procedure
Run the
tests/cond_orderon a board that:pexpectbeforeresetThis broke the test on
cc2650-launchpadas shown on this output as the lines before resetting are already matched by the test and then the board resets:make RIOT_CI_BUILD=1 --no-print-directory -C /srv/ilab-builds/workspace/git/RIOT/tests/cond_order testThis also allows testing on
esp32(when disablingRESETas it gets locked).The test fails due to a different priority on T3 but at least can run (test does not handle that
THREAD_PRIORITY_MAINcan be different.RIOT_CI_BUILD=1 RESET=true BOARD=esp32-wroom-32 make -C tests/cond_order/ flash testWhere in master the output was completely missed:
All the modified tests still work (would be checked by murdock when running tests).
Issues/PRs references
Running tests on boards for
2019.07-RC1tests RIOT-OS/Release-Specs#128 (comment)Work with @MrKevinWeiss to allow running tests on
esp32that cannot reset usingesptoolwhentermis open.