Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

MAVSDK testing improvements #14296

Merged
merged 51 commits into from Mar 18, 2020
Merged

MAVSDK testing improvements #14296

merged 51 commits into from Mar 18, 2020

Conversation

julianoes
Copy link
Contributor

@julianoes julianoes commented Mar 4, 2020 •
edited

This includes MAVSDK testing improvements.

The changes include:

  • SITL tests defined in a JSON file.
  • less verbose output, no logs, unless an error happens
  • show test summary at the end
  • support verbose log output while still writing to log files
  • added colors to output
  • added cpp files to astyle check and added license header
  • properly exit on Ctrl+C
  • added type hints in Python script

And this is what it looks like now:

Selection_006

@julianoes julianoes force-pushed the pr-mavsdk-tests-improvements branch 2 times, most recently from 37423b7 to 5c8c922 Compare March 6, 2020 14:06
@LorenzMeier
Copy link
Member

Nice! What is needed to get this from WIP to ready for review?

@julianoes
Copy link
Contributor Author

@LorenzMeier The output is not like I want it yet. There should be minimal output unless something fails and if so, all the information is required to see what is going on.

@julianoes julianoes force-pushed the pr-mavsdk-tests-improvements branch 3 times, most recently from e9a39a1 to 07e9aa1 Compare March 17, 2020 14:54
@julianoes julianoes marked this pull request as ready for review March 17, 2020 14:54
@julianoes julianoes mentioned this pull request Mar 17, 2020
Merged
@julianoes
Copy link
Contributor Author

@LorenzMeier ready for review and merging (given CI passes).

@julianoes julianoes force-pushed the pr-mavsdk-tests-improvements branch 3 times, most recently from 78e1c05 to d8defb6 Compare March 17, 2020 16:33
LorenzMeier
LorenzMeier reviewed Mar 17, 2020
View reviewed changes
Copy link
Member

@LorenzMeier LorenzMeier left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice structure, looks good to me. I have one question regarding the idle item.

src/modules/navigator/navigator_main.cpp Outdated Show resolved Hide resolved
@julianoes julianoes force-pushed the pr-mavsdk-tests-improvements branch from d8defb6 to 9e25910 Compare March 17, 2020 17:21
@julianoes
Copy link
Contributor Author

Merging depends on #14134.

Jaeyoung-Lim
Jaeyoung-Lim previously approved these changes Mar 18, 2020
View reviewed changes
@Jaeyoung-Lim Jaeyoung-Lim mentioned this pull request Mar 18, 2020
Merged
@julianoes
mavsdk_tests: MAVSDK is required
89e9ce6 
Without the library installed, we can't build and run the tests.
Silently ignoring this just leads to confusion.
@julianoes
mavsdk_tests: PEP8 fixes
7453599
@julianoes
simulator: remove redundant semicolon
c6c259a
@julianoes
mavsdk_tests: add max speed factor
d0eac74 
This is required because some tests don't work at more than 1x.
@julianoes
mavsdk_tests: remove opt_flow test
42bc412 
It doesn't actually seem to work.
@julianoes
mavsdk_tests: use the min of the speed factors
2a5093b 
Facepalm.
@julianoes
mavsdk_tests: remove plane tests
7a80308 
We can put them back once it's supported.
@julianoes
mavsdk_tests: move config into json file
c1bc9ee
@julianoes
mavsdk_tests: split up Python script
07c30cc 
Otherwise, the script will get a tangled mess.
@julianoes
mavsdk_tests: import naming, easier coloring
48f0339
@julianoes
mavsdk_tests: further argument cleanup, some types
10d0b6c
@julianoes
mavsdk_tests: satisfy mypy type checks
0a444af
@julianoes
mavsdk_tests: print overall results
7f8dac5
@julianoes
mavsdk_tests: improve log file folder structure
d439552
@julianoes
mavsdk_tests: add combined log
47ad85c
@julianoes
mavsdk_tests: improve and fix colors/result
8307d4b
@julianoes
mavsdk_tests: print error on failure
8f0aca4
@julianoes
mavsdk_tests: print where to find logfiles
3859746
@julianoes
mavsdk_tests: fix missing space after number
a011996
@julianoes
mavsdk_tests: add filter for cases
d892d70
@julianoes
Tools: check style of mavsdk_tests files
0f3454b
@julianoes
mavsdk_tests: add header and fix style
e817992
@julianoes
mavsdk_tests: name files consistently
92374b4
@julianoes
mavsdk_tests: fix error for Python < 3.8
355f15a
@julianoes
mavsdk_tests: remove unused import
92beffa
@julianoes
github: new check for MAVSDK tester Python scripts
3093a22
@julianoes
github: try to get latest tools using pip
f8a7f84
@julianoes
mavsdk_tests: a timeout of 5min should be enough
44cc8f6
@julianoes
mavsdk_tests: remove commented out code
b1db077
@julianoes
mavsdk_tests: pass PX4_HOME_ env variables on
4aaaa47 
This way tests can be run at different locations.
@julianoes
workflows: add more tests outside of Europe
d123a42
@julianoes
mavsdk_tests: only env vars that are set
e9708b4 
Otherwise this raises a KeyError.
@julianoes
mavsdk_tests: c++ test_runner is now mavsdk_tests
3aaa432
@julianoes julianoes force-pushed the pr-mavsdk-tests-improvements branch from 202fae4 to 3aaa432 Compare March 18, 2020 15:14
@julianoes
Copy link
Contributor Author

Merging depends on #14134.

Resolved, ready to be merged.

@TSC21 TSC21 merged commit 89599f3 into master Mar 18, 2020
@TSC21 TSC21 deleted the pr-mavsdk-tests-improvements branch March 18, 2020 17:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@LorenzMeier LorenzMeier LorenzMeier left review comments

@TSC21 TSC21 TSC21 approved these changes

@Jaeyoung-Lim Jaeyoung-Lim Jaeyoung-Lim left review comments

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
4 participants
@julianoes @LorenzMeier @TSC21 @Jaeyoung-Lim