Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improves reliability of a few wiring/no_fixture tests #2406

Merged
merged 2 commits into from Mar 16, 2022

Conversation

technobly
Copy link
Member

@technobly technobly commented Mar 14, 2022

Problem

  • wiring/no_fixture SYSTEM/THREAD/TIME all had some intermittent tests

Solution

  • improve reliability of SYSTEM/THREAD/TIME tests

Steps to Test

  • for SYSTEM/THREAD tests, run wiring/no_fixture via integration test runner as that was what was required to see the issue
    device-os/user/tests/integration $ device-os-test run boron wiring/no_fixture -v

  • run TIME tests via TEST=wiring/no_fixture with only TIME_19 compiled in, and use the new repeat feature to run this test 5000 times. It should not fail.

Repeat tests how many times?: 5000
Repeating tests 5000 times!
Running tests
...
Test TIME_19_LocalTimeIsCloseToNtpTime passed.
Test [5000/5000]:1 passed, 0 failed, 0 skipped, out of 1 tests(s).
Total [5000/5000]: 5000 passed, 0 failed
Average [5000/5000]: 100.0% passed, 0.0% failed

NOTE: for now, the repeat unit-test feature was removed as it was increasing the user app size and making the slo/startup integration test fail.

References

  • sc-98866
  • sc-97562

Completeness

  • User is totes amazing for contributing!
  • Contributor has signed CLA (Info here)
  • Problem and Solution clearly stated
  • Run unit/integration/application tests on device
  • [N/A] Added documentation
  • Added to CHANGELOG.md after merging (add links to docs and issues)

Copy link
Contributor

@keeramis keeramis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving to unblock the release - Yet to verify the tests are fully passing across all the rigs.

cc: @technobly

@technobly technobly merged commit 403926a into develop Mar 16, 2022
@technobly technobly deleted the fix/sc98866-tests branch March 16, 2022 19:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
2 participants