Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[test] improves wiring/no_fixture_long_running NETWORK_XX test timing #2415

Merged
merged 1 commit into from Mar 31, 2022

Conversation

technobly
Copy link
Member

@technobly technobly commented Mar 30, 2022

Problem

wiring/no_fixture_long_running NETWORK_02 and possibly 01 or 03 can fail when a device takes longer than expected to register. We test connect_time cold/warm boots separately with a 75th percentile criteria after 8 runs. Here, we only run through once and it would be kind of lengthy to run this test 8 or even 4 times.

Solution

Instead of running multiple times and taking a percentile of runs, like 3 out of 4, if we increase the wait time from 10 minutes to 15 minutes, we give Device OS a chance to go through it's 10 minute "Resetting the modem due to the network registration timeout", and a bit extra in case of other network related timers causing an issue.

Steps to Test

  • run wiring/no_fixture_long_running NETWORK_02 in a loop on multiple devices at once while logging the output.

Completeness

  • User is totes amazing for contributing!
  • Contributor has signed CLA (Info here)
  • Problem and Solution clearly stated
  • Run unit/integration/application tests on device
  • Added documentation
  • Added to CHANGELOG.md after merging (add links to docs and issues)

@technobly technobly added this to the 3.3.0 milestone Mar 30, 2022
@technobly technobly merged commit 669703f into develop Mar 31, 2022
@technobly technobly deleted the fix/tests branch March 31, 2022 00:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
2 participants