[test] improves wiring/no_fixture_long_running NETWORK_XX test timing #2415
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Problem
wiring/no_fixture_long_running
NETWORK_02 and possibly 01 or 03 can fail when a device takes longer than expected to register. We test connect_time cold/warm boots separately with a 75th percentile criteria after 8 runs. Here, we only run through once and it would be kind of lengthy to run this test 8 or even 4 times.Solution
Instead of running multiple times and taking a percentile of runs, like 3 out of 4, if we increase the wait time from 10 minutes to 15 minutes, we give Device OS a chance to go through it's 10 minute "Resetting the modem due to the network registration timeout", and a bit extra in case of other network related timers causing an issue.
Steps to Test
wiring/no_fixture_long_running
NETWORK_02 in a loop on multiple devices at once while logging the output.Completeness