Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Revert TSCH config changes for nightly tests #1169

Merged

Conversation

@simonduq
Copy link
Member

simonduq commented Jan 3, 2020

The original commit was introduced to only enable TSCH multi-choffset explicitly once switching to TSCH_CONFIG_OPTIMS3. But the commit did enable multi-choffset for CONFIG_TSCH, by mistake.

This PR reverts the commit, so that we're back to the following:

  • CONFIG_TSCH: all defaults
  • CONFIG_TSCH_OPTIMS: all defaults + basic optims
  • CONFIG_TSCH_OPTIMS2: all defaults + more optims (not run in testbed so far, but will enable this soon)

All will now include multi-choffset.

Note: it does look like multi-choffset enhanced the performance for CONFIG_TSCH. See results in https://contiki-ng.github.io/setups/test-tsch.html, and compare runs prior to the merge on October 30, 2019, to the runs after this date. I only scrolled, did not run stats, but would say PDR is higher.

… multi-choffset Orchestra"

This reverts commit 27d7ce3.
@atiselsts

This comment has been minimized.

Copy link
Member

atiselsts commented Jan 8, 2020

Hi Simon,
The results have indeed improved slightly. I checked the results, old ones going back to the start of August. After excluding one outlier data point on 7th October:
PDR: old=99.83 new=99.89

I ran ANOVA test and the difference between the PDR restuls is (barely) statistically significant, with p value 0.04

But I still don't how did the PR enable multiple channel offsets; when the condition #if CONFIG_OPTIMS < 3 is true, just one channel offset is used. Are you absolutely sure multiple channel offsets were used? The difference in PDR could be explained by other factors, you know.

@simonduq

This comment has been minimized.

Copy link
Member Author

simonduq commented Jan 8, 2020

Great :)

I do think we have multi-choffset enabled, because the test for #if CONFIG_OPTIMS < 3 is nested inside #if CONFIG_OPTIMS >= 1, and in the CONFIG_TSCH makefile flag leaves CONFIG_OPTIMS unset.

@simonduq

This comment has been minimized.

Copy link
Member Author

simonduq commented Jan 8, 2020

Also note that PDR diffs always less important than they actually are when nearing 100%.
If you look at the end-to-end loss rate this is 0.17% vs 0.11%.

@simonduq

This comment has been minimized.

Copy link
Member Author

simonduq commented Jan 9, 2020

BTW new "optim2" setup is doing pretty good so far!
https://contiki-ng.github.io/setups/test-tsch-optims2.html

This is without multi-choffset yet (will be after this PR is merged) but with fine-tuned RPL and TSCH traffic, as well as more frequent link probing.

@atiselsts atiselsts merged commit 194943f into contiki-ng:develop Jan 9, 2020
1 check passed
1 check passed
continuous-integration/travis-ci/pr The Travis CI build passed
Details
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants
You can’t perform that action at this time.