Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Increase reproducability of Lin Thread tests #24

Merged
merged 4 commits into from
Mar 22, 2022
Merged

Conversation

jmid
Copy link
Collaborator

@jmid jmid commented Mar 22, 2022

This splits the negative tests of Lin into

  • Domain tests and
  • Thread tests

In order to increase reproducability it then

  • uses an allocating, non-tail-recursive interp_thread function
  • delays the first thread until the second is up and running
  • increases the rep_count

For the latter I spent a bit of time developing a statistical test to decide whether a candidate change would
constitute an improvement. I then recorded for 10.000 arbitrary cmd-triples whether the property would hold,
and used these numbers for a z-statistic test following https://www.itl.nist.gov/div898/handbook/prc/section3/prc33.htm
The three above changes proved to yield statistically significant increases.

Things that didn't pan out

  • Thread.delay instead of Thread.yield to maximize concurrency issues
  • extra Thread.yields in the Thread.create lambdas
  • ...

- have interp_thread allocate
- have first thread wait until the second is ready
- repeat tests 100 times
@jmid jmid merged commit 9f3ce45 into main Mar 22, 2022
@jmid jmid deleted the split-domain-thread branch March 22, 2022 20:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant