Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

"Snapshot tests" of analysis results #299

Closed
mattwigway opened this issue Jul 9, 2017 · 1 comment
Closed

"Snapshot tests" of analysis results #299

mattwigway opened this issue Jul 9, 2017 · 1 comment

Comments

@mattwigway
Copy link
Contributor

It would be good to make integration tests that snapshot analysis results the same way we snapshot JS components, with files that are autogenerated and can be easily updated (rather than constants hardwired in code). I hadn't initially done this due to the randomness in the results, but it occurs to me that (a) when not using frequencies, there is no randomness; (b) we might be able to seed the Mersenne Twister so that randomness was deterministic (this may not work, though, because the order we pull numbers out of the RNG to make random timetables may depend on the iteration order of a hashmap somewhere, but I don't think so); (c) even if we can't isolate the randomness, we built a thing (the bootstrap code) to determine if changes are statistically significant.

@ansoncfit
Copy link
Member

Closing in favor of #714

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants