You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
It would be good to make integration tests that snapshot analysis results the same way we snapshot JS components, with files that are autogenerated and can be easily updated (rather than constants hardwired in code). I hadn't initially done this due to the randomness in the results, but it occurs to me that (a) when not using frequencies, there is no randomness; (b) we might be able to seed the Mersenne Twister so that randomness was deterministic (this may not work, though, because the order we pull numbers out of the RNG to make random timetables may depend on the iteration order of a hashmap somewhere, but I don't think so); (c) even if we can't isolate the randomness, we built a thing (the bootstrap code) to determine if changes are statistically significant.
The text was updated successfully, but these errors were encountered:
It would be good to make integration tests that snapshot analysis results the same way we snapshot JS components, with files that are autogenerated and can be easily updated (rather than constants hardwired in code). I hadn't initially done this due to the randomness in the results, but it occurs to me that (a) when not using frequencies, there is no randomness; (b) we might be able to seed the Mersenne Twister so that randomness was deterministic (this may not work, though, because the order we pull numbers out of the RNG to make random timetables may depend on the iteration order of a hashmap somewhere, but I don't think so); (c) even if we can't isolate the randomness, we built a thing (the bootstrap code) to determine if changes are statistically significant.
The text was updated successfully, but these errors were encountered: