Conversation
Codecov Report
@@ Coverage Diff @@
## main #2891 +/- ##
=======================================
+ Coverage 99.7% 99.7% +0.1%
=======================================
Files 302 302
Lines 28388 28392 +4
=======================================
+ Hits 28292 28296 +4
Misses 96 96
Continue to review full report at Codecov.
|
| ): | ||
| X, y = X_y_regression | ||
|
|
||
| random_state = 2 |
There was a problem hiding this comment.
Setting the random state to avoid flaky tests since these distributions are randomly generated.
There was a problem hiding this comment.
Can you explain this a bit more--I'm assuming random_state = 0 will not give us a distribution that would raise a warning? If that's true, I'm curious if there's a good way to make sure that this test is not relying on something flaky... or at the very least, we should comment on this so we don't come back 6 months from now wondering what this value is 😂
There was a problem hiding this comment.
Yep! So something both @ParthivNaresh and I noticed was that when the sample gets smaller, the distribution can end up looking more normal compared to lognormal, especially after dropping values outside 3 st_devs. This can be seen here:

We are setting the random_state here so that the tests don't flake for our expected values. I can add a further comment in the file
angela97lin
left a comment
There was a problem hiding this comment.
Really cool that this didn't take too much to implement!! Left a comment about the flakiness but otherwise, LGTM 🥳
| ): | ||
| X, y = X_y_regression | ||
|
|
||
| random_state = 2 |
There was a problem hiding this comment.
Can you explain this a bit more--I'm assuming random_state = 0 will not give us a distribution that would raise a warning? If that's true, I'm curious if there's a good way to make sure that this test is not relying on something flaky... or at the very least, we should comment on this so we don't come back 6 months from now wondering what this value is 😂
chukarsten
left a comment
There was a problem hiding this comment.
This looks good to me. I am also a little concerned that there might be some flakiness with the random seed, but I think we'll find out over time and it is documented enough that I think we'd easily find out why it's failing. I think the docstring could probably use a little info to let someone reading it know what's happening with the switching of tests at a glance, but nice work!
fix #2886