Zscore fix

[caser]

I saw this post in the open issues, and upon closer inspection, realized that the z_score algorithm wasn't calculating z-scores correctly.

I re-implemented the z_score algorithm with the correct math, and then refactored it out into its own module. This effectively decouples the z_score calculation from the alternative class. The alternative class now implements its own z_score method which simply passes data along to the calculate method in the new module.

I also tweaked the z_score specs to work with the new implementation, and changed the UI output in the case that the experiment doesn't yet have sufficient data.

Finally, I updated the documentation with a section about statistical validity. It may make sense in future versions to implement a minimum sample size calculator in Split that hides significance data until the experiment has run its course, which would minimize false positives (and is best practices in A/B testing). Until then, it seems prudent to have some sort of warning or advice to users so that they are running statistically valid tests.

Happy to tweak things if need be.

Cheers,
Casey

[andrew]

That looks awesome, very nicely done. I'll get it merged in, and also add you as a contributor.

Thanks! 🤘

[andrew]

I think I'm going to bundle this up with dropping 1.8.7 support into a 1.0 release within the next couple weeks.

[nclzz]

@andrew How can i reference the last version of split in my Gemfile with this pull request merged?

[caser]

Thanks Andrew - I've been busy, but will open up a new thread with ideas for other features related to statistical validity this weekend. Cool project.

[wadtech]

@nclzz add the github hash key to your gemfile entry

gem 'split', :github => 'andrew/split'

[nclzz]

Thanks

Peter Mellett wrote:

@nclzz https://github.com/nclzz add the github hash key to your
gemfile entry

|gem 'split', :github => 'andrew/split'|

—
Reply to this email directly or view it on GitHub
#221 (comment).

[nfm]

Thanks @caser! ❤️ ❤️ ❤️