Skip to content

auduno/SeGLiR

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

59 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

SeGLiR

npm version

SeGLiR is a javascript library for rapid A/B-testing with Sequential Generalized Likelihood Ratio Tests.

Sequential GLR tests are a family of sequential hypothesis tests, i.e. tests that stop as soon as a significant result has been detected. Compared to classical fixed samplesize tests, sequential GLR tests may give a significant decrease in the needed samplesize, while keeping the same type-1 and type-2 error guarantees. Below is a comparison of the expected samplesize for fixed samplesize and sequential GLR tests at the same levels. For a more detailed explanation of this family of tests (as well as the graph below), see the reference.

Expected samplesize comparison

Try out this demo to compare samplesize from the fixed samplesize test versus the sequential GLR test yourself.

SeGLiR currently contain these tests:

  • comparing two bernoulli proportions (for instance for comparing conversion on a website)
  • comparing two normal means (with equal, known or unknown variance)
  • choosing the best arm in a multi-armed bandit setting (with δ-PAC guarantees)

Improvements and additions are welcome, take a look at issues for suggested improvements.

Installation

The recommended way is to install SeGLiR via node package manager (install node.js if you don't already have it):

npm install seglir

Usage

If you're not running SeGLiR in node, note that SeGLiR requires jStat.js available.

var glr = require("seglir");
// create an instance of a two-sided test comparing bernoulli proportions, with indifference region with size 0.01, alpha-level = 0.05, beta-level = 0.10
var test = new glr.test("bernoulli", "two-sided", 0.01, 0.05, 0.10);
// add data as it comes in
...
test.addData({x : 0});
test.addData({x : 0, y : 0});
test.addData({y : 1});
...
// when addData() returns string 'true' or 'false', the test is concluded
// 'true' means the null hypothesis was accepted
// 'false' means the null hypothesis was rejected (and the alternative hypothesis accepted)

// get bias-adjusted estimates
test.estimate()

For a complete function reference, see the reference.

Building from source

Make sure you have grunt and node installed. To install the development dependencies run npm install and to build it run grunt in the root directory.

License

SeGLiR is distributed under the MIT License.

About

Javascript library for rapid A/B-testing with Sequential Generalized Likelihood Ratio tests

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published