New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make ab-tests create 50/50 cohorts #3674

Closed
ianb opened this Issue Oct 25, 2017 · 1 comment

Comments

Projects
None yet
2 participants
@ianb
Contributor

ianb commented Oct 25, 2017

We should change ab-tests.js to do cohort selection, then 50/50 control/experiment (or 33/33/33 for three tests, etc). The cohort selection might be 99/1, 90/10, 90/10 except only en-US, etc.

I think we should change from just ("control", experiment_name) to ("excluded", "control", experiment_name).

@ianb ianb self-assigned this Oct 25, 2017

@johngruen

This comment has been minimized.

Show comment
Hide comment
@johngruen

johngruen Oct 26, 2017

Collaborator

putting this in 58, but this is kick-outable

Collaborator

johngruen commented Oct 26, 2017

putting this in 58, but this is kick-outable

@johngruen johngruen modified the milestones: Launch 58, Bucket ☕️, Sprint 1 (59-0) Oct 26, 2017

ianb added a commit that referenced this issue Nov 3, 2017

Fix #3674, put 50% of people into control
This separates out people who are excluded from a test, from those who are in the control for the test. 50% of people always go to control, and 50% go into some experiment branch. People who aren't in either go into the 'exclude' category.

@6a68 6a68 closed this in a583d99 Nov 6, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment