You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
"pooled" is actually the estimated prob under the null of equal proportion if the true DGP is the one specified by the alternative.
using prop2 or prop1 for var_null might not correspond to any hypothesis test method, unless the hypothesis specified a fixed level/value of prop2 under the null, e.g. H0: prob1 = prob2 = 0.05.
The generalization of "pooled" if there is a nonzero null diff (null margin as in noninferiority testing) would be the estimate under the null hypothesis restriction, eg. p2 + p1 - value = 0. We would need a more generic name for the option.
This might differ under wald versus score test.
We need some MonteCarlo studies if we want to match up the power and sample size computation directly with the hypothesis test method.
Aside:
"pooled" uses the props specified by the alternative. Those are expected values under the alternative.
In the actual hypothesis test we use the sample estimates so var_null is a random variable. I don't think using the expected variance is "exact" (or equal to the one we would get with simulations) because of the nonlinearities.
PASS has some "averaged power" but, AFAIR, that refers to a distribution of props under the alternative (and not sampling variation of variance estimate, although using random props also changes the implied var.)
https://stats.stackexchange.com/questions/605466/how-to-get-python-statsmodels-to-match-evan-millers-famous-ab-test-sample-size/605540#605540
I'm not sure what the name of the options should be.
The text was updated successfully, but these errors were encountered: