SUMM: score tests and similar

[josef-pkt]

I need a summary issue for the score_test, lagrange multiplier test, artificial regression, moment restriction tests

They are all similar or asymptotically the same with different assumption on robustness to misspecification. There are many open issues for this
#1163 basic issue
#1756 conditional moment restriction
#2054 score_tests with robust covariance assumptions

related:

RESET test for GLM, Logit, Probit (example also in Papke Wooldridge for proportion response model)

special cases
overdispersion test in Poisson (and related count models)

implementation*

what do we have available for a test?

two (or three) cases:

• if we have a fully specified model: we can use score_test directly
  we have score_obs, score, hessian w.r.t. all parameters that we want to test
  no distinction in handling score from parameters used in estimation from those under test
  e.g. variable addition tests
• we only have a partial model without fully specified alternative in the model: we can use artificial regression, "scores" defined outside of the model or conditional moment tests, or embed in GMM
  • the only generic setup I can see is conditional moment tests cm_test
  • testing covariance assumption in mean models: a large battery of LM test for OLS, but in a similar way should be extendable to entire LEF, I have it for GLM reusing the methods from the link and family.
  • special case tests: e.g. score tests for poisson versus zero inflated or hurdle model.
• general tests on properties: Hausman, Information Matrix equality, ...
  can be formulated as LM test in linear case, but I don't know how they fit in "generically"

issues on infrastructure
#2072 get_robustcov to be reused for score tests

artificial regression: Gauss-Newton regression of Davidson-MacKinnon
generic "one-step" estimators (see for example Newey McFadden 1994 Handbook of Econometrics
-> reuse Newton optimizer, iterative WLS results ?

multiple hypothesis, multiple testing
just an idea: using normal based score test for single restriction
simplest case: we have many possibly omitted variables that we want to test
we can test all of them at the same time - standard score test, like results.wald_test
we could also test the individual restrictions/additions - like results.t_test
questions: correlation of the omitted variables, jointly or in loop?
for example Pagan and Vella 1989 report separate t-statistics for a list of moment restrictions (omitted variables or heteroscedasticity)
Stata also reports separate tests with optional p-value correction in some of the estat specification tests after regress

Where are the score_test examples that I already did?

https://github.com/statsmodels/statsmodels/pull/1781/files#diff-724e15f4019b2a24906f6b271d67e370R433
Poisson overdispersion are not in repo

edit
variable addition score test: reduced rank #5364
We need to handle the case when some added variables are already implicitly included in the restricted model. GLM.score_test uses the full number of variables and not the rank of the addition.
(found while running an example for GAM in #5296 )

[josef-pkt]

good read for link between conditional moment tests and score tests, although a bit dated:
Pagan, Adrian, and Frank Vella. 1989. “Diagnostic Tests for Models Based on Individual Data: A Survey.” Journal of Applied Econometrics 4 (S1): S29–59. doi:10.1002/jae.3950040504.

[josef-pkt]

a reference as reminder:
Chapter 5 in Cameron Trivedi has a good discussion of MLE, QMLE based LM tests and related conditional moment tests (I read it a year ago and had forgotten about it.)

In general it's not easy to see what the maintained assumptions in a test are. Wooldridge tries to be robust to higher moment misspecification, Cameron Trivedi mix them up a lot more.
My impression is that Wooldridge's multistep artificial regression based tests (1990, 1991) use a lot of case specific proofs instead of basing it on a "generic" principle (like GMM for example). But I didn't read the main 1990 paper again.
Maybe generic GMM based test would be easier to implement, than the Wooldridge procedures where one of the wiping step replicates the inner part of the sandwich calculation. see #1790

correction: GMM based tests (cm-tests) might be easy, but "too robust". It might be difficult to add additional maintained assumptions to get less than fully robust tests. (tradeoff small sample performance and robustness)

[josef-pkt]

R package statmod has glm.scoretest as a omitted/added variable test with dispersion as optional parameter.

[josef-pkt]

another model: goodness-of-fit in logistic matched case-control (i.e. Conditional Logit)
It looks like it essentially reduces to a conditional moment test or to a variable addition test where we include the squares or the regressors (dropping singular or duplicate, especially categorical variables)
Chen, Li-Ching, and Jiun-Yi Wang. 2013. “Testing the Fit of the Logistic Model for Matched Case-Control Studies.” Computational Statistics & Data Analysis 57 (1): 309–19. doi:10.1016/j.csda.2012.07.001.

looks a bit like a complement to Hosmer-Lemeshow because it only considers continuous variables.
(e.g. curvature of link function doesn't matter in categorical/saturated models. I haven't checked the details.) see also #1061 for related issue in current diagnostic tests.

[josef-pkt]

(random thought for categorical explanatory variables)

FAQ: How do we do a score test for an interaction effect?
side issue: construct the exog columns for the interaction effect that is orthogonal (for the given data) to the main effects. (not the balance case where we can construct ex ante orthogonal constrasts)

score test conditions on included variables, but we need both z/t test for single parameters and chi2/F test for joint hypothesis.

this is mainly a use case and example for multi-column joint tests, everything is generic, when we have the equivalent of t_test and wald_test for score/LM tests. One possible special issue: how do we create the raw design matrix for the interaction effect before conditioning, e.g. using patsy. Also if we don't remove collinearities, then we need to handle them in the score_test (e.g. adjust degrees of freedom based on rank)

[josef-pkt]

another reference,
This has a collection of several tests for binary endog models all implemented with auxiliary regression (Davidson and MacKinnon type), i.e. LM or CM test using plain residuals (not weighted/scaled)
The only exception is a P-test for non-nested models for testing between two specified link functions.
For RESET test they only compare across number of polynomial terms. Minimum sample size in Monte Carlo is 500.

Ramalho, Esmeralda A., and Joaquim J. S. Ramalho. 2012. “Alternative Versions of the RESET Test for Binary Response Index Models: A Comparative Study*.” Oxford Bulletin of Economics and Statistics 74 (1): 107–30. doi:10.1111/j.1468-0084.2011.00654.x.

[josef-pkt]

SAS seems to have generalized/robust score_test for GEE/GLM #1739 (comment)

[josef-pkt]

to separate tests of CMT, normal instead of joint chisquare

CMTNewey and CMTTauchen have ztest methods for the individual constraints.
Those are designed for GMM
The individual lm or cm tests don't have an option for individual hypothesis, only the joint chisquare based results

Can I use them for the a specification test for MLE, e.g. excess probability tests like zero-inflation or binned prob test in count models.?

[josef-pkt]

score_test robust to misspecification cov_type

newer article I found (I didn't read it yet)

Bera, Anil K., Yannis Bilias, Mann J. Yoon, Süleyman Taşpınar, and Osman Doğan. 2020. “Adjustments of Rao’s Score Test for Distributional and Local Parametric Misspecifications.” Journal of Econometric Methods 9 (1). https://doi.org/10.1515/jem-2017-0022.

already referenced in #1163 (comment)

They do not cite Boos which was my main reference

I have problems finding the Boos article in issues and doesn't seem to be in docs
Boos reference is

Boos, Dennis D. "On generalized score tests." The American Statistician 46, no. 4 (1992): 327-333.