ENH: Item Response Theory/Models - categorical PCA/FA

[josef-pkt]

(mainly to get started with an issue and to park two links)

https://stats.stackexchange.com/questions/252317/similarities-and-differences-between-irt-model-and-logistic-regression-model
https://stats.stackexchange.com/questions/215404/is-there-factor-analysis-or-pca-for-ordinal-or-binary-data

I saw it mentioned several times in the PCA and factor analysis literature as alternative when variables are categorical or mixed instead of continuous.

Browsing a bit for the underlying statistical methods (given what little I know):

• basic structure is Logit or Probit, either univariate for binary/binomial outcomes or multivariate/multinomial, multinomial version is usually for ordered categories
• the multiparameter version is nonlinear, not linear plus link as in GLM or Logit
• the unobserved factors, item and individual random effects, require mixed GLM or mixed Logit/Probit/Multinomial
• there might also be a connection to fractional models, i.e. logistic regression or similar for "continuous" data (many discrete points in bounded interval)

So, we need to add some extensions and new models, but IRT might also provide test cases for the generic, not application specific models.

There are some articles and books that make directly the link to mixed effects GLM. Most of the general IRT literature is more on the application to multiple-choice testing.

[josef-pkt]

bump
It looks like IRT are essentially mixed effects GLM/Logit models with one, two or several random effects (intercept, slope of indicator variables)

https://github.com/pluralsight/irt_parameter_estimation/tree/master
Apache license, seems to have stalled 2 years ago.
references Kim, Frank B. Baker, Seock-Ho

however, De Boeck et al, and similar articles directly put it into the GLMM framework
i.e. we just need MixedGLM, or MixedMixin
and maybe some (parameterized) link functions that can map into a subintervals of (0, 1) (guessing effect might introduce lower bound > 0, e.g. multiple choice question with purely random choice)

De Boeck, Paul, and Mark Wilson, eds. Explanatory Item Response Models: A Generalized Linear and Nonlinear Approach. New York, NY: Springer, 2004. https://doi.org/10.1007/978-1-4757-3990-9.

Kim, Frank B. Baker, Seock-Ho, ed. Item Response Theory: Parameter Estimation Techniques, Second Edition. 2nd ed. Boca Raton: CRC Press, 2014. https://doi.org/10.1201/9781482276725.

Aside: we need "prediction" for the values of individual random effects (person ability)

semi-random list of articles mainly using mixed models:

Baayen, R. H., D. J. Davidson, and D. M. Bates. “Mixed-Effects Modeling with Crossed Random Effects for Subjects and Items.” Journal of Memory and Language, Special Issue: Emerging Data Analysis, 59, no. 4 (November 1, 2008): 390–412. https://doi.org/10.1016/j.jml.2007.12.005.

Chalmers, R. Philip. “Extended Mixed-Effects Item Response Models With the MH-RM Algorithm.” Journal of Educational Measurement 52, no. 2 (2015): 200–222.

De Boeck, Paul. “Random Item IRT Models.” Psychometrika 73, no. 4 (December 1, 2008): 533–59. https://doi.org/10.1007/s11336-008-9092-x.

Kim, Jinho, and Mark Wilson. “Polytomous Item Explanatory Item Response Theory Models.” Educational and Psychological Measurement 80, no. 4 (August 1, 2020): 726–55. https://doi.org/10.1177/0013164419892667.

Locker, Lawrence, Lesa Hoffman, and James A. Bovaird. “On the Use of Multilevel Modeling as an Alternative to Items Analysis in Psycholinguistic Research.” Behavior Research Methods 39, no. 4 (November 1, 2007): 723–30. https://doi.org/10.3758/BF03192962.

Rose, Norman, Gabriel Nagy, Benjamin Nagengast, Andreas Frey, and Michael Becker. “Modeling Multiple Item Context Effects With Generalized Linear Mixed Models.” Frontiers in Psychology 10 (2019). https://www.frontiersin.org/articles/10.3389/fpsyg.2019.00248.

a recent comparison of methods:
Park, Jung Yeon, Klest Dedja, Konstantinos Pliakos, Jinho Kim, Sean Joo, Frederik Cornillie, Celine Vens, and Wim Van den Noortgate. “Comparing the Prediction Performance of Item Response Theory and Machine Learning Methods on Item Responses for Educational Assessments.” Behavior Research Methods 55, no. 4 (June 1, 2023): 2109–24. https://doi.org/10.3758/s13428-022-01910-8.
with replication repo https://github.com/E-IRT-team/E-IRT-ML-comparison

[josef-pkt]

other python packages that include IRT:
https://github.com/eribean/girth MIT, last change 1.5 years ago
https://github.com/bigdata-ustc/EduCDM Apache license, last change 5 months ago
https://github.com/inuyasha2012/pypsy MIT, last change 5 years ago

and there are other, Bayesian focused packages