An R package for optimal item calibration in computerized achievement tests.
The restricted optimal design method is implemented to optimally allocate a set of items that require calibration to a group of examinees. The optimization process is based on the method described in detail by Ul Hassan and Miller in their works published in 2019 and 2021. To use the method, preliminary item characteristics must be provided as input. These characteristics can either be expert guesses or based on previous calibration with a small number of examinees. The item characteristics should be described in the form of parameters for an Item Response Theory (IRT) model. These models can include the Rasch model, the 2-parameter logistic model, the 3-parameter logistic model, or a mixture of these models. The output consists of a set of rules for each item that determine which examinees should be assigned to each item. The efficiency or gain achieved through the optimal design is quantified by comparing it to a random allocation. This comparison allows for an assessment of how much improvement or advantage is gained by using the optimal design approach. This work was supported by the Swedish Research Council (Vetenskapsr?det) Grant 2019-02706.
The easiest way to install the optical package from CRAN using:
install.packages("optical")You can install the development version of optical from GitHub with the following code:
# if not installed already on your computer, install devtools
install.packages("devtools")
# Install the package
devtools::install_github("scenic555/optical")
# Load the optical package
library(optical)This is a basic example which shows you how to solve a common problem:
library(optical)
# 2PL-model for three items with parameters (a, b) equal to (1.6,-2),
# (1.6,0.5), and (1.6, 2) for the first, second, and third items,
# respectively.The calibration of these three items with the 2PL model
# is conducted in one block.
a<-c(1.6,1.6,1.6); b<-c(-2,0.5,2)
ip <-cbind(a,b)
yyy <- optical(ip)
yyy
#> Block1
#> ===================================================================
#> Table of interval boundaries for D-optimal design with items and
#> probabilities (expected proportion of examinees in this interval)
#> ===================================================================
#> Lower Upper Item Probability
#> 1 -Inf -1.0957 1 0.13662
#> 2 -1.0957 -0.3965 2 0.20925
#> 3 -0.3965 0.0705 1 0.18223
#> 4 0.0705 0.6431 3 0.21183
#> 5 0.6431 1.4099 2 0.18077
#> 6 1.4099 Inf 3 0.07929
# Graph for (optimal) design
drawdesign(yyy=yyy, ylowl=-1000, refline=0.002, layout=1)
This package is free and open source software, licensed under GPL (>= 3).
This work was supported by the Swedish Research Council (Vetenskapsrådet) Grant 2019-02706.
Ul Hassan and Miller (2019). Optimal item calibration for computerized achievement tests. Psychometrika, 84, 1101-1128.
Ul Hassan and Miller (2021). An exchange algorithm for optimal calibration of items in computerized achievement tests. Computational Statistics and Data Analysis, 157: 107177.
Bjermo, Fackle-Fornius, and Miller (2021). Optimizing Calibration Designs with Uncertaintyin Abilities. Manuscript.