Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add validation analysis #21

Merged
merged 5 commits into from
Oct 17, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -182,3 +182,6 @@ firestore.indexes.json

# Typedoc generated docs
docs

validation/data
validation/simulation/resp
15 changes: 14 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,12 +37,25 @@ const numItems = cat.nItems;

// find the next available item from an input array of stimuli based on a selection method

const stimuli = [{difficulty: -3, item: 'item1'}, {difficulty: -2, item: 'item2'}];
const stimuli = [{difficulty: -3, item: 'item1'}, {difficulty: -2, item: 'item2'}];

const nextItem = cat.findNextItem(stimuli, 'MFI');
```

## Validations
### Validation of theta estimate and theta standard error
Reference software: mirt (Chalmers, 2012)
![img.png](validation/plots/jsCAT_validation_1.png)

### Validation of MFI algorithm
Reference software: catR (Magis et al., 2017)
![img_1.png](validation/plots/jsCAT_validation_2.png)


## References
Chalmers, R. P. (2012). mirt: A multidimensional item response theory package for the R environment. Journal of Statistical Software.

Magis, D., & Barrada, J. R. (2017). Computerized adaptive testing with R: Recent updates of the package catR. Journal of Statistical Software, 76, 1-19.

Lucas Duailibe, irt-js, (2019), GitHub repository, https://github.com/geekie/irt-js

Expand Down
26 changes: 13 additions & 13 deletions paper/paper.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,16 +9,18 @@ authors:
affiliation: 1
- name: Jason D. Yeatman
orcid: 0000-0002-2686-1293
affiliation: "1, 2"
affiliation: "1, 2, 3"
- name: Adam Richie-Halford
orcid: 0000-0001-9276-9084
affiliation: "1, 2"
affiliations:
- name: # TODO (Anya) Add GSE affiliation
index: 1
- name: # TODO (Anya) Add DBP affiliation
- name: Stanford University Graduate School of Education
index: 1
- name: Stanford University School of Medicine, Division of Developmental Behavioral Pediatrics
index: 2
date: 13 March 2023
- name: Stanford University Department of Psychology
index: 3
date: 5 October 2023
bibliography: paper.bib
---

Expand All @@ -33,13 +35,11 @@ bibliography: paper.bib

# Summary

jsCAT is a JavaScript library to support a fully client-side adaptive testing
environment. Computerized Adaptive Testing (CAT) is a type of assessment that
uses computer algorithms to dynamically adjust the difficulty of questions based
on a test-taker's performance. Although CAT has been widely researched and
applied in areas such as education and psychology (e.g., GRE and Duolingo),
designing a CAT solution suitable for large-scale school administration remains
a challenge. Imagine we are administering 500 children in a school at the same
jsCAT is a JavaScript library to support fully client-side computerized adaptive testing. Computerized Adaptive Testing
is an approach to assessment that uses an algorithm to select the most informative item to present to a test-taker
based on their previous responses. Although CAT has been widely researched and applied in areas such as education
and psychology (e.g., GRE and Duolingo), designing a CAT solution suitable for large-scale school administration
remains a challenge. Imagine we are administering 500 children in a school at the same
time, where reliable wifi is not guaranteed at all times, which means that we
should minimize data communications between the client and servers to prevent
disruptions. Most existing adaptive testing solutions, written in R or Python,
Expand Down Expand Up @@ -78,4 +78,4 @@ Funding Acquisition: JY.

# References

<!-- TODO (Anya): Add references in the paper.bib file -->
<!-- TODO (Anya): Add references in the paper.bib file -->
Loading