The irrCAC is a Python package that provides several functions for calculating various chance-corrected agreement coefficients. This package closely follows the general framework of inter-rater reliability assessment presented by Gwet (2014).
The functionality covers calculations for various chance-corrected agreement coefficients (CAC) among 2 or more raters. Among the CAC coefficients covered are Cohen's kappa, Conger's kappa, Fleiss' kappa, Brennan-Prediger coefficient, Gwet's AC1/AC2 coefficients, and Krippendorff's alpha. Multiple sets of weights are proposed for computing weighted analyses.
The functions included in this package can handle 2 types of input data. Those types with the corresponding coefficients are in the following list:
- Contingency Table
- Brennar-Prediger
- Cohen's kappa
- Gwet AC1/AC2
- Krippendorff's Alpha
- Percent Agreement
- Schott's Pi
- Raw Data
- Fleiss' kappa
- Gwet AC1/AC2
- Krippendorff's Alpha
- Conger's kappa
- Brennar-Prediger
Note
All of these statistical procedures are described in details in Gwet, K.L. (2014,ISBN:978-0970806284): "Handbook of Inter-Rater Reliability," 4th edition, Advanced Analytics, LLC.
This package is a port (with permission) to Python of the irrCAC library for R by Gwet, K.L.
Important
This is a work in progress and does not have (yet) the full functionality found in the R library.
To install the package, run:
pip install irrCAC
To use the code for development it is recommended to install poetry and run:
poetry install
And add the pre-commit hook:
pre-commit install
and update the hooks:
pre-commit autoupdate
To update the project dependencies, run:
poetry update
Next run the tests:
poetry run pytest
There is also a config file for tox so you can automatically run the tests for various python versions like this:
tox
The documentation of the project is available at the following page: http://irrcac.readthedocs.io/