Add support for Krippendorff's alpha reliability coefficient #13

Open
ludwigschubert opened this Issue Mar 16, 2013 · 2 comments

Projects

None yet

2 participants

@ludwigschubert

(First of all, thanks so much for this package!
While it took me some time to get used to it, it's super handy even for simple statistics in business applications.)

I might lack the knowledge to combine the pieces of this framework, but I'm missing explicit support for measuring reliability between raters, e.g. Krippendorff's alpha.

R provides a nice interface for this here.

Thanks again for your efforts and time in supporting this project!

@clbustos
Owner

Can you provide me a reference? A journal or book reference will be cool :)

@ludwigschubert

Yes.

The Original paper by Prof. Krippendorf is available here.
Together with a colleague he also provided a (to me unreadable) implementation in SAS here.

Instead, I'd recommend working of Matlab code and the wikipedia article on the subject.
I'm currently using R's package "irr"'s implementation, which can be found here - I extracted the specific file for you in this GIST.

Let me know if there's anything else I can help with - unfortunately I had about one introductory class in statistics, so the finer points of Krippendorf's alpha are entirely lost on me.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment