Gwet's gamma coefficient

Jeffrey M Girard edited this page Apr 5, 2016 · 16 revisions

Overview

The gamma coefficient is a chance-adjusted index for the reliability of categorical measurements. It estimates chance agreement using a hybrid (category- and average-distribution-based) approach. It is also called Gwet's agreement coefficient (AC1 when nominal and AC2 when non-nominal).

MATLAB Functions

  • mGAMMA %Calculate gamma using vectorized formulas

Simplified Formulas

Use these formulas with two raters and two (dichotomous) categories:


p_o

m_1

m_2

p_c

gamma


n_11 is the number of items both raters assigned to category 1

n_22 is the number of items both raters assigned to category 2

n is the total number of items

n_1+ is the number of items rater 1 assigned to category 1

n_2+ is the number of items rater 1 assigned to category 2

n_+1 is the number of items rater 2 assigned to category 1

n_+2 is the number of items rater 2 assigned to category 2

Contingency Table

Generalized Formulas

Use these formulas with multiple raters, multiple categories, and any weighting scheme:


rstar_ik

p_o

T_w

pi_k

p_c

gamma


q is the total number of categories

w_kl is the weight associated with two raters assigning an item to categories k and l

r_il is the number of raters that assigned item i to category l

n' is the number of items that were coded by two or more raters

r_ik is the number of raters that assigned item i to category k

r_i is the number of raters that assigned item i to any category

n is the total number of items

References

  1. Gwet, K. L. (2008). Computing inter-rater reliability and its variance in the presence of high agreement. The British Journal of Mathematical and Statistical Psychology, 61(1), 29–48.
  2. Gwet, K. L. (2014). Handbook of inter-rater reliability: The definitive guide to measuring the extent of agreement among raters (4th ed.). Gaithersburg, MD: Advanced Analytics.
You can’t perform that action at this time.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.
Press h to open a hovercard with more details.