Cohen's Kappa Calculator (Agreement Between Two Raters)

What this tool does

Two raters agreeing 85 % of the time sounds good, but if both say yes nine times out of ten they would agree about 82 % of the time by luck alone. Kappa measures agreement beyond chance: 0 means no better than chance, 1 is perfect agreement and negative values mean worse than chance. The conventional verbal labels of Landis and Koch (1977) are only rough guides. Kappa also falls when one answer is very rare even though the raters agree on almost everything — the 'kappa paradox' — which is why the prevalence- and bias-adjusted figure is shown beside it. The interval is a large-sample approximation.

Formula

κ = (pₒ − pₑ) ÷ (1 − pₑ) · pₒ = (a + d) ÷ N · pₑ = ((a+b)(a+c) + (c+d)(b+d)) ÷ N² · SE ≈ √(pₒ(1 − pₒ) ÷ (N(1 − pₑ)²)) · PABAK = 2pₒ − 1

Variables

Worked example

Limitations

• The calculation runs at full precision and only the display is rounded. If you copy an intermediate value and retype it, small differences can appear.
• The default values are typical reference figures, not measurements of your situation. Replace them with your own data whenever you have it.