Kappa Agreement Analysis
The Kappa calculator measures agreement between two or more raters on categorical data, supporting both two-rater and multi-rater scenarios, nominal and ordinal categories.
Step 1 — Choose the Kappa variant
| Type | Description | When enabled |
|---|---|---|
| Cohen's Kappa | Standard kappa for two raters, nominal | 2 raters, nominal |
| Weighted Kappa (Linear) | Linear weights for ordinal | 2 raters, ordinal |
| Weighted Kappa (Quadratic) | Quadratic weights for ordinal | 2 raters, ordinal |
| Fleiss Kappa | Multi-rater kappa | 3+ raters |
A Data Type selector switches between Nominal and Ordinal.
Step 2 — Provide your data
- Excel Import — auto-detects two-rater vs multi-rater format
- Sample Data Generator — built-in datasets
- Manual Data Input — build custom table
Step 3 — Calculate
Results panel shows:
- Kappa value (kappa) with 95% CI
- Observed agreement (p_o)
- Expected agreement (p_e)
- Standard error and z-statistic
- p-value
- Strength of agreement — Landis & Koch (1977):
| Kappa | Strength |
|---|---|
| < 0.00 | Poor |
| 0.00 - 0.20 | Slight |
| 0.21 - 0.40 | Fair |
| 0.41 - 0.60 | Moderate |
| 0.61 - 0.80 | Substantial |
| 0.81 - 1.00 | Almost perfect |
For Fleiss Kappa: category-specific kappas also shown.
Export Results button available. A UniversalChatBot is available below.
Statistical methods used
Cohen's Kappa
kappa = (p_o - p_e) / (1 - p_e)
where p_o = sum of diagonal / N, p_e = sum of (row_i * col_i) / N^2.
Weighted Kappa
- Linear weights:
w[i][j] = 1 - |i-j| / (max(R,C) - 1) - Quadratic weights:
w[i][j] = 1 - (i-j)^2 / (max(R,C) - 1)^2
kappa_w = (weighted_p_o - weighted_p_e) / (1 - weighted_p_e)
Fleiss Kappa
For N subjects, n raters, k categories:
P_bar = mean subject agreement
P_e = sum(p_j^2) where p_j = category proportion
Fleiss kappa = (P_bar - P_e) / (1 - P_e)
Hypothesis: H0: kappa = 0, H1: kappa != 0, alpha = 0.05.