Intraclass Correlation Coefficient (ICC)
ICC measures the consistency or agreement of continuous measurements made by multiple raters or across repeated measurement occasions. The standard reliability statistic for inter-rater and test-retest reliability.
Step 1 — Provide your data
Matrix of subjects x raters (numeric). Each row = one subject, each column = one rater/occasion.
Supports Excel Import, Sample Data Generator, and Manual Entry.
Step 2 — Choose ICC model and type
Model:
| Model | When to use |
|---|---|
| One-way random | Different raters per subject |
| Two-way random | Same raters, want to generalise |
| Two-way mixed | Same fixed raters, only these matter |
Type:
| Type | When to use |
|---|---|
| Single | Using one rater's score in practice |
| Average | Using average of all raters |
Step 3 — Results
- ICC value with specific form label (e.g. ICC(2,1))
- 95% confidence interval
- F-statistic with df and p-value
- Reliability interpretation
- ANOVA table (SS, df, MS)
- Rater means and SDs
A UniversalChatBot is available for discussion.
Statistical methods used
Six ICC forms (Shrout & Fleiss, 1979):
- ICC(1,1), ICC(1,k) — one-way
- ICC(2,1), ICC(2,k) — two-way random (absolute agreement)
- ICC(3,1), ICC(3,k) — two-way mixed (consistency)
Built on two-way ANOVA decomposition (MS_between, MS_within, MS_error, MS_raters).
Interpretation (Koo & Li, 2016):
| ICC | Label |
|---|---|
| >= 0.90 | Excellent |
| 0.75 - 0.90 | Good |
| 0.50 - 0.75 | Moderate |
| 0.00 - 0.50 | Poor |
| < 0.00 | No agreement |
H0: ICC = 0. H1: ICC > 0. alpha = 0.05.