Percent agreement

The simplest agreement measure, why it always looks good on skewed data, and why no serious reliability claim rests on it alone.

The share of items two raters answered identically. Count the matches, divide by the total. Nothing is corrected for.

percent agreement = matching items / total items

Why it flatters

Two raters who both say pass to everything, without reading anything, score 100 percent on data where everything passes. The measure cannot tell skill from a shared habit.

Because most conversations pass most rules, almost any pair of reviewers will score above 90 percent. The number is real and tells you very little.

What it is still good for

It is the only agreement number a non-specialist reads correctly on sight, and it is the honest companion to a chance-corrected coefficient.

Reporting it beside kappa is what stops a reader misreading a low kappa as reviewers contradicting each other.

Reporting it alone, as a reliability claim, is the most common mistake in this whole vocabulary. The kappa paradox