Prevalence

How often the thing you are looking for actually happens, and why a rare failure wrecks precision no matter how good the check is.

The share of conversations that really do contain the thing being looked for. Also called the base rate. It is the number that makes every other number readable.

Why rare things are hard

Take a check with 90 percent recall and 95 percent specificity, run on 10,000 conversations where 1 percent really fail.

100 real failures      ->  90 caught
9,900 real passes      -> 495 false positives

precision = 90 / (90 + 495) = 15%

The check is good. The precision is terrible. Nothing about the check changed, only how rare the target was.

This is why a vendor quoting precision without prevalence is quoting a number you cannot use, and why precision measured on a balanced test set does not survive contact with production.

Correcting for it

A measured failure rate from an imperfect check is biased by that check's own error rates. The bias can be removed if you know its sensitivity and specificity.