What gets scored

Why the fast rules run on every conversation and the ones that cost money run on a share.

Two things decide whether a rule ran on a conversation: whether that rule needs an AI judge to read the conversation, and, if it does, whether the conversation was picked for the immediate reading or left for the nightly one.

Rules that need no AI run on everything

A rule that can be settled by arithmetic, by a timestamp or by matching text runs on every finished conversation, every time. There is nothing to ration, so nothing is rationed.

Rules that need a judge run on a share, then on the rest

As a conversation closes, one in five goes to the judge immediately. That share is a workspace setting: put it at everything and everything is read at once, put it at nothing and the nightly pass does all of it.

Which one in five is not a coin toss. It is decided from the conversation's own identifier, so the same conversation is always on the same side of the line, and the decision can be reproduced later rather than only trusted.

Overnight, the nightly pass takes the day's finished conversations, plans the reading nothing has done yet, and sends it at the providers' batch prices. It opens at 2am and stops at 7am in the workspace's own timezone by default, and it reconsiders the last seven days on every run.

When something looks wrong, the share becomes everything

While an alert is open on a client, an agent or a channel, every conversation in that part of the traffic goes to the judge immediately, whatever the share says. The escalation holds for as long as the alert keeps being seen, an hour by default, which is the hour you most want the detail.

Some conversations are never scored, deliberately

Before any AI is paid for, a screening step drops what cannot be scored: no reply from the customer at all, two characters of one, a wrong number, or the same line repeated over and over. The conversation carries which of those it was.

We have not measured this yet. The reason lives in the sentence next to it, not the label. A blank is never a zero.

A pass is re-read too

One passing result in twenty goes to the other judges anyway. Re-reading only the failures finds a judge that flags too much, and never finds one that has started waving everything through.

When the budget runs out, the order is published

The nightly pass works to a spend cap. What it can afford it takes in a fixed order: reading tied to an open alert first, then rules that can cap a score on their own, then the remainder ordered by a hash, so a second run of the same night makes the same choices.

What did not fit is written down as reading that was planned and not done. A day nobody could afford and a day nobody had conversations on are different facts, and they read differently.

How much of the last seven days actually got the fuller reading is a number on the scoring quality screen, per workspace. It is measured after the fact, not promised in advance.
Note: Every figure on this page is a default. A workspace can move the share, the hours, the timezone and the cap, and what it sets is what runs.

Read next

  • What a score is · What a number here means, what it does not mean, and what you must not use it for.
  • How it works · The six stages a conversation passes through, from arriving to being scored.
  • Why an outside judge · Why the thing scoring your agent should not be the thing that built it.
This page as markdown