Skip to content

What gets scored

Why the fast rules run on every conversation and the ones that cost money run on a share.

Two things decide whether a rule ran on a conversation: whether that rule needs an AI judge, and, if it does, whether the conversation was picked for the immediate reading or left for the nightly one.

Rules that need no AI run on everything

A rule settled by arithmetic, by a timestamp or by matching text runs on every finished conversation, every time. There is nothing to ration, so nothing is rationed.

Rules that need a judge run on a share, then on the rest

As a conversation closesOne in five goes to the judge immediately
That shareA workspace setting. Put it at everything, or at nothing, and that is what runs
Which one in fiveDecided from the conversation's own identifier, so the same conversation is always on the same side of the line and the choice can be reproduced
The nightly pass2am to 7am in the workspace's own timezone, by default
How far back it looksThe last seven days, on every run

Overnight, the nightly pass takes the day's finished conversations, plans the reading nothing has done yet, and sends it at the providers' batch prices.

When something looks wrong, the share becomes everything

While an alert is open on a client, an agent or a channel, every conversation in that traffic goes to the judge immediately, whatever the share says. It holds for as long as the alert keeps being seen, an hour by default.

Some conversations are never scored, deliberately

Before any AI is paid for, a screening step drops what cannot be scored: no reply from the customer, two characters of one, a wrong number, or the same line repeated. The conversation carries which it was.

We have not measured this yet. The reason lives in the sentence next to it, not the label. A blank is never a zero.

A pass is re-read too

One passing result in twenty goes to the other judges anyway. Re-reading only the failures finds a judge that flags too much, and never finds one that has started waving everything through.

When the budget runs out, the order is published

The nightly pass works to a spend cap. What it can afford it takes in a fixed order: reading tied to an open alert, then rules that can cap a score on their own, then the remainder ordered by a hash.

So a second run of the same night makes the same choices. What did not fit is written down as reading that was planned and not done, because a day nobody could afford and a day with no conversations are different facts.

How much of the last seven days got the fuller reading is published per workspace on the scoring quality screen. It is measured after the fact, not promised in advance.

Note: Every figure on this page is a default. A workspace can move the share, the hours, the timezone and the cap, and what it sets is what runs.

Read next

  • What a score is · What a number here means, what it does not mean, and what you must not use it for.
  • How it works · The six stages a conversation passes through, from arriving to being scored.
  • Why an outside judge · Why the thing scoring your agent should not be the thing that built it.
This page as markdown