Dimensions and severity

The six things a check can be about, and the four levels of how much it matters.

Every rule carries two labels: what it is about, and how much a failure matters. Both come from a fixed list in the code, so no rule can invent a third.

What a rule is about

Six subjects, and every rule sits in exactly one. The code's word for this is a dimension. On screen the product calls it a score area, because that is the phrase a reader recognises.

  • compliance
  • accuracy
  • resolution
  • progression
  • experience
  • communication
complianceA rule the agent is held to by someone outside the company: a regulator, a contract, or a privacy law.
accuracyWhether what the agent said is true against a source that can be checked.
resolutionWhether the customer left with the thing they asked for.
progressionWhether the conversation moved forward: the questions asked, the steps taken, the booking reached.
experienceWhat it felt like to be on the other end. Waiting, dead air, repetition.
communicationWhether the words worked: the right language, plain enough, and an answer to what was actually asked.

How much a failure matters

Severity names what a failure does to the customer or to the business. It does not record how firmly the source document worded the rule.

The order below is also the order used when two near-duplicate rules are merged into one. The higher level wins.

  • critical
  • high
  • medium
  • low
criticalSomeone is harmed, misled, or owed a remedy. Legal, safety or money.
highThe customer did not get what they came for, and would notice.
mediumThe outcome held but the experience did not. Worth fixing this quarter.
lowNoticeable to a reviewer reading closely, not to the customer.

The four kinds of rule

How a rule is answered, which decides what it costs to run and how far it can be trusted on its own.

  • deterministic
  • judged
  • end_state
  • guard
deterministicSettled by arithmetic, a timestamp or a text match. Runs on every conversation, costs nothing, and gives the same answer twice.
judgedNeeds a judge to read the conversation and form an opinion. This is what the sampling share and the nightly pass exist to ration.
end_stateAsks what the conversation ended as, not what happened inside it: booked, abandoned, handed to a person.
guardA line the agent must not cross. A single breach fails the conversation whatever else went well.

Where a rule stands

A rule reaches a score only once a reviewer has approved it. These are the states it moves through.

  • shadow
  • approved
  • rejected
  • superseded
shadowScored and counted, but nobody has agreed to it yet. Run one this way to see what it would have caught before it can fail anything.
approvedA reviewer agreed to it. It scores conversations and it can fail one.
rejectedA reviewer turned it down. Kept, so the same proposal does not come back as though it were new.
supersededReplaced by a newer revision of the same pack check. Distinct from rejected, which records a reviewer taking the rule out of what we check altogether; this one records the rule surviving under a newer row.
Note: Where a value has no line beside it, the code carries no description for it yet. We leave that gap where you can see it rather than invent a sentence here.

Read next

  • Platform matrix · All twelve platforms side by side: channel, audio, timestamps, delivery, retries.
  • API keys · What a key is, the two scopes, and how to revoke one.
  • Webhooks · The one URL a platform posts conversations to.
This page as markdown