Prove a fix worked

Show that a change moved the number, over the same time range and the same traffic.

Use this when you have shipped a prompt change or a new flow, and somebody wants evidence that the number moved rather than your word for it.

The steps

1. Write down the day and time you shipped. Every figure here is bucketed in UTC, and the charts label them that way.
2. Open Overview in the left rail. Metrics sits indented under it. Open Metrics.
3. Pick a bucket size. By hour for a change that landed today, By day for a change a week old.
4. Read What changed in this range. Anything listed there is drawn as a dashed line across every chart, so a step in a number can be read against the change that landed with it.
5. Find the card for the measure you expect to have moved, and open the link at its foot to read that one measure in depth.
6. In depth you get the reading over time, the spread of scores behind it, the same cut by channel, the same cut by agent, and The worst of them, which opens the conversations behind the worst slice.
7. Set the top bar to a period of the same length before your change, and read the same figure again. Write both numbers down.
8. Cross-check on Abandonment. Its reason rows do the arithmetic for you, against the period of the same length immediately before.

What to do with what you find

Two figures of the same length, over the same traffic, with the change marked between them, is a claim you can defend. One figure after the change is not.

Before you compare two scores, confirm they came from the same Rules version. A score only compares with another score frozen under the same set of rules, and each conversation carries its own version beside the rule.

If a measure moved but the cut by agent shows one agent carrying all of it, you changed one agent and not the product. Say which.