Audit chat agents running on Intercom

Evidova asks Intercom what changed, reads each conversation in full, and scores it.

What it catches

Intercom runs the inbox and hands the easy questions to an agent. It does not tell you whether the agent answered them. Evidova reads every turn, the bot's and your teammates', and scores the conversation against the checks you approved: whether the agent answered the question asked, whether it said it was not a person when asked, whether it promised something nobody authorised, and where the customer stopped replying.

Connect Intercom

In Intercom, create an access token for the workspace you want read.
In Evidova, open Setup, choose Intercom, and paste the token.
There is nothing to paste back into Intercom. About once a minute we ask Intercom which conversations were updated since the last time we asked, oldest first, and then read each one in full, because Intercom's search result leaves the messages out. The first pass starts at the beginning of the workspace and works forward until it catches up, so history arrives on its own.
Leave the page size and the parallel fetches alone unless Intercom asks you to slow down. A page holds 25 conversations by default and Intercom refuses more than 150; fetches run 4 at a time and stop at 8.

What we read from Intercom

type	Must be conversation. Any other value produces no turns
id	Which conversation this is
updated_at	How we know it changed. The newest one we have read is where the next read starts, and we deliberately read a second back over that edge, because a repeat costs nothing and a gap is a conversation nobody scored
contacts.contacts[0].id	The customer, and what the conversation is grouped under. It falls back to the author of the opening message, then to the conversation's own id
created_at, source.body, source.author	The opening message: when the conversation started, what it said, and who wrote it
conversation_parts.conversation_parts[].part_type	Which entries count as turns. comment, open and close are read, and no other kind is
conversation_parts.conversation_parts[].body	What was said. Intercom stores it as HTML, so we keep the words and drop the markup
conversation_parts.conversation_parts[].created_at	When it was said, in whole seconds
conversation_parts.conversation_parts[].id	Which entry it is, so a conversation read twice does not become two copies of itself
author.type	Who wrote it. user, lead and contact are the customer, bot is your agent, and anything else is one of your people
author.id, author.name	Which teammate, when a teammate replied. Read the trap below

What we ignore, and why

Every part type except comment, open and close is left out, because those three are what the customer actually saw. An entry whose words come out empty is left out as well, so a part that is only an image, a file or a reaction produces no turn rather than an empty one. And a conversation Intercom refuses us, because it was deleted or because the token may not read it, is dropped from that pass and comes back the next time it changes. If Intercom refuses every one of them, we stop and keep our place, so a token with the wrong access reads as a failure you can see rather than as a gap nobody noticed.

One trap worth knowing

Only an author Intercom marks as bot is scored as your agent. Every other reply, from an admin or from a team, is stored as a person's and stays out of the agent's score, so your teammates cannot drag the agent's numbers down. Who we name is the less clean half. Intercom puts the author on the entry itself, so we keep the author's id and the name the customer saw, but when that author is a team the id names a shared inbox and not whoever actually typed. A name on a turn here is the account Intercom credited, which is the truest thing Intercom said about it, and not a promise that one named person wrote those words.

What Intercom gives us

Channel	chat
Audio	No
Timestamps	Seconds, on Intercom's clock
Delivery	Polled by us
Retries	Intercom does not retry