The first year of a support centre is the only chance to capture a baseline against which every later year gets judged. What is worth tracking is fixed by what the routine actually needs to know: how much comes in, how much of it ships, how often the agent gets it wrong, and whether the weekly routine is holding under real volume.
This is the first-year view of a larger loop. For the whole thing end to end, see running support with a coding agent.
What should get measured from day one?
Five things, and they are worth instrumenting before the first request arrives rather than reconstructed later from memory.
- Volume. How many requests, bugs and pieces of feedback arrive, over what period, and from what source, so a rising or falling trend is visible rather than felt.
- Throughput. What fraction of what comes in gets a real answer, gets merged into something else, ships, or is closed unshipped, against what fraction is still open.
- Accuracy. How often the agent's triage, drafted replies or duplicate flags are wrong on review, sampled on a fixed schedule rather than only checked when something goes visibly wrong.
- Routine health. Whether the weekly pass is actually holding: how often a week gets shortened or missed, and what happens to the backlog when it does.
- Permissions. Whether the scope granted at the start still matches what the agent is actually doing, or has quietly grown past what anyone decided.
How do you sample an agent's error rate honestly?
Pull a fixed-size random sample on a fixed schedule, not a sample chosen because something looked wrong, and check it against the category definitions and the source of truth cold, the same way described in letting an agent answer without letting it invent. A sample chosen only after a mistake is already visible measures nothing, because it only ever confirms what was already known. The mechanisms that constrain an agent reduce invention rather than remove it, which is why the sample is a standing job and not a launch task. The number worth keeping is not a single figure but a trend: whether the rate is falling as the categories and the source of truth get more complete, or holding steady because something structural is still wrong.
What does throughput actually tell you that volume doesn't?
Volume on its own says how busy the queue has been. It says nothing about whether the routine is keeping pace with it. A backlog that is small because little arrived, and a backlog that is small because everything got answered, look identical in a headline count and mean completely different things for the next quarter. Throughput needs three figures held apart, not blended into one: what shipped, what closed without shipping, and what is still open.
| Outcome | What it should mean | What it can also mean if unchecked |
|---|---|---|
| Shipped | The request became something that was built | Not applicable |
| Closed, unshipped | Correctly filtered: a duplicate, already answered, out of scope | Or quietly ignored, with nobody checking why |
| Still open | Genuinely pending a decision | Or forgotten, with nothing forcing a second look |
A high number of closed-without-shipping is not automatically a bad sign. It can mean triage is correctly filtering out duplicates and already-answered requests rather than a backlog quietly being ignored. Reading the figure honestly means knowing which of the two it is, and that only comes from the same sampling habit that checks the agent's accuracy.
One payment, no subscription, unlimited products.
When is the right time to revisit the routine or the permission model?
Not on a fixed calendar date, and not only after something visibly breaks. Three signals are worth watching instead: the sampled error rate moving in either direction for more than a few consecutive checks, a triage category swelling or emptying in a way the original definitions did not predict, and the weekly routine getting shortened or missed often enough that it stops feeling like an exception. The support inbox as a git repository is the shape that makes revisiting the permission model cheap: widening or narrowing what the agent can merge is a change to a few lines of configuration, not a system migration.
What does a first year usually get wrong about its own numbers?
The number easiest to report and least useful on its own is a raw count of requests handled, because it says nothing about whether they were handled well. A figure worth keeping alongside it is what fraction was sampled and checked, not just what fraction was processed. A support centre that can state its own error rate, checked on a fixed schedule rather than estimated, has a number worth trusting. One that can only report volume has a number that flatters activity, not accuracy.
The same discipline applies to the calendar itself. Fix what will be measured and how often before the year starts, not after the first quarter's numbers are already in front of you. A definition chosen once the answer is already known stops being a measurement and starts being a description of whatever happened to look good.
Frequently asked questions
What is the single most useful number to track from day one?
There isn't one. Volume without throughput hides whether the routine is keeping pace, and throughput without a sampled error rate hides whether the pace is actually correct. The five categories, volume, throughput, accuracy, routine health and permissions, only tell the truth read together.
How often should the agent's error rate be sampled?
On a fixed schedule agreed in advance, not only after something looks wrong. A sample taken because a mistake was already spotted only ever confirms what was already known. A regular, fixed-size sample is what actually shows whether the rate is improving or holding steady.
What should trigger a review of the permission model, if not a calendar date?
A sampled error rate moving for more than a few consecutive checks, a triage category behaving in a way the original definitions did not predict, or a weekly routine that keeps getting shortened rather than run in full. Any of the three is a better trigger than a date on a calendar chosen in advance.
Is a high "closed without shipping" number a bad sign?
Not necessarily. It can mean triage is correctly filtering duplicates and already-answered requests, which is exactly what it should be doing. It becomes a bad sign only when the same sampling used to check accuracy shows those closures were not actually correct.
What is the difference between tracking volume and tracking throughput?
Volume counts what arrives. Throughput tracks what happens to it afterwards: shipped, closed without shipping, or still open. A queue can have falling volume and still have a throughput problem, and the reverse, so neither figure substitutes for the other.