An agent answering customers will, unprompted, invent a feature, a date, a price or a policy, because a fluent wrong answer and a fluent right one look identical to the system producing them. Nothing here removes that risk. What follows are the mechanisms that reduce it, in order of how much they actually help, and what each one misses.
This matters most once an agent sits on top of a real setup, such as a self-hosted feature request board, where a wrong answer reaches a customer, not a test conversation.
This page is one part of a larger guide. For the whole subject in one place, see running support with a coding agent.
What actually goes wrong when an agent answers customers?
Not usually a factual error the agent could have looked up. The failure is invention: asked something the source does not cover, an agent tends to produce a plausible answer rather than say it does not know. Ask when a feature shipped and it may offer a date that sounds right, confident, specific and wrong, the worst combination for a reader with no way to tell. It has already happened in public: in April 2025 the support bot for the coding tool Cursor invented a subscription policy that did not exist, and customers cancelled over it.
Give it a single source of truth, and forbid anything outside it
This is the whole ballgame, and everything else on this page is a backstop for what it misses. Point the agent at one file, or one small set of files, holding every fact it may state: prices, features, policies, dates. If a fact is not in that file, the answer is "I do not know," never a guess dressed up as one. It works because it changes what the agent is doing: unbounded, it generates the most plausible continuation of the conversation, which is exactly what invention looks like; bounded, it looks something up. Keeping that source in the same repository as a support inbox stored the same way keeps the fact an agent reads and the record a human audits as one file.
Make refusal a success, not a failure
An agent that says it cannot answer, and hands the conversation to a person, has done its job correctly. Most invention happens because the system around it implicitly rewards producing an answer over admitting it lacks one, the way a support rep under pressure to close tickets starts guessing rather than escalating. Fixing this is about what you measure: if "handled without escalation" is the number anyone optimises, an agent will find a way to hit it. Measure instead whether a low-confidence question gets correctly routed to a person.
Why does review before send change the category of risk?
Because a draft a person reads before it goes out is a fundamentally different risk from a reply that posts on its own, not a smaller version of the same risk. A wrong answer a human catches costs nothing; one that reaches a customer costs trust, sometimes a refund. Review also catches what nothing else can: a confident, plausible, wrong answer is precisely the shape a good source of truth and a good refusal policy are built to prevent, but neither is airtight. If you can afford only one mechanism on this page, it is this one.
Constrain the shape of the answer
An open text box gives an agent room to improvise a sentence that sounds right. A fixed set of replies it selects from leaves far less room: if the only outputs are "yes, here is the documented answer," "no, that is not a feature we offer," or "I am passing this to a person," there is no fourth option to improvise. This will not fit a genuinely open question, but wherever the question is one of a known, finite set, a fixed shape removes the chance to improvise at all.
Ban the categories that cause damage
Some categories of fact do more damage per wrong answer than others, so the cheapest fix is a flat rule: the agent may never state a date, a price, a refund decision, a promise about future behaviour, or anything about a different customer's account, however confident it sounds. Those five cover most of the money and trust an agent can cost you when it errs, and the rule only works as a flat prohibition, never a suggestion to be careful.
Check afterwards, on a schedule
Sample real replies on a regular cadence and check them against the source of truth by hand: how many strayed, and in what direction. This is the only mechanism that tells you whether the other five are actually working, and it is the slowest to set up and easiest to skip, which is why it tends to be missing. Keep the sample small enough to actually read, and treat a rising rate of stray answers as a signal to tighten one of the mechanisms above.
What does each mechanism actually catch?
| Mechanism | What it catches | What it misses |
|---|---|---|
| Single source of truth | Facts the agent would otherwise invent from general knowledge | A fact that is wrong, missing, or stale inside the source itself |
| Refusal as success | Confident guessing under pressure to always produce an answer | An agent confidently wrong but still within its allowed source |
| Review before send | A plausible wrong answer that passed every earlier check | Reviewer fatigue, and any reply that goes out unreviewed |
| Constrain the shape | Free-text invention on questions with a known, finite set of answers | Genuinely open questions, which still need a written reply |
| Ban the categories | The specific facts causing the most damage per wrong answer | Every other category not on the banned list |
| Check afterwards | Whether the other five are holding up in practice | Anything that happened between checks and was never sampled |
What can this project itself show, honestly?
This site is written against a single file of verified facts: any figure a draft wants to use either comes from that file, or it does not appear at all, checked by a script before anything is treated as finished. It exists because an earlier research pass produced a fabricated quote, caught before publication. That is the shape worth copying: a bounded source, and something automated that enforces it.
One payment, no subscription, unlimited products.
How do you combine these without turning support into a bottleneck?
Layer them. The source of truth and the banned categories constrain what the agent can say at all, typically set once in an instructions file the agent reads before every reply. Refusal as success and constrained shapes change day-to-day behaviour. Review before send stays manual because it catches what nothing automated can: a confident answer wrong in a way nothing upstream flagged. Checking afterwards is the audit that tells you whether the first six are doing their job. None of this is a guarantee: a constrained agent can still answer wrongly, just more narrowly, and the honest goal is reducing how often invention happens, not eliminating it.
while these techniques significantly reduce hallucinations, they don't eliminate them entirely
Anthropic, Reduce hallucinations
Frequently asked questions
Isn't a single source of truth enough on its own?
It removes most of the risk, not all of it. An agent can misread the source, combine two true facts into a false conclusion, or answer confidently within the source when the honest answer is that the source does not cover this case.
Why treat refusal as a success instead of a failure to fix?
Because the alternative teaches the agent, and whoever measures it, that producing an answer matters more than producing a correct one. Once "handled without escalation" is the number people optimise, both will find ways to answer questions they should have routed to a person.
Does review before send not defeat the point of using an agent at all?
No, it changes what the agent does, not whether it is used. Drafting a reply from a bounded source is still real work an agent does well; sending it without a human glance is the part that changes the risk.
What is the difference between constraining the shape and banning a category?
Shape is the form an answer can take: fixed responses instead of an open paragraph. Category is the content: specific topics, like refund amounts or dates, off limits regardless of phrasing. A tightly shaped answer can still name a banned category if nobody separately ruled it out.
How often should you check replies against the source of truth?
Often enough that a new failure mode gets caught before it becomes a pattern, on a fixed schedule rather than only after something goes wrong. The cadence matters less than doing it at all, since an unchecked rule quietly stops being followed.