Why I made an AI say "I don't know" more often

I made an AI say "I don't know" more often. On purpose. And the product got better.

Most people get this exactly backwards, so stay with me.

Two ways to handle a missing fact

The receipt reader I built pulls facts off a receipt. Vendor, date, tax, total. Sometimes a receipt simply has no tax line on it at all.

The AI can handle that two ways. It can say "no tax printed." Honest and correct. Or it can invent a tax number that was never there. That's a hallucination, and the invented number looks every bit as trustworthy as a real one.

Think of two doctors. One says "I'm not certain, let's run the test." The other gives you a confident diagnosis he's quietly guessing at. The second one is far more dangerous, and he sounds far more sure.

A blank you can catch. A confident lie you can't.

So I made a call that felt odd to say out loud. An AI admitting it doesn't know counts as a correct answer. An AI making something up counts as the worst failure we track, kept in its own separate column, away from every other kind of mistake.

Let me repeat that, because it's the heart of it: a blank you can catch. A confident lie you can't.

That decision shows up in how we grade. Every extraction lands in one of four buckets, not two:

• Correct — it matches the truth, including when both agree nothing was there.
• Miss — it left a blank where there should have been a value. Honest, but incomplete.
• Hallucination — it invented a value that was never on the receipt. The dangerous one.
• Wrong — it gave an answer that simply disagrees with the truth.

Most teams mash all of that into a single "accuracy" number. That's exactly how a hallucination problem hides inside a healthy-looking score. Ninety-five percent accurate sounds fine until you learn that the missing five percent is the model confidently making things up.

The bucket isn't just for the record, either. A hallucination doesn't quietly average itself away like a normal error would. It gets pulled out and put in front of a person before anything downstream trusts it. A wrong-but-honest miss can wait for the next review cycle. An invention can't.

Where responsible AI actually lives

Responsible AI isn't a slide in a governance deck. It's a hundred small decisions like this one, made at the level of a single field on a single screen. Which mistakes you're willing to tolerate, and which you refuse to let hide.

So let me ask you. Would your AI rather guess, or admit it's stuck? You've already answered that, whether you meant to or not. The only real question is whether you answered on purpose.