Spotting and avoiding AI mistakes
AI makes mistakes in predictable patterns. Once you know the failure modes, you can catch most errors in under a minute — before they embarrass you or cause real problems.
The four failure modes to know
AI mistakes aren't random — they cluster into a small number of recognisable types. Knowing them lets you scan for the right things rather than reading every word with equal suspicion.
1. Hallucination. The model invents facts, statistics, citations, names, or events that don't exist. These are the most dangerous errors because they look exactly like real facts. The tell: specific details — a precise number, a named study, a direct quote — that you haven't seen elsewhere and can't immediately verify. These deserve the most scrutiny.
2. Stale information. AI models have a training cut-off date. Anything that changed after that date may be wrong or outdated. Prices, laws, statistics, software versions, company leadership, market conditions — all of these change. The tell: any claim about "current" state or recent events. Always check these against live sources.
3. Confident vagueness. The model produces fluent, confident-sounding text that says very little. It's grammatically flawless and sounds authoritative but doesn't commit to specifics. The tell: sentences that could describe almost anything — lots of "it is important to" and "one must consider" without actually stating what or how. This is not factually wrong; it's just useless.
4. Missing the real question. The model answers a close-but-wrong version of what you asked. You asked for pros and cons; it gave you only pros. You asked for plain English; it produced jargon. You asked for three options; it gave you one. The tell: compare what you asked against what you got. They don't always match.
The red-flag checklist
Before using AI output, scan for these specifically:
- Any specific number, statistic, or percentage
- Any named person, organisation, study, book, or publication
- Any date or timeline
- Any claim about what is "currently" true
- Any legal, medical, or financial advice
For anything on this list that matters — verify it independently. This doesn't mean verify everything; it means verify the things that would matter if wrong.
When not to use AI at all
Some tasks are unsuited to AI without heavy oversight:
- High-stakes factual claims you'll publish or present as true
- Personal advice for others (legal, medical, financial) — liability sits with you, not the model
- Anything requiring real-time information — stock prices, breaking news, live availability
- Nuanced interpersonal situations where knowing the specific person and history matters
None of these are absolute prohibitions — AI can help with research and drafting even in these areas. But the human making the final call needs to be fully in the loop, not just copy-pasting the output.
A useful mental model: treat AI output like a very smart intern's first draft. You wouldn't send an intern's draft to a client without reading it. The intern might be brilliant — but they don't have your context, your relationships, or your judgement. Neither does the model.
What to remember
- The four failure modes: hallucination, stale information, confident vagueness, missing the real question.
- Scrutinise specific details — numbers, names, citations, dates — more than general claims.
- Any "current" claim should be verified against a live source.
- High-stakes factual, legal, medical, and financial content always requires human review before use.