Why AI chatbots make things up and how to check their answers
TUREFF• •0
Ask an AI chatbot a question and you get a confident, well-written answer in seconds. Most of the time it is useful. Sometimes it is simply wrong, and it reads exactly as convincingly as when it is right. These invented answers are usually called hallucinations, and understanding why they happen is the best way to protect yourself from them.
Why it happens
A chatbot is built on a language model trained to predict likely text. It learns patterns from enormous amounts of writing, which is why it can explain, summarise and draft so fluently. But producing text that sounds right is not the same as looking up a fact that is right.
- It fills gaps with plausible guesses. If it does not have reliable information, it may produce something that fits the pattern of an answer, such as a book title that sounds real or a statistic in the right range.
- It rarely signals uncertainty well. The tone is usually the same whether the content is solid or invented.
- Its knowledge has a cut-off. Unless it searches the web, it may not know about recent events, new rules or changed prices.
- It tries to be helpful. If you ask a leading question, such as what year did a law change, it may supply a year even if the law never changed.
- Search does not remove the problem. When a chatbot reads web pages, it can still misread them, mix up sources or cite a page that does not say what it claims.
Where errors are most common
Be especially careful with:
- Specific numbers: dates, prices, percentages, statistics, phone numbers.
- Citations and quotations: references to papers, books, court cases or articles that may not exist, or quotes the person never said.
- Niche or local topics with little written about them, including many Korea-specific procedures described in English.
- Recent events and anything that changes yearly, such as tax thresholds or visa rules.
- Legal, medical and financial details, where a small error matters.
- Maths and counting in longer problems, though this varies a lot between tools.
How to check an answer
- Ask for sources, then open them. A link or reference is only useful if you confirm it exists and says what the chatbot claims.
- Go to the primary source for anything important: the government agency, the official rules, the company's own help page, the original study.
- Cross-check with a second, independent source. Another chatbot is not independent if both may be repeating the same error.
- Ask the chatbot to show its working for calculations and check each step yourself.
- Ask it what it is unsure about. It will not always know, but the question sometimes surfaces weak points.
- Rephrase the question neutrally. Instead of when did the rule change, ask has this rule changed, and if so when.
- Test it on something you know. Ask about your own field to get a feel for how reliable it is.
Better ways to use it
Chatbots are strongest when you do not need them to be the final authority.
- Explaining a concept you can then check in a textbook.
- Drafting and rewriting, where you judge the result.
- Brainstorming options, questions or angles you had not considered.
- Summarising a document you provide, then spot-checking the summary against the original.
- Turning your own notes into structure.
Use it as a fast, well-read assistant who sometimes bluffs, not as an encyclopaedia.
A quick checklist before you rely on an answer
- Would it cause harm or cost money if this were wrong?
- Have I seen the claim in a primary source myself?
- Are names, numbers and dates confirmed, not just plausible?
- Is the information current?
If the answer to the first question is yes, the rest are not optional.
What is the strangest thing a chatbot has confidently told you? Share it in the comments.