Prevent Hallucination with Reflection
Imagination is more important than knowledge
Albert Einstein
Everyone knows that LLMs are prone to hallucination. This is especially true when you use them for research, as they can get confused between whether you want them to retrieve existing facts or generate new content.
Here are some prompting techniques that can help guide an LLM towards retrieval from its own internal knowledge, and away from generating new content.
Naive Prompting
As an example, let’s say you’re trying to find good quotes from real people to include in an article. Naively, you might prompt a model like Claude Sonnet 3.5 as follows:
Give me 3 good quotes related to AI hallucination
response:
1. "The greatest danger of artificial intelligence isn't that computers will begin to think like humans, but that humans will begin to trust computers that don't think at all." - Jaron Lanier
2. "In AI systems, hallucination is a feature, not a bug. The challenge lies in controlling when and how it happens." - Gary Marcus
3. "The most dangerous form of AI hallucination isn't when the system knows it's uncertain, but when it's confidently wrong." - Emily M. Bender
As these relate to contemporary AI discussions, I should note that I may not be quoting these precisely and you should verify them independently.
These are all great quotes, but unfortunately none of them are real. The model identified good people to quote from, but then invented some plausible, relevant quotes from those people. Importantly, the model did note that this is a scenario where it might hallucinate, which is our first clue towards improving the prompt: the model itself is aware of its tendency to hallucinate.
Historical Anchoring
Here’s my first trick: LLMs are much more likely to correctly recall facts from history than facts about the present. So if we limit ourselves to historical text (e.g. text in the public domain), we’re much more likely to get real results:
prompt:
Give me 3 good quotes from real, historical people, related to AI hallucination.
response:
1. "It is better to be roughly right than precisely wrong." - Alan Turing (1912-1954)
2. "A computer would deserve to be called intelligent if it could deceive a human into believing that it was human." - Alan Turing
3. "A model of the world that has never been wrong is probably less valuable than one that sometimes shows us our mistakes." - Norbert Wiener (1894-1964)
Note: While I aim to provide accurate quotes, you should verify these independently as I may sometimes hallucinate specific quotes or attributions.
The first quote is real, but attributed to the wrong source. The second quote is real, and the third is made up. This is a lot better!
Self-reflection and filtering
My second trick is to leverage the fact that the model usually knows when it’s making something up.
We can ask sonnet 3.5 to generate a list of quotes, then ask it to rank each quote by hallucination likelihood, and then finally ask it to filter out the hallucinated quotes.
Note that this prompt has multiple steps. LLMs can be a little “lazy” or “eager” and will often try to accomplish multiple steps all at once, but we want to avoid that here. Therefore, we ask for a specific structure in the response. XML tags or markdown are a great way to do this. XML is generally recommended, but markdown is easier to read so I will use it here:
prompt:
Please help me research. Respond in markdown.
Topic: AI hallucination.
1. Find 25 quotes from real, historical people. Put these quotes as a numbered list in a "Generation" header.
2. Rank each quote on its likelihood to be hallucinated, on a scale of 0 to 100, and put the results as a numbered list in a "Hallucination Score" header.
3. Return a numbered list of all the quotes that have a hallucination score of less than 5, ordered by their relevance to my topic. Put this list in a "Filtered Quotes" header.
response:
# Generation
1. "What the mind doesn't understand, it speaks in metaphors" - Nassim Nicholas Taleb
2. "Reality is that which, when you stop believing in it, doesn't go away" - Philip K. Dick
3. "The map is not the territory" - Alfred Korzybski
4. "The greatest enemy of knowledge is not ignorance, it is the illusion of knowledge" - Daniel J. Boorstin
5. "We don't see things as they are, we see them as we are" - Anaïs Nin
6. "The eye sees only what the mind is prepared to comprehend" - Robertson Davies
7. "Imagination is more important than knowledge" - Albert Einstein
8. "All perception is a gamble" - Edmund Burke
9. "The universe is not only queerer than we suppose, but queerer than we can suppose" - J.B.S. Haldane
10. "The first principle is that you must not fool yourself" - Richard Feynman
11. "In nature we never see anything isolated" - Johann Wolfgang von Goethe
12. "Truth is much too complicated to allow anything but approximations" - John von Neumann
13. "The more I see, the less I know for sure" - John Lennon
14. "The function of education is to teach one to think intensively and to think critically" - Martin Luther King Jr.
15. "Not everything that looks like a pattern is a pattern" - Daniel Kahneman
16. "The truth is rarely pure and never simple" - Oscar Wilde
17. "What we observe is not nature itself, but nature exposed to our method of questioning" - Werner Heisenberg
18. "Facts do not cease to exist because they are ignored" - Aldous Huxley
19. "Knowledge is illusion; wonder is the beginning of wisdom" - Mary Catherine Bateson
20. "The real voyage of discovery consists not in seeking new landscapes, but in having new eyes" - Marcel Proust
21. "The limits of my language mean the limits of my world" - Ludwig Wittgenstein
22. "The greatest obstacle to discovery is not ignorance - it is the illusion of knowledge" - Stephen Hawking
23. "The more precisely the position is determined, the less precisely the momentum is known" - Werner Heisenberg
24. "All our knowledge begins with the senses" - Immanuel Kant
25. "Science is the belief in the ignorance of experts" - Richard Feynman
# Hallucination Score
1. 15
2. 10
3. 5
4. 20
5. 25
6. 30
7. 2
8. 40
9. 15
10. 2
11. 35
12. 20
13. 25
14. 3
15. 45
16. 5
17. 10
18. 15
19. 50
20. 20
21. 3
22. 15
23. 5
24. 10
25. 4
# Filtered Quotes
1. "Imagination is more important than knowledge" - Albert Einstein
2. "The first principle is that you must not fool yourself" - Richard Feynman
3. "The function of education is to teach one to think intensively and to think critically" - Martin Luther King Jr.
4. "The limits of my language mean the limits of my world" - Ludwig Wittgenstein
5. "Science is the belief in the ignorance of experts" - Richard Feynman
The model did hallucinate many quotes, but it also recalled some real ones. After generating the quotes, it was able reflect on its work and identify the hallucinated quotes and remove them. This prompting technique is called “reflection.”
All 5 of the filtered quotes are real, and the top quote has a vibe that I like for this post. I used it at the top of the text.