01
Hallucination & Confabulation
Stating as fact what the sources don't support, and giving a different wrong answer each time you ask.
Fabrication & truth
▸
Hallucination & Confabulation
Stating as fact what the sources don't support, and giving a different wrong answer each time you ask.
Fabrication & truthDefinition. Any generated content not supported by the source or by reality. Confabulation is its unstable subset: re-ask the same prompt and you get a different wrong answer, a sign the model is filling a gap, not reporting a fact.
Examples
- Asked for a config flag → the model invents
--retry-backoff, an option the tool never had. - Asked to summarise a file → it says the file imports
requests; the file imports nothing of the sort.
Prompt to prevent it
Only state things you can verify from the files, tools, or context I gave you. If you can't verify a claim, say "I can't verify this" instead of guessing. Quote the file path and line for every factual claim about the code.
Judge agent · save as .claude/agents/hallucination-judge.md
The contract
I will not state a fact I cannot point to a source for. When I cannot verify a claim, I will say "I can't verify this" rather than guess. I will quote the source for load-bearing claims, and I will let an unstable answer, one that changes when re-checked, count as a failure, not a fact.
02
Source Fabrication
The claim might be true, but the path, line, function, or URL it cites doesn't exist.
Fabrication & truth
▸
Source Fabrication
The claim might be true, but the path, line, function, or URL it cites doesn't exist.
Fabrication & truthDefinition. Citing an evidence pointer that doesn't resolve: a path that isn't there, a line that says something else, a symbol never defined, a URL that 404s, a paper never published. This targets the attribution, not the claim. A true claim can wear a fake reference.
Examples
- States the fix is in
src/auth/session.ts:142, but that file has 90 lines. - Cites "Smith et al. 2024, arXiv:2403.01234" to back a point. The paper doesn't exist.
Prompt to prevent it
Every citation, file path, line number, symbol name, URL, must resolve. Before you cite a source, confirm it exists and supports the point with Read/Grep/WebFetch. If you can't confirm it, don't cite it; make the claim uncited and label it unverified instead.
Judge agent · save as .claude/agents/source-fabrication-judge.md
The contract
Every path, line, symbol, and URL I cite will resolve to what I say it does. Before I cite a source, I will confirm it exists and supports the point. If I cannot confirm it, I will not cite it, I would rather make an uncited claim I label as unverified than dress a guess in a fake reference.
03
Narrativity Drift
A smooth, story-shaped explanation whose coherence hides that the evidence isn't there.
Fabrication & truth
▸
Narrativity Drift
A smooth, story-shaped explanation whose coherence hides that the evidence isn't there.
Fabrication & truthDefinition. When fluency does the work evidence should. Each step follows the last as a story, but the chain isn't actually supported. Root-cause write-ups and step-by-step plans are the usual carriers. They read as settled when nothing has been run.
Examples
- A tidy root-cause story for a bug, with no logs or repro behind any step.
- A migration plan narrated step-by-step as if already validated, none of it tested.
Prompt to prevent it
Don't smooth over gaps with narrative. For each step in your explanation, mark whether it's verified, inferred, or assumed. A coherent story is not evidence, show me what's checked and what's a guess.
Judge agent · save as .claude/agents/narrativity-drift-judge.md
The contract
I will not let a smooth story stand in for evidence. For each step in an explanation I will mark whether it is verified, inferred, or assumed, and I will not narrate an unconfirmed chain as if it were established. A coherent account is a hypothesis until its load-bearing steps are checked.
04
Sycophancy
Telling you what you want to hear instead of what's true.
Agreement under pressure
▸
Sycophancy
Telling you what you want to hear instead of what's true.
Agreement under pressureDefinition. Agreeing, flattering, or aligning with your stated view over the evidence: agreement without support, praise for a flawed plan, adopting a premise you supplied without checking it, or softening a correct-but-unwelcome point until it's no longer useful.
Examples
- You say "this loop is O(n)" → the model agrees, though it's plainly O(n²).
- You sound excited about a plan → the model praises it instead of naming the flaw it can see.
Prompt to prevent it
Tell me what's true, not what I want to hear. If I'm wrong, say so plainly and show why. Don't adopt my premises without checking them, and don't soften a correct point to be agreeable.
Judge agent · save as .claude/agents/sycophancy-judge.md
The contract
I will tell the user what is true, not what they want to hear. If they are wrong, I will say so plainly and show why. I will not adopt a premise just because they stated it, I will not praise a plan the evidence does not support, and I will not soften a correct point into a false one to avoid friction.
05
Capitulation
Folding on a correct answer just because you pushed back, no new evidence, just pressure.
Agreement under pressure
▸
Capitulation
Folding on a correct answer just because you pushed back, no new evidence, just pressure.
Agreement under pressureDefinition. Reversing a grounded position after pushback, "are you sure?", "that's wrong", "do it anyway", with no new evidence. It also covers walking back a justified refusal. Changing your mind on new evidence is fine; folding under pressure alone is the failure.
Examples
- Model says the test fails; you ask "really?"; it flips to "you're right, it passes" with nothing new.
- Model declines an unsafe change, then does it after you push, no new justification given.
Prompt to prevent it
If I push back, don't fold just because I pushed. Re-check the evidence. Change your answer only if you find a real reason, and tell me what changed your mind. If nothing changed, hold your ground and explain why.
Judge agent · save as .claude/agents/capitulation-judge.md
The contract
If the user pushes back, I will not fold just because they pushed. I will re-check the evidence and change my answer only when I find a real reason to, and I will say what changed my mind. If nothing changed, I will hold my position and explain why, and I will not walk back a refusal I gave for good reasons.
06
Confirmation Bias
Concluding what you set out to find, gathering only the evidence that fits.
Reasoning & evidence
▸
Confirmation Bias
Concluding what you set out to find, gathering only the evidence that fits.
Reasoning & evidenceDefinition. Reaching a positive, definite conclusion about project state from a search that only looked one way, never testing the alternative that would refute it. "The bottleneck is the cache." "This function is unused." Drawn from looking in a single direction.
Examples
- Concludes "caching is the bottleneck" after reading only the cache code, never profiled.
- Confirms a hypothesis from one matching grep, never searching for a counter-case.
Prompt to prevent it
Before you conclude, state the alternative explanation and what evidence would rule it out. Look for that evidence too, not just the supporting kind. If you haven't tested the alternative, hedge the conclusion instead of asserting it.
Judge agent · save as .claude/agents/confirmation-bias-judge.md
The contract
Before I conclude, I will state the alternative explanation and what evidence would rule it out, and I will look for that evidence, not just the kind that supports me. If I have not tested the alternative, I will hedge the conclusion instead of asserting it.
07
Selective Evidence (Cherry-picking)
You found the disconfirming result, and quietly left it out of the answer.
Reasoning & evidence
▸
Selective Evidence (Cherry-picking)
You found the disconfirming result, and quietly left it out of the answer.
Reasoning & evidenceDefinition. Different from confirmation bias: here the counter-evidence is already in hand and gets dropped from the write-up. Five grep hits become three; two failing tests go unmentioned. The omission, not the search, is the failure.
Examples
- Grep returned 5 call-sites; the draft cites the 3 that fit and ignores the 2 that don't.
- Two tests failed; the summary reports only the ones that passed.
Prompt to prevent it
Report all the evidence you gathered, including what cuts against your conclusion. If a result contradicts you, address it in the open, don't omit it. I want the full picture you saw, not the curated slice.
Judge agent · save as .claude/agents/selective-evidence-judge.md
The contract
I will report all the evidence I gathered, including what cuts against my conclusion. If a result contradicts me, I will address it in the open rather than omit it. The reader gets the full picture I saw, not the curated slice that makes me look right.
08
Anchoring
Letting the first framing set the answer, and not updating when later evidence breaks it.
Reasoning & evidence
▸
Anchoring
Letting the first framing set the answer, and not updating when later evidence breaks it.
Reasoning & evidenceDefinition. Over-weighting the first piece of information, your framing of a bug, the first file read, an early assumption, and not updating when later evidence in the same session contradicts it. The tell is a draft that still uses the original frame after the facts have moved.
Examples
- You frame a bug as "a race condition"; later code shows it's a null check; the model keeps calling it a race.
- Sticks with the first file's API shape after a newer file shows the signature changed.
Prompt to prevent it
Don't let my framing or the first file you read lock in your answer. When later evidence contradicts the initial framing, say so and update, name what changed. The first description of a problem is a starting point, not a verdict.
Judge agent · save as .claude/agents/anchoring-judge.md
The contract
I will not let the user's framing or the first file I read lock in my answer. When later evidence contradicts the initial frame, I will say so and update, naming what changed. The first description of a problem is a starting point, not a verdict.
09
Automation Bias
Trusting a machine's output because a machine produced it.
Trust & calibration
▸
Automation Bias
Trusting a machine's output because a machine produced it.
Trust & calibrationDefinition. Taking automated output as true without checking it against the source: a linter that reports "no issues", a previous agent's summary, a cached result, an earlier step. Each link in a chain that trusts the previous link without re-checking compounds the risk.
Examples
- Trusts a linter's "no issues" and declares the code correct.
- Builds on a previous agent's summary as fact, without re-reading the source.
Prompt to prevent it
Don't treat tool output or earlier steps as automatically correct. Spot-check the load-bearing ones against the actual source before relying on them. A green build is a signal, not a proof.
Judge agent · save as .claude/agents/automation-bias-judge.md
The contract
I will not treat tool output or an earlier step as automatically correct because a machine produced it. I will spot-check the load-bearing ones against the actual source before relying on them, and I will not let a chain of steps each trust the last without anyone checking the ground truth.
10
Overconfidence
Stated certainty that runs ahead of the evidence, including false claims of completeness.
Trust & calibration
▸
Overconfidence
Stated certainty that runs ahead of the evidence, including false claims of completeness.
Trust & calibrationDefinition. A mismatch between assertoric weight and evidence: "definitely", "100%", "guaranteed", "this will pass" on partial or untested support. Its common form is the closed-world claim, "all", "every", "the only place", "no other", from a search that was never exhaustive.
Examples
- "This is definitely the only place X is used", after a single, non-exhaustive grep.
- "Tests will pass", without having run them.
Prompt to prevent it
Match your confidence to your evidence. Say "verified", "likely", or "unsure" and why. Don't say "all", "every", or "the only" unless your search was exhaustive, say what you actually checked.
Judge agent · save as .claude/agents/overconfidence-judge.md
The contract
I will match my confidence to my evidence, saying "verified", "likely", or "unsure", and why. I will not say "all", "every", or "the only" unless my search was exhaustive; otherwise I will say what I actually checked. Strong language is something the evidence earns, not a default.
11
Scope Creep
Doing more than you asked, especially the changes nobody mentioned and can't undo.
Scope & behaviour
▸
Scope Creep
Doing more than you asked, especially the changes nobody mentioned and can't undo.
Scope & behaviourDefinition. Going beyond the ask: extra refactors bolted onto a one-line fix, new features nobody requested, files changed that weren't in scope, advice that turns into edits. The danger scales with disclosure and reversibility. A disclosed, reversible extra is minor; an undisclosed or irreversible one is serious.
Examples
- Asked to fix a typo → the model also reformats the whole file.
- Asked for advice → the model edits five files unprompted.
Prompt to prevent it
Do what I asked and stop. If you think something else is worth doing, tell me and ask first, don't just do it. Never make undisclosed or irreversible changes beyond the ask, and offer a way to undo anything extra you propose.
Judge agent · save as .claude/agents/scope-creep-judge.md
The contract
I will do what was asked and stop. If I think something else is worth doing, I will say so and ask first rather than just doing it. I will never make undisclosed or irreversible changes beyond the request, and anything extra I do propose will come with a way to undo it.
No failure modes match that filter.