Negative-instruction backfire in style prompts
Parent: Prompt-side control of LLM lexical tics · Published reference · snapshot 2026-09-23
↓ Facts as markdownall context files
Source-anchored research for Negative-instruction backfire in style prompts. Run status: COMPLETED.
These notes link each claim to its source. Some citations identify archived research reports that are no longer hosted on this site. A published reference means the content is available; it does not certify independent review or accuracy.Read the editorial policy and follow the sources before relying on a claim.
Structure and components
- In a 40,000-sample study of forbidden-word constraints, 87.5% of violations were 'priming failures' (naming the word activated it) and 12.5% were 'override failures' from late-layer feed-forward networks; violation probability follows a logistic curve in the model's intrinsic probability of the forbidden token. (low confidence) [source]
How it works
- Mentioning a topic in the prompt makes the model more likely to mention it even when told to avoid it (the Pink Elephant problem); prompting alone was outperformed by a Llama-2-13B model fine-tuned with Direct Principle Feedback, which matched GPT-4 on the authors' test set. (medium confidence) [source]
Problems, failure modes and limitations
- Listing banned words in a style prompt is an antipattern: the list itself places the tokens in context and primes them; pair any unavoidable prohibition with a named replacement. (medium confidence) [source]
Comparisons and alternatives
- Circuit analysis suggests sparse middle-layer attention heads amplify forbidden tokens while early layers suppress them, so ironic rebound reflects competing internal representations rather than simple suppression. (low confidence) [source]
- Anthropic's prompting docs advise telling Claude what to do instead of what not to do (e.g. replace 'Do not use markdown' with a positive description of the wanted prose format). (medium confidence) (verified as of 2026-09-23) [source]
- The backfire is scoped, not universal: a small code-review test found prohibitions matched the positive reframe on discrete actions and underperformed only on judgment calls (2/3 false positives with prohibition vs 0/3 with reframe); earlier inverse-scaling evidence came from classification tasks on older models. (low confidence) [source]
Facts and statements
- Negation instructions such as 'do not mention X' can paradoxically increase the accessibility of X in LLMs (ironic rebound); rebound appears immediately after negation and worsens with longer or semantic distractor text, while repetition supports suppression. (medium confidence) [source]
Children
- No children recorded.