Inter-model homogeneity in open-ended generation
Parent: Why LLMs converge on the same words: mode collapse and post-training lexical effects · Published reference · snapshot 2026-09-23
↓ Facts as markdownall context files
Source-anchored research for Inter-model homogeneity in open-ended generation. Run status: COMPLETED.
These notes link each claim to its source. Some citations identify archived research reports that are no longer hosted on this site. A published reference means the content is available; it does not certify independent review or accuracy.Read the editorial policy and follow the sources before relying on a claim.
Definitions
- Homogeneity is a distributional property, not a lack of individual quality: in the PNAS Nexus study LLMs matched or slightly beat humans on individual originality for two of three tasks, while population-level diversity was much lower. The Hivemind paper likewise finds comparable overall quality but poor calibration of LMs, reward models and LM judges on generations that draw differing idiosyncratic human preferences. (medium confidence) [source]
Problems, failure modes and limitations
- Scope limits on the homogeneity findings: the PNAS Nexus study covers only publicly hosted, aligned LLMs and three divergent-thinking tasks. The longitudinal study cannot separate distillation, shared training data or shared techniques, samples one response per model per prompt, and notes some Infinity-Chat prompts themselves request similar outputs. (medium confidence) [source]
Comparisons and alternatives
- In a PNAS Nexus study of 22 LLMs and 102 humans, LLM response variability was lower than human variability on all three divergent-thinking tasks: Alternative Uses 0.459 vs 0.699, Forward Flow 0.518 vs 0.642, Divergent Association 0.632 vs 0.791 (p<0.01). LLM responses mirror other LLM responses more than humans mirror other humans. (medium confidence) [source]
Facts and statements
- The Artificial Hivemind paper defines two effects: intra-model repetition (one model repeats similar outputs) and inter-model homogeneity (different models independently converge on similar ideas with minor phrasing variations). It uses Infinity-Chat, 26K real-world open-ended queries with 31,250 human annotations (25 annotators per example). (high confidence) [source]
- A longitudinal study of 68 models from 12 providers (March 2023 to July 2026) reports cross-family semantic distance falling over time: on the Alternate Uses Task from about 0.50 to below 0.40, and on Infinity-Chat100 from about 0.34 to just above 0.32. All 1,000 resampling iterations gave negative slopes. (medium confidence) (verified as of 2026-09-23) [source]
- The Hivemind paper measures similarity with cosine similarity of OpenAI text-embedding-3-small embeddings; secondary summaries note this is sensitive to surface phrasing and could overstate homogeneity, though verbatim-overlap examples suggest it is not a pure artifact. (low confidence) [source]
Children
- No children recorded.