Human baselines and control words for tic detection
Parent: Measuring LLM lexical tics: excess vocabulary, baselines, drift · Published reference · snapshot 2026-09-23
↓ Facts as markdownall context files
Source-anchored research for Human baselines and control words for tic detection. Run status: COMPLETED.
These notes link each claim to its source. Some citations identify archived research reports that are no longer hosted on this site. A published reference means the content is available; it does not certify independent review or accuracy.Read the editorial policy and follow the sources before relying on a claim.
How it works
- Kobak et al. build a counterfactual baseline by linear extrapolation from earlier years (q = p_{Y-2} + 2*max{p_{Y-2}-p_{Y-3},0}) and deliberately exclude 2023 because it may already be LLM-affected. (medium confidence) [source]
Measurements and reference values
- Control by word class: authors hand-annotated 900 excess words from 2013-2024 as content (about 51%) or style (about 45%); Covid-era excess was mostly content nouns, 2024 excess mostly style verbs and adjectives, which separates topical drift from LLM tics. (low confidence) [source]
Problems, failure modes and limitations
- Excess-word counts are only a lower bound on LLM use, and cannot distinguish LLM use from authors independently adopting LLM-preferred vocabulary. (low confidence) [source]
Facts and statements
- Excess words are flagged using two metrics, frequency gap and frequency ratio; thresholds were chosen so most pre-Covid years fall well below them, giving conservative estimates. (low confidence) [source]
- Human baselines for model-output tic detection can be built from pre-2022 corpora (wordfreq for single words; Reddit creative writing and Project Gutenberg for n-grams in Antislop), with overuse expressed as a ratio of model frequency to baseline frequency. (medium confidence) [source]
- Post-2021 web text is contaminated by generative-AI output, so frequency resources built from it no longer measure human language; wordfreq's maintainer stopped updating it for this reason, forcing baselines to pre-2022 archives. (medium confidence) (verified as of 2024-09-15) [source]
- Confound check: the hypothesis that 'delve' reflects Nigerian English annotators was not supported by International Corpus of English data (small corpus); RLHF-based Llama experiments fit better as the source of overrepresentation. (medium confidence) [source]
Children
- No children recorded.