# Excess-vocabulary frequency method > Source-anchored research for Excess-vocabulary frequency method. Run status: COMPLETED. Parent: [Measuring LLM lexical tics: excess vocabulary, baselines, drift](https://llms-explorer.com/tree/ai-isms-tic-measurement/) · 4 facets · 7 facts · page: https://llms-explorer.com/tree/excess-vocabulary-frequency-method/ ## Measurements and reference values - Headline estimate: lower bound of at least 13.5% of 2024 PubMed abstracts processed with LLMs, ranging roughly 5-40% across fields and countries. Earlier preprint versions reported 10%; the figure is a lower bound and moved between versions. (low confidence) (verified as of 2025-07-02) — [source](https://arxiv.org/html/2406.07016v3) ## Problems, failure modes and limitations - Frequency-based measurement is valid at population level only; it cannot label individual texts, and results from PubMed/arXiv abstracts may not generalise to other genres. (medium confidence) — [source](https://arxiv.org/html/2502.09606v2) ## Comparisons and alternatives - Excess words in 2024 are predominantly style words (about 66% verbs, 14% adjectives, e.g. 'meticulously', 'intricate', 'pivotal'), whereas COVID-era excess was mostly content nouns; this style-vs-content split is the diagnostic that separates LLM shift from topic shift. (low confidence) — [source](https://arxiv.org/html/2406.07016v3) - Model-side comparison of Llama Base vs Instruct on continuations of 9,853 PubMed abstracts finds large post-alignment increases (e.g. 'nuanced' +8,342%), and 400 raters preferred variants with more such words (52.4% vs 47.6%), linking part of the overuse to RLHF; scope is Llama only, English abstracts, with some false-positive words. (low confidence) — [source](https://arxiv.org/html/2508.01930v1) ## Facts and statements - The method projects each word's expected 2024 frequency by linear extrapolation from 2021-2022 (2023 excluded as possibly already LLM-influenced), then flags words whose observed frequency exceeds the projection; it needs no labelled LLM text. (high confidence) — [source](https://arxiv.org/html/2406.07016v3) - Two complementary metrics are used: frequency ratio r (observed/expected, surfaces rare words) and frequency gap delta (observed minus expected, surfaces common words); thresholds such as delta > 0.01 or a log-scaled ratio criterion were chosen so pre-pandemic years show minimal excess. (low confidence) — [source](https://arxiv.org/html/2406.07016v3) - Once a word list is publicised, frequencies change: 'delve' and 'intricate' dropped sharply on arXiv from March-April 2024, while less suspicious words like 'significant' and 'additionally' kept rising. Excess counts of famous tic words therefore understate later LLM use. (low confidence) (verified as of 2025-02-13) — [source](https://arxiv.org/html/2502.09606v2)