# Marginal evidence gain research stopping > Source-anchored research for Marginal evidence gain research stopping. Run status: BUDGET_EXHAUSTED. Parent: [Concept knowledge architecture strategies](https://llms-explorer.com/tree/concept-knowledge-architecture-strategies/) · 3 facets · 19 facts · page: https://llms-explorer.com/tree/marginal-evidence-gain-research-stopping/ ## How it works - During screening, true recall remains unknown until all candidate documents have been assessed because the total relevant-document count is unknown. (low confidence) — [source](https://link.springer.com/article/10.1186/s13643-020-01521-4) ## Comparisons and alternatives - SAFE explicitly presents its numerical percentages and record counts as adaptable examples rather than universally applicable rules. (low confidence) — [source](https://link.springer.com/article/10.1186/s13643-024-02502-7) ## Facts and statements - Callaghan and Müller-Hansen formulate stopping as rejecting a null hypothesis that achieved recall is below a chosen target. (low confidence) — [source](https://link.springer.com/article/10.1186/s13643-020-01521-4) - Their random-sampling criterion uses a hypergeometric model for relevant records drawn without replacement from the finite unscreened pool. (low confidence) — [source](https://link.springer.com/article/10.1186/s13643-020-01521-4) - SAFE combines random training-set screening, active learning, a different screening model, and a final quality-check phase. (low confidence) — [source](https://link.springer.com/article/10.1186/s13643-024-02502-7) - Callaghan and Müller-Hansen report inconsistent achieved recall and work reductions across datasets for previously proposed stopping methods. (low confidence) — [source](https://link.springer.com/article/10.1186/s13643-020-01521-4) - The 2024 stopping-criteria commentary warns that observed reliability can depend on datasets, ranking algorithms, hyperparameters, and implementation details. (low confidence) — [source](https://link.springer.com/article/10.1186/s13643-024-02699-7) - SAFE warns that using expert-selected key papers as training seeds can bias the screening model. (low confidence) — [source](https://link.springer.com/article/10.1186/s13643-024-02502-7) - The SAFE paper states that it does not present a direct comparison with existing stopping heuristics. (low confidence) — [source](https://link.springer.com/article/10.1186/s13643-024-02502-7) - Guest, Namey, and Chen calculate new information over consecutive observation runs relative to unique themes in a specified initial base. (low confidence) — [source](https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0232076) - Guest, Namey, and Chen explicitly state that their new-information thresholds lack a probability-theory guarantee of saturation. (low confidence) — [source](https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0232076) - Their saturation method was tested on single-tier codebooks; hierarchical applications require choosing which coding level contributes to the calculation. (low confidence) — [source](https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0232076) - Hennink, Kaiser, and Marconi distinguish code saturation, where new issues stop appearing, from meaning saturation, where new dimensions of issues stop appearing. (low confidence) — [source](https://pmc.ncbi.nlm.nih.gov/articles/PMC9359070/) - In the Hennink, Kaiser, and Marconi study, some conceptual codes reached meaning saturation between interviews 16 and 24, while the conceptual code responsibility for health never reached meaning saturation. (low confidence) — [source](https://pmc.ncbi.nlm.nih.gov/articles/PMC9359070/) - In that study, higher code prevalence did not indicate earlier meaning saturation. (low confidence) — [source](https://pmc.ncbi.nlm.nih.gov/articles/PMC9359070/) - Repke and colleagues evaluated stopping methods using 223 simulated screening runs over 81 real-world datasets. (low confidence) — [source](https://www.ebi.ac.uk/europepmc/webservices/rest/PMC12825451/fullTextXML) - In those experiments, CMH never stopped before the specified recall target, but often screened substantially beyond the optimal stopping point. (low confidence) — [source](https://www.ebi.ac.uk/europepmc/webservices/rest/PMC12825451/fullTextXML) - Repke and colleagues evaluate both missed recall targets and additional screening after the target's theoretically optimal stopping point. (low confidence) — [source](https://www.ebi.ac.uk/europepmc/webservices/rest/PMC12825451/fullTextXML) - Repke and colleagues disclose that evaluation co-author Finn Müller-Hansen also co-authored CMH. (low confidence) — [source](https://www.ebi.ac.uk/europepmc/webservices/rest/PMC12825451/fullTextXML)