# Concept knowledge architecture strategies > Source-anchored research for Concept knowledge architecture strategies. Run status: BUDGET_EXHAUSTED. Parent: [Research & Knowledge Tooling](https://llms-explorer.com/tree/research-knowledge-tooling/) · 6 facets · 129 facts · page: https://llms-explorer.com/tree/concept-knowledge-architecture-strategies/ ## Structure and components - AIS evaluates whether a generated statement is attributable to identified parts of an underlying source corpus. (low confidence) — [source](https://arxiv.org/html/2112.12870v2) - AIS permits support from multiple source parts combined through reasoning such as arithmetic. (low confidence) — [source](https://arxiv.org/html/2112.12870v2) - TREC test collections contain a document corpus, information needs, and relevance judgments. (low confidence) — [source](https://trec.nist.gov/howto.html) ## How it works - Bitemporal history separates when a fact applies in the modeled world from when the system recorded it. (medium confidence) — [source](https://docs.xtdb.com/about/time-in-xtdb.html) - ALCE assigns statement-level citation recall only when the cited passages jointly entail the whole statement. (low confidence) — [source](https://aclanthology.org/2023.emnlp-main.398.pdf) - ALCE's automatic precision method can falsely penalize a citation that partially supports a statement when other citations also cover that information. (low confidence) — [source](https://aclanthology.org/2023.emnlp-main.398.pdf) - Claimify can decline extraction when context does not resolve competing interpretations with sufficient confidence. (low confidence) — [source](https://aclanthology.org/2025.acl-long.348.pdf) - The late-chunking paper finds no benefit when surrounding document text is irrelevant to the target information. (low confidence) — [source](https://arxiv.org/html/2409.04701v3) - Long late chunking uses overlapping long-context windows when the document exceeds the embedding model's context capacity. (low confidence) — [source](https://arxiv.org/html/2409.04701v3) - During screening, true recall remains unknown until all candidate documents have been assessed because the total relevant-document count is unknown. (low confidence) — [source](https://link.springer.com/article/10.1186/s13643-020-01521-4) - The query embedding model must match the intended stored embedding space when switching indexes. (medium confidence) — [source](https://www.mongodb.com/docs/voyageai/tutorials/migrate-embedding-model.md) - BEIR selected nDCG because it accommodates binary and graded relevance while respecting rank. (low confidence) — [source](https://arxiv.org/html/2104.08663) - NIST requires the document collection and relevance-judgment set to match when evaluating a run. (low confidence) — [source](https://trec.nist.gov/data/reljudge_eng.html) - Bruch et al. reduce evaluation cutoff for small corpora because recall at 1000 approaches one across methods. (low confidence) — [source](https://arxiv.org/html/2210.11934v2) ## Measurements and reference values - OpenSearch's six-dataset comparison reported RRF nDCG@10 averaging 3.86% below its score-based hybrid comparator. (low confidence) — [source](https://opensearch.org/blog/introducing-reciprocal-rank-fusion-hybrid-search/) ## Problems, failure modes and limitations - Elasticsearch alias actions can partially succeed unless the request requires missing aliases to fail the complete action list. (low confidence) (verified as of 2026-10-22) — [source](https://www.elastic.co/docs/manage-data/data-store/aliases) ## Comparisons and alternatives - PROV validity checks a consistent provenance history rather than establishing the factual truth of its assertions. (low confidence) — [source](https://www.w3.org/TR/prov-constraints/) - The RRF rank constant differs from a nearest-neighbor count or a candidate-depth limit. (low confidence) — [source](https://learn.microsoft.com/azure/search/hybrid-search-ranking) - DnDScore verifies a specified atomic subclaim with separate contextual information instead of treating all added context as the target claim. (low confidence) — [source](https://aclanthology.org/2025.emnlp-main.1205.pdf) - Claimify's evaluation framework assesses element-level coverage of verifiable information, rather than only whether a sentence produces any claim. (low confidence) — [source](https://aclanthology.org/2025.acl-long.348.pdf) - WiCE preserves alternative sufficient supporting-sentence sets rather than requiring one unique evidence set. (low confidence) — [source](https://arxiv.org/html/2303.01432v2) - Anthropic reports limited gains from generic document summaries added to chunks, unlike its chunk-specific context. (low confidence) — [source](https://www.anthropic.com/research/contextual-retrieval) - SAFE explicitly presents its numerical percentages and record counts as adaptable examples rather than universally applicable rules. (low confidence) — [source](https://link.springer.com/article/10.1186/s13643-024-02502-7) - Elasticsearch 8.19 reindexing reads a source snapshot, rather than continuously applying later writes. (low confidence) — [source](https://www.elastic.co/guide/en/elasticsearch/reference/8.19/docs-reindex.html) - BEIR evaluates retrieval over heterogeneous tasks and domains rather than one homogeneous question-answering setting. (low confidence) — [source](https://arxiv.org/html/2104.08663) ## Facts and statements - PROV represents a revised document as an entity derived from an earlier document entity. (medium confidence) — [source](https://www.w3.org/TR/prov-dm/) - A retrospective correction changes the current account of valid history while preserving earlier recorded beliefs. (medium confidence) — [source](https://docs.xtdb.com/about/time-in-xtdb.html) - Valid time can describe a future-effective change recorded before that change takes effect. (low confidence) — [source](https://docs.xtdb.com/about/time-in-xtdb.html) - XTDB's ordinary query defaults to current valid time and the best currently known system-time view. (low confidence) (verified as of 2026-10-22) — [source](https://docs.xtdb.com/about/time-in-xtdb.html) - An audit query can select a past system time to exclude corrections learned later. (low confidence) — [source](https://docs.xtdb.com/about/time-in-xtdb.html) - PROV entities have fixed aspects; changing an aspect can require a different entity. (medium confidence) — [source](https://www.w3.org/TR/prov-dm/) - The temporal database glossary distinguishes uncertain valid time from transaction time maintained by the database. (low confidence) — [source](https://www2.cs.arizona.edu/~rts/pubs/TRmerged.pdf) - A fact transferred between databases can carry several transaction-time histories. (low confidence) — [source](https://www2.cs.arizona.edu/~rts/pubs/TRmerged.pdf) - Different modeled perspectives can assign different valid times to the same fact. (low confidence) — [source](https://www2.cs.arizona.edu/~rts/pubs/TRmerged.pdf) - XTDB v1's bitemporality does not itself implement persisted-query replay and cascading retroactive corrections. (low confidence) — [source](https://v1-docs.xtdb.com/concepts/bitemporality/) - An immutable record of an action's inputs can satisfy some audit needs without a complete bitemporal property history. (low confidence) — [source](https://martinfowler.com/articles/bitemporal-history.html) - PROV quotation describes copied content, whereas revision describes a revised version of an earlier entity. (low confidence) — [source](https://www.w3.org/TR/prov-dm/) - XTDB provides ERASE for hard deletion of history; ordinary historical retention is therefore not an unconditional permanence guarantee. (low confidence) (verified as of 2026-10-22) — [source](https://docs.xtdb.com/quickstart/sql-overview.html) - RRF adds reciprocal rank contributions from the input rankings; raw score magnitudes do not enter its formula. (high confidence) — [source](https://cormack.uwaterloo.ca/cormacksigir09-rrf.pdf) - In truncated-list RRF, an item absent from one input list contributes zero from that list. (low confidence) — [source](https://www.elastic.co/search-labs/blog/improving-information-retrieval-elastic-stack-hybrid) - Cormack et al. fixed k=60 during a pilot and retained it during later validation. (low confidence) — [source](https://cormack.uwaterloo.ca/cormacksigir09-rrf.pdf) - Elastic's 2023 BEIR subset experiment favored k=20 and candidate depth 1000 for the tested dense retrievers. (low confidence) — [source](https://www.elastic.co/search-labs/blog/improving-information-retrieval-elastic-stack-hybrid) - Bruch et al. found tuned convex score combination superior to RRF in their in-domain and out-of-domain experiments. (low confidence) — [source](https://arxiv.org/html/2210.11934v2) - Elasticsearch separates the per-retriever rank window from the final response size. (low confidence) (verified as of 2026-10-22) — [source](https://www.elastic.co/docs/reference/elasticsearch/rest-apis/reciprocal-rank-fusion) - Bruch et al.'s min-max normalization uses the minimum and maximum scores in the retrieved candidate union. (low confidence) — [source](https://arxiv.org/html/2210.11934v2) - Bruch et al. compute missing candidate scores and approximate otherwise unknown ranks within the candidate union. (low confidence) — [source](https://arxiv.org/html/2210.11934v2) - Increasing the RRF rank constant flattens relative differences between high and low ranks. (low confidence) — [source](https://www.elastic.co/docs/reference/elasticsearch/rest-apis/reciprocal-rank-fusion) - Bruch et al. exclude near-zero score variance from their empirical normalization analysis and identify it as a challenge for some normalizers. (low confidence) — [source](https://arxiv.org/html/2210.11934v2) - The original RRF definition models each input ranking as a permutation, with one rank per document. (low confidence) — [source](https://cormack.uwaterloo.ca/cormacksigir09-rrf.pdf) - The original RRF experiments include a TREC-3 case where CombMNZ outscored RRF, so the paper does not show universal dominance. (low confidence) — [source](https://cormack.uwaterloo.ca/cormacksigir09-rrf.pdf) - Elasticsearch requires a fixed rank window for stable RRF pagination; changing that window can change previously seen ordering. (low confidence) (verified as of 2026-10-22) — [source](https://www.elastic.co/docs/reference/elasticsearch/rest-apis/reciprocal-rank-fusion) - Exact-term matching depends on indexed field analysis; Elasticsearch's standard text analyzer removes most punctuation and lowercases tokens. (low confidence) (verified as of 2026-10-22) — [source](https://www.elastic.co/docs/reference/query-languages/query-dsl/query-dsl-term-query) - Bruch et al.'s normalization rank-equivalence result for one query does not by itself establish equivalent learned weights across all queries. (low confidence) — [source](https://arxiv.org/html/2210.11934v2) - Azure documents a fused RRF score bound that increases with the number of query lists being fused. (low confidence) (verified as of 2026-10-22) — [source](https://learn.microsoft.com/azure/search/hybrid-search-ranking) - Azure semantic reranking runs after RRF and reports its score separately from the fusion score. (low confidence) (verified as of 2026-10-22) — [source](https://learn.microsoft.com/azure/search/hybrid-search-ranking) - The EAHR preprint bounds unread RRF contributions and requests deeper channel ranks while they can change complete-list top-K membership or order. (low confidence) — [source](https://arxiv.org/abs/2608.07152) - EAHR's reported anti-correlated cases exhaust both input lists; its authors do not guarantee a speedup on every request. (low confidence) — [source](https://arxiv.org/abs/2608.07152) - AIS treats whether a source is correct or trustworthy as a separate evaluation from whether the generated text accurately represents it. (low confidence) — [source](https://arxiv.org/html/2112.12870v2) - FActScore computes the fraction of a generation's atomic facts that a reliable knowledge source supports. (low confidence) — [source](https://aclanthology.org/2023.emnlp-main.741.pdf) - ALCE's citation-precision evaluation penalizes irrelevant citations without requiring the citation set to be minimal. (low confidence) — [source](https://aclanthology.org/2023.emnlp-main.398.pdf) - ALCE evaluates fluency, correctness, and citation quality as separate dimensions. (low confidence) — [source](https://aclanthology.org/2023.emnlp-main.398.pdf) - AIS resolves a sentence's interpretation using its linguistic context and the relevant time before judging attribution. (low confidence) — [source](https://arxiv.org/html/2112.12870v2) - FActScore does not measure factual recall and does not penalize a response merely for providing fewer facts. (low confidence) — [source](https://aclanthology.org/2023.emnlp-main.741.pdf) - FActScore's authors exclude nuanced, debatable facts and frequently conflicting knowledge sources from the metric's stated applicability. (low confidence) — [source](https://aclanthology.org/2023.emnlp-main.741.pdf) - Wallat and colleagues define citation faithfulness to require causal influence from the cited document in addition to citation correctness. (low confidence) — [source](https://arxiv.org/html/2412.18004v1) - Their proposed necessary-condition tests include altering cited information and adding irrelevant documents while examining the answer and attribution. (low confidence) — [source](https://arxiv.org/html/2412.18004v1) - DnDScore identifies a tension between isolating atomic subclaims and adding the context needed to interpret them. (low confidence) — [source](https://aclanthology.org/2025.emnlp-main.1205.pdf) - Decomposition Dilemmas identifies harmful decomposition errors including omitted context, over-decomposition, ambiguity, and alteration of the original meaning. (low confidence) — [source](https://aclanthology.org/2025.naacl-long.320.pdf) - WiCE annotates supported, partially supported, and not-supported claims and subclaims. (low confidence) — [source](https://arxiv.org/html/2303.01432v2) - Voyage contextualized embedding accepts grouped chunks so each chunk is encoded with its document peers. (low confidence) (verified as of 2026-10-22) — [source](https://www.mongodb.com/docs/voyageai/models/contextualized-chunk-embeddings.md) - Voyage contextualized embedding preserves chunk order in the returned document embeddings. (low confidence) (verified as of 2026-10-22) — [source](https://www.mongodb.com/docs/voyageai/models/contextualized-chunk-embeddings.md) - Voyage supports query and document input types that add task-specific prompts before embedding. (low confidence) (verified as of 2026-10-22) — [source](https://www.mongodb.com/docs/voyageai/models/contextualized-chunk-embeddings.md) - Anthropic contextual retrieval prepends chunk-specific explanatory context before embedding and BM25 indexing. (low confidence) — [source](https://www.anthropic.com/research/contextual-retrieval) - Late chunking contextualizes document tokens before pooling token spans into individual chunk vectors. (low confidence) — [source](https://arxiv.org/html/2409.04701v3) - Dense X defines propositions as minimal facts that include the context required to interpret them independently. (low confidence) — [source](https://arxiv.org/html/2312.06648v3) - Late chunking can be applied to suitable long-context embedding models without additional training. (low confidence) — [source](https://arxiv.org/html/2409.04701v3) - Dense X reports retrieval and QA improvements from proposition units under its evaluated computation budgets. (low confidence) — [source](https://arxiv.org/html/2312.06648v3) - Chunk boundaries, size and overlap remain empirical design choices in Anthropic's contextual retrieval recipe. (low confidence) — [source](https://www.anthropic.com/research/contextual-retrieval) - Dense X's evaluated retrieval corpus is Wikipedia, limiting direct evidence about other corpora. (low confidence) — [source](https://arxiv.org/html/2312.06648v3) - The late-chunking paper's overlap ablation does not show a clear retrieval advantage from overlapping chunks. (low confidence) — [source](https://arxiv.org/html/2409.04701v3) - Dense X's manual proposition audit found both unsupported propositions and propositions lacking standalone context. (low confidence) — [source](https://arxiv.org/html/2312.06648v3) - Callaghan and Müller-Hansen formulate stopping as rejecting a null hypothesis that achieved recall is below a chosen target. (low confidence) — [source](https://link.springer.com/article/10.1186/s13643-020-01521-4) - Their random-sampling criterion uses a hypergeometric model for relevant records drawn without replacement from the finite unscreened pool. (low confidence) — [source](https://link.springer.com/article/10.1186/s13643-020-01521-4) - SAFE combines random training-set screening, active learning, a different screening model, and a final quality-check phase. (low confidence) — [source](https://link.springer.com/article/10.1186/s13643-024-02502-7) - Callaghan and Müller-Hansen report inconsistent achieved recall and work reductions across datasets for previously proposed stopping methods. (low confidence) — [source](https://link.springer.com/article/10.1186/s13643-020-01521-4) - The 2024 stopping-criteria commentary warns that observed reliability can depend on datasets, ranking algorithms, hyperparameters, and implementation details. (low confidence) — [source](https://link.springer.com/article/10.1186/s13643-024-02699-7) - SAFE warns that using expert-selected key papers as training seeds can bias the screening model. (low confidence) — [source](https://link.springer.com/article/10.1186/s13643-024-02502-7) - The SAFE paper states that it does not present a direct comparison with existing stopping heuristics. (low confidence) — [source](https://link.springer.com/article/10.1186/s13643-024-02502-7) - Guest, Namey, and Chen calculate new information over consecutive observation runs relative to unique themes in a specified initial base. (low confidence) — [source](https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0232076) - Guest, Namey, and Chen explicitly state that their new-information thresholds lack a probability-theory guarantee of saturation. (low confidence) — [source](https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0232076) - Their saturation method was tested on single-tier codebooks; hierarchical applications require choosing which coding level contributes to the calculation. (low confidence) — [source](https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0232076) - Hennink, Kaiser, and Marconi distinguish code saturation, where new issues stop appearing, from meaning saturation, where new dimensions of issues stop appearing. (low confidence) — [source](https://pmc.ncbi.nlm.nih.gov/articles/PMC9359070/) - In the Hennink, Kaiser, and Marconi study, some conceptual codes reached meaning saturation between interviews 16 and 24, while the conceptual code responsibility for health never reached meaning saturation. (low confidence) — [source](https://pmc.ncbi.nlm.nih.gov/articles/PMC9359070/) - In that study, higher code prevalence did not indicate earlier meaning saturation. (low confidence) — [source](https://pmc.ncbi.nlm.nih.gov/articles/PMC9359070/) - Repke and colleagues evaluated stopping methods using 223 simulated screening runs over 81 real-world datasets. (low confidence) — [source](https://www.ebi.ac.uk/europepmc/webservices/rest/PMC12825451/fullTextXML) - In those experiments, CMH never stopped before the specified recall target, but often screened substantially beyond the optimal stopping point. (low confidence) — [source](https://www.ebi.ac.uk/europepmc/webservices/rest/PMC12825451/fullTextXML) - Repke and colleagues evaluate both missed recall targets and additional screening after the target's theoretically optimal stopping point. (low confidence) — [source](https://www.ebi.ac.uk/europepmc/webservices/rest/PMC12825451/fullTextXML) - Repke and colleagues disclose that evaluation co-author Finn Müller-Hansen also co-authored CMH. (low confidence) — [source](https://www.ebi.ac.uk/europepmc/webservices/rest/PMC12825451/fullTextXML) - Equal vector dimensions do not establish relevance compatibility between embeddings from different models. (low confidence) — [source](https://www.mongodb.com/docs/voyageai/tutorials/migrate-embedding-model.md) - Voyage documents compatibility within its voyage-4 model family while still recommending corpus regeneration during migration. (low confidence) (verified as of 2026-10-22) — [source](https://www.mongodb.com/docs/voyageai/tutorials/migrate-embedding-model.md) - MongoDB's self-managed migration procedure keeps new embeddings in a separate field while retaining the existing field and index. (low confidence) (verified as of 2026-10-22) — [source](https://www.mongodb.com/docs/voyageai/tutorials/migrate-embedding-model.md) - Qdrant supports building a second collection in the background before atomically changing its collection alias. (low confidence) (verified as of 2026-10-22) — [source](https://qdrant.tech/documentation/concepts/collections/) - Elasticsearch can remove and add an index alias in one atomic multi-action request. (low confidence) (verified as of 2026-10-22) — [source](https://www.elastic.co/docs/manage-data/data-store/aliases) - MongoDB recommends evaluating a successor embedding model on representative application data before production migration. (low confidence) — [source](https://www.mongodb.com/docs/voyageai/tutorials/migrate-embedding-model.md) - A successor model with a shorter input context can truncate text that the old model accepted. (low confidence) — [source](https://www.mongodb.com/docs/voyageai/tutorials/migrate-embedding-model.md) - SentenceTransformer can load a model at an explicit repository commit, tag or branch revision. (low confidence) (verified as of 2026-10-22) — [source](https://www.sbert.net/docs/package_reference/sentence_transformer/SentenceTransformer.html) - SentenceTransformer's encoding interface exposes prompt, normalization, output precision and dimension truncation settings. (low confidence) (verified as of 2026-10-22) — [source](https://www.sbert.net/docs/package_reference/sentence_transformer/SentenceTransformer.html) - Elasticsearch reindexing does not copy source index settings or mappings into the destination. (low confidence) (verified as of 2026-10-22) — [source](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-reindex) - Qdrant's migration tutorial warns that deleting a not-yet-backfilled point can let later backfill recreate it. (low confidence) (verified as of 2026-10-22) — [source](https://qdrant.tech/documentation/tutorials-operations/embedding-model-migration/) - Keeping the old Qdrant collection current during an observation period preserves rollback after the initial switch. (low confidence) (verified as of 2026-10-22) — [source](https://qdrant.tech/documentation/tutorials-operations/embedding-model-migration/) - Stopping writes to the old collection makes later rollback lose intervening changes. (low confidence) (verified as of 2026-10-22) — [source](https://qdrant.tech/documentation/tutorials-operations/embedding-model-migration/) - Qdrant's incremental update tutorial hashes normalized chunk text to distinguish changed content from unchanged content. (low confidence) (verified as of 2026-10-22) — [source](https://qdrant.tech/documentation/tutorials-operations/incremental-embedding-updates/) - Qdrant's incremental update tutorial separates a chunk's location-based point identifier from its content hash. (low confidence) (verified as of 2026-10-22) — [source](https://qdrant.tech/documentation/tutorials-operations/incremental-embedding-updates/) - BEIR found no retrieval approach consistently strongest on every included dataset. (low confidence) — [source](https://arxiv.org/html/2104.08663) - TREC pooling combines top results from multiple runs for relevance assessment. (low confidence) — [source](https://trec.nist.gov/howto.html) - Traditional pooled evaluation often scores unjudged documents as nonrelevant. (low confidence) — [source](https://trec.nist.gov/pubs/trec30/papers/Overview-2021.pdf) - BEIR's manual TREC-COVID hole annotation increased measured scores for non-lexical approaches. (low confidence) — [source](https://arxiv.org/html/2104.08663) - A TREC-8 study found near-identical system rankings after adding relevance judgments for modern neural retrieval results. (low confidence) — [source](https://arxiv.org/abs/2201.11086) - trec_eval reciprocal rank measures the first relevant retrieved document and suits a user needing one result. (low confidence) — [source](https://github.com/usnistgov/trec_eval/blob/main/m_recip_rank.c) - TREC separates the development period from evaluation data released without relevance labels. (low confidence) — [source](https://trec.nist.gov/howto.html) - Recall divides retrieved relevant items by all relevant items; precision and recall ignore their order. (low confidence) — [source](https://trec.nist.gov/pubs/trec15/appendices/CE.MEASURES06.pdf) - BEIR's TREC-COVID hole assessors did not know which retrieval system supplied each judged result. (low confidence) — [source](https://arxiv.org/html/2104.08663) - Upadhyay et al. report successful LLM hole filling in simulations that remove relevance judgments from three TREC DL datasets. (low confidence) — [source](https://arxiv.org/abs/2405.04727) - Soboroff argues that model-generated relevance labels make the model's judgments the evaluation target, limiting the evaluation's validity. (low confidence) — [source](https://tsapps.nist.gov/publication/get_pdf.cfm?pub_id=958495) - The TREC 2024 RAG assessment study found UMBRELA-based system rankings highly correlated with manual rankings across 77 submitted runs. (low confidence) — [source](https://arxiv.org/abs/2411.08275) - ALCE evaluates generated text on separate fluency, correctness, and citation-quality dimensions. (low confidence) — [source](https://arxiv.org/abs/2305.14627) - RAGAS separates relevant focused retrieval, faithful use of context, and generation quality as distinct evaluation dimensions. (low confidence) — [source](https://arxiv.org/abs/2309.15217) - MultiHop-RAG includes queries, answers, and supporting evidence so evidence retrieval and reasoning given evidence can be evaluated separately. (low confidence) — [source](https://arxiv.org/abs/2401.15391) - The original BEIR neural retrieval experiments use only the first 512 word pieces of each document. (low confidence) — [source](https://arxiv.org/html/2104.08663)