Keyset (resume) pagination for restartable cursors
Parent: Long-running secondary reads in sharded clusters · Published reference · snapshot 2026-10-31
↓ Facts as markdownall context files
Source-anchored research for Keyset (resume) pagination for restartable cursors. Run status: COMPLETED.
These notes link each claim to its source. Some citations identify archived research reports that are no longer hosted on this site. A published reference means the content is available; it does not certify independent review or accuracy.Read the editorial policy and follow the sources before relying on a claim.
Definitions
- Keyset pagination is not a promise of an unchanging result set. Sequin documents missed late-committing rows in PostgreSQL and repeated or skipped rows after sort-key changes; its transaction-specific explanation must not be assumed to describe MongoDB internals. (low confidence) [source]
How it works
- The recovery example advances lastProcessedId only after processItem completes, but stores that checkpoint in memory. Process-crash recovery requires an additional durable checkpoint design; the example itself does not provide one. (low confidence) [source]
- MongoDB v8.3 documentation recommends an application resume mechanism for long-running secondary reads. Its example reissues the original state filter with zipcode greater than the last successfully processed value, retains ascending sorting and secondary read preference, and assumes unique zipcodes. (low confidence) [source]
- For orphan-cleanup recovery, match code 175 together with errmsg containing "Read has been terminated due to orphan range cleanup". QueryPlanKilled has other causes, so code 175 alone is insufficient to identify this recovery path. (low confidence) [source]
- MongoDB’s secondary-read example processes readDoc and then assigns its zipcode to latestZip. When it catches code 175 with the orphan-range-cleanup message, it creates a new secondary query that retains state: NY and adds zipcode greater than latestZip, sorted by zipcode ascending. The example uses local variables for progress. (medium confidence) [source]
How-to and procedures
- Use an index aligned with equality filters and the pagination ordering. Mixmax illustrates {name:1, city:1, _id:1}; string pagination also requires compatible query and index collations to use the intended index. (low confidence) [source]
Problems, failure modes and limitations
- The resume order must uniquely position every document. If the primary sort field repeats, include a unique tie-breaker such as _id and preserve both values in the checkpoint; a timestamp alone cannot safely traverse a group larger than the page size. (high confidence) [source]
Comparisons and alternatives
- MongoDB range pagination stores the last-seen key, uses $gt with ascending sort or $lt with descending sort, and limits each query. Recovery can create a new query after the last successfully processed key rather than retain the failed server cursor. (high confidence) [source]
- For large offsets, skip() scans preceding results before returning the page. Indexed range queries can avoid that work; the benefit depends on a usable index rather than pagination syntax alone. (high confidence) [source]
Facts and statements
- For ascending (createdAt, _id) order, a continuation predicate is createdAt > t OR (createdAt = t AND _id > id), combined with the original filter and sort on both fields. Both cursor values are required to traverse ties. (medium confidence) [source]
- A pagination sort field should be immutable and consistently typed. Changing a sort key between pages can move a document across the boundary, producing duplicates or omissions; a deterministic tiebreaker alone does not prevent this. (high confidence) [source]
- ObjectId is useful as a unique pagination key, but MongoDB documents that it is not necessarily monotonic in creation time: temporal resolution is one second and clients can have different clocks. Do not interpret an _id checkpoint as a guaranteed insertion-time watermark. (low confidence) [source]
- Indexed range pagination stores the last-seen key and seeks strictly beyond it: use $gt with ascending order or $lt with descending order. This avoids the growing prefix scan required by skip(), although performance still depends on index support. (high confidence) [source]
- For a defined, non-null paginated-field cursor, Mixmax’s generateCursorQuery ascending default branch returns {$or:[{k:{$gt:lastK}},{k:{$eq:lastK},_id:{$gt:lastId}}]}, and generateSort uses {k:1,_id:1}. Its descending default branch reverses the comparisons and also includes a null-or-undefined-field branch. (medium confidence) [source]
- Build a compound index that supports the fixed equality filter followed by the pagination sort fields. MongoDB illustrates {state:1,zipcode:1}; Mixmax illustrates {name:1,city:1,_id:1} for name equality and city/_id ordering. (high confidence) [source]
- Mutable sort fields can make documents repeat or disappear behind the resume boundary. Prefer immutable keys; keyset pagination alone does not establish a fixed snapshot across separate queries. (high confidence) [source]
Children
- Durable traversal checkpoints and idempotent processing (frontier)
- Compound keyset predicates (frontier)
- Snapshot and secondary visibility across pages (frontier)
Frontier under this node: Compound keyset predicates, Durable traversal checkpoints and idempotent processing, Snapshot and secondary visibility across pages