Hidden indexes and safe index removal

Published reference · snapshot 2026-10-20 · skill mongodb-expert/references/hidden-indexes-and-safe-index-removal.md

Also known as: Hidden indexes as a drop trial, Hide-before-drop index workflow, Index pruning with hideIndex, Safe index-drop workflow with $indexStats, collMod index.hidden option

↓ Facts as markdown all context files

Source-anchored research for Hidden indexes and safe index removal. Run status: COMPLETED.

These notes link each claim to its source. Some citations identify archived research reports that are no longer hosted on this site. A published reference means the content is available; it does not certify independent review or accuracy.Read the editorial policy and follow the sources before relying on a claim.

Definitions

Dropping on short evidence is the main removal mistake. Counters restart at zero after a restart, so an index that serves a monthly or quarterly job looks unused until that job runs. Percona warns that dropping an index used once a month for heavy reporting can be problematic. Netdata says not to drop within 24 hours of a restart. (high confidence) [source]

Structure and components

Do not drop a single-field index only because a compound index contains the same field in a non-leading position. Atlas Performance Advisor treats { b: -1 } as not redundant next to { a: 1, b: -1 }. Only a leading prefix makes the smaller index redundant. (medium confidence) [source]
Unneeded indexes cost every write. Each insert and delete updates every index, and an update to an indexed field removes and re-adds entries in each index that contains the field. Indexes also compete with documents for WiredTiger cache. MongoDB says an index on every field bloats a collection and hurts write performance; Percona says extra indexes hurt write performance, disk and memory. A collection can hold at most 64 indexes. (high confidence) (verified as of 2026-10-14) [source]

How it works

Unhiding is immediate and cheaper than rebuilding. Because the index is maintained while hidden, it is available for use the moment it is unhidden. Atlas states that unhiding takes less time than rebuilding a dropped index. Percona notes that building an index is expensive on large collections or when memory is short. (high confidence) [source]
dropIndexes takes an exclusive lock on the collection for the duration of the operation, and all later operations on the collection wait until it finishes. Netdata advises dropping during a low-traffic window and watching globalLock.currentQueue and lock waits, because a drop under heavy write load can stall writes. (high confidence) [source]
$indexStats returns usage per index. accesses.ops counts user operations that used the index, and accesses.since is when MongoDB began gathering statistics. Internal operations such as TTL deletion and chunk split or migration are not counted. The counters reset on mongod restart, on index drop and recreation, and when collMod modifies the index. (high confidence) [source]
The sources give different minimum observation windows. Netdata asks for at least 24 hours of uptime under representative load that includes peak and off-peak traffic. Atlas Performance Advisor flags an index after 7 days without a supported query. Percona and Pythian say to observe long enough to cover every query pattern, such as monthly reports, and to check uptime with db.serverStatus().uptime. Choose a window longer than your longest business cycle. (medium confidence) [source]
Redundancy is a prefix test. Atlas Performance Advisor flags an index as redundant when it matches the prefix of another index in the same collection. With indexes { a: 1 }, { b: -1 } and { a: 1, b: -1 }, only { a: 1 } is redundant. The advisor lists the covering indexes under each redundant index so you can verify the drop, and it does not recommend removing the covering indexes. (high confidence) [source]
Persisting $indexStats in monitoring lets the observation window outlast mongod restarts. Percona's 2018 post used the PMM mongodb exporter option -collect.indexusage, which is off by default because it runs an aggregate on the source instance. Retention must match or exceed your full application cycle. (low confidence) (verified as of 2018-11-21) [source]
Hiding is a mechanism with several equivalent forms. Use db.collection.hideIndex(<name or key document>) and unhideIndex(), or the collMod command with index.hidden, or createIndex(keys, { hidden: true }). mongosh hideIndex is a wrapper for collMod. Identify a text index by name, not by key document. (high confidence) [source]
When the hidden value changes, collMod returns hidden_old and hidden_new. When it does not change (hiding an already hidden index or unhiding an already unhidden one), collMod omits both fields. Hiding an already hidden index still writes an empty oplog entry. (high confidence) [source]
Three commands control visibility. collMod with index: {name | keyPattern, hidden: true | false} hides or unhides an existing index. The mongosh helpers db.collection.hideIndex() and unhideIndex() wrap collMod and accept an index name or a key specification (a text index must be named). createIndex(spec, {hidden: true}) builds an index that starts hidden. getIndexes() shows the hidden field only when it is true. collMod returns hidden_old and hidden_new only when the value actually changed. (high confidence) [source]
Hiding is cheap to undo; dropping is not. A hidden index is fully maintained, so unhiding makes it usable immediately. A dropped index must be rebuilt, which Percona calls quite expensive for large collections or when memory is short, and Atlas notes that unhiding takes less time than rebuilding. A hidden index can be dropped directly; unhiding it first is not required. (high confidence) [source]
Hiding an index clears the plan cache of its collection. The query-plans manual lists 'creating, deleting, or hiding an index' as DDL events that clear that collection's plan cache. SERVER-102775 says hiding any index clears the entire collection plan cache, forces every later query to replan, and can cause a 'multiplan storm'. SERVER-103054 says dropping an already-hidden index also clears the entire plan cache. As of 2026-10-20 both tickets are in Backlog with no fix version. Inference: hide large, query-heavy collections off-peak. (high confidence) (verified as of 2026-10-20) [source]
$indexStats reports accesses.ops (the number of user operations that used the index) and accesses.since (when MongoDB began gathering that index's statistics). The count excludes internal operations: TTL deletions and chunk split and migration are not counted. Counters reset on mongod restart, on index drop and recreate, and on collMod changes. So ops: 0 means only that no user operation used the index since the since timestamp. (high confidence) [source]
ops: 0 is not proof that an index is unnecessary unless the observation window spans the longest workload cycle. Percona warns that an index used once a month for heavy reporting looks idle in between. Netdata notes that a nightly batch job may use an index that daytime OLTP never touches. The Ops Manager docs keep an infrequently used { stars: -1 } index so that a leaderboard query does not scan every document. (high confidence) [source]
A unique index can show accesses.ops of 0 while still protecting data, because the uniqueness check on insert does not increment the counter. Dropping such an index silently allows duplicate keys. Exclude unique indexes from ops-based candidate lists. (medium confidence) [source]
A TTL index can show accesses.ops of 0 while it actively expires documents, because $indexStats does not count TTL deletions. Do not treat a zero count on a TTL index as evidence that it is unused. (high confidence) [source]
Prefix redundancy is the safest drop candidate. A single-field index that is the leading prefix of a compound index (for example { a: 1 } next to { a: 1, b: -1 }) is redundant. Atlas flags only prefix matches and lists the covering indexes so you can verify the drop; the covering indexes are not recommended for removal. The Ops Manager example drops { last_name: 1 } because { last_name: 1, first_name: 1 } covers it. (high confidence) [source]
Atlas Performance Advisor flags an index as unused when it has not supported a query for 7 or more days since creation or since the last server restart. It inspects only the 20 most active collections for unused indexes, and it lists hidden indexes only from the top 20 most active namespaces at page load, so an unused or hidden index in a quieter namespace may not be flagged. (low confidence) (verified as of 2026-10-20) [source]

Problems, failure modes and limitations

A hidden index is not visible to the query planner and cannot support a query. The planner does not evaluate it during plan selection, and it does not appear even in rejectedPlans. Hiding therefore shows what would happen if the index were dropped, and unhiding undoes it without a rebuild. (high confidence) [source]
Judge the effect of a hide with application latency, slow-query log entries and explain() on the critical query shapes, not with the hidden index's own counter. The planner cannot choose a hidden index, so its counter is no evidence of safety. After a removal, watch for COLLSCAN entries and a rising docsExamined to docsReturned ratio. (medium confidence) [source]
Practitioner guides add safeguards around the drop. Record index definitions with db.collection.getIndexes() as a rollback plan. Take a backup of the affected collections. Run the drop in a low-traffic window. Afterward, check the slow-query log for COLLSCAN, run explain('executionStats') on critical query shapes, and compare opLatencies.writes before and after. (medium confidence) [source]
Zero accesses.ops does not prove an index is useless when it enforces a constraint. A uniqueness check on insert does not increment the counter. Dropping a unique index removes the constraint and can let duplicate keys in. A hidden unique index keeps enforcing uniqueness, and a hidden TTL index keeps expiring documents, so a hide-and-observe test cannot show what a drop would remove. (medium confidence) [source]
Since MongoDB 6.0, dropIndexes raises an error if you try to remove the last remaining shard-key-compatible index. Passing index: "*" drops every index except _id and that last shard-key-compatible index. (low confidence) (verified as of 2026-10-14) [source]
Before MongoDB 8.3, dropIndexes returns an IndexNotFound error when a named index does not exist, so a re-run of a drop script fails on indexes it already removed. From 8.3, dropIndexes is idempotent: it ignores missing names and drops the rest of an array. (low confidence) (verified as of 2026-10-14) [source]
dropIndexes can drop one index while another index build runs on the same collection. The manual says earlier versions instead failed with a BackgroundOperationInProgressForNamespace error. The manual does not give the version where that changed. (low confidence) (verified as of 2026-10-14) [source]
Before MongoDB 8.1, some $geoNear queries failed with an IndexNotFound error when a hidden 2d or 2dsphere index existed, because the server was unsure which index to use. MongoDB 8.1 made some of those queries succeed. Test $geoNear paths before you hide a geospatial index on an older server. (low confidence) [source]
The hide trial cannot test the removal of a unique or TTL index. A hidden unique index still rejects duplicates and a hidden TTL index still deletes expired documents, so hiding shows nothing about what dropping the constraint or the expiry would do. (high confidence) [source]
On a sharded collection, MongoDB refuses to drop or hide an index that is the only non-hidden index supporting the shard key. The exception is a hashed shard key: since 7.0.3 (also 6.0.12 and 5.0.22) its index can be dropped. The manual advises dropping it only if a supporting non-hashed index on the shard key exists, and says dropping it disables balancing for the collection until the hashed index is recreated. The _id index cannot be dropped at all. (high confidence) [source]
Because $indexStats resets on restart, export it to a time-series store. Percona's mongodb_exporter reads $indexStats with --collector.indexstats and limits it with --mongodb.indexstats-colls=db.coll,...; --collector.collstats-limit=<n> disables the indexstats collector above n collections. Percona's 2018 post says metrics kept in PMM survive an instance restart, provided retention matches or exceeds the application cycle. The -collect.indexusage flag in that post does not appear in the current exporter reference. (medium confidence) (verified as of 2026-10-20) [source]
Use more than $indexStats to judge impact. MongoDB's Measure Index Use page points to explain() in executionStats and allPlansExecution modes, serverStatus metrics.queryExecutor.scanned and metrics.operation.scanAndOrder, collStats totalIndexSize and indexSizes, and dbStats indexes and indexSize. Netdata adds post-removal checks: new COLLSCAN entries in the slow query log, a rising docsExamined-to-docsReturned ratio, opLatencies.writes, and the WiredTiger cache dirty ratio. (medium confidence) [source]
Judge a hide trial by workload signals, not by the hidden index's own counters. Inference: a hidden index cannot accumulate ops (the planner cannot choose it, hint() on it fails, and its counters reset when it is hidden), so its $indexStats says nothing about demand. Watch the slow query log or profiler for COLLSCAN, the docsExamined-to-docsReturned ratio, tail latency, and explain('executionStats') on critical query shapes. (medium confidence) [source]
Version-specific dropIndex behavior from the manual: starting in MongoDB 8.3, dropIndex is idempotent and ignores a missing index. In earlier versions it returns an IndexNotFound error. Earlier versions also return BackgroundOperationInProgressForNamespace when you drop a different index during an in-progress build; current versions allow it. (low confidence) (verified as of 2026-10-14) [source]
If hideIndex or collMod fails, check four things. (1) The index must exist: collMod errors with 'cannot find index <name|keyPattern> for ns <db.collection>'. (2) Privilege: collMod needs the collMod action (built-in role dbAdmin); on Atlas, creating, dropping or hiding indexes needs Project Owner, Organization Owner or Project Data Access Admin. (3) featureCompatibilityVersion must be high enough: Percona's 4.4 walkthrough hit code 13 Unauthorized ('not authorized on restaurants to execute command { collMod: ... hidden: true }') until it ran setFeatureCompatibilityVersion '4.4'. (4) The _id index cannot be hidden, and a text index must be named. (medium confidence) [source]

Comparisons and alternatives

You can drop a hidden index directly. You do not need to unhide it first. If the evaluation shows a negative impact, unhide it instead of dropping it. (medium confidence) [source]
Hiding shows the fallback plan. In Percona's test, hiding { cuisine: 1, borough: 1 } made the winning plan an IXSCAN on { borough: 1 } with a FETCH filter on cuisine, and rejectedPlans was empty. Netdata makes the same point: after a drop the planner may pick another index instead of a COLLSCAN, so verify critical shapes with explain(). (medium confidence) [source]
A hidden index cannot be targeted with cursor.hint(), and active index filters cannot use it. Any operation that hints a hidden index returns an exception. Hiding an index that application code hints therefore makes those queries fail instead of only slowing them. Search the code base for hint() calls on the index before you hide it. (high confidence) [source]
Two hard restrictions apply. The _id index cannot be hidden. A query that passes hint() naming a hidden index returns an error instead of falling back to another plan; the server test pins this to error code 2 (BadValue). (high confidence) [source]
Hidden indexes shipped in MongoDB 4.4, and hiding required featureCompatibilityVersion 4.4 or greater at that time. The collMod reference still says 4.4 in its option description. Hidden-index pages for other manual versions print a different number (see disagreements), so check the cluster's actual FCV instead of trusting one page. (medium confidence) (verified as of 2026-10-20) [source]
Audit application code before hiding. Any query that passes hint() with the name or key pattern of a hidden or dropped index fails with an error instead of choosing another index. Search drivers, ODM models and saved queries for hint usage. An error during the hide window exposes such a dependency, and unhiding restores the index immediately. (medium confidence) [source]
Hiding a text index breaks $text queries instead of slowing them. The server test hides a text index and then expects a find with $text to fail with error code 291 (NoQueryExecutionPlans). Hide-before-drop on a text index is therefore an outage risk, not a performance experiment. A text index must be hidden by name, not by key pattern. (medium confidence) [source]

Facts and statements

A hidden index is fully maintained. Every write still updates it, and it keeps consuming disk space and memory. A hidden unique index still enforces its unique constraint, and a hidden TTL index still expires documents. The index still appears in listIndexes, db.collection.getIndexes(), db.collection.stats() and $indexStats. (high confidence) [source]
Hiding an index does not reduce write cost or storage. Only dropping the index reclaims disk space and improves write performance. An index left hidden keeps its full maintenance cost. (high confidence) [source]
The current MongoDB 9.0 manual says hiding an index, or creating one with the hidden: true option, requires featureCompatibilityVersion 7.0 or greater. Percona's 2020 test on MongoDB 4.4 reports FCV 4.4 as the requirement. Read the manual that matches your server version. (high confidence) (verified as of 2026-10-18) [source]
The default _id index can be neither hidden nor dropped. To remove the _id index you must drop the whole collection. (high confidence) [source]
dropIndexes kills only the queries that use the index being dropped. That can include queries that are still considering the index during query planning. The pages read document this kill behavior for dropIndexes only, not for hiding. (low confidence) [source]
The Query Plans page lists hiding an index as a DDL event that clears the plan cache for that collection. Queries on the collection therefore plan again after a hide. (low confidence) [source]
Unnecessary indexes cost write throughput, cache and disk. Each insert and delete updates every index, and an update that changes an indexed field updates each index containing that field. In WiredTiger, index pages compete with documents for cache space. Unused indexes also add disk and memory use. (high confidence) [source]
The MongoDB manual's workflow for removing an index has three steps. First, evaluate usage with $indexStats. Second, hide the candidate indexes with hideIndex(). Third, once no harm shows, drop them with dropIndexes([...]). The Drop an Index page says to consider hiding before dropping, and the Atlas page repeats that advice. (high confidence) [source]
Hiding an unhidden index, or unhiding a hidden index, resets that index's $indexStats. Hiding an already hidden index, or unhiding an already unhidden one, does not reset them. Read and record accesses.ops and accesses.since before you hide. (high confidence) [source]
$indexStats reports usage only for the node where the query runs. Run it on every data-bearing node, secondaries included, before you call an index unused. The Atlas Data Explorer Usage metrics show the primary only, so they miss indexes that serve secondary reads. (high confidence) [source]
On a sharded cluster, an index can be idle on one shard and busy on another. Netdata advises collecting $indexStats from each shard primary and summing accesses.ops across shards before calling an index globally unused. The manual documents a shard field in $indexStats output for sharded clusters. (medium confidence) [source]
On a deployment with access control, hiding an index needs the collMod privilege on the collection's database, which the built-in dbAdmin role provides. In the Atlas UI, creating, dropping or hiding an index needs the Project Owner, Organization Owner or Project Data Access Admin role. (medium confidence) (verified as of 2026-10-19) [source]
Atlas Performance Advisor drop-index recommendations cover three classes. Unused: no supported query in 7 or more days after the index was created or the server restarted. Redundant: a prefix of another index. Hidden: always recommended for dropping, with the advice to drop once you are confident the index is unnecessary. Cloud Manager and the Atlas CLI describe the same classes. (high confidence) (verified as of 2026-10-14) [source]
Atlas Performance Advisor inspects only the 20 most active collections for unused indexes. Its page shows hidden indexes from the top 20 most active namespaces at page load, so a cluster with more namespaces may not list every hidden index. (low confidence) (verified as of 2026-10-14) [source]
The Performance Advisor drop dialog gives a copyable mongosh command to drop the index and another to hide it. The Atlas UI Indexes tab has a Hide Index button (closed-eye icon) and marks a hidden index with a Hidden badge in the Properties column. (medium confidence) (verified as of 2026-10-14) [source]
The Remove Unnecessary Indexes page says Atlas Performance Advisor is available for M10 clusters or higher, and that Compass can also help determine, hide and drop unnecessary indexes. (low confidence) (verified as of 2026-10-18) [source]
The Atlas CLI command atlas api performanceAdvisor listDropIndexSuggestions --clusterName <name> --groupId <id> returns the indexes Performance Advisor suggests dropping. It maps 1:1 to the Atlas Admin API endpoint and defaults to API version 2024-08-05. The CLI page lists no thresholds or response fields. (low confidence) (verified as of 2026-10-20) [source]
In Cloud Manager, the Performance Advisor drop-index results by default describe one of the primary hosts. Pick another host from the drop-down to see its recommendations. This matches the per-node nature of $indexStats. (medium confidence) (verified as of 2026-10-14) [source]
In Percona's 2020 test on MongoDB 4.4, hideIndex first failed with errmsg 'not authorized on restaurants to execute command { collMod: "restaurants", index: { keyPattern: { cuisine: 1.0, borough: 1.0 }, hidden: true }, ... }', code 13, codeName 'Unauthorized'. After db.adminCommand({ setFeatureCompatibilityVersion: "4.4" }), the same hideIndex returned { hidden_old: false, hidden_new: true, ok: 1 }. If hideIndex returns code 13 for a user who holds dbAdmin, check featureCompatibilityVersion. (low confidence) [source]
A hidden index is invisible to the query planner: the planner does not consider it for plan selection, and it does not even appear in rejectedPlans. Apart from that, it behaves like a normal index. Writes still update it, it still uses disk and memory, listIndexes and getIndexes() still list it with hidden: true, a hidden unique index still enforces uniqueness, and a hidden TTL index still expires documents. (high confidence) [source]
Hiding an unhidden index, or unhiding a hidden index, resets that index's $indexStats counters. Hiding an already-hidden index or unhiding an already-visible index does not reset them. More generally, $indexStats counters also reset on mongod restart and on index drop and recreate, and the manual states that any collMod modification of an existing index resets them. (high confidence) [source]
Hiding validates only the read side of a drop. A hidden index is still fully maintained, so write amplification, WiredTiger cache use and disk use do not fall until the index is dropped. For this reason the Atlas Performance Advisor always recommends dropping hidden indexes. Treat hiding as a time-boxed trial, not a resting state. (high confidence) [source]
MongoDB's documented removal workflow has three steps. (1) Evaluate usage with the $indexStats stage, on each node. (2) Hide the candidates with hideIndex(). (3) If the impact is acceptable, drop with dropIndex() or dropIndexes([...]); if the impact is negative, unhide. The manual warns that dropping an index that is actively used in production can degrade performance, and it recommends hiding first. (high confidence) [source]
$indexStats is node-local. Each node reports only the operations that ran on that node, so an index used only by secondary reads can show ops: 0 on the primary. MongoDB says to run $indexStats on every node in the cluster. The Atlas Data Explorer usage column reflects only the primary. (high confidence) [source]
The Atlas Performance Advisor Drop Indexes card reports three categories: unused, redundant and hidden. It offers copyable mongosh commands to drop or to hide each index, and the Atlas UI has Hide Index and Drop Index controls. The same suggestions are available through the Atlas Admin API operation listDropIndexSuggestions (CLI: atlas api performanceAdvisor listDropIndexSuggestions). Performance Advisor is available only on M10 and larger clusters. MongoDB Compass can also hide and drop indexes. (high confidence) (verified as of 2026-10-20) [source]
Before hiding, save the evidence that hiding will erase. Inference: record $indexStats output (ops, since, host) from every node and save db.collection.getIndexes() output. Hiding resets the counters, and a dropped index must be rebuilt from its saved definition. Netdata lists getIndexes() as the rollback plan; Pythian lists a backup of the affected collections. (medium confidence) [source]
SERVER-92843: in 7.0.9 and 7.0.12 a hidden 2d index still affected planning. With two 2d indexes on the same field and one of them hidden, a $geoNear aggregation failed with IndexNotFound: 'There is more than one 2d index on admin.data; unsure which to use for $geoNear'. The ticket is Closed/Fixed in 8.1.0-rc0, with backports requested to 8.0, 7.0, 6.0 and 5.0; whether those backports shipped is unverified. (low confidence) (verified as of 2026-10-20) [source]
Idempotent index-ensure scripts can collide with a hidden index. The server test asserts that createIndex on the same key with hidden: false, against an existing hidden index, fails with IndexOptionsConflict (code 85). Inference: pause schema-sync jobs, or give them the same hidden option, during a hide trial. (medium confidence) [source]
dropIndex takes an exclusive lock on the collection for the duration of the operation, and all later operations on that collection wait for it. Netdata warns that concurrent writes can stall, advises a low-traffic window, and suggests watching globalLock.currentQueue during the drop. If the target index is still building, dropIndex attempts to stop the build. (high confidence) [source]

Children

$indexStats usage statistics and their reset conditions (frontier)
Atlas Performance Advisor drop-index recommendations (frontier)
Atlas Performance Advisor drop-index recommendations (unused, redundant, hidden) (frontier)
Index filters and query settings versus hidden indexes (frontier)
Index rebuild cost after a wrong drop (frontier)
Index usage measurement with $indexStats (frontier)
Plan cache flushes on index DDL (frontier)
Plan cache invalidation on index DDL (hide, drop, create) (frontier)
Redundant (prefix) index detection for compound indexes (frontier)
Redundant and prefix index detection (frontier)
Shard key index constraints when dropping or hiding indexes (frontier)
Unique and TTL index side effects on removal (frontier)
dropIndexes locking and in-flight query behavior (frontier)

Frontier under this node: $indexStats usage statistics and their reset conditions, Atlas Performance Advisor drop-index recommendations (unused, redundant, hidden), Index filters and query settings versus hidden indexes, Index rebuild cost after a wrong drop, Index usage measurement with $indexStats, Plan cache flushes on index DDL, Plan cache invalidation on index DDL (hide, drop, create), Redundant (prefix) index detection for compound indexes, Shard key index constraints when dropping or hiding indexes, Unique and TTL index side effects on removal, dropIndexes locking and in-flight query behavior

← the whole tree · 3D view· how to read this page