# Crash Stack Trace Symbolization > Depth-first rabbithole dossier for Crash Stack Trace Symbolization; source-anchored research pack. Parent: [KB-Backed Troubleshooting](https://llms-explorer.com/tree/kb-backed-troubleshooting/) · 4 facets · 24 facts · page: https://llms-explorer.com/tree/crash-stack-trace-symbolization/ ## How it works - [low] When mongod and mongos processes receive a SIGUSR2 signal, they generate a backtrace for crash stack trace symbolization. Sources: https://www.mongodb.com/docs/manual/tutorial/manage-mongodb-processes.md — source: `~/.global-ai-hub/research/.pipeline/mongodb-2026-10-14/frb/crash-stack-trace-symbolization-b43876a274/rabbithole-synthesis.md#troubleshooting` - [low] The symbolization process extracts the MongoDB version from the processInfo field of the trace document. Sources: https://raw.githubusercontent.com/mongodb/mongo/master/buildscripts/mongosymb.py — source: `~/.global-ai-hub/research/.pipeline/mongodb-2026-10-14/frb/crash-stack-trace-symbolization-b43876a274/rabbithole-synthesis.md#methodology` - [low] If the binary name does not end with .debug, the symbolization process appends .debug to the binary name to locate the debug symbols file. Sources: https://raw.githubusercontent.com/mongodb/mongo/master/buildscripts/mongosymb.py — source: `~/.global-ai-hub/research/.pipeline/mongodb-2026-10-14/frb/crash-stack-trace-symbolization-b43876a274/rabbithole-synthesis.md#methodology` - [low] The symbolization process retries if the ratio of not-found frames is 0.5 or higher, waiting 60 seconds between retries. Sources: https://raw.githubusercontent.com/mongodb/mongo/master/buildscripts/mongosymb.py — source: `~/.global-ai-hub/research/.pipeline/mongodb-2026-10-14/frb/crash-stack-trace-symbolization-b43876a274/rabbithole-synthesis.md#methodology` - [low] Symbol lines are formatted as 'CODE {path:} {addr:}' during the symbolization process. Sources: https://raw.githubusercontent.com/mongodb/mongo/master/buildscripts/mongosymb.py — source: `~/.global-ai-hub/research/.pipeline/mongodb-2026-10-14/frb/crash-stack-trace-symbolization-b43876a274/rabbithole-synthesis.md#methodology` ## Measurements and reference values - [low] The symbolization tool identifies backtrace sections using the indicator '{"backtrace":' and skips lines containing 'Frame: 0x'. Sources: https://raw.githubusercontent.com/mongodb/mongo/master/buildscripts/mongosymb.py — source: `~/.global-ai-hub/research/.pipeline/mongodb-2026-10-14/frb/crash-stack-trace-symbolization-b43876a274/rabbithole-synthesis.md#methodology` ## Problems, failure modes and limitations - [low] WiredTiger panics, such as WT_PANIC with error code -31804, can trigger crash stack traces in mongod. Sources: https://www.percona.com/blog/resolving-mongodb-stack-trace/ — source: `~/.global-ai-hub/research/.pipeline/mongodb-2026-10-14/frb/crash-stack-trace-symbolization-b43876a274/rabbithole-synthesis.md#troubleshooting` - [low] The symbolization tool exits with an error if it cannot find a JSON backtrace object in the input. Sources: https://raw.githubusercontent.com/mongodb/mongo/master/buildscripts/mongosymb.py — source: `~/.global-ai-hub/research/.pipeline/mongodb-2026-10-14/frb/crash-stack-trace-symbolization-b43876a274/rabbithole-synthesis.md#troubleshooting` ## Facts and statements - [low] The default symbolizer path for crash stack trace symbolization is /opt/mongodbtoolchain/v5/bin/llvm-symbolizer. (verified as of 2026-10-15) Sources: https://raw.githubusercontent.com/mongodb/mongo/master/buildscripts/mongosymb.py — source: `~/.global-ai-hub/research/.pipeline/mongodb-2026-10-14/frb/crash-stack-trace-symbolization-b43876a274/rabbithole-synthesis.md#tools` - [low] Crash stack traces may include stripped mongod binaries, such as ELF 64-bit LSB executable x86-64 for GNU/Linux 2.6.9. Sources: https://www.percona.com/blog/resolving-mongodb-stack-trace/ — source: `~/.global-ai-hub/research/.pipeline/mongodb-2026-10-14/frb/crash-stack-trace-symbolization-b43876a274/rabbithole-synthesis.md#troubleshooting` - [low] Crash stack traces may include non-stripped mongod binaries with BuildID information, such as ELF 64-bit LSB shared object x86-64 for GNU/Linux 2.6.32. Sources: https://www.percona.com/blog/resolving-mongodb-stack-trace/ — source: `~/.global-ai-hub/research/.pipeline/mongodb-2026-10-14/frb/crash-stack-trace-symbolization-b43876a274/rabbithole-synthesis.md#troubleshooting` - [low] Crash stack traces include mongod frames with mangled names like _ZN5mongo15printStackTraceERSo and demangled names like mongo::printStackTrace. Sources: https://www.percona.com/blog/resolving-mongodb-stack-trace/ — source: `~/.global-ai-hub/research/.pipeline/mongodb-2026-10-14/frb/crash-stack-trace-symbolization-b43876a274/rabbithole-synthesis.md#troubleshooting` - [low] Crash stack traces include frames for assertion failures, such as mongo::fassertFailedNoTraceWithLocation and __wt_err_func. Sources: https://www.percona.com/blog/resolving-mongodb-stack-trace/ — source: `~/.global-ai-hub/research/.pipeline/mongodb-2026-10-14/frb/crash-stack-trace-symbolization-b43876a274/rabbithole-synthesis.md#troubleshooting` - [low] Crash stack traces include frames for WiredTiger panic handling, such as __wt_panic, and system library frames like gsignal, abort, and clone. Sources: https://www.percona.com/blog/resolving-mongodb-stack-trace/ — source: `~/.global-ai-hub/research/.pipeline/mongodb-2026-10-14/frb/crash-stack-trace-symbolization-b43876a274/rabbithole-synthesis.md#troubleshooting` - [low] Crash stack traces may include unresolved mongod frames with only offset addresses, such as mongod(+0x22744D9). Sources: https://www.percona.com/blog/resolving-mongodb-stack-trace/ — source: `~/.global-ai-hub/research/.pipeline/mongodb-2026-10-14/frb/crash-stack-trace-symbolization-b43876a274/rabbithole-synthesis.md#troubleshooting` - [low] Crash stack traces for WiredTiger panics include frames for printStackTrace, fassertFailed, __wt_panic, __wt_bm_read, and WiredTigerIndexStandard::_insert. Sources: https://jira.mongodb.org/browse/SERVER-25067 — source: `~/.global-ai-hub/research/.pipeline/mongodb-2026-10-14/frb/crash-stack-trace-symbolization-b43876a274/rabbithole-synthesis.md#troubleshooting` - [low] Crash stack traces for WiredTiger panics include frames for replication operations like applyOperation_inlock and ThreadPool::_workerThreadBody. Sources: https://jira.mongodb.org/browse/SERVER-25067 — source: `~/.global-ai-hub/research/.pipeline/mongodb-2026-10-14/frb/crash-stack-trace-symbolization-b43876a274/rabbithole-synthesis.md#troubleshooting` - [low] Crash stack traces for WiredTiger panics include frames for status conversion like wtRCToStatus_slow and cursor operations like WiredTigerRecordStore6Cursor::seekExact. Sources: https://jira.mongodb.org/browse/SERVER-25067 — source: `~/.global-ai-hub/research/.pipeline/mongodb-2026-10-14/frb/crash-stack-trace-symbolization-b43876a274/rabbithole-synthesis.md#troubleshooting` - [low] Crash stack traces for WiredTiger panics include frames for query execution stages like IDHackStage::work, UpdateStage::work, and update operations. Sources: https://jira.mongodb.org/browse/SERVER-25067 — source: `~/.global-ai-hub/research/.pipeline/mongodb-2026-10-14/frb/crash-stack-trace-symbolization-b43876a274/rabbithole-synthesis.md#troubleshooting` - [low] If no trace document is provided via stdin, the symbolization tool prompts the user to provide the backtrace through stdin. Sources: https://raw.githubusercontent.com/mongodb/mongo/master/buildscripts/mongosymb.py — source: `~/.global-ai-hub/research/.pipeline/mongodb-2026-10-14/frb/crash-stack-trace-symbolization-b43876a274/rabbithole-synthesis.md#troubleshooting` - [low] The symbolization tool defaults to using the 'pr' option for --debug-file-resolver and 'classic' for --input-format. Sources: https://raw.githubusercontent.com/mongodb/mongo/master/buildscripts/mongosymb.py — source: `~/.global-ai-hub/research/.pipeline/mongodb-2026-10-14/frb/crash-stack-trace-symbolization-b43876a274/rabbithole-synthesis.md#tools` - [low] llvm-symbolizer accepts positional arguments or stdin values preceded by 'DATA' or 'CODE' to specify symbolization type, defaulting to 'CODE' if unspecified. Sources: https://llvm.org/docs/CommandGuide/llvm-symbolizer.html — source: `~/.global-ai-hub/research/.pipeline/mongodb-2026-10-14/frb/crash-stack-trace-symbolization-b43876a274/rabbithole-synthesis.md#tools` - [low] Address calculation for symbolization determines addr_base based on ELF type (0 for type 3, frame['b'] for type 2, or soinfo vmaddr) and adds the frame offset. Sources: https://raw.githubusercontent.com/mongodb/mongo/master/buildscripts/mongosymb.py — source: `~/.global-ai-hub/research/.pipeline/mongodb-2026-10-14/frb/crash-stack-trace-symbolization-b43876a274/rabbithole-synthesis.md#methodology` - [low] Frames missing the 'b' field are ignored during symbolization, with a reference to SERVER-58863 for discussions. Sources: https://raw.githubusercontent.com/mongodb/mongo/master/buildscripts/mongosymb.py — source: `~/.global-ai-hub/research/.pipeline/mongodb-2026-10-14/frb/crash-stack-trace-symbolization-b43876a274/rabbithole-synthesis.md#troubleshooting` ## Related concepts - Trace — is a part of Crash Stack Trace Symbolization - Crash — is a part of Crash Stack Trace Symbolization - Stack — is a part of Crash Stack Trace Symbolization - Symbolization — is a part of Crash Stack Trace Symbolization