# Streaming BSON Parser Architecture > Depth-first rabbithole dossier for Streaming BSON Parser Architecture; source-anchored research pack. Parent: [Diagnostic Tool Design](https://llms-explorer.com/tree/diagnostic-tool-design/) · 5 facets · 22 facts · page: https://llms-explorer.com/tree/streaming-bson-parser-architecture/ ## Definitions - [low] BSON is a binary format storing zero or more ordered key/value pairs as a single entity, with each type serialized in little-endian format. Sources: https://bsonspec.org/spec.html — source: `~/.global-ai-hub/research/.pipeline/mongodb-2026-10-14/frb/streaming-bson-parser-architecture-a92d73555b/rabbithole-synthesis.md#core` - [low] A BSON document is defined as an `int32` total byte count followed by an element list and a trailing `unsigned_byte(0)`, where the `int32` is the total number of bytes constituting the document. Sources: https://bsonspec.org/spec.html — source: `~/.global-ai-hub/research/.pipeline/mongodb-2026-10-14/frb/streaming-bson-parser-architecture-a92d73555b/rabbithole-synthesis.md#core` - [low] BSON `ObjectId` is a `signed_byte(7)` followed by an element name and 12 bytes of value, and `datetime` is a `signed_byte(9)` followed by an element name and an `int64` representing UTC milliseconds since the Unix epoch. Sources: https://bsonspec.org/spec.html — source: `~/.global-ai-hub/research/.pipeline/mongodb-2026-10-14/frb/streaming-bson-parser-architecture-a92d73555b/rabbithole-synthesis.md#core` - [low] BSON `Timestamp` is a `signed_byte(17)` followed by a `uint64`, where the first 4 bytes are an increment and the second 4 bytes are a timestamp, used internally by MongoDB replication and sharding. Sources: https://bsonspec.org/spec.html — source: `~/.global-ai-hub/research/.pipeline/mongodb-2026-10-14/frb/streaming-bson-parser-architecture-a92d73555b/rabbithole-synthesis.md#core` - [low] BSON `string` is an `int32` length followed by bytes and a trailing null byte, where the `int32` is the number of bytes in the string plus one for the trailing null byte. Sources: https://bsonspec.org/spec.html — source: `~/.global-ai-hub/research/.pipeline/mongodb-2026-10-14/frb/streaming-bson-parser-architecture-a92d73555b/rabbithole-synthesis.md#core` - [low] BSON `binary` is an `int32` length followed by a subtype and bytes, where the `int32` is the number of bytes in the binary data. Sources: https://bsonspec.org/spec.html — source: `~/.global-ai-hub/research/.pipeline/mongodb-2026-10-14/frb/streaming-bson-parser-architecture-a92d73555b/rabbithole-synthesis.md#core` - [low] A BSON array is a document with integer values for keys starting at 0 and continuing sequentially. Sources: https://bsonspec.org/spec.html — source: `~/.global-ai-hub/research/.pipeline/mongodb-2026-10-14/frb/streaming-bson-parser-architecture-a92d73555b/rabbithole-synthesis.md#core` - [low] A BSON vector is a dense array of numeric values stored in a binary format efficient for storage and retrieval, where all values within the vector must be of the same type. Sources: https://bsonspec.org/spec.html — source: `~/.global-ai-hub/research/.pipeline/mongodb-2026-10-14/frb/streaming-bson-parser-architecture-a92d73555b/rabbithole-synthesis.md#core` ## Structure and components - [low] The `bson_reader_t` structure is used for reading a sequence of BSON documents from a file descriptor, memory region, or custom callbacks. Sources: https://mongoc.org/libbson/current/bson_reader_t.html — source: `~/.global-ai-hub/research/.pipeline/mongodb-2026-10-14/frb/streaming-bson-parser-architecture-a92d73555b/rabbithole-synthesis.md#core` - [low] BSON regex consists of a first cstring for the pattern and a second cstring for the options string. Sources: https://bsonspec.org/spec.html — source: `~/.global-ai-hub/research/.pipeline/mongodb-2026-10-14/frb/streaming-bson-parser-architecture-a92d73555b/rabbithole-synthesis.md#core` ## How it works - [low] Parsing large blobs of BSON is difficult with certain libraries because they require pulling the entirety of the BSON blob into memory. Sources: https://jira.mongodb.org/browse/NODE-2793 — source: `~/.global-ai-hub/research/.pipeline/mongodb-2026-10-14/frb/streaming-bson-parser-architecture-a92d73555b/rabbithole-synthesis.md#core` - [low] In `wirebson`, accessing an index out of bounds or encountering invalid BSON types causes a panic. Sources: https://pkg.go.dev/github.com/FerretDB/wire/wirebson — source: `~/.global-ai-hub/research/.pipeline/mongodb-2026-10-14/frb/streaming-bson-parser-architecture-a92d73555b/rabbithole-synthesis.md#troubleshooting` ## Problems, failure modes and limitations - [low] The `wirebson` `Equal` function panics if an invalid BSON type or decoding error is encountered, so it should not be used with user input. Sources: https://pkg.go.dev/github.com/FerretDB/wire/wirebson — source: `~/.global-ai-hub/research/.pipeline/mongodb-2026-10-14/frb/streaming-bson-parser-architecture-a92d73555b/rabbithole-synthesis.md#antipatterns` ## Facts and statements - [low] The `BSONEachRow` format parses data as a sequence of Binary JSON (BSON) documents without separators, where each row is a single document and each column is a BSON field with the column name as a key. Sources: https://clickhouse.com/docs/interfaces/formats/BSONEachRow — source: `~/.global-ai-hub/research/.pipeline/mongodb-2026-10-14/frb/streaming-bson-parser-architecture-a92d73555b/rabbithole-synthesis.md#core` - [low] The `BSONEachRow` format supports parsing big integers and decimals such as `Int128`, `UInt128`, `Int256`, `UInt256`, `Decimal128`, and `Decimal256` from BSON Binary values with the `\x00` subtype, validating that the binary data size equals the expected value size. Sources: https://clickhouse.com/docs/interfaces/formats/BSONEachRow — source: `~/.global-ai-hub/research/.pipeline/mongodb-2026-10-14/frb/streaming-bson-parser-architecture-a92d73555b/rabbithole-synthesis.md#core` - [low] The `wirebson` package implements BSON encoding and decoding, passing composite types (Document and Array) by pointers while raw composite types and scalars are passed by values. Sources: https://pkg.go.dev/github.com/FerretDB/wire/wirebson — source: `~/.global-ai-hub/research/.pipeline/mongodb-2026-10-14/frb/streaming-bson-parser-architecture-a92d73555b/rabbithole-synthesis.md#core` - [low] BSON data types include `int32` (4 bytes), `int64` (8 bytes), `double` (8 bytes), and `decimal128` (16 bytes). Sources: https://bsonspec.org/spec.html — source: `~/.global-ai-hub/research/.pipeline/mongodb-2026-10-14/frb/streaming-bson-parser-architecture-a92d73555b/rabbithole-synthesis.md#core` - [low] BSON binary subtypes include `unsigned_byte(0)` for generic binary, `unsigned_byte(4)` for UUID, `unsigned_byte(6)` for encrypted BSON, `unsigned_byte(8)` for sensitive data, and `unsigned_byte(9)` for vectors. Sources: https://bsonspec.org/spec.html — source: `~/.global-ai-hub/research/.pipeline/mongodb-2026-10-14/frb/streaming-bson-parser-architecture-a92d73555b/rabbithole-synthesis.md#core` - [low] Compressed BSON column data types use delta and delta-of-delta compression and run-length-encoding for efficient element storage. Sources: https://bsonspec.org/spec.html — source: `~/.global-ai-hub/research/.pipeline/mongodb-2026-10-14/frb/streaming-bson-parser-architecture-a92d73555b/rabbithole-synthesis.md#core` - [low] The `deserializeStream` method in some BSON libraries does not accept piped streams and still requires a buffer to be passed in. Sources: https://jira.mongodb.org/browse/NODE-2793 — source: `~/.global-ai-hub/research/.pipeline/mongodb-2026-10-14/frb/streaming-bson-parser-architecture-a92d73555b/rabbithole-synthesis.md#core` - [low] The `bsondump` tool converts BSON files into human-readable formats, including JSON, using Extended JSON v2.0 (Canonical mode) to format its data. Sources: https://www.mongodb.com/docs/database-tools/bsondump.md — source: `~/.global-ai-hub/research/.pipeline/mongodb-2026-10-14/frb/streaming-bson-parser-architecture-a92d73555b/rabbithole-synthesis.md#tools` - [low] By default, `bsondump` enables `--objcheck` to validate each BSON object before outputting it in JSON format; starting in MongoDB 9.0, this option has no effect on server behavior as `bsondump` always validates BSON objects. (verified as of 2026-10-15) Sources: https://www.mongodb.com/docs/database-tools/bsondump.md — source: `~/.global-ai-hub/research/.pipeline/mongodb-2026-10-14/frb/streaming-bson-parser-architecture-a92d73555b/rabbithole-synthesis.md#tools` ## Related concepts - BSON — is a part of Streaming BSON Parser Architecture