Grammar-aware draft filtering for ngram and draft-model drafters
Parent: Mac local LLMs: Speculative decoding and MTP · Published reference · snapshot 2026-10-05
↓ Facts as markdownall context files
speculative-decoding-with-structured-output-gram.md already holds verifier-side grammar handling and its asserted statement that no grammar-aware draft filtering exists; this file adds the drafter-side evidence for that statement.
These notes link each claim to its source. A source may be a research report hosted on this site rather than the primary document. A published reference means the content is available; it does not certify independent review or accuracy.Read the editorial policy and follow the sources before relying on a claim.
Facts
- speculative-decoding-with-structured-output-gram.md already holds verifier-side grammar handling and its asserted statement that no grammar-aware draft filtering exists; this file adds the drafter-side evidence for that statement. [source]
- In `common/speculative.cpp` the draft-model drafter builds its per-sequence samplers with `top_k = 10` and `samplers = { TOP_K }` only, with no grammar field set, so the drafter proposes tokens without any grammar mask. [source]
- The draft sampler rebuild for rejection-style verification uses `samplers = { TOP_K, TEMPERATURE }` with top_k 10 at the target's temperature and a seed XORed with 0x85ebca6b, and again sets no grammar. [source]
- `common/speculative.cpp` registers five n-gram drafter types by name: ngram-simple, ngram-map-k, ngram-map-k4v, ngram-mod and ngram-cache; the file includes ngram-cache.h, ngram-map.h and ngram-mod.h. [source]
- A grep of the file for grammar or GBNF finds no match, so neither the n-gram drafters nor the draft-model drafter filter proposals against a grammar; filtering happens only at target verification. [source]
Children
- No children recorded.