Tool-calling benchmarks for local models

Facts

• BFCL V1 (2024) introduced AST matching plus executable checks over simple, multiple, parallel and parallel-multiple function calls in several languages. V2 (2024-08-19) added enterprise and OSS-contributed "live" data. V3 (2024-09-19) added multi-turn and multi-step. V4 adds holistic agentic evaluation: web search, memory (key-value, vector, recursive summarization) and format sensitivity. [source]
• The BFCL V4 table also carries hallucination columns (relevance, irrelevance), live and non-live single-turn columns, cost, and latency (mean, SD, P95). [source]
• Docker model-test scores F1 over three dimensions (tool invocation, tool selection, parameter accuracy), accepts several valid tool sequences per case, and runs an agent loop capped at 5 rounds. [source]
• BFCL leaderboard page last updated 2026-04-12 and is headed "From Tool Use to Agentic Evaluation of Large Language Models". [source]
• Docker's study (2025-06-30) predates Qwen3.5/3.6 and is the only Mac-run (M4 Max 128 GB) tool-calling comparison found across 21 models. [source]
• BFCL's overall score mixes agentic columns: a model can lead single-turn and still rank low overall. Qwen3-8B shows about 87-89 on non-live single turn but 12-13.5 on web search; xLAM-2-8b-fc-r shows 6.5 on web search yet 70 on multi-turn. [source]
• Scores for the same model appear in two rows (function-calling mode versus prompt mode), differing by several points; the rows can be mistaken for duplicates. [source]
• Docker found high BFCL-listed specialist models (xLAM-2-8b-fc-r, watt-tool-8B) failing a simple chat-cart task with eager invocation, wrong tool selection and bad arguments, and ranked them last (llama-xlam 0.570, watt-tool 0.484). [source]
• Specialist versus general: BFCL ranks xLAM-2-32b-fc-r (overall 54.66) above Qwen3-32B (48.71 and 46.78); Docker ranked llama-xlam:8B (0.570) far below qwen3:8B (0.919-0.933). Different task shapes (BFCL breadth versus a five-tool shopping loop) and neither averaged. [source]
• Bigger is not better in Docker's table: llama3.3:70B Q4_K_M scored 0.607, below qwen2.5:7B (0.753). [source]
• No benchmark found that runs BFCL or similar through llama-server, mlx-lm or Ollama on a Mac with the chat template and parser the agent will actually use. [source]
• Whether BFCL V4 multi-turn scores on a local endpoint depend on the harness's own prompt-mode handler rather than the server's tool parser. [source]
• BFCL V3 (released 2024-09-19) added multi-turn and multi-step function-calling evaluation. [source]
• BFCL V2 (released 2024-08-19) added enterprise-contributed "live" data. [source]
• The BFCL V4 leaderboard groups columns as Agentic (web search, memory), Multi Turn, Single Turn (non-live, live), Hallucination Measurement and Format Sensitivity, plus cost and latency. [source]
• BFCL V4 agentic memory is split into KV, Vector and Recursive Summarization backends. [source]
• The BFCL V4 leaderboard was last updated 2026-04-12; rank 1 was Claude-Opus-4-5-20251101 at 77.47 overall. [source]
• On that leaderboard Qwen3-8B scores 42.57 and 40.43 overall in its two listed rows, Qwen3-14B 41.03 and 37.77, Qwen3-32B 48.71 and 46.78, Qwen3-30B-A3B-Instruct-2507 41.39 and 36.70, Qwen3-4B-Instruct-2507 35.68 and 35.52. [source]
• On that leaderboard xLAM-2-8b-fc-r scores 46.68 overall with 6.5 on web search and 70 on multi-turn. [source]
• Docker's model-test measured 21 models over 3,570 test cases in 210 batch runs on a MacBook Pro M4 Max with 128 GB. [source]
• Docker's F1 ranking: gpt-4 0.974, qwen3:14B-Q4_K_M 0.971, qwen3:14B-Q6_K 0.943, qwen3:8B-F16 0.933, qwen3:8B-Q4_K_M 0.919, llama3.1:8B-F16 0.835, llama3.3:70B-Q4_K_M 0.607, llama-xlam:8B-Q4_K_M 0.570, watt-tool:8B-Q4_K_M 0.484. [source]
• Docker's model-test caps its simulated agent loop at 5 rounds and counts a case failed if the model still wants to continue. [source]
• Docker's model-test accepts multiple valid tool-call variants per case, for example direct add or search then add. [source]
• Docker concluded that even the Qwen3 8B variant beat every other local model it tested and that higher accuracy meant higher latency. [source]
• MCP-Bench reports overall scores of 0.678 for qwen3-235b-a22b-2507, 0.627 for qwen3-30b-a3b-instruct-2507 and 0.428 for llama-3-1-8b-instruct. [source]