v1.0.0

What's Changed

• Integrate ExtractBench extraction benchmark by @boyang-zhang1 in #1
• Strip Datalab _meta sidecars from extracted_data by @boyang-zhang1 in #3
• feat: add gemini-3.7-flash configs (and 3.6) by @markmcd in #4
• fix nuextract3 inference/template conversion by @Alex-C137E in #6
• Update NuExtract3 leaderboard row after converter fix by @boyang-zhang1 in #8
• Intern JSON list and object cells in the array-record cost matrix by @xujustinj in #7
• Run extend_extract_max via Extend async API by @boyang-zhang1 in #9
• Add NuExtract-2.0-8B self-hosted vLLM extract pipeline by @boyang-zhang1 in #10
• Add InternVL3.5-8B self-hosted vLLM extract pipeline by @boyang-zhang1 in #11
• Add InternVL3.5-14B self-hosted vLLM extract pipeline by @boyang-zhang1 in #12
• Add MiniCPM-V-4.5 self-hosted vLLM extract pipeline by @boyang-zhang1 in #13
• Add Qwen3.5-35B-A3B self-hosted vLLM extract pipeline by @boyang-zhang1 in #14
• Add Qwen3.5-9B self-hosted vLLM extract pipeline by @boyang-zhang1 in #15
• Land the full open-weight stack on main by @boyang-zhang1 in #29
• Add GLM-5.3 Flash extraction pipeline by @boyang-zhang1 in #30
• Align GLM-5.3 Flash leaderboard name by @boyang-zhang1 in #31
• Add Reducto Extract leaderboard result by @boyang-zhang1 in #32
• Add Extend Extract leaderboard result by @boyang-zhang1 in #33
• Add Qwen3.8-Flash-Next-FP8 one-shot extract pipeline by @boyang-zhang1 in #34
• Score nested object fields as individual extract cells by @xujustinj in #35
• Add Gemini 3.8 Flash extract one-shot pipeline by @boyang-zhang1 in #36
• Add GPT-6 Astra extract one-shot pipeline by @boyang-zhang1 in #37
• Score nested arrays, pairing cost, and schema defaults in extract value F1 by @xujustinj in #38
• Grade wrapped extract citations against coarse evidence envelopes by @xujustinj in #39
• Remove extract metrics that never score on published gold by @xujustinj in #41
• Add DeepSeek one-shot extract pipeline by @boyang-zhang1 in #43
• fix: Align the extract scorer with the internal fixes by @logan-markewich in #45
• Port loader, dataset lint, score_case and confidence coverage by @logan-markewich in #46
• Score evaluator failures and make the eval stall timeout fire by @logan-markewich in #47
• Port provider and inference fixes from the internal harness by @logan-markewich in #48
• Package extract-bench for PyPI by @logan-markewich in #49
• Add the parse-bench extension points and extra symbols by @logan-markewich in #53
• Add Pulse schema pipelines and benchmark results by @morgan-pulse in #42
• Refresh three LlamaExtract leaderboard scores by @boyang-zhang1 in #54
• Fix Pulse citation field paths by @boyang-zhang1 in #55
• Rename package by @logan-markewich in #56
• feat: final alignment fixes before publishing by @logan-markewich in #57
• Add evidence extract pipelines and refresh leaderboard by @boyang-zhang1 in #58
• test: cover the extract metric entry point for downstream harnesses by @logan-markewich in #59
• perf: normalize extract comparison cells once per cell, not once per pair by @logan-markewich in #60
• Add Claude Code Opus 5.5 extract pipelines and leaderboard result by @boyang-zhang1 in #63
• Add Codex GPT-6 Sol and Luna evidence pipelines and leaderboard rows by @boyang-zhang1 in #64
• Pair over-cap array-record rows inside identity buckets by @xujustinj in #62
• Use listed Anthropic prices for Claude parse and codegen cost by @boyang-zhang1 in #66
• Add Claude Code Sonnet 5.5 evidence pipeline; pass large schemas as a file by @boyang-zhang1 in #67
• Restore LlamaExtract cost columns by @boyang-zhang1 in #68
• Update Reducto Deep Extract leaderboard row by @boyang-zhang1 in #69
• Add Reducto Extract v4 and update Reducto rows by @boyang-zhang1 in #70
• Revert "Restore LlamaExtract cost columns" by @boyang-zhang1 in #71

New Contributors

• @markmcd made their first contribution in #4
• @Alex-C137E made their first contribution in #6
• @xujustinj made their first contribution in #7
• @logan-markewich made their first contribution in #45
• @morgan-pulse made their first contribution in #42

Full Changelog: https://github.com/run-llama/ExtractBench/commits/v1.0.0