Releases · run-llama/ExtractBench

What's Changed

Integrate ExtractBench extraction benchmark by @boyang-zhang1 in #1
Strip Datalab _meta sidecars from extracted_data by @boyang-zhang1 in #3
feat: add gemini-3.7-flash configs (and 3.6) by @markmcd in #4
fix nuextract3 inference/template conversion by @Alex-C137E in #6
Update NuExtract3 leaderboard row after converter fix by @boyang-zhang1 in #8
Intern JSON list and object cells in the array-record cost matrix by @xujustinj in #7
Run extend_extract_max via Extend async API by @boyang-zhang1 in #9
Add NuExtract-2.0-8B self-hosted vLLM extract pipeline by @boyang-zhang1 in #10
Add InternVL3.5-8B self-hosted vLLM extract pipeline by @boyang-zhang1 in #11
Add InternVL3.5-14B self-hosted vLLM extract pipeline by @boyang-zhang1 in #12
Add MiniCPM-V-4.5 self-hosted vLLM extract pipeline by @boyang-zhang1 in #13
Add Qwen3.5-35B-A3B self-hosted vLLM extract pipeline by @boyang-zhang1 in #14
Add Qwen3.5-9B self-hosted vLLM extract pipeline by @boyang-zhang1 in #15
Land the full open-weight stack on main by @boyang-zhang1 in #29
Add GLM-5.3 Flash extraction pipeline by @boyang-zhang1 in #30
Align GLM-5.3 Flash leaderboard name by @boyang-zhang1 in #31
Add Reducto Extract leaderboard result by @boyang-zhang1 in #32
Add Extend Extract leaderboard result by @boyang-zhang1 in #33
Add Qwen3.8-Flash-Next-FP8 one-shot extract pipeline by @boyang-zhang1 in #34
Score nested object fields as individual extract cells by @xujustinj in #35
Add Gemini 3.8 Flash extract one-shot pipeline by @boyang-zhang1 in #36
Add GPT-6 Astra extract one-shot pipeline by @boyang-zhang1 in #37
Score nested arrays, pairing cost, and schema defaults in extract value F1 by @xujustinj in #38
Grade wrapped extract citations against coarse evidence envelopes by @xujustinj in #39
Remove extract metrics that never score on published gold by @xujustinj in #41
Add DeepSeek one-shot extract pipeline by @boyang-zhang1 in #43
fix: Align the extract scorer with the internal fixes by @logan-markewich in #45
Port loader, dataset lint, score_case and confidence coverage by @logan-markewich in #46
Score evaluator failures and make the eval stall timeout fire by @logan-markewich in #47
Port provider and inference fixes from the internal harness by @logan-markewich in #48
Package extract-bench for PyPI by @logan-markewich in #49
Add the parse-bench extension points and extra symbols by @logan-markewich in #53
Add Pulse schema pipelines and benchmark results by @morgan-pulse in #42
Refresh three LlamaExtract leaderboard scores by @boyang-zhang1 in #54
Fix Pulse citation field paths by @boyang-zhang1 in #55
Rename package by @logan-markewich in #56
feat: final alignment fixes before publishing by @logan-markewich in #57
Add evidence extract pipelines and refresh leaderboard by @boyang-zhang1 in #58
test: cover the extract metric entry point for downstream harnesses by @logan-markewich in #59
perf: normalize extract comparison cells once per cell, not once per pair by @logan-markewich in #60
Add Claude Code Opus 5.5 extract pipelines and leaderboard result by @boyang-zhang1 in #63
Add Codex GPT-6 Sol and Luna evidence pipelines and leaderboard rows by @boyang-zhang1 in #64
Pair over-cap array-record rows inside identity buckets by @xujustinj in #62
Use listed Anthropic prices for Claude parse and codegen cost by @boyang-zhang1 in #66
Add Claude Code Sonnet 5.5 evidence pipeline; pass large schemas as a file by @boyang-zhang1 in #67
Restore LlamaExtract cost columns by @boyang-zhang1 in #68
Update Reducto Deep Extract leaderboard row by @boyang-zhang1 in #69
Add Reducto Extract v4 and update Reducto rows by @boyang-zhang1 in #70
Revert "Restore LlamaExtract cost columns" by @boyang-zhang1 in #71

No results found