v1.0.0rc1

What's Changed

• Adapter plugin discovery by @PaulScemama in #25
• Fix small bug (doesn't change scores, only edge case verdict from invented_field to fabricated by @PaulScemama in #27
• Add absent kwarg to score to handle nulls in custom way. by @PaulScemama in #28
• order_matters can also be bool by @PaulScemama in #30
• Pytest runner by @PaulScemama in #31
• Consistent abstractions by @PaulScemama in #40

Full Changelog: v0.1.7...v1.0.0rc1

v0.1.7

What's Changed

• Docs + comments by @PaulScemama in #23

Full Changelog: v0.1.6...v0.1.7

v0.1.6

What's Changed

• Manifest path instead of --repo so you can use different manifests of single repo by @PaulScemama in #21
• fix test for CI by @PaulScemama in #22

Full Changelog: v0.1.5...v0.1.6

v0.1.5

What's Changed

• Simplify schema + prediction harness code by @PaulScemama in #13
• catch errors better for your own benchmark datasets by @PaulScemama in #14
• update llama extract in light of API changes by @PaulScemama in #17
• move collapse union to single caller - extend by @PaulScemama in #18
• bump to 0.1.5 by @PaulScemama in #20

Full Changelog: v0.1.4...v0.1.5

v0.1.4

What's Changed

• Default root to huggingface cache by @PaulScemama in #12

Full Changelog: v0.1.3...v0.1.4

v0.1.3

What's Changed

• big refactor; primitives are score() and predict() and you can run the benchmark with one cli oeb benchmark ... by @PaulScemama in #10

Full Changelog: v0.1.2...v0.1.3

Build fixes

v0.1.1

Bump version

Initial release

Initial benchmark release.

What's Changed

• Bring the public scorer to parity with the 660-document run by @VikParuchuri in #1
• Publish the harness that generated the predictions by @VikParuchuri in #2
• v0 by @PaulScemama in #3
• update readme by @PaulScemama in #4
• Ready the package for PyPI by @PaulScemama in #5
• Release on a version tag, and gate it on the tests by @VikParuchuri in #6

New Contributors

• @VikParuchuri made their first contribution in #1
• @PaulScemama made their first contribution in #3

Full Changelog: https://github.com/datalab-to/omni_extract_bench/commits/v0.1.0