Unitxt 1.26.7
What's Changed
- Fix examples by @elronbandel in #1907
- Fix inference tests by @elronbandel in #1912
- Minor text2sql metric fixes by @oktie in #1913
- fix mtrag by @dafnapension in #1918
- fix xlam's schema issues by @dafnapension in #1917
- Add ReflectionToolCallingMetricSyntactic for evaluating tool call predictions referenceless by @korenLazar in #1923
- Divide biggen bench into multigual and not multilingual by @martinscooper in #1922
- remove redundant split from airbench2024 by @dafnapension in #1928
- Revert BigGen Benchmark partition by @martinscooper in #1924
- fixed spit names in wiki_bio by @dafnapension in #1925
- Fix erroneous prompts in evaluation tasks (and clean some json-schema-wise) by @dafnapension in #1920
- fix the only 4 erroneous global_mmlu cards that do not pass _source_to_dataset by @dafnapension in #1916
- Normalize llm judge bench target variable by @martinscooper in #1933
- Improved multi turn evaluation to be self contained and use LLM as judge by @yoavkatz in #1929
- Add more RAG judges by @arielge in #1934
- Add ReflectionToolCallingMetric and update related metrics by @korenLazar in #1931
- potential fix for preparation file: prepare/cards/mtrag.py by @dafnapension in #1938
- Lazy load vectara hhem model because it is gated in HF by @yoavkatz in #1946
- Fixed missing sampling_seed in DiverseLabelsSampler by @yoavkatz in #1941
- Correct reflection based tool calling metrics so valid results will be 1. by @yoavkatz in #1940
- Rag metric update again by @dafnapension in #1948
- fix gpt-oss classification inference engines by @lilacheden in #1952
- Update version to 1.26.7 by @elronbandel in #1953
New Contributors
- @korenLazar made their first contribution in #1923
Full Changelog: 1.26.6...1.26.7