v0.1.3

Scripts:

• Reverted the top_k=20 / presence_penalty=1.5 tuning for phi-4-mini-reasoning after it caused infinite thinking, restoring previous decoding params 6cb32ea)
• Added the deepseek_r1 reasoning-parser flag to the microsoft/Phi-4-mini-reasoning vLLM serving script (49c2d36)
• Added scripts/compile_final_results.py to compile experiment results across runs into a single CSV (1d7d908)

Docs:

• Added the generate-release-notes skill (a4cf71c)
• Added Claude Code skills support via .claude/skills (e88a5b1)
• Added test-quality and test-coverage skills (d4a52ad)

v0.1.2

Scripts:

• Set top_k=20 and presence_penalty=1.5 for phi-4-mini-reasoning eval to stop the model from thinking indefinitely (9f78682)

Build & Packaging:

• The package is now published to PyPI/TestPyPI via a new release workflow, so it can be installed with pip install virex-bench (3f53d2f)
• Version numbers are now derived automatically from git tags via hatch-vcs — no more manual bumping (a2b1d3b)
• Serving dependencies are split into optional vllm and sglang extras, and torch/torchvision/transformers moved into the serving group, so a
  plain install stays lightweight and pulls those only when you need to serve models (b94cd87, 54c2051)
• Declared support for Python up to 3.14.x (579bb9d)

Docs:

• Added a Zenodo DOI badge to the README (bad6805)

v0.1.2rc2

Full Changelog: v0.1.2rc1...v0.1.2rc2

v0.1.2rc1

Full Changelog: v0.1.1...v0.1.2rc1

v0.1.1

Features:

• Support specifying a custom self-hosted judge model via the hosted_vllm/ prefix (hosted_vllm/deepseek-v4-pro, hosted_vllm/deepseek-v4-flash), backed by a new VIREX_BENCH_JUDGE_BASE_URL env var that points the LLM-as-a-judge metric at an OpenAI-compatible serving endpoint (d940208)

Scripts:

• Qwen3-4B → Qwen3-4B-Instruct-2507 in the SGLang serving script (f044801)
• Add eval scripts for large models: Gemma 4 (26b-a4b-it, 31b-it), Qwen3.5 (9b, 27b, 35b-a3b, 122b-a10b-fp8), Qwen3.6 (27b, 35b-a3b) (af994c1)
• Add serving script for phi-4-mini-reasoning (vLLM + SGLang) (5ae9969)
• Add vLLM serving scripts for NVIDIA-Nemotron-3-Nano-4B-BF16, NVIDIA-Nemotron-3-Super-120B-A12B-FP8, and NVIDIA-Nemotron-3-Nano-30B-A3B-BF16 (b5a8931, b09a304, d9c2662)
• Add eval scripts for the Nemotron-3 family and phi-4-mini-reasoning (e17b525)
• Add SGLang serving script for Qwen3.5-122b-a10b-fp8 and update the Nemotron-3-Super-120B vLLM script (6f9e158)

Fixes:

• Set EXTRA_PARAMS in the large-model eval scripts (Gemma 4, Qwen3.5/3.6 variants) (eef0bc7)
• Fall back an empty-string API key to <empty> across the run CLI, judge LM builder, and model backends, since some providers reject an empty API key (1e4938b)

Docs:

• Correct the README usage guide for VIREX_BENCH_JUDGE_MODEL (41700d6)

v0.1.0

Initial release of ViREx-Bench.