Evalution v0.0.17

What's Changed

• Automatically optimize transformer loglikelihood scoring by @Qubitium in #142

Full Changelog: v0.0.16...v0.0.17

Evalution v0.0.16

Full Changelog: v0.0.15...v0.0.16

What's Changed

• docs: refresh README with latest release notes and engine updates by @Qubitium in #140
• add ZML/LLMD inference engine by @Qubitium in #141

Full Changelog: v0.0.14...v0.0.16

Evalution v0.0.14

What's Changed

• feat: add agentic benchmark scaffolds including PoolSide Laguna S 2.1 suites and Docker isolation by @Qubitium in #135
• feat: add AgentRuntimeConfig with Docker and smolvm agent runtimes by @Qubitium in #137
• feat: generic runtime opts and native tool-call compat for GLM/Laguna/MiniMax by @Qubitium in #138
• Sync llama.cpp and tinygrad stable runtimes by @Qubitium in #139

Full Changelog: v0.0.13...v0.0.14

Evalution v0.0.13

What's Changed

• feat: include package versions in evaluation results by @ZX-ModelCloud in #133
• Update pyproject.toml by @Qubitium in #134

Full Changelog: v0.0.12...v0.0.13

Evalution v0.0.12

What's Changed

• fix(sglang): reuse engine loop for continuous generation by @ZX-ModelCloud in #132

Full Changelog: v0.0.11...v0.0.12

Evalution v0.0.11

What's Changed

• fix: use locally cached dataset when stream=True by @Qubitium in #130
• Update pyproject.toml by @Qubitium in #131

Full Changelog: v0.0.10...v0.0.11

Evalution v0.0.10

What's Changed

• fix(engines/base): default use_cuda_graph=False for paged attention continuous batching by @Qubitium in #129

Full Changelog: v0.0.9...v0.0.10

Evalution v0.0.9

What's Changed

• Preserve model EOS and chat-template defaults by @Qubitium in #128

Full Changelog: v0.0.8...v0.0.9

Evalution v0.0.8

What's Changed

• [CI] split tests into 2 groups, fix 255 limit by @CSY-ModelCloud in #116
• add maturin requirement by @CSY-ModelCloud in #117
• fix 4090 score by @CSY-ModelCloud in #118
• [CI] refactor ci. sync logic/codes from gptqmodel by @CSY-ModelCloud in #119
• [CI] don't clean whl for uv cache by @CSY-ModelCloud in #120
• [CI] use new docker image by @CSY-ModelCloud in #121
• [CI] all use cuda 130 docker image by @CSY-ModelCloud in #122
• increase timing, fix flaky failures by @CSY-ModelCloud in #124
• fix old transformers has a different func name by @CSY-ModelCloud in #125
• fix 2 positional arguments but 3 were given by @CSY-ModelCloud in #126
• Fix cb compat for Transformers 5.14+ by @Qubitium in #127

Full Changelog: v0.0.7...v0.0.8

Evalution v0.0.7

Notable Changes:

• Add Tinygrad LLM to Engines by @Qubitium in #112

What's Changed

• [CI] refactor ci & fix streaming tests by @CSY-ModelCloud in #110
• [CI] install device smi & fix build_server_info has no arg by @CSY-ModelCloud in #111
• Update pyproject.toml by @Qubitium in #113
• Update Transformers fa launch monkeypatch and fold rogue scorer into pkg by @Qubitium in #114
• Update ci scores by @Qubitium in #115

Full Changelog: v0.0.6...v0.0.7