Skip to content

v0.5.12

Choose a tag to compare

View all tags
@yifanmai yifanmai released this 28 Jan 00:35
· 196 commits to main since this release
v0.5.12
23a2e13
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Models

  • Add Kimi K2 Instruct 0905 (#3955)
  • Add DeepSeek v3.1 (#3956)
  • Add Cohere Labs Command A (#3957)
  • Allow using Together models without configuration (#3980)
  • Allow using LiteLLM models without configuration (#3981, #3982)
  • Allow using Hugging Face Hub models without configuration (#3983)
  • Remove tokenizer from OpenAIResponseClient (#4020)

Scenarios

  • Update schema, model metadata and scenario metadata for HELM Arabic (#3959)
  • Set subject to all by default for MMLU-Pro (#3975)
  • Add missing language entry for MMLU-Winogrande-Afr (#3973)
  • Fix scenario description in ThaiExam (#3972)
  • Support newer versions of MedCalc-Bench (#3921)
  • Fix broken BoolQ file download URLs (#4039)

Framework

  • Allow omitting the tokenizer and disabling request truncation (#3974)
  • Support plugins that use --plugin or project.entry-points.helm (#3916)

Contributors

Thank you to the following contributors for your work on this HELM release!

Contributors

yifanmai, Erotemic, and 4 other contributors
Assets 2
Loading