Repository navigation
Release v0.5.2
·
28 commits
to release/0.5
since this release
Highlight features
- Support Multi-modal models evaluation (VLM Eval)
- Transform the synchronous API to asynchronous for OpenAI's API format, speed up the evaluation process up to 10x .
- Support installation with format:
pip install evalscope[opencompass]orpip install evalscope[vlmeval]
Breaking Changes
None
What's Changed
- Support Multi-modal models evaluation (VLM Eval)
- Transform the synchronous API to asynchronous for OpenAI's API format, speed up the evaluation process up to 10x .
- Support installation with format:
pip install evalscope[opencompass]orpip install evalscope[vlmeval] - Update README
- Add UT cases for VLM eval
- Update examples for
OpenCompassandVLMEvaleval backends - Update version restrictions for ms-opencompass and ms-vlmeval dependencies.