Skip to content

Tags: EvolvingLMMs-Lab/lmms-eval

Tags

v0.3.5

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
Update pyproject.toml

v0.3.4

Update datasets dependency in pyproject.toml to allow for newer versi…

…ons (>=2.16.1).

v0.3.3

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
[Fix] add more model examples (#644)

* Update OpenAI compatibility script for Azure integration

- Set environment variables for Azure OpenAI API configuration.
- Modify model arguments to use the new GPT-4o model version and enable Azure OpenAI support.
- Clean up commented installation instructions for clarity.

* Refactor imports and clean up code in various utility files

- Consolidated import statements in `plm.py` and `utils.py` for better organization.
- Removed redundant blank lines in `eval_utils.py`, `fgqa_utils.py`, `rcap_utils.py`, `rdcap_utils.py`, `rtloc_utils.py`, and `sgqa_utils.py` to enhance readability.
- Ensured consistent import structure across utility files for improved maintainability.

* Update README and add example scripts for model evaluations

- Revised installation instructions to facilitate direct package installation from Git.
- Added detailed usage examples for various models including Aria, LLaVA, and Qwen2-VL.
- Introduced new example scripts for model evaluations, enhancing user guidance for running specific tasks.
- Improved clarity in environmental variable setup and common issues troubleshooting sections.

* Update README to reflect new example script locations and remove outdated evaluation instructions

- Changed paths for model evaluation scripts to point to the new `examples/models` directory.
- Added a note directing users to find more examples in the updated location.
- Removed outdated evaluation instructions for LLaVA on multiple datasets to streamline the documentation.

* Update README to reflect new script locations and enhance evaluation instructions

- Replaced outdated evaluation commands with new script paths in the `examples/models` directory.
- Updated sections for evaluating larger models, including the introduction of new scripts for tensor parallel and SGLang evaluations.
- Streamlined instructions for model evaluation to improve clarity and usability.

v0.3.2

[Fix] Regular Linting

- Added missing comma in AVAILABLE_MODELS for consistency.
- Reordered import statements in vora.py for better readability.
- Simplified input_data generation by condensing method calls into a single line.
- Ensured default generation parameters are set correctly in the VoRA class.

v0.3.1

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
Update README.md

v0.3.0

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
Merge pull request #432 from EvolvingLMMs-Lab/pufanyi/pypi_0.3.0

PyPI 0.3.0

v0.2.4

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
[Feat] Add support for evaluation of InternVideo2-Chat && Fix evaluat…

…ion for mvbench (#280)

* [add] add internvideo2 support && change mvbench to video branch

* [add] answer_prompt of internvideo2

* [add] change video type of internvideo2

* [fix] update template of mvbench

* [reformat]

* [fix] generate_until_multi_round

* [Feat] videochat2 support

---------

Co-authored-by: heyinan <heyinan@pjlab.org.cn>

v0.2.3.post1

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
Merge pull request #226 from EvolvingLMMs-Lab/dev/quick-fix-wandb

fix: fix wrong args in wandb logger

v0.2.3

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
Update README.md

v0.2.2

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
Merge pull request #184 from EvolvingLMMs-Lab/pufanyi/pypi/0.2.2

New pypi