Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

building context takes too long #110

Open
simplelifetime opened this issue Jun 13, 2024 · 1 comment
Open

building context takes too long #110

simplelifetime opened this issue Jun 13, 2024 · 1 comment
Labels
question Further information is requested

Comments

@simplelifetime
Copy link

Almost takes 10 minutes to build context for MMMU, I wonder if it's a problem with huggingface connection
zjysteven pushed a commit to zjysteven/lmms-eval that referenced this issue Jun 13, 2024
@Luodian
[WIP] check video datasets, in progress (EvolvingLMMs-Lab#110)
a81e30b 
* Refactor process_results function to handle full_docs in videochatgpt task

* Refactor models/__init__.py to add "Reka" model

* chore: Add "Llava_OneVision" model to lmms_eval/models/__init__.py

* update onevision model interface.

* Fix error message when importing claude in lmms_eval/models/claude.py

* Refactor lmms_eval/models/llava_vid.py for safer object type handling in llama_3 conv_template

* Refactor llava_vid.py to handle pad_token_ids for "llama_3" conv_template

* Refactor lmms_eval/api/task.py to update datasets.config for streaming read retries and intervals
@Luodian Luodian added the question Further information is requested label Jun 15, 2024
@jzhang38
Copy link
Contributor

It does not take that long on my end.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
question Further information is requested
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@Luodian @jzhang38 @simplelifetime