-
Notifications
You must be signed in to change notification settings - Fork 460
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[feat] Add finetune code for Yi-VL model #368
base: main
Are you sure you want to change the base?
Conversation
Prerequistes -> Prerequisites
* [doc][feat] modified readme_CN. * [doc][feat] modified readme_CN. --------- Co-authored-by: YShow <66633207+Yimi81@users.noreply.github.com>
@minlik Thank you for your PR, I will test it |
* [doc][feat] modified readme. * [doc][feat] modified readme. * [doc][feat] modified readme. * [doc][feat] modified readme. * [doc][feat] modified readme. * [doc][feat] modified readme.
Can you provide your environment? Both the official configuration of llava and the requirements. txt you provided reported errors. It would be great if you could provide a step in readme to reproduce your environment
|
I have updated the requirements in the latest commit here. I suggest you reinstall the requirements.txt under VL folder, and these errors should be resolved.
I have added PYTHONPATH=../../:$PYTHONPATH in the training scripts to ensure python environment
I use transformers==4.34.0 because transformers removed ShardedDDPOption since 4.35. I will fix it later, but for now, 4.34.0 is okay
I think the error is related to the torch version and cuda version. You may need to reinstall some packages following here. Some of my environment is listed here, with cuda version 11.7 accelerate==0.26.1 |
my sh script:
error:
How did you specify a vision_tower? Are you the same as me? |
Hello, I've been trying to run the finetune script as per the instructions in VL/scripts/finetune.sh, but I keep encountering an error that I haven't been able to resolve. The script fails to execute properly, and the error seems to originate from the llama_flash_attn_monkey_patch.py file, specifically lines 87 to 89: Thank you. |
I set the parameter |
I think it is related to the accelerate==0.26.1 |
could you provide a sample of you training data? Is the data format same as llava and just change <image> to <image_placeholder> ? |
Yes. See here |
你好,最新的微调代码哪里可以看到呢? |
The code is mostly modified from LLaVA