How to run InstructBLIP #313
Comments
|
Ok I'm a dumb dumb, this was discussed and apparently solved in another issue. |
|
In case anyone has trouble with this in the future, here are the steps I took to run InstructBLIP-7B: Download LlamaThis can be done quickly via import os
from transformers import LlamaTokenizer, LlamaForCausalLM
save_folder = "/path/to/llms/folder"
tokenizer = LlamaTokenizer.from_pretrained("decapoda-research/llama-7b-hf")
tokenizer.save_pretrained(os.path.join(save_folder, "tokenizer"))
model = LlamaForCausalLM.from_pretrained("decapoda-research/llama-7b-hf")
model.save_pretrained(os.path.join(save_folder, "model"))Note that we are manually saving both the tokenizer and model in a folder. This can also be achieved with the Download AlpacaNow that you have Llama locally, you'll need to add the delta weights to convert this into Alpaca. This is done installing FastChat and then following the Vicuna 7b instructions. When you follow FastChat's instructions make sure that Install LAVISYou need to install LAVIS manually. This won't work with the git clone https://github.com/salesforce/LAVIS.git
cd LAVIS
pip install -e .Now go into Use InstructBLIPimport torch
from lavis.models import load_model_and_preprocess
device = torch.device("cuda") if torch.cuda.is_available() else "cpu"
model, vis_processors, _ = load_model_and_preprocess(
name="blip2_vicuna_instruct",
model_type="vicuna7b",
is_eval=True,
device=device
)This code snippet should work, meaning you are ready to play with If you want to use the 13B version, follow the same steps but change |
|
nice, it works! |
|
Guys what size GPU's are you running it at? My does hangs at.. 'Loading model...'; I have a 3090 24GB. Should be enough, I hope? |
|
@Don-Chad I'm using a 3090 with 24GB too, and at least the 7B model worked flawlessly. I haven't tried with bigger ones though. |
|
I'm complete newbie on python, but, would it be possible you help me on running this on a bigger model… Or how can I run this, let's say I want to use falcon 30b model (which is amazing results as an llm chat) My goal is to be able to use a big llm to throw some questions about a specific image using blip2. |
|
Do you really have to go through all of those steps to get the instructBLIP to work? I'm confused why the huggingface model card doesn't list any of that or mention LAVIS. (https://huggingface.co/Salesforce/instructblip-vicuna-7b) Also, for 7b this seems like a huge download. Maybe I'm used to using the quantized models from TheBloke, but this seems to big to run on my 24g 3090. I can't say I fully understand what's going on here from a technical standpoint, but is there a way to quantize this? Is there an easier, more out-of-the-box way to get this up and running? |
|
Thank you very much for your explanation. After I followed the steps you mentioned and started using instructBLIP, there were two errors as follows: May I ask how to solve it |
|
Can I confirm that the yaml file to edit is: blip2_instruct_vicuna7b.yaml these files: are stored in this folder: so should I change: |
Hello!
I'm trying to run Vicuna InstructBLIP, but sadly, I can't make it work.
I installed LAVIS directly from your repo following the step 3 of the installation guide, and I'm using the following code:
With which I get the following error:
I guess this happens because inside the
blip2/blip2_instruct_vicuna7b.yamlconfigurationllm_modelis set to"./llm/vicuna-7b".I tried to fix this by manually downloading the model from the
pretrainedvalue present in saidyaml, and create a./llm/vicuna-7bfolder to store it but it still fails.Any ideas on how can I run this?
The text was updated successfully, but these errors were encountered: