有人成功了吗？

[Vector-Cross]

我试图用我的3060（12G）跑，内存不够，需要什么配置的卡才能成功呀，多卡行不行

我在huggingface上建立了一个空间，MiniGPT-4的docker部署，有没有成功的大佬来指点一下
https://huggingface.co/spaces/zylj/MiniGPT-4

[2132660698]

成功了，比较耗显存。

[Vector-Cross]

怎么17G显存就能跑呀，你这卡也太多了吧

[huangzhongzhong]

卡在准备模型这一步了

[TsuTikgiau]

@huangzhongzhong It looks like the name of your tokenizer is incorrect. It should be 'LlamaTokenizer' instead of 'LLaMATokenizer'. You can update it in the tokenizer config file ' tokenizer_config.json' in your llama weight folder

[Viewerand]

怎么17G显存就能跑呀，你这卡也太多了吧

int4量化了的应该是

[Kizai]

我运行报这个错误

[TsuTikgiau]

Hi @Kizai , from the error info I see the placeholder of vicuna weight '/path/to/vicuna/weights', that means the vicuna weight path is not set. Please check the Readme to know how to set it. Thanks!

[WangRongsheng]

@TsuTikgiau

1. I have set vicuna weights in minigpt4/configs/models/minigpt4.yaml
2. I have set minigpt4 weights in eval_configs/minigpt4_eval.yaml
3. How should the llama weights be configured to load?

[TsuTikgiau]

@WangRongsheng Hello! How do you prepare the vicuna weights? The vicuna weight from huggingface is a delta version and cannot be directly used. We provide a guide PrepareVicuna.md to show you how to prepare the final working vicuna weight. LLAMA weight is used only in this preparation. Thanks!

[Kizai]

I encountered the same problem as the first floor，I'm not worthy

[Vector-Cross]

I encountered the same problem as the first floor，I'm not worthy

yes, we are

[WangRongsheng]

@WangRongsheng Hello! How do you prepare the vicuna weights? The vicuna weight from huggingface is a delta version and cannot be directly used. We provide a guide PrepareVicuna.md to show you how to prepare the final working vicuna weight. LLAMA weight is used only in this preparation. Thanks!

good! I have solved it!

[Kizai]

very vague，numb

[2214962083]

Successfully run on windows 11! It took me 6 hours to complete, and I encountered many problems, which were finally solved.

My pc is rtx4090, and I use vicuna 13B. When uploading pictures and asking questions, it takes up 20G of video memory, and it takes about 5-30 seconds to wait for a reply.

[Vector-Cross]

我在huggingface上建立了一个空间，MiniGPT-4的docker部署，有没有成功的大佬来指点一下
https://huggingface.co/spaces/zylj/MiniGPT-4

[created-Bi]

楼主你好，请问LLaMA-13B的原始模型参数可以分享一下吗？十分感谢

[Vector-Cross]

楼主你好，请问LLaMA-13B的原始模型参数可以分享一下吗？十分感谢

我也没有申请到模型，但是我直接在hugging face上找了别人的，也不知道是不是原始模型参数
https://huggingface.co/decapoda-research

[HunterShenSmzh]

I encountered the same problem as the first floor，I'm not worthy

大佬你是如何解决repo_name问题的呀。我无论是使用相对路径还是绝对路径都会报错。附图：

[Gary-yeh]

I use rtx4090 on windows 11 as well, but I got error message
AssertionError: Torch not compiled with CUDA enabled

Loading VIT
Loading VIT Done
Loading Q-Former
Loading Q-Former Done
Loading LLAMA

============BUG REPORT===================================
Welcome to bitsandbytes. For bug reports, please submit your error trace to: https://github.com/TimDettmers/bitsandbytes/

Loading checkpoint shards: 0%| | 0/3 [00:08<?, ?it/s]
╭────── Traceback (most recent call last) ────────────────────────────────╮
│ C:\MiniGPT-4\demo.py:60 in
│ ****
│ 57 model_config = cfg.model_cfg
│ 58 model_config.device_8bit = args.gpu_id
│ 59 model_cls = registry.get_model_class(model_config.arch)
│ ❱ 60 model = model_cls.from_config(model_config).to('cuda:{}'.format(args.gpu_id))
│ 61
│ 62 vis_processor_cfg = cfg.datasets_cfg.cc_sbu_align.vis_processor.train
│ 63 vis_processor = registry.get_processor_class(vis_processor_cfg.name).from_config(vis_pro
│
│ C:\MiniGPT-4\minigpt4\models\mini_gpt4.py:243 in from_config
│
│ 240 max_txt_len = cfg.get("max_txt_len", 32)
│ 241 end_sym = cfg.get("end_sym", '\n')
│ 242
│ ❱ 243 model = cls(
│ 244 vit_model=vit_model,
│ 245 q_former_model=q_former_model,
│ 246 img_size=img_size,
│
│ C:\MiniGPT-4\minigpt4\models\mini_gpt4.py:90 in init
│
│ 87 self.llama_tokenizer.pad_token = self.llama_tokenizer.eos_token
│ 88
│ 89 if self.low_resource:
│ ❱ 90 self.llama_model = LlamaForCausalLM.from_pretrained(
│ 91 llama_model,
│ 92 torch_dtype=torch.float16,
│ 93 load_in_8bit=True,
│
│ C:\Users\User\anaconda3\envs\minigpt4\lib\site-packages\transformers\modeling_utils.py:2795 in │
│ from_pretrained
│
│ 2792 mismatched_keys,
│ 2793 offload_index, │
│ 2794 error_msgs, │
│ ❱ 2795 ) = cls._load_pretrained_model(
│ 2796 model,
│ 2797 state_dict,
│ 2798 loaded_state_dict_keys, # XXX: rename?
│
│ C:\Users\User\anaconda3\envs\minigpt4\lib\site-packages\transformers\modeling_utils.py:3124 in
│ _load_pretrained_model
│
│ 3121 )
│ 3122
│ 3123 if low_cpu_mem_usage:
│ ❱ 3124 new_error_msgs, offload_index, state_dict_index = _load_state_dict_i
│ 3125 model_to_load,
│ 3126 state_dict,
│ 3127 loaded_keys,
│
│ C:\Users\User\anaconda3\envs\minigpt4\lib\site-packages\transformers\modeling_utils.py:706 in
│ load_state_dict_into_meta_model
│
│ 703 fp16_statistics = None
│ 704
│ 705 if "SCB" not in param_name:
│ ❱ 706 set_module_8bit_tensor_to_device(
│ 707 model, param_name, param_device, value=param, fp16_statistics=fp16_s
│ 708 )
│ 709
│
│ C:\Users\User\anaconda3\envs\minigpt4\lib\site-packages\transformers\utils\bitsandbytes.py:87 in
│ set_module_8bit_tensor_to_device
│
│ 84 if value is None:
│ 85 new_value = old_value.to(device)
│ 86 elif isinstance(value, torch.Tensor):
│ ❱ 87 new_value = value.to(device)
│ 88 else:
│ 89 new_value = torch.tensor(value, device=device)
│ 90
│ C:\Users\User\anaconda3\envs\minigpt4\lib\site-packages\torch\cuda_init.py:239 in _lazy_init
│
│ 236 "Cannot re-initialize CUDA in forked subprocess. To use CUDA with "
│ 237 "multiprocessing, you must use the 'spawn' start method")
│ 238 if not hasattr(torch._C, '_cuda_getDeviceCount'):
│ ❱ 239 raise AssertionError("Torch not compiled with CUDA enabled")
│ 240 if _cudart is None:
│ 241 raise AssertionError(
│ 242 "libcudart functions unavailable. It looks like you have a broken build?
╰───────────────────────────────────────────
AssertionError: Torch not compiled with CUDA enabled

Can someone know how to fix it? Thanks

[WangRongsheng]

#81 maybe you can see it!☺️

[Vector-Cross]

I use rtx4090 on windows 11 as well, but I got error message AssertionError: Torch not compiled with CUDA enabled

Loading VIT Loading VIT Done Loading Q-Former Loading Q-Former Done Loading LLAMA

============BUG REPORT===================================

Welcome to bitsandbytes. For bug reports, please submit your error trace to: https://github.com/TimDettmers/bitsandbytes/
Loading checkpoint shards: 0%| | 0/3 [00:08<?, ?it/s] ╭────── Traceback (most recent call last) ────────────────────────────────╮ │ C:\MiniGPT-4\demo.py:60 in │ **** │ 57 model_config = cfg.model_cfg │ 58 model_config.device_8bit = args.gpu_id │ 59 model_cls = registry.get_model_class(model_config.arch) │ ❱ 60 model = model_cls.from_config(model_config).to('cuda:{}'.format(args.gpu_id)) │ 61 │ 62 vis_processor_cfg = cfg.datasets_cfg.cc_sbu_align.vis_processor.train │ 63 vis_processor = registry.get_processor_class(vis_processor_cfg.name).from_config(vis_pro │ │ C:\MiniGPT-4\minigpt4\models\mini_gpt4.py:243 in from_config │ │ 240 max_txt_len = cfg.get("max_txt_len", 32) │ 241 end_sym = cfg.get("end_sym", '\n') │ 242 │ ❱ 243 model = cls( │ 244 vit_model=vit_model, │ 245 q_former_model=q_former_model, │ 246 img_size=img_size, │ │ C:\MiniGPT-4\minigpt4\models\mini_gpt4.py:90 in init │ │ 87 self.llama_tokenizer.pad_token = self.llama_tokenizer.eos_token │ 88 │ 89 if self.low_resource: │ ❱ 90 self.llama_model = LlamaForCausalLM.from_pretrained( │ 91 llama_model, │ 92 torch_dtype=torch.float16, │ 93 load_in_8bit=True, │ │ C:\Users\User\anaconda3\envs\minigpt4\lib\site-packages\transformers\modeling_utils.py:2795 in │ │ from_pretrained │ │ 2792 mismatched_keys, │ 2793 offload_index, │ │ 2794 error_msgs, │ │ ❱ 2795 ) = cls._load_pretrained_model( │ 2796 model, │ 2797 state_dict, │ 2798 loaded_state_dict_keys, # XXX: rename? │ │ C:\Users\User\anaconda3\envs\minigpt4\lib\site-packages\transformers\modeling_utils.py:3124 in │ _load_pretrained_model │ │ 3121 ) │ 3122 │ 3123 if low_cpu_mem_usage: │ ❱ 3124 new_error_msgs, offload_index, state_dict_index = load_state_dict_i │ 3125 model_to_load, │ 3126 state_dict, │ 3127 loaded_keys, │ │ C:\Users\User\anaconda3\envs\minigpt4\lib\site-packages\transformers\modeling_utils.py:706 in │ load_state_dict_into_meta_model │ │ 703 fp16_statistics = None │ 704 │ 705 if "SCB" not in param_name: │ ❱ 706 set_module_8bit_tensor_to_device( │ 707 model, param_name, param_device, value=param, fp16_statistics=fp16_s │ 708 ) │ 709 │ │ C:\Users\User\anaconda3\envs\minigpt4\lib\site-packages\transformers\utils\bitsandbytes.py:87 in │ set_module_8bit_tensor_to_device │ │ 84 if value is None: │ 85 new_value = old_value.to(device) │ 86 elif isinstance(value, torch.Tensor): │ ❱ 87 new_value = value.to(device) │ 88 else: │ 89 new_value = torch.tensor(value, device=device) │ 90 │ C:\Users\User\anaconda3\envs\minigpt4\lib\site-packages\torch\cuda__init.py:239 in _lazy_init │ │ 236 "Cannot re-initialize CUDA in forked subprocess. To use CUDA with " │ 237 "multiprocessing, you must use the 'spawn' start method") │ 238 if not hasattr(torch._C, '_cuda_getDeviceCount'): │ ❱ 239 raise AssertionError("Torch not compiled with CUDA enabled") │ 240 if _cudart is None: │ 241 raise AssertionError( │ 242 "libcudart functions unavailable. It looks like you have a broken build? ╰─────────────────────────────────────────── AssertionError: Torch not compiled with CUDA enabled

Can someone know how to fix it? Thanks

It seems like pytorch version is cpu version. Maybe change torch version and try again can work.

[created-Bi]

乱码问题，这个要改一下Chrome编码方式？

[ljeff97]

乱码问题，这个要改一下Chrome编码方式？

我也都是乱码，想问下应该怎么解决呀

[WeileiZeng]

楼主你好，请问LLaMA-13B的原始模型参数可以分享一下吗？十分感谢

llamA weights

magnet:?xt=urn:btih:ZXXDAUWYLRUXXBHUYEMS6Q5CE5WA3LVA&dn=LLaMA

vicuna weight magnet

https://huggingface.co/lmsys/vicuna-13b-delta-v0 add to the header

magnet:?xt=urn:btih:a7fac57094561a63d53eed943f904abf24c6969d&dn=Vicuna-13B-HF-fp16-delta-merged_2023-04-03&tr=udp%3a%2f%2ftracker.opentrackr.o
rg%3a1337%2fannounce&tr=udp%2ftracker%2 -udp.gbitt.info%3a80%2fannounce&tr=udp%3a%2f%2ftracker1.bt.moack.co.kr%3a80%2fannounce&tr=udp%3a%2f%2f
tracker.tiny-vps.com%3a6969%2fannounce&tr=udp %3a%2f%2ftracker2.dler.org%3a80%2fannounce&tr=udp%3a%2f%2fopentracker.i2p.rocks%3a6969%2fannounc
e&tr=udp%3a%2f%2ftracker.altrosky.nl%3a6969%2fannounce&tr=udp%3a %2f%2ftracker.theoks.net%3a6969%2fannounce&tr=udp%3a%2f%2ftracker.dler.org%3a
6969%2fannounce&tr=udp%3a%2f%2ftracker.torrent.eu.org%3a451%2fannounce&tr=udp%3a %2f%2ftracker.openbittorrent.com%3a6969%2fannounce&tr=https%3
a%2f%2fopentracker.i2p.rocks%3a443%2fannounce&tr=http%3a%2f%2ftracker.openbittorrent.com%3a80%2fannounce&tr=udp%3a%2f%2ftracker.moeking.me%3a6
969%2fannounce&tr=udp%3a%2f%2ftracker.monitorit4.me%3a6969%2fannounce&tr=udp%3a%2f%2f9.rarbg.com% 3a2810%2fannounce

magnet:?xt=urn:btih:1e0c3dbeefe82483f81bd4e7ea959e4953c8081f&dn=Vicuna-13B-ggml-4bit-delta-merged_2023-04-03&tr=udp%3a%2f%2ftracker.opentrackr
.org%3a1337%2fannounce&tr=udp%2fa%2 .rarbg.com%3a2810%2fannounce&tr=udp%3a%2f%2ftracker.monitorit4.me%3a6969%2fannounce&tr=udp%3a%2f%2ftracker
2.dler.org%3a80%2fannounce&tr=udp%3a%2f%2fopentracker.i2p .rocks%3a6969%2fannounce&tr=udp%3a%2f%2ftracker.torrent.eu.org%3a451%2fannounce&tr=u
dp%3a%2f%2ftracker.openbittorrent.com%3a6969%2fannounce&tr=udp%3a%2f%2ftracker.dler .org%3a6969%2fannounce&tr=udp%3a%2f%2ftracker-udp.gbitt.in
fo%3a80%2fannounce&tr=udp%3a%2f%2ftracker1.bt.moack.co.kr%3a80%2fannounce&tr=https%3a%2f %2fopentracker.i2p.rocks%3a443%2fannounce&tr=udp%3a%2
f%2ftracker.altrosky.nl%3a6969%2fannounce&tr=udp%3a%2f%2ftracker.tiny-vps.com%3a6969%2fannounce&tr=udp%3a%2f%2ftracker.moeking.me%3a6969%2fann
ounce&tr=http%3a%2f%2ftracker.openbittorrent.com%3a80%2fannounce&tr=udp%3a%2f%2ftracker.theoks.net% 3a6969%2fannounce

[WangRongsheng]

乱码问题，这个要改一下Chrome编码方式？

可能你的权重加载错误，你可以参考 #81

[mayyalove]

使用的Windows 上面的docker跑的吗？我的会报错

[cnxupupup]

乱码问题，这个要改一下Chrome编码方式？

同样乱码，我配置的是7B的模型，请问如何解决？

[chwshuang]

window10下的经验：
git clone https://github.com/Vision-CAIR/MiniGPT-4.git
cd MiniGPT-4
conda env create -f environment.yml
conda activate minigpt4
pip install torch==1.13.1+cu117 torchvision==0.14.1+cu117 torchaudio==0.13.1 --extra-index-url https://download.pytorch.org/whl/cu117
pip uninstall bitsandbytes
pip install https://github.com/jllllll/bitsandbytes-windows-webui/releases/download/wheels/bitsandbytes-0.39.1-py3-none-win_amd64.whl
其中有2个模型配置修改，和模型权重下载，可以参考：https://github.com/rbbrdckybk/MiniGPT-4
run python demo.py --cfg-path eval_configs/minigpt4_eval.yaml --gpu-id 0