Weights not initializing for opt

[Hritikpatel]

I'm using opt facebook/opt-6.7b model on my CPU and I'm getting this line in CLI and I'm not getting any response for prompt. What should I do?

Some weights of OPTForCausalLM were not initialized from the model checkpoint at .\bentoml\models\pt-facebook-opt-6-7b\a45aa65bbeb77c1558bc99bedc6779195462dab0 and are newly initialized: ['lm_head.weight'] You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.

[aarnphm]

This warning can be ignored.

I'm using opt facebook/opt-6.7b model on my CPU and I'm getting this line in CLI and I'm not getting any response for prompt. What should I do?

Can you send the traceback here and what the terminal shows?

[Hritikpatel]

C:\Users\qwert>openllm start opt --model-id facebook/opt-6.7b
2023-08-15T11:36:24+0530 [INFO] [cli] Prometheus metrics for HTTP BentoServer from "_service.py:svc" can be accessed at http://localhost:3000/metrics.
2023-08-15T11:36:26+0530 [INFO] [cli] Starting production HTTP BentoServer from "_service.py:svc" listening on http://0.0.0.0:3000 (Press CTRL+C to quit)
'NoneType' object has no attribute 'cadam32bit_grad_fp32'
Loading checkpoint shards: 100%|████████████████████████████████████████████████████████| 2/2 [04:14<00:00, 127.04s/it]
Some weights of OPTForCausalLM were not initialized from the model checkpoint at C:\Users\qwert\bentoml\models\pt-facebook-opt-6-7b\a45aa65bbeb77c1558bc99bedc6779195462dab0 and are newly initialized: ['lm_head.weight']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.

[Denys120]

Thank you guys

[aarnphm]

I belive this is not a bug, as this warning can be safely ignored. Feel free to reopen if you run into different problem.