Skip to content
This repository was archived by the owner on Oct 9, 2024. It is now read-only.

Issues: huggingface/transformers-bloom-inference

Clear current search query, filters, and sorts
20 Open 44 Closed
20 Open 44 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

does this work for llama 65B
#98 by GradientGuru was closed Jul 31, 2023
1
Can I combine fastertransformer to make it faster
#93 by xsj4cs was closed May 31, 2023
1
ds_inference success but OOM when use tp_presharded_mode=True
#92 by LiuShixing was closed Jun 1, 2023
2
accelerate in bloom-inference-scripts?
#91 by jeromeku was closed May 23, 2023
1
Bloom176B RuntimeError: expected scalar type Half but found BFloat16
#89 by wohenniubi was closed Jun 9, 2023
3
question regarding the float16 and bfloat
#87 by allanj was closed May 10, 2023
1
why no use deepspeed.init_inference in zero benchmark
#86 by tingshua-yts was closed May 31, 2023
2
Unable to reload a quantized model
#85 by moonlightian was closed May 10, 2023
4
The details of hf-accelerate pp.
#83 by tohneecao was closed May 20, 2023
2
BUILD ERROR with nvcc
#81 by tohneecao was closed May 10, 2023
3
Can not generate text correctly after loading an int8 model
#80 by moonlightian was closed Jul 8, 2023
4
Why does ds-inference int8 run slower than ds-inference fp16?
#79 by DominickZhang was closed May 10, 2023
3
tokenizer.json 乱码怎么解析
#78 by hongshengxin was closed May 10, 2023
1
concurrent requests
#75 by ustclan was closed Apr 7, 2023
2
Is there a way to initialize a random weight for
#74 by PannenetsF was closed Apr 19, 2023
3
beam search
#73 by syp1997 was closed Apr 19, 2023
2
Short response for bloom inferring
#70 by raihan0824 was closed Mar 29, 2023
3
Should I use bf16 or fp16?
#69 by richarddwang was closed Mar 23, 2023
3
"bloom-ds-zero-inference.py" works but "inference_server.cli --deployment_framework ds_zero" fails
#68 by richarddwang was closed Jun 17, 2024
4
how to run the server?
#64 by raihan0824 was closed Mar 29, 2023
2
Distributed Training using the same loading method
#61 by ananda1996ai was closed Mar 14, 2023
2
OOM of CUDA when using one GPU
#60 by xiongjun19 was closed May 31, 2023
4
Why is the throughput of DS-inference doubled when using 4 A100 GPUs compared to 8 A100 GPUs
#59 by DominickZhang was closed Apr 6, 2023
3
Max tokens generated remains constant for whatever the input token size
#55 by vamsikrishnav was closed Feb 21, 2023
2
How do build a web api for deepspeed inference
#52 by vamsikrishnav was closed Feb 17, 2023
22
ProTip! no:milestone will show everything without a milestone.