Search results

ValueError: Couldn't instantiate the backend tokenizer from one of:

Status: Open.

#101 In huggingface/transformers-bloom-inference;

· SeekPoint opened on Jun 30, 2023

It does not work with Falcon-40B correctly

Status: Open.

#100 In huggingface/transformers-bloom-inference;

· AGrosserHH opened on Jun 23, 2023

How to understand this note: "note: Since Deepspeed-ZeRO can process multiple generate streams in parallel its throughput can be further divided by 8 or 16 ..."

Status: Open.

#99 In huggingface/transformers-bloom-inference;

· HuipengXu opened on Jun 14, 2023

When deploying the Bloom model, I noticed that the POST method is used for the generation task. Is it possible to modify it to perform question-answering instead?

Status: Open.

#97 In huggingface/transformers-bloom-inference;

· dizhenx opened on Jun 5, 2023

The Makefile execution was successful, but there is no response when entering text.

Status: Open.

#96 In huggingface/transformers-bloom-inference;

· dizhenx opened on Jun 2, 2023

AttributeError: 'BloomForCausalLM' object has no attribute 'module'

Status: Open.

#95 In huggingface/transformers-bloom-inference;

· detectiveJoshua opened on Jun 1, 2023

Are there fine-tuning and inference scripts available for int4 quantization in bloom-7b? Is it possible to limit the GPU memory usage to within 10GB?

Status: Open.

#94 In huggingface/transformers-bloom-inference;

· dizhenx opened on May 31, 2023

Inference(chatbot) does not work as expected on 2 gpus with bigscience/bloom-7b1 model

Status: Open.

#90 In huggingface/transformers-bloom-inference;

· dantalyon opened on May 19, 2023

pip install command does not work as expected

Status: Open.

#88 In huggingface/transformers-bloom-inference;

· Billijk opened on May 9, 2023

Big batchsize cause OOM in bloom-ds-inference.py, how to adjust max_split_size_mb value

Status: Open.

#84 In huggingface/transformers-bloom-inference;

· tohneecao opened on Apr 27, 2023

root_dir in TemporaryCheckpointsJSON is redundant

Status: Open.

#82 In huggingface/transformers-bloom-inference;

· dc3671 opened on Apr 24, 2023

[Bug] Int8 quantize inference failed using bloom-inference-scripts/bloom-ds-inference.py with deepspeed==0.9.0 on multi-gpus

Status: Open.

#77 In huggingface/transformers-bloom-inference;

· hanrui1sensetime opened on Apr 17, 2023