Issues: huggingface/text-generation-inference
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Author
Label
Projects
Milestones
Assignee
Sort
Issues list
Logging has no formating when using docker enviroment instead of command
#1880
opened May 11, 2024 by
onel
1 of 4 tasks
text generation details not working when stream=False
#1876
opened May 10, 2024 by
uyeongkim
2 of 4 tasks
How to share memory among 2 GPUS for distributed inference?
#1875
opened May 10, 2024 by
martinigoyanes
how do I adjust the logging level when launching via the docker container?
#1872
opened May 8, 2024 by
bitsofinfo
2 of 4 tasks
llama3-70B-Instruct-AWQ causing CUDA error: an illegal memory access was encountered
#1871
opened May 8, 2024 by
anindya-saha
4 tasks
Cannot use Inference Endpoint: UnprocessableEntityError: Error code: 422 - {'error': 'Template error: template not found', 'error_type': 'template_error'}
#1870
opened May 8, 2024 by
rvoak
1 of 4 tasks
"docker run --gpus all --shm-size 1g -p 8080:80 -v $volume:/data -e HUGGING_FACE_HUB_TOKEN={your_token} ghcr.io/huggingface/text-generation-inference:latest --model-id $model --num-shard $num_shard" showing error with my token id that "Unable to find image 'ghcr.io/huggingface/text-generation-inference:latest' locally latest: Pulling from huggingface/text-generation-inference docker: no matching manifest for linux/arm64/v8 in the manifest list entries. See 'docker run --help'."
#1868
opened May 7, 2024 by
anushka192001
4 tasks
Use pre-built FA2, vllm, quantization kernels in the dockerfiles
#1867
opened May 7, 2024 by
fxmarty
Encounter install error when install vllm package.
#1862
opened May 6, 2024 by
for-just-we
2 of 4 tasks
Serverless inference API endpoints fails to return logprobs via chat completions
#1852
opened May 2, 2024 by
ggbetz
2 of 4 tasks
UserWarning: You are using a Backend <class 'text_generation_server.utils.dist.FakeGroup'> as a ProcessGroup. This usage is deprecated since PyTorch 2.0
#1847
opened May 2, 2024 by
fxmarty
2 of 4 tasks
Previous Next
ProTip!
Type g p on any issue or pull request to go back to the pull request listing page.