Skip to content

Issues: microsoft/DeepSpeed-MII

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

175 Open 107 Closed
175 Open 107 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

[QUERY] Expert Parallelism Supported?
#498 opened Jun 27, 2024 by Shamauk
Attempting to flush sequence N which does not exist
#497 opened Jun 24, 2024 by aagontuk
Compute perplexity
#496 opened Jun 21, 2024 by Sh1gechan
Configure server log level
#495 opened Jun 19, 2024 by sedletsky-f5
@loadams
2
few questions regarding the implementation of streaming and batching
#494 opened Jun 14, 2024 by KimMinSang96
[BUG] MII Backend Hangs After 9999 Exceptions in MIIAsyncPipeline.put_request
#489 opened Jun 5, 2024 by weiqisun
1
support stream
#488 opened Jun 3, 2024 by ZZhangxian
support Qwen1.5
#487 opened Jun 3, 2024 by ZZhangxian
Support LLava next stronger
#483 opened May 20, 2024 by thesby
How can I use the same prompt to produce the same text output as vllm
#482 opened May 19, 2024 by Greatpanc
Tf32 support
#481 opened May 17, 2024 by Chasapas
DeepSpeed-MII 能加载量化的int4或者int8的模型吗？
#479 opened May 16, 2024 by wangyongpenga
Does deepspeed-mii support prefix_allowed_tokens_fn?
#477 opened May 14, 2024 by zcakzhuu
[REQUEST] LLAMA-3 support
#475 opened May 13, 2024 by MRYingLEE
[REQUEST] Mixtral-8x22B support
#474 opened May 11, 2024 by y-live-koba
Cannot run Yi-34B-Chat => ValueError: Unsupported q_ratio: 7
#472 opened May 9, 2024 by joeking11829
2
BUG in run_batch_processing
#471 opened May 8, 2024 by zhihui96
ValueError: Unsupported model type phi3
#469 opened Apr 30, 2024 by abpani
error when using Qwen1.5-32B
#468 opened Apr 29, 2024 by puppet101
Performance with vllm
#467 opened Apr 26, 2024 by littletomatodonkey
Only running one replica even though setting many replicas
#465 opened Apr 24, 2024 by thesby
RuntimeError: The server socket has failed to listen on any local network address
#464 opened Apr 19, 2024 by thesby
1
[FEATURE] Access to logits and final hidden layer
#463 opened Apr 17, 2024 by lshamis
1
How is the prompt segmentation specifically implemented for Dynamic SplitFuse? Is there any code implement or code snippet ？
#462 opened Apr 17, 2024 by wenyangchou
How do I launch the api on a graphics card other than cuda: 0
#460 opened Apr 15, 2024 by Stark-zheng
1
ProTip! What’s not been updated in a month: updated:<2024-06-02.