Support DeepSpeed FastGen

[thiner]

Is your feature request related to a problem? Please describe.

No.

Describe the solution you'd like

DeepSpeed FastGen is an inference framework developed by MicroSoft. They claim that it's two times faster than vllm. https://github.com/microsoft/DeepSpeed/tree/master/blogs/deepspeed-fastgen

Describe alternatives you've considered

No.

Additional context

I haven't tested FastGen, just attracted by their blog. I searched in this repo, seems no one mentioned this framework yet, so I'd like to bring it to the attention of community.

[thiner]

Glad to see you have added it to the roadmap.

[mudler]

Glad to see you have added it to the roadmap.

sounds a solid backend to have, thanks for the tip 👍 good to see that there is interest in this backend being added. Definetly a good addition for LocalAI