New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Question] Onnx support in TGI #1873

Open

Ben-Epstein opened this issue May 9, 2024 · 0 comments

Ben-Epstein commented May 9, 2024

Feature request

Apologies if this should be elsewhere, but I'm curious if you plan on adding support for onnx models like https://huggingface.co/microsoft/Phi-3-mini-128k-instruct-onnx

Motivation

onnx is highly optimized for throughput, which aligns well with the goals of this repo

Your contribution

I would be happy to help contribute to this if needed!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment