-
Notifications
You must be signed in to change notification settings - Fork 5.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add support for Intel Gaudi Backend #40561
Conversation
Signed-off-by: Jerome <janand@habana.ai>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Lg
Signed-off-by: Jerome <janand@habana.ai>
Signed-off-by: Jerome Anand <88475913+jerome-habana@users.noreply.github.com>
Signed-off-by: Jerome <janand@habana.ai>
Signed-off-by: Jerome <janand@habana.ai>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Have you tested this on the machine with Gaudi?
Signed-off-by: Jerome <janand@habana.ai>
Signed-off-by: Jerome Anand <88475913+jerome-habana@users.noreply.github.com>
Signed-off-by: Jerome Anand <88475913+jerome-habana@users.noreply.github.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Last comment
Lint failure:
|
Signed-off-by: Jerome Anand <88475913+jerome-habana@users.noreply.github.com>
Signed-off-by: Jerome <janand@habana.ai>
Lint failure:
|
Signed-off-by: Jerome <janand@habana.ai>
Signed-off-by: Jerome <janand@habana.ai>
Signed-off-by: Jerome Anand <88475913+jerome-habana@users.noreply.github.com>
Signed-off-by: Jerome Anand <88475913+jerome-habana@users.noreply.github.com>
* Add Intel gaudi to accelerator list * Add check for backend initialization with updated test Signed-off-by: Jerome <janand@habana.ai>
Lint failure:
|
Signed-off-by: Jerome <janand@habana.ai>
might be nice to have auto corrector |
@@ -7,6 +7,7 @@ | |||
NVIDIA_TESLA_A10G = "A10G" | |||
INTEL_MAX_1550 = "Intel-GPU-Max-1550" | |||
INTEL_MAX_1100 = "Intel-GPU-Max-1100" | |||
INTEL_GAUDI = "Intel-GAUDI" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you also add INTEL_GAUDI2 here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sure. I've kept it generic for now. Lets update post closure of the right instance usage ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hi all, I'm working on LLM serving on Gaudi 2. Is Gaudi 2 not supported yet?
Signed-off-by: Jerome Anand <88475913+jerome-habana@users.noreply.github.com>
Added support for intel gaudi backend based on new interfaces defined in #40286
Why are these changes needed?
Related issue number
Checks
git commit -s
) in this PR.scripts/format.sh
to lint the changes in this PR.method in Tune, I've added it in
doc/source/tune/api/
under thecorresponding
.rst
file.