Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

V1.2.2 release #129

Closed
wants to merge 4 commits into from
Closed

Conversation

yafshar
Copy link

@yafshar yafshar commented Apr 15, 2024

What does this PR do?

  • Add extra check to prevent failure on models (like mixtral) which do not have bypass_hpu_graphs optional arguments in the interface.
  • The PR has been tested successfully on

Fixes # (issue)

Before submitting

  • This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
  • Did you read the contributor guideline,
    Pull Request section?
  • Was this discussed/approved via a Github issue or the forum? Please add a link
    to it if that's the case.
  • Did you make sure to update the documentation with your changes? Here are the
    documentation guidelines, and
    here are tips on formatting docstrings.
  • Did you write any new necessary tests?

Who can review?

@kdamaszk

@yafshar
Copy link
Author

yafshar commented Apr 15, 2024

@kdamaszk, if you are working on this branch. I can help following on this PR

@yafshar
Copy link
Author

yafshar commented Apr 19, 2024

This PR has the solution for HPU Graph issue reported in #130

A temp solution to address overriding issue installing dill with habana
torch from gaudi-docker/1.15.0
- Having `import __main__ as _main_module` in the global space of the
  dill module causes some overriding issue on hpu graph destructor
@hsubramony
Copy link
Collaborator

hsubramony commented Apr 30, 2024

@yafshar did you try the env variable

ENABLE_HPU_GRAPH | True/False | True | Enable hpu graph or not

also please retry the issue on latest docker 1.15.1 + #134

@yafshar
Copy link
Author

yafshar commented May 1, 2024

@yafshar did you try the env variable

ENABLE_HPU_GRAPH | True/False | True | Enable hpu graph or not

For workloads which have an issue, disabling the hpu graph by env variable works. But it has the performance effect.

also please retry the issue on latest docker 1.15.1 + #134
The issue exists there as well.

One point is that #134 uses the merge from tgi_gaudi:habana-main which has the dill patch for dill 0.3.7, we should update that to 0.3.8 to have the correct dependencies.

@yafshar
Copy link
Author

yafshar commented May 1, 2024

@regisss, we can close this PR in favor of #134

@regisss regisss closed this May 1, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants