chore: add optimum-neuron 0.0.18 image #55
Conversation
|
@dacorvo Could you please also modify |
Done |
| @@ -21,7 +21,7 @@ | |||
| { | |||
| "device": "inf2", | |||
| "min_version": "0.0.16", | |||
| "max_version": "0.0.17", | |||
| "max_version": "0.0.18", | |||
There was a problem hiding this comment.
Please also include 0.0.18 as a release candidate. I.e. put it into "releases"
There was a problem hiding this comment.
OK: I did not know what to put there, so I did what was done in the other pull-request for TGI 1.4.0. Hope I did it right.
| RUN pip3 install ./optimum-neuron | ||
|
|
||
| # TGI base env | ||
| ENV HUGGINGFACE_HUB_CACHE=/data \ |
There was a problem hiding this comment.
Sorry - another comment. This passes since optimum seems to be ok with being model data being downloaded to a smaller file system for the test case we have. But this might be a concern if there is a larger model that does not fit into \data. Could you please change to \tmp? cc. @nicolas, @david Holtz, @philschmid
dacorvo
force-pushed
the
optimum-0.0.18
branch
from
8f69b78
to
7e4df73
Compare
Description of changes:
This adds a new docker image for Neuron X TGI based on
optimum-neuron0.0.18.