Skip to content

Add NeuronModel classes for inference on nlp tasks#45

Merged
JingyaHuang merged 32 commits intomainfrom
add-neuron-model-nlp
Jun 6, 2023
Merged

Add NeuronModel classes for inference on nlp tasks#45
JingyaHuang merged 32 commits intomainfrom
add-neuron-model-nlp

Conversation

@JingyaHuang
Copy link
Collaborator

@JingyaHuang JingyaHuang commented Apr 24, 2023

  • Add NeuronModel base class architecture.
  • Add NeuronModelForXXX subclasses for tasks: feature-extraction, fill-mask, multiple-choice, question-answering, text-classification, token-classification.
  • Support padding when input shapes is not equal to the compilation static shapes.
  • Add tests for modeling
  • Update precompiled examples to HF hub

Upcoming

  • Maybe cache for each export (so far neuron compiled models won't be cache unless it exits on the hub)
  • Some models have accuracy issues that need to further investigate (c.f comments in test_modeling.py)
  • All remaining tests passed for neuronx on INF2, need to test with INF1 and Trainium, might be some patches.
  • Add documentation for NeuronModel(s)

@HuggingFaceDocBuilderDev

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint.

@JingyaHuang JingyaHuang marked this pull request as ready for review June 4, 2023 11:11
Copy link
Contributor

@philschmid philschmid left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thats an awesome but big PR! Didn't went into detail on all of the code changes.
Can we add a documentation Page for similar to https://huggingface.co/docs/optimum/onnxruntime/usage_guides/models as a guide and
the classes as reference: https://huggingface.co/docs/optimum/onnxruntime/package_reference/modeling_ort

@JingyaHuang
Copy link
Collaborator Author

JingyaHuang commented Jun 5, 2023

Thats an awesome but big PR! Didn't went into detail on all of the code changes. Can we add a documentation Page for similar to https://huggingface.co/docs/optimum/onnxruntime/usage_guides/models as a guide and the classes as reference: https://huggingface.co/docs/optimum/onnxruntime/package_reference/modeling_ort

@philschmid I will add the doc in a separate PR as this one is huge enough.

@JingyaHuang
Copy link
Collaborator Author

Issue opened for logging performance problems that we found in this PR: #90

@JingyaHuang
Copy link
Collaborator Author

The suggestions are addressed, gently ping you for a final check then I am gonna to merge it. @michaelbenayoun

Copy link
Member

@michaelbenayoun michaelbenayoun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!
Huge work @JingyaHuang

@JingyaHuang JingyaHuang merged commit bdddd10 into main Jun 6, 2023
@JingyaHuang JingyaHuang deleted the add-neuron-model-nlp branch June 6, 2023 12:08
@philschmid
Copy link
Contributor

Congrats 👯‍♀️ @JingyaHuang

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants