Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for HuggingFace's TensorFlow models #127

Merged
merged 30 commits into from Jan 3, 2023

Conversation

valeriosofi
Copy link
Collaborator

@valeriosofi valeriosofi commented Dec 21, 2022

New Feature:

This PR implements support for HuggingFace's TensorFlow models, solving #125.
It also adds a notebook that shows a use case with BERT.

Example of usage:

from speedster import optimize_model
from transformers import TFAutoModel, AutoTokenizer

# Make sure to always load a matching tokenizer and model!
tokenizer = AutoTokenizer.from_pretrained("bert-base-cased")
model = TFAutoModel.from_pretrained("bert-base-cased")

# Let's load some data and tokenize it
test_string = "This is a sentence!"
tokenized_inputs = tokenizer(test_string, return_tensors="np")

dynamic_info = {
    "inputs": [
        {0: "batch", 1: "num_tokens"},
        {0: "batch", 1: "num_tokens"},
        {0: "batch", 1: "num_tokens"},
    ],
    "outputs": [{0: "batch", 1: "num_tokens"}, {0: "batch"}],
}

# Run Speedster optimization
optimized_model = optimize_model(
  model, input_data=[tokenized_inputs], dynamic_info=dynamic_info
)

Minor changes:

  • Added a warning when dynamic_info is not provided with HuggingFace models
  • The final report on the optimized model now contains also the name of the compiler used
  • Fix a tvm issue when using the PyTorch implementation with gpu
  • Added dynamic shape support for optimized models converted from TensorFlow to ONNX
  • Fixed an issue that caused a wrong output shape for the models optimized with the neural compressor
  • Enabled openVINO compiler when using python 3.10

Copy link
Collaborator

@nebuly-ai nebuly-ai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@diegofiori
Copy link
Collaborator

@valeriosofi why was the azure test cancelled?

@valeriosofi
Copy link
Collaborator Author

@valeriosofi why was the azure test cancelled?

Because there was a bug, so there was no reason to perform the tests, they would have failed

apps/accelerate/speedster/requirements.txt Outdated Show resolved Hide resolved
apps/accelerate/speedster/setup.py Outdated Show resolved Hide resolved
apps/accelerate/speedster/speedster/root_op.py Outdated Show resolved Hide resolved
@diegofiori
Copy link
Collaborator

Closes #125 and #103

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants