Added support for Tapas Model #520

JuheonChu · 2022-11-27T09:57:30Z

What does this PR do?

This PR adds BetterTransformerAPI for Tapas model.

Fixes #20372

Before submitting

This PR adds new support for BetterTransformer integration for the Tapas model.
This PR adds documentation that indicates BetterTransofrmer integration for Tapas is added.

Questions

Can I ask you how I can test to add Bettertransformer feature for Tapas Model?

To: @younesbelkada, @sgugger

JuheonChu · 2022-11-27T09:58:37Z

Do you mind if I ask you whether I am on the right track?
Any comment or feedback will be very appreciated.

…r-Tapas

…hub.com/JuheonChu/optimum into Add-Bettertransformer-support-for-Tapas

michaelbenayoun

For me it looks like you did everything well!
Left a few nits and comments.
@younesbelkada will be back on Wednesday, so we should wait for his approval, but great work!

tests/bettertransformer/test_bettertransformer_encoder.py

optimum/bettertransformer/models/encoder_models.py

michaelbenayoun · 2022-11-28T10:57:27Z

optimum/bettertransformer/models/__init__.py

@@ -23,6 +23,8 @@
    ViTLayerBetterTransformer,
    Wav2Vec2EncoderLayerBetterTransformer,
    WhisperEncoderLayerBetterTransformer,
+    TapasLayerBetterTransformer,
+


Suggested change

HuggingFaceDocBuilderDev · 2022-11-28T11:18:58Z

The documentation is not available anymore as the PR was closed or merged.

test_better_encoder Co-authored-by: Michael Benayoun <mickbenayoun@gmail.com>

Co-authored-by: Michael Benayoun <mickbenayoun@gmail.com>

…r-Tapas

JuheonChu · 2022-11-28T13:55:57Z

Thanks for your suggestion and feedback! I reformatted with black, and can I proceed with the CircleCI test one more time to check if my reformatting file works?

sgugger

Thanks for your contribution! @younesbelkada good to merge if you are satisfied.

tests/bettertransformer/test_bettertransformer_encoder.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

fxmarty

Waiting for the tests to run but it looks good to me!

fxmarty · 2022-11-28T15:28:32Z

Apparently a test is not passing: https://github.com/huggingface/optimum/actions/runs/3566116329/jobs/5992236636

You can run them with pytest tests/bettertransformer/test_XXX.py -k "name_of_test" --exitfirst -s locally to see if a specific test passes.

Can you run as well make style for the code quality?

…hub.com/JuheonChu/optimum into Add-Bettertransformer-support-for-Tapas

JuheonChu · 2022-11-28T16:33:19Z

@fxmarty I applied make style command and made commits with it!
Thanks for your suggestion and I will try to test it out based on your suggestion.
I sincerely appreciate with your guidance.

JuheonChu · 2022-11-28T17:16:35Z

It seems to me that module packages in tests/bettertransformer/test_bettertransformer_encoder.py are importing all fail. However, I would not like to change the structure of a file system based on the suggestion from here. May I ask you for any suggestions?

Thank you for your time.

optimum/bettertransformer/models/encoder_models.py

Co-authored-by: fxmarty <9808326+fxmarty@users.noreply.github.com>

optimum/bettertransformer/models/encoder_models.py

Call super() in the _init() to inherit from BetterTransformerBaseLayer Co-authored-by: fxmarty <9808326+fxmarty@users.noreply.github.com>

…r-Tapas

younesbelkada

Thanks so much for your efforts @JuheonChu !
I left few comments, I think that the conversion is currently failing because the keys that you are trying to retrieve (for eg tapas_layer.attention.query.weight does not match, here you should use tapas_layer.attention.self.query.weight instead. Check the detailed modeling file here. )
One way to make sure that your conversion worked is to constantly run this snippet and debug further 💪 :

from transformers import AutoModel
from optimum.bettertransformer import BetterTransformer

model_id = "hf-internal-testing/tiny-random-TapasModel"
model = AutoModel.from_pretrained(model_id)
bt_model = BetterTransformer.transform(model)

Thanks and let us know if you face into any issue!

younesbelkada · 2022-11-29T09:30:42Z

tests/bettertransformer/test_bettertransformer_encoder.py

@@ -49,6 +49,7 @@
 ALL_ENCODER_DECODER_MODELS_TO_TEST = [
    "hf-internal-testing/tiny-random-FSMTModel",
    "hf-internal-testing/tiny-random-BartModel",
+    "hf-internal-testing/tiny-random-TapasModel",


Could you move this to ALL_ENCODER_MODELS_TO_TEST instead? Tapas is an encoder-only model ;)

Reading https://huggingface.co/docs/transformers/v4.24.0/en/model_doc/tapas#transformers.TapasModel

The model can behave as an encoder (with only self-attention) as well as a decoder, in which case a layer of cross-attention is added between the self-attention layers, following the architecture described in Attention is all you need by Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser and Illia Polosukhin.

Oh I see! Thanks for an insightful explanation! I just moved it! :)

tests/bettertransformer/test_bettertransformer_encoder.py

fxmarty · 2022-11-29T09:36:30Z

optimum/bettertransformer/models/encoder_models.py

+        self.in_proj_weight = nn.Parameter(
+            torch.cat(
+                [
+                    tapas_layer.attention.query.weight,


looking at the Tapas implementation, https://github.com/huggingface/transformers/blob/28247e78819ab9756b81f8df39611c333d099400/src/transformers/models/tapas/modeling_tapas.py#L442 , I think we need here and in the rest tapas_layer.attention.self.query.weight, that is why the test is failing!

Oh........ thanks you for providing me with your insights!

fxmarty · 2022-11-29T09:42:30Z

Actually, looking at https://github.com/huggingface/transformers/blob/28247e78819ab9756b81f8df39611c333d099400/src/transformers/models/tapas/modeling_tapas.py#L539

I think the TapasLayerBetterTransformer is not needed. I think we simply need to use the mapping "TapasLayer": BetLayerBetterTransformer,

What do you think @younesbelkada ?

Co-authored-by: Younes Belkada <49240599+younesbelkada@users.noreply.github.com>

younesbelkada · 2022-11-29T11:00:50Z

yes probably, thanks for the heads up @fxmarty !
@JuheonChu , you can also modify the mapping with "TapasLayer": BertLayerBetterTransformer and let us know if the conversion works fine!

JuheonChu · 2022-11-29T11:22:06Z

Just made changes accordingly! @younesbelkada Thank you so much! I will try the conversions and let you know!
Do you mind if you can check the commit above is the one you intended to say?

fxmarty · 2022-11-29T13:17:21Z

optimum/bettertransformer/models/__init__.py

@@ -19,6 +19,7 @@
    BertLayerBetterTransformer,
    DistilBertLayerBetterTransformer,
    FSMTEncoderLayerBetterTransformer,
+    TapasLayerBetterTransformer,


To be removed

optimum/bettertransformer/models/__init__.py

optimum/bettertransformer/models/encoder_models.py

fxmarty · 2022-11-29T13:18:48Z

Seems all good now @JuheonChu ! Thank you for this PR! So in the end using BertLayerBetterTransformer was enough to work with Tapas. If you can just address the points above, and we are ready to merge :)

JuheonChu · 2022-11-29T13:26:43Z

Thank you so much, everyone!! @fxmarty @younesbelkada @sgugger @michaelbenayoun.
I just made changes accordingly! @fxmarty.
Do you mind if you can double check if my commits are correct as you intended?

optimum/bettertransformer/models/__init__.py

fxmarty · 2022-11-29T13:49:48Z

optimum/bettertransformer/models/encoder_models.py

@@ -17,7 +17,6 @@
 from .base import BetterTransformerBaseLayer


-class TapasLayerBetterTransformer(BetterTransformerBaseLayer):
    def __init__(self, tapas_layer, config):


I meant to remove the class TapasLayerBetterTransformer in full 😅 It is not needed in the end since BertLayerBetterTransformer can handle Tapas.

I think I removed class TapasLayerBetterTransformer. Is there anything to be adjusted in this file?

On your branch, with optimum dev installed (pip install -e . in the optimum top folder), can you try to run the script @younesbelkada provided you and see what happens?

from transformers import AutoModel from optimum.bettertransformer import BetterTransformer model_id = "hf-internal-testing/tiny-random-TapasModel" model = AutoModel.from_pretrained(model_id) bt_model = BetterTransformer.transform(model)

younesbelkada · 2022-11-30T10:27:19Z

Hey @JuheonChu !
I have pushed directly on your branch ;) The tests should now pass 💪

younesbelkada

Thanks a lot @JuheonChu for adding Tapas support for BetterTransformer! Great job!
Looking forward to your next contributions 💪

michaelbenayoun

LGTM!

JuheonChu · 2022-12-10T03:58:49Z

@younesbelkada May I ask you where I can check the merged BetterTransformer feature of Tapas model?

Added support for Tapas Model

7939e13

JuheonChu and others added 3 commits November 27, 2022 06:07

Merge branch 'huggingface:main' into Add-Bettertransformer-support-fo…

2db0d45

…r-Tapas

Added support for Tapas Model

283186a

Merge branch 'Add-Bettertransformer-support-for-Tapas' of https://git…

b874831

…hub.com/JuheonChu/optimum into Add-Bettertransformer-support-for-Tapas

michaelbenayoun reviewed Nov 28, 2022

View reviewed changes

This was referenced Nov 28, 2022

Community contribution - BetterTransformer integration for more models! huggingface/transformers#20372

Closed

Community contribution - BetterTransformer integration for more models! #488

Open

Juheon Chu and others added 4 commits November 28, 2022 08:50

reformatted files with black

7c24a42

Update tests/bettertransformer/test_bettertransformer_encoder.py

12d6154

test_better_encoder Co-authored-by: Michael Benayoun <mickbenayoun@gmail.com>

Update optimum/bettertransformer/models/encoder_models.py

14b35ca

Co-authored-by: Michael Benayoun <mickbenayoun@gmail.com>

Merge branch 'huggingface:main' into Add-Bettertransformer-support-fo…

9632f98

…r-Tapas

sgugger approved these changes Nov 28, 2022

View reviewed changes

tests/bettertransformer/test_bettertransformer_encoder.py Outdated Show resolved Hide resolved

Update tests/bettertransformer/test_bettertransformer_encoder.py

d2e6ebb

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

fxmarty approved these changes Nov 28, 2022

View reviewed changes

Juheon Chu added 2 commits November 28, 2022 11:25

Styled optimum files

762a804

Merge branch 'Add-Bettertransformer-support-for-Tapas' of https://git…

ba1079e

…hub.com/JuheonChu/optimum into Add-Bettertransformer-support-for-Tapas

fxmarty reviewed Nov 28, 2022

View reviewed changes

optimum/bettertransformer/models/encoder_models.py Outdated Show resolved Hide resolved

Update optimum/bettertransformer/models/encoder_models.py

4047267

Co-authored-by: fxmarty <9808326+fxmarty@users.noreply.github.com>

fxmarty reviewed Nov 28, 2022

View reviewed changes

optimum/bettertransformer/models/encoder_models.py Outdated Show resolved Hide resolved

JuheonChu and others added 2 commits November 28, 2022 16:39

Update optimum/bettertransformer/models/encoder_models.py

ac716cc

Call super() in the _init() to inherit from BetterTransformerBaseLayer Co-authored-by: fxmarty <9808326+fxmarty@users.noreply.github.com>

Merge branch 'huggingface:main' into Add-Bettertransformer-support-fo…

add72dd

…r-Tapas

younesbelkada reviewed Nov 29, 2022

View reviewed changes

fxmarty reviewed Nov 29, 2022

View reviewed changes

JuheonChu and others added 2 commits November 29, 2022 04:52

Update tests/bettertransformer/test_bettertransformer_encoder.py

bbd625e

Co-authored-by: Younes Belkada <49240599+younesbelkada@users.noreply.github.com>

Moved Tapas Encoder model to Encoder

fded9ac

change mapping in __init_.py

7fcc616

fxmarty reviewed Nov 29, 2022

View reviewed changes

optimum/bettertransformer/models/__init__.py Outdated Show resolved Hide resolved

fxmarty reviewed Nov 29, 2022

View reviewed changes

optimum/bettertransformer/models/encoder_models.py Outdated Show resolved Hide resolved

JuheonChu added 2 commits November 29, 2022 08:21

deleted

579f889

Update __init__.py

fe547eb

fxmarty reviewed Nov 29, 2022

View reviewed changes

optimum/bettertransformer/models/__init__.py Outdated Show resolved Hide resolved

fxmarty reviewed Nov 29, 2022

View reviewed changes

JuheonChu and others added 2 commits November 29, 2022 08:56

Update __init__.py

1d64709

refactor doc + remove class + styling

64e644c

younesbelkada approved these changes Nov 30, 2022

View reviewed changes

younesbelkada requested a review from michaelbenayoun November 30, 2022 10:30

michaelbenayoun approved these changes Nov 30, 2022

View reviewed changes

michaelbenayoun merged commit d33801d into huggingface:main Nov 30, 2022

miyu386 mentioned this pull request Apr 12, 2023

[BT] Added BT for Speech2Text model #969

Open

3 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Added support for Tapas Model #520

Added support for Tapas Model #520

JuheonChu commented Nov 27, 2022 •

edited

JuheonChu commented Nov 27, 2022 •

edited

michaelbenayoun left a comment

michaelbenayoun Nov 28, 2022

HuggingFaceDocBuilderDev commented Nov 28, 2022 •

edited

JuheonChu commented Nov 28, 2022

sgugger left a comment

fxmarty left a comment

fxmarty commented Nov 28, 2022 •

edited

JuheonChu commented Nov 28, 2022

JuheonChu commented Nov 28, 2022 •

edited

younesbelkada left a comment •

edited

younesbelkada Nov 29, 2022

fxmarty Nov 29, 2022

JuheonChu Nov 29, 2022

fxmarty Nov 29, 2022

JuheonChu Nov 29, 2022

fxmarty commented Nov 29, 2022

younesbelkada commented Nov 29, 2022

JuheonChu commented Nov 29, 2022 •

edited

fxmarty Nov 29, 2022

fxmarty commented Nov 29, 2022

JuheonChu commented Nov 29, 2022 •

edited

fxmarty Nov 29, 2022

JuheonChu Nov 29, 2022

fxmarty Nov 29, 2022

younesbelkada commented Nov 30, 2022

younesbelkada left a comment

michaelbenayoun left a comment

JuheonChu commented Dec 10, 2022

Added support for Tapas Model #520

Added support for Tapas Model #520

Conversation

JuheonChu commented Nov 27, 2022 • edited

What does this PR do?

Before submitting

Questions

JuheonChu commented Nov 27, 2022 • edited

michaelbenayoun left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

HuggingFaceDocBuilderDev commented Nov 28, 2022 • edited

JuheonChu commented Nov 28, 2022

sgugger left a comment

Choose a reason for hiding this comment

fxmarty left a comment

Choose a reason for hiding this comment

fxmarty commented Nov 28, 2022 • edited

JuheonChu commented Nov 28, 2022

JuheonChu commented Nov 28, 2022 • edited

younesbelkada left a comment • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

fxmarty commented Nov 29, 2022

younesbelkada commented Nov 29, 2022

JuheonChu commented Nov 29, 2022 • edited

Choose a reason for hiding this comment

fxmarty commented Nov 29, 2022

JuheonChu commented Nov 29, 2022 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

younesbelkada commented Nov 30, 2022

younesbelkada left a comment

Choose a reason for hiding this comment

michaelbenayoun left a comment

Choose a reason for hiding this comment

JuheonChu commented Dec 10, 2022

JuheonChu commented Nov 27, 2022 •

edited

JuheonChu commented Nov 27, 2022 •

edited

HuggingFaceDocBuilderDev commented Nov 28, 2022 •

edited

fxmarty commented Nov 28, 2022 •

edited

JuheonChu commented Nov 28, 2022 •

edited

younesbelkada left a comment •

edited

JuheonChu commented Nov 29, 2022 •

edited

JuheonChu commented Nov 29, 2022 •

edited