Add TFBartForConditionalGeneration #5411

sshleifer · 2020-06-30T19:52:10Z

adds TFBartForConditionalGeneration, which can generate summaries that are equivalent to pytorch.

TODO this PR:

Future PRs:

blender/pegasus/mBART/marian etc.
BART/TFBart: allow decoder_input_ids.shape[-1] > 1 + use_cache = True #7814

LysandreJik

Great, very clean! The testing suite is impressive.

There are only a few items left to do before we merge:

Could you add TFBartModel and TFBartForConditionalGeneration to the auto models?
Please remove the "defaults to :obj:None" in docstrings, we don't do that anymore
Please make sure that docstrings are not larger than 119 characters like we do in the rest of the repo. You can just add a line return and ensure the following line is at the same indentation level to make sphinx happy.
Most assertions do not have an error message. Please add messages so that they're easier to debug for users and for us.

Also, you seem to have added all the if/else statement corresponding to mBART, Pegasus and Blenderbot. Have you tried using these models with this code? If it works, doing PR on the port of all of these models in TF would be big!

src/transformers/modeling_tf_bart.py

LysandreJik · 2020-10-16T08:19:48Z

src/transformers/modeling_tf_bert.py

-            Check out the :meth:`~transformers.PreTrainedModel.from_pretrained` method to load the model weights.
+            Check out the :meth:`~transformers.TFPreTrainedModel.from_pretrained` method to load the model weights.
 """


Nice catch 👌

LysandreJik · 2020-10-16T08:21:55Z

tests/test_modeling_bart.py

+
+@require_torch
+# @slow
+class FastIntegrationTests(unittest.TestCase):


Very cool test!

tests/test_modeling_tf_bart.py

LysandreJik · 2020-10-16T08:23:38Z

tests/test_modeling_tf_bart.py

+    def test_compile_tf_model(self):
+        # This passes for TFBartForConditionalGeneration, fails for TFBartModel
+        pass


Why does it fail? Compilation seems like something necessary in TF

This passes for TFBartForConditionalGeneration.

To make it pass for TFBartModel, the decoder and encoder need to always return Tuple and a bunch of other hacks @patrickvonplaten had to add for T5 (like https://github.com/huggingface/transformers/blob/master/src/transformers/modeling_tf_t5.py#L1149)

Since so few people use BartModel directly, I decided that supporting this compilation case (that would likely never be used) was not worth losing the readability benefit of ModelOutputs and also adding 30 lines of annoying/hacky code. And that if we wanted to add that feature later we could revisit my decision.

What do you think?

This test is one of the most necessary because compile is a very important function in TF, basically if you cannot compile you cannot train it with .fit() which will become a big problem knowing that the TF Trainer will move to .compile() + .fit() to train a model.

OK. I added test coverage to make sure that TFBartForConditionalGeneration can be compiled.

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>

sshleifer · 2020-10-16T17:28:41Z

Thanks for the review @LysandreJik !

mBART, Pegasus and Blenderbot, and Marian will be in the next PR. (this is too big already for me to hold in my tiny brain).
Your 4 bullets: Will do!

sgugger

Thanks for all the cleanup! LGTM!

LysandreJik

Great, thanks a lot for iterating!

* half done * doc improvement * Cp test file * brokedn * broken test * undo some mess * ckpt * borked * Halfway * 6 passing * boom boom * Much progress but still 6 * boom boom * merged master * 10 passing * boom boom * Style * no t5 changes * 13 passing * Integration test failing, but not gibberish * Frustrated * Merged master * 4 fail * 4 fail * fix return_dict * boom boom * Still only 4 * prepare method * prepare method * before delete classif * Skip tests to avoid adding boilerplate * boom boom * fast tests passing * style * boom boom * Switch to supporting many input types * remove FIXMENORM * working * Fixed past_key_values/decoder_cached_states confusion * new broken test * Fix attention mask kwarg name * undo accidental * Style and reviewers * style * Docs and common tests * Cleaner assert messages * copy docs * style issues * Sphinx fix * Simplify caching logic * test does not require torch * copy _NoLayerEmbedTokens * Update src/transformers/modeling_tf_bart.py Co-authored-by: Lysandre Debut <lysandre@huggingface.co> * Update tests/test_modeling_tf_bart.py Co-authored-by: Lysandre Debut <lysandre@huggingface.co> * Update src/transformers/modeling_tf_bart.py Co-authored-by: Lysandre Debut <lysandre@huggingface.co> * Update src/transformers/modeling_tf_bart.py Co-authored-by: Lysandre Debut <lysandre@huggingface.co> * Update src/transformers/modeling_tf_bart.py Co-authored-by: Lysandre Debut <lysandre@huggingface.co> * Line length and dont document None * Add pipeline test coverage * assert msg * At parity * Assert messages * mark slow * Update compile test * back in init * Merge master * Fix tests Co-authored-by: Lysandre Debut <lysandre@huggingface.co>

This reverts commit eb42d58.

sshleifer added 23 commits March 10, 2020 18:08

half done

b8e21a8

doc improvement

06897c8

Merge branch 'master' into doc-and-tf

37800ec

Merge branch 'master' into doc-and-tf

e17fb6e

Cp test file

891f99c

brokedn

9204174

broken test

6759d83

undo some mess

3ca1c2c

ckpt

cee3eea

borked

bba2455

Merged master

34bc4fa

Halfway

689f2c1

6 passing

7de9917

boom boom

0cf0625

Much progress but still 6

9e7fcf4

boom boom

3f7e97c

Merge branch 'master' into bart-tensorflow

14fc540

merged master

9af10ac

10 passing

b6563b4

boom boom

3ba7990

Style

fe8ac15

no t5 changes

416cbd9

Merge branch 'master' into bart-tensorflow

88c4fe9

sshleifer changed the title ~~Bart tensorflow~~ [WIP, dont merge] TFBart Jun 30, 2020

sshleifer linked an issue Jun 30, 2020 that may be closed by this pull request

TF BART ? #4001

Closed

sshleifer added 5 commits July 1, 2020 13:35

Merge branch 'master' into bart-tensorflow

d58c517

13 passing

c5d9b5f

Integration test failing, but not gibberish

4509e74

Merge branch 'master' into bart-tensorflow

1b0a6a5

Frustrated

a9e4290

sshleifer added 2 commits October 15, 2020 11:17

Sphinx fix

0dae1f3

Simplify caching logic

ec9e1c3

sshleifer linked an issue Oct 15, 2020 that may be closed by this pull request

Does bart need to cache prev_key_padding_mask? #7749

Closed

sshleifer added 2 commits October 15, 2020 12:54

test does not require torch

b1f4d04

copy _NoLayerEmbedTokens

a9fb456

sshleifer changed the title ~~[WIP] TFBart~~ Add TFBartForConditionalGeneration Oct 15, 2020

LysandreJik reviewed Oct 16, 2020

View reviewed changes

sshleifer and others added 3 commits October 16, 2020 11:22

Update src/transformers/modeling_tf_bart.py

22affd8

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>

Update tests/test_modeling_tf_bart.py

8baecc8

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>

Update src/transformers/modeling_tf_bart.py

6c16715

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>

sshleifer mentioned this pull request Oct 16, 2020

ProphetNet #7157

Merged

sshleifer and others added 2 commits October 16, 2020 13:22

Update src/transformers/modeling_tf_bart.py

27dea6d

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>

Update src/transformers/modeling_tf_bart.py

f93ce52

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>

sshleifer added 11 commits October 16, 2020 13:42

Line length and dont document None

d117eb8

Add pipeline test coverage

dcc1013

assert msg

58df751

At parity

26744b0

Assert messages

dee048b

mark slow

8f17d3a

Update compile test

78c6b9b

back in init

e71d519

Merge master

6b07f8b

Fix tests

8cc2c8b

Fix merge conflict

e5bc9bd

sgugger approved these changes Oct 19, 2020

View reviewed changes

LysandreJik approved these changes Oct 21, 2020

View reviewed changes

LysandreJik merged commit 8298421 into huggingface:master Oct 21, 2020

fabiocapsouza added a commit to fabiocapsouza/transformers that referenced this pull request Nov 15, 2020

Revert "Add TFBartForConditionalGeneration (huggingface#5411)"

623d6bd

This reverts commit eb42d58.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add TFBartForConditionalGeneration #5411

Add TFBartForConditionalGeneration #5411

sshleifer commented Jun 30, 2020 •

edited

LysandreJik left a comment

LysandreJik Oct 16, 2020

LysandreJik Oct 16, 2020

LysandreJik Oct 16, 2020

sshleifer Oct 16, 2020

jplu Oct 16, 2020

sshleifer Oct 16, 2020

sshleifer commented Oct 16, 2020 •

edited

sgugger left a comment

LysandreJik left a comment

Add TFBartForConditionalGeneration #5411

Add TFBartForConditionalGeneration #5411

Conversation

sshleifer commented Jun 30, 2020 • edited

TODO this PR:

Future PRs:

LysandreJik left a comment

Choose a reason for hiding this comment

LysandreJik Oct 16, 2020

Choose a reason for hiding this comment

LysandreJik Oct 16, 2020

Choose a reason for hiding this comment

LysandreJik Oct 16, 2020

Choose a reason for hiding this comment

sshleifer Oct 16, 2020

Choose a reason for hiding this comment

jplu Oct 16, 2020

Choose a reason for hiding this comment

sshleifer Oct 16, 2020

Choose a reason for hiding this comment

sshleifer commented Oct 16, 2020 • edited

sgugger left a comment

Choose a reason for hiding this comment

LysandreJik left a comment

Choose a reason for hiding this comment

sshleifer commented Jun 30, 2020 •

edited

sshleifer commented Oct 16, 2020 •

edited