Feed forward chunking others #6365

Pradhy729 · 2020-08-09T15:13:46Z

Adding feed forward chunking to other models. Based on #6024

Update from source

fix the shuffle agrument usage and the default (huggingface#6307)

Update from source

codecov · 2020-08-09T15:22:29Z

Codecov Report

Merging #6365 into master will increase coverage by 2.04%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master    #6365      +/-   ##
==========================================
+ Coverage   78.42%   80.47%   +2.04%     
==========================================
  Files         156      156              
  Lines       28129    28152      +23     
==========================================
+ Hits        22061    22655     +594     
+ Misses       6068     5497     -571

Impacted Files	Coverage Δ
src/transformers/configuration_reformer.py	`100.00% <ø> (ø)`
src/transformers/modeling_bert.py	`88.26% <ø> (-0.17%)`	⬇️
src/transformers/modeling_utils.py	`87.35% <ø> (+0.19%)`	⬆️
src/transformers/configuration_utils.py	`96.62% <100.00%> (+1.38%)`	⬆️
src/transformers/modeling_albert.py	`83.50% <100.00%> (+0.13%)`	⬆️
src/transformers/modeling_distilbert.py	`97.84% <100.00%> (+1.65%)`	⬆️
src/transformers/modeling_longformer.py	`92.02% <100.00%> (+0.07%)`	⬆️
src/transformers/modeling_reformer.py	`96.09% <100.00%> (ø)`
src/transformers/modeling_xlm.py	`91.31% <100.00%> (+0.07%)`	⬆️
src/transformers/modeling_xlnet.py	`83.42% <100.00%> (+0.11%)`	⬆️
... and 17 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update fb7330b...80c6b27. Read the comment docs.

patrickvonplaten · 2020-08-11T17:17:02Z

#6024 is merged :-) Great work @Pradhy729! It would be a good idea to rebase this PR to current master so that you can easily leverage the tests that were added in #6024 just by setting the flag test_chunking=True for all models you want to add here.

Pradhy729 · 2020-08-11T19:02:58Z

Yes - definitely will do. Was just waiting for the merge. Thanks for adding the tests.

Update from source

Pradhy729 · 2020-08-12T21:29:18Z

@patrickvonplaten Feed forward chunking has been added for the following:

Albert
Distillbert
Longformer
XLNet
XLM

Also, changed model signature to have callable as first positional argument.

Pradhy729 · 2020-08-14T22:23:52Z

Hi @patrickvonplaten --> Can you review and approve if this looks good?

patrickvonplaten · 2020-08-17T18:24:01Z

src/transformers/configuration_utils.py

@@ -188,6 +188,7 @@ def __init__(self, **kwargs):
        self.pad_token_id = kwargs.pop("pad_token_id", None)
        self.eos_token_id = kwargs.pop("eos_token_id", None)
        self.decoder_start_token_id = kwargs.pop("decoder_start_token_id", None)
+        self.chunk_size_feed_forward = kwargs.pop("chunk_size_feed_forwar", 0)


great, thanks for adding this!

Can you move the docstring from Reformer to this file and delete the corresponding docstring / config variable from reformer?

actually it's aleady done - never mind

patrickvonplaten · 2020-08-17T18:27:43Z

src/transformers/modeling_utils.py

@@ -1447,7 +1447,7 @@ def prune_layer(


 def apply_chunking_to_forward(
-    chunk_size: int, chunk_dim: int, forward_fn: Callable[..., torch.Tensor], *input_tensors
+    forward_fn: Callable[..., torch.Tensor], chunk_size: int, chunk_dim: int, *input_tensors


Thanks for changing that. @LysandreJik - as you said this is the better order of the arguments and should be fine in terms of breaking backward compatibility

patrickvonplaten · 2020-08-17T18:34:21Z

tests/test_modeling_common.py

@@ -60,7 +60,7 @@ class ModelTesterMixin:
    test_resize_embeddings = True
    test_head_masking = True
    test_missing_keys = True
-    test_chunking = False
+    test_chunking = True


Can you remove the test_chunking=True statement in other test files as well?

patrickvonplaten · 2020-08-17T18:39:18Z

Hey @Pradhy729 - this looks great!

Can you add the docstrings for chunk_size_feed_forward as explained in the comment above and delete the corresponding config param in Reformer and the Reformer docstring (You can just cut & paste the Reformer docstring here)
Can you please remove the test_chunking=True statements in the model specific test files -> I think it's only in test_modeling_bert.py actually.
It would be awesome if you try to rebase the branch to master (git fetch upstream master, git rebase upstream/master).
If you have too many merge conflicts - then I'll do it :-)

Update from source

Pradhy729 · 2020-08-18T05:30:10Z

@patrickvonplaten
Done. Please review and let me know if there's anything else.

patrickvonplaten · 2020-08-18T20:52:08Z

LGTM! @Pradhy729 - great work!

sgugger

Great addition, thanks a lot!

patrickvonplaten · 2020-08-19T12:31:07Z

Merging! Good job @Pradhy729

* Feed forward chunking for Distilbert & Albert * Added ff chunking for many other models * Change model signature * Added chunking for XLM * Cleaned up by removing some variables. * remove test_chunking flag Co-authored-by: patrickvonplaten <patrick.v.platen@gmail.com>

This reverts commit c45253a.

Pradhy729 added 8 commits July 14, 2020 21:48

Merge pull request #1 from huggingface/master

62788a7

Update from source

Merge pull request #2 from huggingface/master

b413437

Update from source

Merge pull request #3 from huggingface/master

fcaa3aa

Update from source

Merge pull request #4 from huggingface/master

623352f

Update from source

Merge pull request #5 from huggingface/master

5a26a2d

Update from source

Merge pull request #6 from huggingface/master

c96bc33

Update from source

Merge pull request #7 from huggingface/master

b85adeb

fix the shuffle agrument usage and the default (huggingface#6307)

Merge pull request #8 from huggingface/master

a258372

Update from source

Merge pull request #9 from huggingface/master

b18cb64

Update from source

Pradhy729 force-pushed the feed_forward_chunking_others branch from 4346fe2 to b35f648 Compare August 12, 2020 06:55

Merge pull request #10 from huggingface/master

04b985f

Update from source

Pradhy729 force-pushed the feed_forward_chunking_others branch from b35f648 to 44efa91 Compare August 12, 2020 19:10

Pradhy729 changed the title ~~[WIP] Feed forward chunking others~~ Feed forward chunking others Aug 12, 2020

patrickvonplaten reviewed Aug 17, 2020

View reviewed changes

Pradhy729 and others added 5 commits August 17, 2020 18:06

Merge pull request #11 from huggingface/master

d36817f

Update from source

Feed forward chunking for Distilbert & Albert

dce34e5

Added ff chunking for many other models

7f370d4

Change model signature

8d69d6f

Added chunking for XLM

82f79df

Pradhy729 force-pushed the feed_forward_chunking_others branch from dfe0497 to 82f79df Compare August 18, 2020 01:34

Cleaned up by removing some variables.

2ac6418

remove test_chunking flag

80c6b27

patrickvonplaten requested review from sgugger and LysandreJik August 18, 2020 20:52

patrickvonplaten approved these changes Aug 18, 2020

View reviewed changes

sgugger approved these changes Aug 19, 2020

View reviewed changes

patrickvonplaten merged commit 2a7402c into huggingface:master Aug 19, 2020

fabiocapsouza added a commit to fabiocapsouza/transformers that referenced this pull request Nov 15, 2020

Revert "Feed forward chunking others (huggingface#6365)"

223fee1

This reverts commit c45253a.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Feed forward chunking others #6365

Feed forward chunking others #6365

Pradhy729 commented Aug 9, 2020

codecov bot commented Aug 9, 2020 •

edited

Loading

patrickvonplaten commented Aug 11, 2020

Pradhy729 commented Aug 11, 2020

Pradhy729 commented Aug 12, 2020 •

edited

Loading

Pradhy729 commented Aug 14, 2020

patrickvonplaten Aug 17, 2020

patrickvonplaten Aug 17, 2020

patrickvonplaten Aug 18, 2020

patrickvonplaten Aug 17, 2020

patrickvonplaten Aug 17, 2020

patrickvonplaten commented Aug 17, 2020

Pradhy729 commented Aug 18, 2020

patrickvonplaten commented Aug 18, 2020

sgugger left a comment

patrickvonplaten commented Aug 19, 2020

Feed forward chunking others #6365

Feed forward chunking others #6365

Conversation

Pradhy729 commented Aug 9, 2020

codecov bot commented Aug 9, 2020 • edited Loading

Codecov Report

patrickvonplaten commented Aug 11, 2020

Pradhy729 commented Aug 11, 2020

Pradhy729 commented Aug 12, 2020 • edited Loading

Pradhy729 commented Aug 14, 2020

patrickvonplaten Aug 17, 2020

Choose a reason for hiding this comment

patrickvonplaten Aug 17, 2020

Choose a reason for hiding this comment

patrickvonplaten Aug 18, 2020

Choose a reason for hiding this comment

patrickvonplaten Aug 17, 2020

Choose a reason for hiding this comment

patrickvonplaten Aug 17, 2020

Choose a reason for hiding this comment

patrickvonplaten commented Aug 17, 2020

Pradhy729 commented Aug 18, 2020

patrickvonplaten commented Aug 18, 2020

sgugger left a comment

Choose a reason for hiding this comment

patrickvonplaten commented Aug 19, 2020

codecov bot commented Aug 9, 2020 •

edited

Loading

Pradhy729 commented Aug 12, 2020 •

edited

Loading