Squeeze and Excitation Network #13316
Conversation
…tests to test for this additional network as well
|
Hi, Thanks for your PR! However, I don't think that we want to add this block to files of other models. It's more appropriate to add a new SesameBERT model (if pretrained weights are available), or add it under the cc @LysandreJik |
|
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread. Please note that issues that do not follow the contributing guidelines are likely to be ignored. |
|
Hi @AdityaDas-IITM, are you interested in working on this PR (making it a research project instead)? |
|
Hey @NielsRogge, Yes I'll get started on it soon |
|
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread. Please note that issues that do not follow the contributing guidelines are likely to be ignored. |
What does this PR do?
This PR implements an optional Squeeze and Excitation Block in Bert and the copied modules (RoBerta, Electra, splinter and layoutlm) in pytorch.
Fixes #11998
Additional tests have been added to the corresponding test scripts and the docs updated.
Before submitting
Pull Request section?
to it if that's the case.
documentation guidelines, and
here are tips on formatting docstrings.
Who can review?
Anyone in the community is free to review the PR once the tests have passed.
@LysandreJik