BartForCausalLM analogs to ProphetNetForCausalLM

[sadakmed]

What does this PR do?

Implementing BartForCausalLM anologs for ProphetNetForCausalLM

Fixes #9066

[patrickvonplaten]

Hy @sadakmed,

let me know if you need help on the issue or if you don't find the time to tackle it. I'll then just make it open to the "public" again :-)

[patrickvonplaten]

It's great to see progress here! Good job. Note that BartForCausalLM should have the exact same functionality in terms of computing the loss as the BartForConditionalGeneration model => this means we can copy the way BartForConditionalGeneration computes the loss into BartForCausalLM.

Let me know if you need help or are stuck :-)

[sadakmed]

@patrickvonplaten The loss function it what I stuck on, thank you very much for your guidance.

Let me know if you need help or are stuck :-)

for sure I will ;-)

[patrickvonplaten]

Hey @sadakmed,

Do you have an update on the PR? It's been three weeks now and it would be great to merge this soon. Sorry, we're very fast-moving in this lib and other community contributors have started asking for this feature. By next week, I'll probably have to take a look myself or redistribute the issue.

[sadakmed]

Hi @patrickvonplaten my apologies,

Could you Please see it now, lemme know if anything is missing.

[sadakmed]

Hi @patrickvonplaten, working on the test 'BartStandaloneCausalLM': I dont know if the self.model_tester in 'setUp' should be 'BartDecoderTester' (needed to be implemented), or do u recommend something else.

[patrickvonplaten]

Hey @sadakmed,

Thanks a lot for you additions here :-)
Yes, we need a new BartStandaloneDecoderModelTester analog to how it's done for ProphetNet in tests/test_modeling_prophetnet.py. Do you want to give it a try? Otherwise, I can go into your PR and see how to add the tests :-)

[sadakmed]

Hi @patrickvonplaten

Do you want to give it a try?

of course, I'm working on it,

[sadakmed]

Hi @patrickvonplaten, I just pushed the test, could you please check it out!
thaaaanks

[sadakmed]

@patrickvonplaten could you check please!

[patrickvonplaten]

UPDATE:

@LysandreJik @sgugger
This PR enables all Bart-like models to be used in combination with the Encoder-Decoder framework. The model BartForCausalLM is added for Bart and then copied to all other models via the copying mechanism. Also, a new model tester is added for all those models.
While working on this I found a small bug for a very edge-case scenario for Bart and corrected it here: https://github.com/huggingface/transformers/pull/9128/files#r569436360 . The newly added tests were failing, which made me aware of the bug.
Also, I had to slightly change the check_repo.py file so that it counts both classes from all_model_classes with 1 and 2 paratheses.

[sgugger]

This is fantastic work! Thanks a lot!

[LysandreJik]

Impressive work, great work @sadakmed and @patrickvonplaten!

Very cool that you completed the templates as well.

There seems to be a few issues with the "copied by" scripts in the templates, but the test still passes, which is weird.

[patrickvonplaten]

Great job @sadakmed

[sadakmed]

Great job @sadakmed

wouldn't happen without you, thank you very much. see u in the next PR ;)