[ProphetNet] Bart-like Refactor

[patrickvonplaten]

What does this PR do?

This PR refactors ProphetNet similar to Bart in that it moves the time dimension to be always at the 2nd place and the batch dimensions always in the first place. Also, the cache is refactored to consists of tuples instead of a dict.

The model is thereby very much aligned with Bart (I cannot really add any " # Copied from" statements though because the weight names are different).

The PR is in spirit very similar to #8900.

I've verified that all slow tests pass. In the next step, I want to make a short notebook, verifying that ProphetNet can be trained since there have been some issues on training: #9804

Benchmarking

The PR doesn't change compute or memory complexity:

On this PR:

====================       INFERENCE - SPEED - RESULT       ====================
--------------------------------------------------------------------------------
          Model Name             Batch Size     Seq Length     Time in s   
--------------------------------------------------------------------------------
microsoft/prophetnet-large-unc       8               8             0.029     
microsoft/prophetnet-large-unc       8               32            0.044     
microsoft/prophetnet-large-unc       8              128            0.175     
microsoft/prophetnet-large-unc       8              512             N/A      
--------------------------------------------------------------------------------

====================      INFERENCE - MEMORY - RESULT       ====================
--------------------------------------------------------------------------------
          Model Name             Batch Size     Seq Length    Memory in MB 
--------------------------------------------------------------------------------
microsoft/prophetnet-large-unc       8               8              2562     
microsoft/prophetnet-large-unc       8               32             2756     
microsoft/prophetnet-large-unc       8              128             3628     
microsoft/prophetnet-large-unc       8              512             N/A      
--------------------------------------------------------------------------------

on master:

====================       INFERENCE - SPEED - RESULT       ====================
--------------------------------------------------------------------------------
          Model Name             Batch Size     Seq Length     Time in s   
--------------------------------------------------------------------------------
microsoft/prophetnet-large-unc       8               8             0.027     
microsoft/prophetnet-large-unc       8               32            0.044     
microsoft/prophetnet-large-unc       8              128            0.172     
microsoft/prophetnet-large-unc       8              512             N/A      
--------------------------------------------------------------------------------

====================      INFERENCE - MEMORY - RESULT       ====================
--------------------------------------------------------------------------------
          Model Name             Batch Size     Seq Length    Memory in MB 
--------------------------------------------------------------------------------
microsoft/prophetnet-large-unc       8               8              2562     
microsoft/prophetnet-large-unc       8               32             2768     
microsoft/prophetnet-large-unc       8              128             3740     
microsoft/prophetnet-large-unc       8              512             N/A      
--------------------------------------------------------------------------------

[sgugger]

Thanks for refactoring this, LGTM!

[LysandreJik]

Very cool! Thanks for running all the slow tests and verifying time/memory complexity.

LGTM!