Fix tests without any real effect in EncoderDecoderMixin

[ydshieh]

What does this PR do?

In test_modeling_encoder_decoder.py, there are 2 places like

enc_dec_model.save_pretrained(tmpdirname)
EncoderDecoderModel.from_pretrained(tmpdirname)

after_outputs = enc_dec_model(...)

Therefore after_outputs will be exactly the same outputs, since enc_dec_model doesn't get the new value.
For the testing purpose, we need to do

enc_dec_model = EncoderDecoderModel.from_pretrained(tmpdirname)

(Hope I get it right)

Who can review?

@patrickvonplaten @sgugger

[sgugger]

LGTM, but let's wait for @patrickvonplaten review as he is the expert on encoder decoders.

[patrickvonplaten]

Thanks a lot @ydshieh !

[LysandreJik]

The model should have been cast to the torch_device as otherwise the inputs/model are on different devices :)

https://github.com/huggingface/transformers/runs/3524847938?check_suite_focus=true

[patrickvonplaten]

I'll take care of it :-)

[ydshieh]

Hey, thank you guys! Sorry about the device issue, I am less familiar with PyTorch, and it seems the checks I had when I opened this PR don't have that test above (probably it only runs when we merge to master?)

[patrickvonplaten]

Don't worry about it @ydshieh :-)