[T5] Fix load weights function

[patrickvonplaten]

What does this PR do?

Fixes #7791

The problem in 7791 was that the code that was used to convert mtf t5 to hf t5 was outdated and had a couple of bugs:

1. weight embedding were not tied -> random weight matrices were used instead
2. the tokenizer didn't add EOS which meant that the input_ids were wrong.

@agemagician I will merge this into master now so that you don't have to work on the hacky branch I did a while back. Your models should be convertible with the new code added here and then everything should work as expected.

The following code now produces the correct results:

#!/usr/bin/env python3
from transformers import T5Tokenizer  # noqa: E402
from transformers.convert_t5_original_tf_checkpoint_to_pytorch import (  # noqa: E402
    convert_tf_checkpoint_to_pytorch,
)
from transformers.modeling_t5 import T5Config, T5ForConditionalGeneration  # noqa: E402
import torch

path_to_tf_checkpoint = "t5_mesh_checkpoints"

config = T5Config.from_pretrained("t5-base")
config.d_ff = 2048
config.d_kv = 64
config.d_model = 512
config.num_decoder_layers = 6
config.num_layers = 6
config.num_heads = 8
config.vocab_size = 32128
config.tie_word_embeddings = True
config.save_pretrained(path_to_tf_checkpoint)

convert_tf_checkpoint_to_pytorch(path_to_tf_checkpoint, path_to_tf_checkpoint + "/config.json", path_to_tf_checkpoint)

input_txt = ["javascript documentation generation: function isStandardBrowserEnv ( ) { if ( typeof navigator !== 'undefined' && ( navigator . product === 'ReactNative' || navigator . product === 'NativeScript' || navigator . product === 'NS' ) ) { return false ; } return ( typeof window !== 'undefined' && typeof document !== 'undefined' ) ; }"]

tok = T5Tokenizer.from_pretrained(path_to_tf_checkpoint)

model = T5ForConditionalGeneration.from_pretrained(path_to_tf_checkpoint, return_dict=True)
model.to("cuda")

input_ids = tok(input_txt, return_tensors="pt").input_ids
outputs = model.generate(input_ids.to("cuda"), num_beams=4)
print(tok.decode(outputs[0]))

=> gives " Returns true if the browser is a native element. "

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Did you read the contributor guideline,
  Pull Request section?
• Was this discussed/approved via a Github issue or the forum? Please add a link
  to the it if that's the case.
• Did you make sure to update the documentation with your changes? Here are the
  documentation guidelines, and
  here are tips on formatting docstrings.
• Did you write any new necessary tests?

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

I checked that the new convert_weights_function works with previous mtf (t5==0.7.1) models.

[patrickvonplaten]

Failing tests are unrelated

[agemagician]

Thanks again @patrickvonplaten , you made my weekend 😄