The docs for this PR live here. All of your documentation changes will be reflected on that endpoint.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

Hey! Thanks for contributing! In the current state I cannot really see the differences between this model and T5. Adding the # Copied from statements would help a lot. However if the model is very similar (and you still want to persue the PR!) I would recommend adding the model to the hub following this tutorial! It will be simpler for you and you won't have to deal with all the red CIs!

@ArthurZucker

You are correct that the basic structure of the model is based on the existing T5. However, there are differences in the implementation between huggingface and MegatronLM (or NeMo) regarding the reshaping of tensors for attention computation, as well as various differences in normalization methods. Due to these differences, I decided to submit the pull request. Simply mapping the model weights wouldn't result in proper functioning, so a custom class was required. I will refer to the guide you provided and give it a try. Thank you :)

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

Hey! Could you refer me to the link of the updated model if you already push it to the hub? 😉 This is in order to keep track of models on the hub!

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.