Add madlad-400 MT models

[jbochi]

What does this PR do?

Fixes #26696

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Did you read the contributor guideline,
  Pull Request section?
• Was this discussed/approved via a Github issue or the forum? Please add a link
  to it if that's the case.
• Did you make sure to update the documentation with your changes? Here are the
  documentation guidelines, and
  here are tips on formatting docstrings.
• Did you write any new necessary tests?

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@ArthurZucker

[amyeroberts]

Thanks for adding @jbochi! Let us know when you're ready for a review.

I've set the failed tests to re-run as it seems there was a transient connection error causing them to fail.

[jbochi]

Thanks for re-running the tests, @amyeroberts .

The PR is ready for review. I don't think the previous failure was related to my changes.

[jbochi]

The tests passed this time. 😌

[amyeroberts]

Thanks for adding!

Just a few small notes on the doc page structure.

[jbochi]

I think we can transfer all MT models now and merge this PR. Maybe I will work on LM models later, but I am not sure. What do you think?

…

On Thu, Nov 16, 2023, 8:15 AM amyeroberts ***@***.***> wrote: ***@***.**** commented on this pull request. ------------------------------ In docs/source/en/model_doc/madlad-400.md <#27471 (comment)> : > +- [jbochi/madlad400-3b-mt](https://huggingface.co/jbochi/madlad400-3b-mt) + +- [jbochi/madlad400-7b-mt](https://huggingface.co/jbochi/madlad400-7b-mt) + +- [jbochi/madlad400-7b-mt-bt](https://huggingface.co/jbochi/madlad400-7b-mt-bt) + +- [jbochi/madlad400-10b-mt](https://huggingface.co/jbochi/madlad400-10b-mt) OK - let me know when it's ready. We'll do this as the last step before merging — Reply to this email directly, view it on GitHub <#27471 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AACHO2FLM2AV6ITIUQ63O23YEYGY5AVCNFSM6AAAAAA7JISLSCVHI2DSMVQWIX3LMV43YUDVNRWFEZLROVSXG5CSMV3GSZLXHMYTOMZUGM2DONBZGE> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[amyeroberts]

Great - thanks for iterating!

Just one little nit on the abstract formatting. I can transfer the checkpoints, the checkpoint paths can be updated and then we can merge 🤗

[amyeroberts]

Thank you for filling out the model cards so beautifully 🙏

[jbochi]

Thanks for the review!

[amyeroberts]

Hi @jbochi. Apologies for the delay in getting back to you - I'm also learning about the intricacies of org weights too! For the model weights, would you prefer that we move the original checkpoint repos directly under google or duplicate, so that yours still exist under your profile?

[jbochi]

Hey. No worries. I think it's better to move them (assuming people will be redirected if they go to mine).

…

On Thu, Nov 23, 2023, 4:50 AM amyeroberts ***@***.***> wrote: Hi @jbochi <https://github.com/jbochi>. Apologies for the delay in getting back to you - I'm also learning about the intricacies of org weights too! For the model weights, would you prefer that we move the original checkpoint repos directly under google or duplicate, so that yours still exist under your profile? — Reply to this email directly, view it on GitHub <#27471 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AACHO2AT3SW7LP3R6SLGTVTYF4L5VAVCNFSM6AAAAAA7JISLSCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMRUGA4DSNBTGA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[amyeroberts]

@jbochi If they're moved then they wouldn't exist under your profile any more at all. I believe it would error-out saying that the repo doesn't exist if someone tried: AutoTokenizer.from_pretrained("jbochi/madlad400-3b-mt"). Is this still OK or would you rather we copy?

[jbochi]

In this case, can we copy them instead? Thanks

…

On Fri, Nov 24, 2023, 12:45 PM amyeroberts ***@***.***> wrote: @jbochi <https://github.com/jbochi> If they're moved then they wouldn't exist under your profile any more at all. I believe it would error-out saying that the repo doesn't exist if someone tried: AutoTokenizer.from_pretrained("jbochi/madlad400-3b-mt"). Is this still OK or would you rather we copy? — Reply to this email directly, view it on GitHub <#27471 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AACHO2CTUUN2QO6SMH62K63YGDMNLAVCNFSM6AAAAAA7JISLSCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMRVHE2TKOBUHA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[amyeroberts]

@jbochi Yep! All done. I've set the doc tests to re-run which should pass now the google checkpoints exist 🤞

[jbochi]

Yay! Thank you!

…

On Mon, Nov 27, 2023, 11:03 AM amyeroberts ***@***.***> wrote: @jbochi <https://github.com/jbochi> Yep! All done. I've set the doc tests to re-run which should pass now the google checkpoints exist 🤞 — Reply to this email directly, view it on GitHub <#27471 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AACHO2ARZDBRUO4SS2AAVBDYGS2VHAVCNFSM6AAAAAA7JISLSCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMRYGEZTGMJTGU> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[amyeroberts]

@jbochi One last final update - the doctests are failing with a timeout. Could you add the model doc page madlad-400.md to the not_doctested.txt page?

[jbochi]

@jbochi One last final update - the doctests are failing with a timeout. Could you add the model doc page madlad-400.md to the not_doctested.txt page?

Sure. Done!

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint.

[amyeroberts]

@jbochi Awesome - thanks again for adding!

[jbochi]

Thanks for the review and all the help, @amyeroberts