Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Doc] add more MBart and other doc #6490

Merged
merged 8 commits into from
Aug 17, 2020
Merged

Conversation

patil-suraj
Copy link
Contributor

This PR

  1. adds example for MBart
  2. adds MBart in pre_trained models list and readme (Pegasus was missing from readme, so also added that).

@sshleifer , @sgugger

@patil-suraj
Copy link
Contributor Author

@sgugger do you think it would be a good idea to add more fine-tuning info for MBart, since it requires input processed in a different way than other models as it is multilingual model ?

Copy link
Collaborator

@sgugger sgugger left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for your PR, it looks good to me. If there is any specific behavior for MBart needed in the preprocessing, it should be documented, yes.

@@ -126,7 +126,7 @@ conversion utilities for the following models:
Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih.
23. `Pegasus <https://github.com/google-research/pegasus>`_ (from Google) released with the paper `PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarization
<https://arxiv.org/abs/1912.08777>`_ by Jingqing Zhang, Yao Zhao, Mohammad Saleh and Peter J. Liu.
24. `MBart <https://github.com/pytorch/fairseq/tree/master/examples/mbart>`_ (from Facebook) released with the paper `Multilingual Denoising Pre-training for Neural Machine Translation <https://arxiv.org/abs/2001.08210>`_ by Yinhan Liu, Jiatao Gu, Naman Goyal, Xian Li, Sergey Edunov
24. `MBart <https://github.com/pytorch/fairseq/tree/master/examples/mbart>`_ (from Facebook) released with the paper `Multilingual Denoising Pre-training for Neural Machine Translation <https://arxiv.org/abs/2001.08210>`_ by Yinhan Liu, Jiatao Gu, Naman Goyal, Xian Li, Sergey Edunov,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like the two lists got out of date with DPR (it should be in 22 in that list). Would you mind fixing this in your PR? Can do in another one if it's a problem.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no worries, will fix it in this PR.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@patil-suraj
Copy link
Contributor Author

@sshleifer ,@sgugger added DPR in readme.

@patil-suraj patil-suraj changed the title [Doc] add more MBart doc [Doc] add more MBart and other doc Aug 17, 2020
@codecov
Copy link

codecov bot commented Aug 17, 2020

Codecov Report

Merging #6490 into master will decrease coverage by 0.46%.
The diff coverage is n/a.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #6490      +/-   ##
==========================================
- Coverage   80.38%   79.91%   -0.47%     
==========================================
  Files         156      156              
  Lines       28058    28058              
==========================================
- Hits        22554    22423     -131     
- Misses       5504     5635     +131     
Impacted Files Coverage Δ
src/transformers/modeling_mbart.py 100.00% <ø> (ø)
src/transformers/modeling_tf_openai.py 22.58% <0.00%> (-72.26%) ⬇️
src/transformers/modeling_tf_flaubert.py 24.53% <0.00%> (-63.81%) ⬇️
src/transformers/tokenization_roberta.py 76.71% <0.00%> (-21.92%) ⬇️
src/transformers/tokenization_utils_base.py 86.58% <0.00%> (-7.19%) ⬇️
src/transformers/tokenization_transfo_xl.py 38.73% <0.00%> (-3.76%) ⬇️
src/transformers/tokenization_auto.py 95.55% <0.00%> (-2.23%) ⬇️
src/transformers/tokenization_utils_fast.py 92.14% <0.00%> (-2.15%) ⬇️
src/transformers/tokenization_openai.py 82.57% <0.00%> (-1.52%) ⬇️
src/transformers/generation_tf_utils.py 84.96% <0.00%> (-1.51%) ⬇️
... and 9 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 895ed8f...e1c522b. Read the comment docs.

@sgugger
Copy link
Collaborator

sgugger commented Aug 17, 2020

Great! Thanks for the PR.

@sgugger sgugger merged commit c9564f5 into huggingface:master Aug 17, 2020
Zigur pushed a commit to Zigur/transformers that referenced this pull request Oct 26, 2020
* add mbart example

* add Pegasus and MBart in readme

* typo

* add MBart in Pretrained models

* add pre-proc doc

* add DPR in readme

* fix indent

* doc fix
fabiocapsouza pushed a commit to fabiocapsouza/transformers that referenced this pull request Nov 15, 2020
* add mbart example

* add Pegasus and MBart in readme

* typo

* add MBart in Pretrained models

* add pre-proc doc

* add DPR in readme

* fix indent

* doc fix
fabiocapsouza added a commit to fabiocapsouza/transformers that referenced this pull request Nov 15, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants