Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

RobertaModel.fill_mask() missing space #1306

Closed
louismartin opened this issue Oct 25, 2019 · 1 comment
Closed

RobertaModel.fill_mask() missing space #1306

louismartin opened this issue Oct 25, 2019 · 1 comment

Comments

@louismartin
Copy link
Contributor

https://github.com/pytorch/fairseq/blob/5b086a0c17a16ed84285ce78017637d47fa50caa/fairseq/models/roberta/hub_interface.py#L174

When using sentencepiece, it seems that there is a missing space before the masked word in the output of fill_mask().

> roberta.fill_mask('My name is <mask>.', topk=1)
[('My name isJennifer.', 0.016039729118347168, 'Jennifer')]

I don't know how to fixing without breaking other stuff.
The sentencepiece encoded sentence is: ▁My ▁n ame ▁is <mask> ▁ .
facebook-github-bot pushed a commit that referenced this issue Nov 10, 2019
@louismartin @facebook-github-bot
Camembert model and code (#904)
b31849a 
Summary:
Check locally that everything works fine.
Model is uploaded to fbaipublicfiles.

I fixed a few inconsistencies in the bpe encoding along the way, e.g. related to #1306..
Pull Request resolved: fairinternal/fairseq-py#904

Reviewed By: ngoyal2707

Differential Revision: D18418345

Pulled By: louismartin

fbshipit-source-id: 53acb4d021581968d70430ee9babee07d6573c17
@myleott
Copy link
Contributor

myleott commented Dec 16, 2019

This seems fixed in your Camembert model code? Please reopen if this is still an issue.

@myleott myleott closed this as completed Dec 16, 2019
moussaKam pushed a commit to moussaKam/language-adaptive-pretraining that referenced this issue Sep 29, 2020
@louismartin @facebook-github-bot
Camembert model and code (facebookresearch#904)
1cfce0f 
Summary:
Check locally that everything works fine.
Model is uploaded to fbaipublicfiles.

I fixed a few inconsistencies in the bpe encoding along the way, e.g. related to facebookresearch#1306..
Pull Request resolved: fairinternal/fairseq-py#904

Reviewed By: ngoyal2707

Differential Revision: D18418345

Pulled By: louismartin

fbshipit-source-id: 53acb4d021581968d70430ee9babee07d6573c17
yzpang pushed a commit to yzpang/gold-off-policy-text-gen-iclr21 that referenced this issue Feb 19, 2021
@louismartin @yzpang
Camembert model and code (#904)
ed4d6d6 
Summary:
Check locally that everything works fine.
Model is uploaded to fbaipublicfiles.

I fixed a few inconsistencies in the bpe encoding along the way, e.g. related to facebookresearch/fairseq#1306..
Pull Request resolved: fairinternal/fairseq-py#904

Reviewed By: ngoyal2707

Differential Revision: D18418345

Pulled By: louismartin

fbshipit-source-id: 53acb4d021581968d70430ee9babee07d6573c17
yzpang pushed a commit to yzpang/gold-off-policy-text-gen-iclr21 that referenced this issue Feb 19, 2021
@louismartin @yzpang
Camembert model and code (#904)
83ec86f 
Summary:
Check locally that everything works fine.
Model is uploaded to fbaipublicfiles.

I fixed a few inconsistencies in the bpe encoding along the way, e.g. related to facebookresearch/fairseq#1306..
Pull Request resolved: fairinternal/fairseq-py#904

Reviewed By: ngoyal2707

Differential Revision: D18418345

Pulled By: louismartin

fbshipit-source-id: 53acb4d021581968d70430ee9babee07d6573c17
yfyeung pushed a commit to yfyeung/fairseq that referenced this issue Dec 6, 2023
@JinZr
fixes for data preparation (facebookresearch#1307)
162ceaf 
Issue: facebookresearch#1306
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@myleott @louismartin