Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HuggingFace conversion script doesn't work #69

Open
mcPear opened this issue Jan 20, 2022 · 1 comment
Open

HuggingFace conversion script doesn't work #69

mcPear opened this issue Jan 20, 2022 · 1 comment

Comments

@mcPear
Copy link

mcPear commented Jan 20, 2022

First of all thanks for this amazing open source work!
Second, I converted eng-pol model to HF format with instruction: https://github.com/huggingface/transformers/tree/master/scripts/tatoeba, but when I use it with transformers library I get sequence of dots ".................." as result. I also tried with pol-eng which is available in HF repo as opus-mt-pl-en but no success again.
Do you have any idea what am I doing wrong?

Thanks

@mcPear mcPear closed this as completed Jan 20, 2022
@mcPear mcPear reopened this Jan 20, 2022
@mcPear mcPear changed the title HuggingFace conversion script HuggingFace conversion script doesn't work Jan 20, 2022
@jorgtied
Copy link
Member

There was a bug in the conversion script that assumed that special tokens are in the beginning of the vocabulary. That has been fixed recently and it should work with the latest source code. Note, that we also work on updating the conversion procedures to make them more flexible with other variants of marian models. That is not yet ready but hopefully will come soon.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants