Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[run_seq2seq] fix nltk lookup #10585

Merged
merged 1 commit into from Mar 8, 2021
Merged

[run_seq2seq] fix nltk lookup #10585

merged 1 commit into from Mar 8, 2021

Conversation

stas00
Copy link
Contributor

@stas00 stas00 commented Mar 8, 2021

Hmm, CI crashes every so often on

try:
    nltk.data.find("tokenizers/punkt")
except LookupError:

introduced in this PR: #10407

https://app.circleci.com/pipelines/github/huggingface/transformers/20635/workflows/989fde0b-e543-4620-9d9a-f213ad53dd9b/jobs/176742

__________________ ERROR collecting examples/test_examples.py __________________
examples/test_examples.py:51: in <module>
    import run_seq2seq
examples/seq2seq/run_seq2seq.py:54: in <module>
    nltk.data.find("tokenizers/punkt")
../.local/lib/python3.6/site-packages/nltk/data.py:539: in find
    return FileSystemPathPointer(p)
../.local/lib/python3.6/site-packages/nltk/compat.py:41: in _decorator
    return init_func(*args, **kwargs)
../.local/lib/python3.6/site-packages/nltk/data.py:315: in __init__
    raise IOError("No such file or directory: %r" % _path)
E   OSError: No such file or directory: '/home/circleci/nltk_data/tokenizers/punkt/PY3'

which is odd, re-running the job fixed the problem.

So trying to mend it with:

try:
    nltk.data.find("tokenizers/punkt")
except (LookupError, OSError):

not sure why the Exception was different here.

@sgugger, @LysandreJik

@stas00
Copy link
Contributor Author

stas00 commented Mar 8, 2021

I'm going to merge this since the issue could be interfering with other PRs.

@stas00 stas00 merged commit e6ce636 into huggingface:master Mar 8, 2021
@stas00 stas00 deleted the nltk-find branch March 8, 2021 06:10
Iwontbecreative pushed a commit to Iwontbecreative/transformers that referenced this pull request Jul 15, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant