Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[examples] fix sys.path in conftest.py #11636

Merged
merged 4 commits into from May 7, 2021

Conversation

stas00
Copy link
Contributor

@stas00 stas00 commented May 7, 2021

the pt/tf reshuffle broke the examples tests setup. This PR fixes it, by

  • fixing sys.path setting now that it's one level down

@sgugger

@sgugger
Copy link
Collaborator

sgugger commented May 7, 2021

That can't work since conftest now believes the module "tensorflow" is the folder in the examples with the same name, which is the reason why I moved it.

Path in conftest need to be adapted instead.

@stas00
Copy link
Contributor Author

stas00 commented May 7, 2021

ah, OK, then I will make copies of it. But you can't move it since sys.path was relying on it being in a top-level subfolder - so will have to fix that or make the code more flexible. It's currently broken.

I will take care of it.

thank you for letting me know tf didn't like it.

@stas00
Copy link
Contributor Author

stas00 commented May 7, 2021

ok,

  1. put conftest.py back and fixed it to set up sys.path correctly
  2. made 2 copies of it for legacy/research.

Copy link
Collaborator

@sgugger sgugger left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The fix in itself LGTM but why add a conftest to legacy and research-projects? Both are not supposed to be tested.

examples/legacy/conftest.py Outdated Show resolved Hide resolved
examples/research_projects/conftest.py Outdated Show resolved Hide resolved
@stas00
Copy link
Contributor Author

stas00 commented May 7, 2021

The fix in itself LGTM but why add a conftest to legacy and research-projects? Both are not supposed to be tested.

I'm adding deepspeed tests for wav2vec2 - it's been complicated to make it work (some parts still don't work). So I want to make sure it doesn't break again.

I don't think it's a good idea to put these tests under common tests.

So I can't run the tests w/o this conftest on dev box.

@sgugger
Copy link
Collaborator

sgugger commented May 7, 2021

I really don't think it's a good idea to add this to research-projects: all research projects have a pinned version of Transformers and this would use the master install to test them, which is incompatible with what is advertised.

Maybe add it specifically to the wav2vec2 research project? Or maybe wav2vec2 should move to a maintained example @patrickvonplaten ?

@stas00
Copy link
Contributor Author

stas00 commented May 7, 2021

I really don't think it's a good idea to add this to research-projects: all research projects have a pinned version of Transformers and this would use the master install to test them, which is incompatible with what is advertised.

Good point!

OK, will recall the 2 other files for now.

Maybe add it specifically to the wav2vec2 research project? Or maybe wav2vec2 should move to a maintained example @patrickvonplaten ?

Yes, totally agree!

@stas00 stas00 changed the title [examples] restore conftest.py [examples] fix sys.path in conftest.py May 7, 2021
@stas00
Copy link
Contributor Author

stas00 commented May 7, 2021

pathlib is easier to use for finding out older ancestors than doing dirname 4 times:

import sys
from pathlib import Path
git_repo_path = Path(__file__).resolve().parents[3] / "src"
sys.path.insert(1, str(git_repo_path))

@stas00 stas00 merged commit ba0d50f into huggingface:master May 7, 2021
@stas00 stas00 deleted the fix-ex-conftest branch May 7, 2021 21:44
Iwontbecreative pushed a commit to Iwontbecreative/transformers that referenced this pull request Jul 15, 2021
* restore conftest.py

* fix conftest and make copies

* remove unneeded parts

* remove unwanted files
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants