Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix CI due to pytest #2408

Merged
merged 2 commits into from
Feb 1, 2024
Merged

Fix CI due to pytest #2408

merged 2 commits into from
Feb 1, 2024

Conversation

muellerzr
Copy link
Collaborator

Pytest released a new version that broke our CI because:

Files and directories are now collected in alphabetical order jointly, unless changed by a plugin. Previously, files were collected before directories. See below for an example.

The failures were coming from deepspeed being the first one being ran instead of later on. This PR change make test to run in the following order:

test:
	$(MAKE) test_core
	$(MAKE) test_cli
	$(MAKE) test_big_modeling
	$(MAKE) test_deepspeed
	$(MAKE) test_fsdp

Instead of the prior:

python -m pytest -s -v ./tests/ --ignore=./tests/test_examples.py $(if $(IS_GITHUB_CI),--report-log "$(PYTORCH_VERSION)_all.log",)

@HuggingFaceDocBuilderDev

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Copy link
Member

@SunMarc SunMarc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's great that we manage to fix the issue without having to pin pytest ! LGTM !

@muellerzr muellerzr merged commit f4b411f into main Feb 1, 2024
5 of 9 checks passed
@muellerzr muellerzr deleted the fix-cli-good branch February 1, 2024 17:28
@akx
Copy link
Contributor

akx commented Feb 15, 2024

Hold on, why is the order of tests significant for the test suite to pass? 😳

@SunMarc
Copy link
Member

SunMarc commented Feb 15, 2024

Yeah, this is definitely strange, but the deepspeed tests seems to mess up with our env. Very simple tests were failing because of that. We will investigate this further when we get the time !

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants