-
Notifications
You must be signed in to change notification settings - Fork 4.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Move slow tests to nightly #4526
Conversation
The documentation is not available anymore as the PR was closed or merged. |
@@ -539,47 +539,6 @@ def test_stable_diffusion_2_pix2pix_inversion(self): | |||
|
|||
assert np.abs(expected_slice - image_slice.cpu().numpy()).max() < 5e-2 | |||
|
|||
def test_stable_diffusion_pix2pix_full(self): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why are we getting rid of it entirely?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The pipeline wasn't prioritised in this list (under highly important and important)
So I moved all the slow tests to nightly.
def test_two_step_model(self): | ||
expected_video = load_numpy( | ||
"https://huggingface.co/datasets/hf-internal-testing/diffusers-images/resolve/main/text_to_video/video.npy" | ||
"https://huggingface.co/datasets/hf-internal-testing/diffusers-images/resolve/main/text_to_video/video_2step.npy" | ||
) | ||
|
||
pipe = TextToVideoSDPipeline.from_pretrained("damo-vilab/text-to-video-ms-1.7b") | ||
pipe.scheduler = DPMSolverMultistepScheduler.from_config(pipe.scheduler.config) | ||
pipe = pipe.to("cuda") | ||
pipe = pipe.to(torch_device) | ||
|
||
prompt = "Spiderman is surfing" | ||
generator = torch.Generator(device="cpu").manual_seed(0) | ||
|
||
video_frames = pipe(prompt, generator=generator, num_inference_steps=25, output_type="pt").frames | ||
video_frames = pipe(prompt, generator=generator, num_inference_steps=2, output_type="pt").frames | ||
video = video_frames.cpu().numpy() | ||
|
||
assert np.abs(expected_video - video).mean() < 5e-2 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Works for me!
@nightly | ||
@skip_mps | ||
@require_torch_gpu | ||
class TextToVideoSDPipelineNightlyTests(unittest.TestCase): | ||
def test_full_model(self): | ||
expected_video = load_numpy( | ||
"https://huggingface.co/datasets/hf-internal-testing/diffusers-images/resolve/main/text_to_video/video_2step.npy" | ||
"https://huggingface.co/datasets/hf-internal-testing/diffusers-images/resolve/main/text_to_video/video.npy" | ||
) | ||
|
||
pipe = TextToVideoSDPipeline.from_pretrained("damo-vilab/text-to-video-ms-1.7b") | ||
pipe = pipe.to("cuda") | ||
pipe.scheduler = DPMSolverMultistepScheduler.from_config(pipe.scheduler.config) | ||
pipe = pipe.to(torch_device) | ||
|
||
prompt = "Spiderman is surfing" | ||
generator = torch.Generator(device="cpu").manual_seed(0) | ||
|
||
video_frames = pipe(prompt, generator=generator, num_inference_steps=2, output_type="pt").frames | ||
video_frames = pipe(prompt, generator=generator, num_inference_steps=25, output_type="pt").frames | ||
video = video_frames.cpu().numpy() | ||
|
||
assert np.abs(expected_video - video).mean() < 5e-2 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If the two step model is passing the assertion, I think it's okay to remove it from nightly unless we have a strong reason to.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just to clarify. Do you mean you want to keep the two step and remove the full test or keep the full and remove the two step?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Remove the full and keep the two steps.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sweet!
* move slow pix2pixzero tests to nightly * move slow panorama tests to nightly * move txt2video full test to nightly * clean up * remove nightly test from text to video pipeline
What does this PR do?
Move certain slow tests to nightly. The following modules are not present in this list of important pipelines and are some of the slowest tests in the CI/CD pipeline.
The
test_full_model
test intests/pipelines/text_to_video/test_text_to_video.py
is a similar test totest_two_step_model
except that it runs for a greater number of inference steps. I propose moving the long running test to nightly and keeping the shorter 2 step test inSlowTests
Fixes # (issue)
Before submitting
documentation guidelines, and
here are tips on formatting docstrings.
Who can review?
Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.