replace remaining asserts #5636

jdsgomes · 2022-03-17T12:37:26Z

Replace torch._asserts as a follow up to this PR #5587

Thanks @datumbox for raising it in #5494

facebook-github-bot · 2022-03-17T12:37:34Z

💊 CI failures summary and remediations

As of commit 5eac9b8 (more details on the Dr. CI page):

7/10 failures introduced in this PR
3/10 broken upstream at merge base 375e4ab since Mar 21

🕵️ 6 new failures recognized by patterns

The following CI failures do not appear to be due to upstream breakages:

unittest_windows_gpu_py3.8 (1/6)

Step: "Run tests" (full log | diagnosis details | 🔁 rerun)

test/test_utils.py::test_flow_to_image_errors[i...low should be of dtype torch.float] SKIPPED [ 98%]

test/test_utils.py::test_draw_keypoints_colored[red] SKIPPED (We're ...) [ 98%]
test/test_utils.py::test_draw_keypoints_colored[#FF00FF] SKIPPED (We...) [ 98%]
test/test_utils.py::test_draw_keypoints_colored[colors2] SKIPPED (We...) [ 98%]
test/test_utils.py::test_draw_keypoints_errors SKIPPED (We're in a C...) [ 98%]
test/test_utils.py::test_flow_to_image[True] SKIPPED (We're in a Cir...) [ 98%]
test/test_utils.py::test_flow_to_image[False] SKIPPED (We're in a Ci...) [ 98%]
test/test_utils.py::test_flow_to_image_errors[input_flow0-Input flow should have shape] SKIPPED [ 98%]
test/test_utils.py::test_flow_to_image_errors[input_flow1-Input flow should have shape] SKIPPED [ 98%]
test/test_utils.py::test_flow_to_image_errors[input_flow2-Input flow should have shape] SKIPPED [ 98%]
test/test_utils.py::test_flow_to_image_errors[input_flow3-Input flow should have shape] SKIPPED [ 98%]
test/test_utils.py::test_flow_to_image_errors[input_flow4-Flow should be of dtype torch.float] SKIPPED [ 98%]
test/test_video_gpu_decoder.py::TestVideoGPUDecoder::test_frame_reading[RATRACE_wave_f_nm_np1_fr_goo_37.avi] SKIPPED [ 98%]
test/test_video_gpu_decoder.py::TestVideoGPUDecoder::test_frame_reading[TrumanShow_wave_f_nm_np1_fr_med_26.avi] SKIPPED [ 98%]
test/test_video_gpu_decoder.py::TestVideoGPUDecoder::test_frame_reading[v_SoccerJuggling_g23_c01.avi] SKIPPED [ 98%]
test/test_video_gpu_decoder.py::TestVideoGPUDecoder::test_frame_reading[v_SoccerJuggling_g24_c01.avi] SKIPPED [ 98%]
test/test_video_gpu_decoder.py::TestVideoGPUDecoder::test_frame_reading[R6llTwEh07w.mp4] SKIPPED [ 98%]
test/test_video_gpu_decoder.py::TestVideoGPUDecoder::test_frame_reading[SOX5yA1l24A.mp4] SKIPPED [ 98%]
test/test_video_gpu_decoder.py::TestVideoGPUDecoder::test_frame_reading[WUzgd7C1pWA.mp4] SKIPPED [ 98%]
test/test_video_gpu_decoder.py::TestVideoGPUDecoder::test_seek_reading[C:\\Users\\circleci\\project\\test\\assets\\videos\\v_SoccerJuggling_g23_c01.avi-8.0-True] SKIPPED [ 98%]
test/test_video_gpu_decoder.py::TestVideoGPUDecoder::test_seek_reading[C:\\Users\\circleci\\project\\test\\assets\\videos\\v_SoccerJuggling_g23_c01.avi-8.0-False] SKIPPED [ 98%]
test/test_video_gpu_decoder.py::TestVideoGPUDecoder::test_seek_reading[C:\\Users\\circleci\\project\\test\\assets\\videos\\v_SoccerJuggling_g24_c01.avi-8.0-True] SKIPPED [ 98%]

unittest_linux_cpu_py3.9 (2/6)

Step: "Run tests" (full log | diagnosis details | 🔁 rerun)

/root/project/torchvision/io/video.py:406: Runt...log: [mov,mp4,m4a,3gp,3g2,mj2] moov atom not found

test/test_datasets_video_utils.py::TestVideo::test_video_clips_custom_fps
  /root/project/torchvision/datasets/video_utils.py:218: UserWarning: There aren't enough frames in the current video to get a clip for the given clip length and frames between clips. The video (and potentially others) will be skipped.
    warnings.warn(

test/test_image.py::test_decode_png[L-ImageReadMode.GRAY-palette_pytorch.png]
test/test_image.py::test_decode_png[RGB-ImageReadMode.RGB-palette_pytorch.png]
  /root/project/env/lib/python3.9/site-packages/PIL/Image.py:945: UserWarning: Palette images with Transparency expressed in bytes should be converted to RGBA images
    warnings.warn(

test/test_io.py::TestVideo::test_read_video_timestamps_corrupted_file
  /root/project/torchvision/io/video.py:406: RuntimeWarning: Failed to open container for /tmp/tmph2kfawjj.mp4; Caught error: [Errno 1094995529] Invalid data found when processing input: '/tmp/tmph2kfawjj.mp4'; last error log: [mov,mp4,m4a,3gp,3g2,mj2] moov atom not found
    warnings.warn(msg, RuntimeWarning)

test/test_models.py::test_memory_efficient_densenet[densenet121]
test/test_models.py::test_memory_efficient_densenet[densenet169]
test/test_models.py::test_memory_efficient_densenet[densenet201]
test/test_models.py::test_memory_efficient_densenet[densenet161]
  /root/project/env/lib/python3.9/site-packages/torch/nn/modules/module.py:1383: UserWarning: positional arguments and argument "destination" are deprecated. nn.Module.state_dict will not accept them in the future. Refer to https://pytorch.org/docs/master/generated/torch.nn.Module.html#torch.nn.Module.state_dict for details.
    warnings.warn(

test/test_models.py::test_memory_efficient_densenet[densenet121]

unittest_linux_cpu_py3.10 (3/6)

Step: "Run tests" (full log | diagnosis details | 🔁 rerun)

/root/project/torchvision/io/video.py:406: Runt...log: [mov,mp4,m4a,3gp,3g2,mj2] moov atom not found

test/test_image.py::test_decode_png[L-ImageReadMode.GRAY-palette_pytorch.png]
test/test_image.py::test_decode_png[RGB-ImageReadMode.RGB-palette_pytorch.png]
  /root/project/env/lib/python3.10/site-packages/PIL/Image.py:945: UserWarning: Palette images with Transparency expressed in bytes should be converted to RGBA images
    warnings.warn(

test/test_io.py::TestVideo::test_probe_video_from_memory
  /root/project/torchvision/io/_video_opt.py:419: UserWarning: The given buffer is not writable, and PyTorch does not support non-writable tensors. This means you can write to the underlying (supposedly non-writable) buffer using the tensor. You may want to copy the buffer to protect its data or make it writable before converting it to a tensor. This type of warning will be suppressed for the rest of this program. (Triggered internally at  /opt/conda/conda-bld/pytorch_1647932909338/work/torch/csrc/utils/tensor_new.cpp:954.)
    video_data = torch.frombuffer(video_data, dtype=torch.uint8)

test/test_io.py::TestVideo::test_read_video_timestamps_corrupted_file
  /root/project/torchvision/io/video.py:406: RuntimeWarning: Failed to open container for /tmp/tmppjc31n8j.mp4; Caught error: [Errno 1094995529] Invalid data found when processing input: '/tmp/tmppjc31n8j.mp4'; last error log: [mov,mp4,m4a,3gp,3g2,mj2] moov atom not found
    warnings.warn(msg, RuntimeWarning)

test/test_models.py::test_memory_efficient_densenet[densenet121]
test/test_models.py::test_memory_efficient_densenet[densenet169]
test/test_models.py::test_memory_efficient_densenet[densenet201]
test/test_models.py::test_memory_efficient_densenet[densenet161]
  /root/project/env/lib/python3.10/site-packages/torch/nn/modules/module.py:1383: UserWarning: positional arguments and argument "destination" are deprecated. nn.Module.state_dict will not accept them in the future. Refer to https://pytorch.org/docs/master/generated/torch.nn.Module.html#torch.nn.Module.state_dict for details.
    warnings.warn(

test/test_models.py::test_memory_efficient_densenet[densenet121]

unittest_linux_cpu_py3.7 (4/6)

Step: "Run tests" (full log | diagnosis details | 🔁 rerun)

/root/project/torchvision/io/video.py:406: Runt...log: [mov,mp4,m4a,3gp,3g2,mj2] moov atom not found

test/test_image.py::test_decode_png[L-ImageReadMode.GRAY-palette_pytorch.png]
test/test_image.py::test_decode_png[RGB-ImageReadMode.RGB-palette_pytorch.png]
  /root/project/env/lib/python3.7/site-packages/PIL/Image.py:946: UserWarning: Palette images with Transparency expressed in bytes should be converted to RGBA images
    "Palette images with Transparency expressed in bytes should be "

test/test_io.py::TestVideo::test_probe_video_from_memory
  /root/project/torchvision/io/_video_opt.py:419: UserWarning: The given buffer is not writable, and PyTorch does not support non-writable tensors. This means you can write to the underlying (supposedly non-writable) buffer using the tensor. You may want to copy the buffer to protect its data or make it writable before converting it to a tensor. This type of warning will be suppressed for the rest of this program. (Triggered internally at  /opt/conda/conda-bld/pytorch_1647932923740/work/torch/csrc/utils/tensor_new.cpp:954.)
    video_data = torch.frombuffer(video_data, dtype=torch.uint8)

test/test_io.py::TestVideo::test_read_video_timestamps_corrupted_file
  /root/project/torchvision/io/video.py:406: RuntimeWarning: Failed to open container for /tmp/tmptozi041n.mp4; Caught error: [Errno 1094995529] Invalid data found when processing input: '/tmp/tmptozi041n.mp4'; last error log: [mov,mp4,m4a,3gp,3g2,mj2] moov atom not found
    warnings.warn(msg, RuntimeWarning)

test/test_models.py::test_memory_efficient_densenet[densenet121]
test/test_models.py::test_memory_efficient_densenet[densenet169]
test/test_models.py::test_memory_efficient_densenet[densenet201]
test/test_models.py::test_memory_efficient_densenet[densenet161]
  /root/project/env/lib/python3.7/site-packages/torch/nn/modules/module.py:1384: UserWarning: positional arguments and argument "destination" are deprecated. nn.Module.state_dict will not accept them in the future. Refer to https://pytorch.org/docs/master/generated/torch.nn.Module.html#torch.nn.Module.state_dict for details.
    " and ".join(warn_msg) + " are deprecated. nn.Module.state_dict will not accept them in the future. "

test/test_models.py::test_memory_efficient_densenet[densenet121]

unittest_linux_gpu_py3.8 (5/6)

Step: "Run tests" (full log | diagnosis details | 🔁 rerun)

test/test_utils.py::test_flow_to_image_errors[i...type torch.float] �[33mSKIPPED�[0m�[31m [ 98%]�[0m

test/test_utils.py::test_draw_keypoints_colored[red] �[33mSKIPPED�[0m (We're ...)�[31m [ 98%]�[0m

test/test_utils.py::test_draw_keypoints_colored[#FF00FF] �[33mSKIPPED�[0m (We...)�[31m [ 98%]�[0m

test/test_utils.py::test_draw_keypoints_colored[colors2] �[33mSKIPPED�[0m (We...)�[31m [ 98%]�[0m

test/test_utils.py::test_draw_keypoints_errors �[33mSKIPPED�[0m (We're in a C...)�[31m [ 98%]�[0m

test/test_utils.py::test_flow_to_image[True] �[33mSKIPPED�[0m (We're in a Cir...)�[31m [ 98%]�[0m

test/test_utils.py::test_flow_to_image[False] �[33mSKIPPED�[0m (We're in a Ci...)�[31m [ 98%]�[0m

test/test_utils.py::test_flow_to_image_errors[input_flow0-Input flow should have shape] �[33mSKIPPED�[0m�[31m [ 98%]�[0m

test/test_utils.py::test_flow_to_image_errors[input_flow1-Input flow should have shape] �[33mSKIPPED�[0m�[31m [ 98%]�[0m

test/test_utils.py::test_flow_to_image_errors[input_flow2-Input flow should have shape] �[33mSKIPPED�[0m�[31m [ 98%]�[0m

test/test_utils.py::test_flow_to_image_errors[input_flow3-Input flow should have shape] �[33mSKIPPED�[0m�[31m [ 98%]�[0m

test/test_utils.py::test_flow_to_image_errors[input_flow4-Flow should be of dtype torch.float] �[33mSKIPPED�[0m�[31m [ 98%]�[0m

test/test_video_gpu_decoder.py::TestVideoGPUDecoder::test_frame_reading[RATRACE_wave_f_nm_np1_fr_goo_37.avi] �[33mSKIPPED�[0m�[31m [ 98%]�[0m

test/test_video_gpu_decoder.py::TestVideoGPUDecoder::test_frame_reading[TrumanShow_wave_f_nm_np1_fr_med_26.avi] �[33mSKIPPED�[0m�[31m [ 98%]�[0m

test/test_video_gpu_decoder.py::TestVideoGPUDecoder::test_frame_reading[v_SoccerJuggling_g23_c01.avi] �[33mSKIPPED�[0m�[31m [ 98%]�[0m

test/test_video_gpu_decoder.py::TestVideoGPUDecoder::test_frame_reading[v_SoccerJuggling_g24_c01.avi] �[33mSKIPPED�[0m�[31m [ 98%]�[0m

test/test_video_gpu_decoder.py::TestVideoGPUDecoder::test_frame_reading[R6llTwEh07w.mp4] �[33mSKIPPED�[0m�[31m [ 98%]�[0m

test/test_video_gpu_decoder.py::TestVideoGPUDecoder::test_frame_reading[SOX5yA1l24A.mp4] �[33mSKIPPED�[0m�[31m [ 98%]�[0m

test/test_video_gpu_decoder.py::TestVideoGPUDecoder::test_frame_reading[WUzgd7C1pWA.mp4] �[33mSKIPPED�[0m�[31m [ 98%]�[0m

test/test_video_gpu_decoder.py::TestVideoGPUDecoder::test_seek_reading[/home/circleci/project/test/assets/videos/v_SoccerJuggling_g23_c01.avi-8.0-True] �[33mSKIPPED�[0m�[31m [ 98%]�[0m

test/test_video_gpu_decoder.py::TestVideoGPUDecoder::test_seek_reading[/home/circleci/project/test/assets/videos/v_SoccerJuggling_g23_c01.avi-8.0-False] �[33mSKIPPED�[0m�[31m [ 98%]�[0m

test/test_video_gpu_decoder.py::TestVideoGPUDecoder::test_seek_reading[/home/circleci/project/test/assets/videos/v_SoccerJuggling_g24_c01.avi-8.0-True] �[33mSKIPPED�[0m�[31m [ 98%]�[0m

unittest_linux_cpu_py3.8 (6/6)

Step: "Run tests" (full log | diagnosis details | 🔁 rerun)

/root/project/torchvision/io/video.py:406: Runt...log: [mov,mp4,m4a,3gp,3g2,mj2] moov atom not found

test/test_image.py::test_decode_png[L-ImageReadMode.GRAY-palette_pytorch.png]
test/test_image.py::test_decode_png[RGB-ImageReadMode.RGB-palette_pytorch.png]
  /root/project/env/lib/python3.8/site-packages/PIL/Image.py:945: UserWarning: Palette images with Transparency expressed in bytes should be converted to RGBA images
    warnings.warn(

test/test_io.py::TestVideo::test_probe_video_from_memory
  /root/project/torchvision/io/_video_opt.py:419: UserWarning: The given buffer is not writable, and PyTorch does not support non-writable tensors. This means you can write to the underlying (supposedly non-writable) buffer using the tensor. You may want to copy the buffer to protect its data or make it writable before converting it to a tensor. This type of warning will be suppressed for the rest of this program. (Triggered internally at  /opt/conda/conda-bld/pytorch_1647932931174/work/torch/csrc/utils/tensor_new.cpp:954.)
    video_data = torch.frombuffer(video_data, dtype=torch.uint8)

test/test_io.py::TestVideo::test_read_video_timestamps_corrupted_file
  /root/project/torchvision/io/video.py:406: RuntimeWarning: Failed to open container for /tmp/tmpxs4gzkd6.mp4; Caught error: [Errno 1094995529] Invalid data found when processing input: '/tmp/tmpxs4gzkd6.mp4'; last error log: [mov,mp4,m4a,3gp,3g2,mj2] moov atom not found
    warnings.warn(msg, RuntimeWarning)

test/test_models.py::test_memory_efficient_densenet[densenet121]
test/test_models.py::test_memory_efficient_densenet[densenet169]
test/test_models.py::test_memory_efficient_densenet[densenet201]
test/test_models.py::test_memory_efficient_densenet[densenet161]
  /root/project/env/lib/python3.8/site-packages/torch/nn/modules/module.py:1383: UserWarning: positional arguments and argument "destination" are deprecated. nn.Module.state_dict will not accept them in the future. Refer to https://pytorch.org/docs/master/generated/torch.nn.Module.html#torch.nn.Module.state_dict for details.
    warnings.warn(

test/test_models.py::test_memory_efficient_densenet[densenet121]

1 failure not recognized by patterns:

Job	Step	Action
^{unittest_prototype}	^{Run tests}	🔁 rerun

🚧 3 ongoing upstream failures:

These were probably caused by upstream breakages that are not fixed yet.

binary_linux_conda_py3.10_cu115 since Mar 21 (fbc8ea4)
- 🔁 rerun
binary_linux_conda_py3.8_cu115 since Mar 21 (fbc8ea4)
- 🔁 rerun
binary_linux_conda_py3.7_cu115 since Mar 21 (fbc8ea4)
- 🔁 rerun

This comment was automatically generated by Dr. CI (expand for details).

Please report bugs/suggestions to the (internal) Dr. CI Users group.

Click here to manually regenerate this comment.

NicolasHug

Thanks @jdsgomes

jdsgomes · 2022-03-17T15:03:39Z

tests are failing because of this change: https://github.com/pytorch/vision/pull/5636/files#diff-678e419692ae76586d9f58ce62d2841be82d607637b7d507ead76eca412fcdd1R244

I am thinking to revert and leave this particular torch._assert because its traceable.
@NicolasHug do you know if I have other options here?

NicolasHug · 2022-03-22T09:16:44Z

Sorry, I thought I had replied to this comment but it looks like I didn't 🤦‍♂️

I don't know if there is a good workaround for this. But I'm surprised that we did not encouter the same error for the changes we did in #5587. Do you know why that might be the case?

datumbox · 2022-03-22T10:06:13Z

I had a very quick look on #5587 and I think it works because all of our Exceptions are either on utils/methods that are not part of the models or on model constructors. I agree with jdgomes that we should probably close the PR and leave the assertions as is.

NicolasHug · 2022-03-22T10:51:48Z

Looks like even in #5587 there were a bunch of changes made to the forward pass of e.g. RAFT, but it looks like we don't test RAFT against fx, which is probably why tests were passing before

jdsgomes · 2022-03-22T14:55:28Z

Abandoning this PR and will follow up with another PR reverting some of the recent exceptions added after the offline discussion with @datumbox

replace remaining asserts

0d089d2

pytorch-bot bot added the ciflow/default label Mar 17, 2022

facebook-github-bot added the cla signed label Mar 17, 2022

jdsgomes mentioned this pull request Mar 17, 2022

Replace asserts with exceptions #5494

Closed

NicolasHug approved these changes Mar 17, 2022

View reviewed changes

fix tests

5198317

Merge branch 'main' into replace_remaining_asserts

5eac9b8

jdsgomes closed this Mar 22, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

replace remaining asserts #5636

replace remaining asserts #5636

Uh oh!

jdsgomes commented Mar 17, 2022

Uh oh!

facebook-github-bot commented Mar 17, 2022 •

edited

Loading

Uh oh!

NicolasHug left a comment

Uh oh!

jdsgomes commented Mar 17, 2022

Uh oh!

NicolasHug commented Mar 22, 2022

Uh oh!

datumbox commented Mar 22, 2022

Uh oh!

NicolasHug commented Mar 22, 2022 •

edited

Loading

Uh oh!

jdsgomes commented Mar 22, 2022

Uh oh!

Uh oh!

replace remaining asserts #5636

replace remaining asserts #5636

Uh oh!

Conversation

jdsgomes commented Mar 17, 2022

Uh oh!

facebook-github-bot commented Mar 17, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

💊 CI failures summary and remediations

🕵️ 6 new failures recognized by patterns

unittest_windows_gpu_py3.8 (1/6)

unittest_linux_cpu_py3.9 (2/6)

unittest_linux_cpu_py3.10 (3/6)

unittest_linux_cpu_py3.7 (4/6)

unittest_linux_gpu_py3.8 (5/6)

unittest_linux_cpu_py3.8 (6/6)

1 failure not recognized by patterns:

🚧 3 ongoing upstream failures:

Uh oh!

NicolasHug left a comment

Choose a reason for hiding this comment

Uh oh!

jdsgomes commented Mar 17, 2022

Uh oh!

NicolasHug commented Mar 22, 2022

Uh oh!

datumbox commented Mar 22, 2022

Uh oh!

NicolasHug commented Mar 22, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jdsgomes commented Mar 22, 2022

Uh oh!

Uh oh!

facebook-github-bot commented Mar 17, 2022 •

edited

Loading

NicolasHug commented Mar 22, 2022 •

edited

Loading