add tests for F.crop and transforms.RandomCrop

[pmeier]

cc @vfdev-5

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/vision/7892

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 13 New Failures, 1 Pending, 19 Unrelated Failures

As of commit 5e1510b with merge base 58f834a ():

NEW FAILURES - The following jobs have failed:

• linux (linux.12xlarge, cpu) / linux-job (gh)
• macos (macos-m1-12) / macos-job (gh)
• onnx / linux-job (gh)
• python-types / linux-job (gh)
• unittests-extended / linux-job (gh)
• unittests-linux (3.10, linux.12xlarge, cpu) / linux-job (gh)
• unittests-linux (3.11, linux.12xlarge, cpu) / linux-job (gh)
• unittests-linux (3.8, linux.12xlarge, cpu) / linux-job (gh)
• unittests-linux (3.8, linux.g5.4xlarge.nvidia.gpu, cuda, 11.8) / linux-job (gh)
• unittests-macos (3.8, macos-m1-12) / macos-job (gh)
• unittests-prototype (3.8, linux.12xlarge, cpu) / linux-job (gh)
• unittests-windows (3.8, windows.g5.4xlarge.nvidia.gpu, cuda, 11.8) / windows-job (gh)
• windows (windows.4xlarge, cpu) / windows-job (gh)

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

• build / linux-job (gh)
• c-source / linux-job (gh)
• linux (linux.g5.4xlarge.nvidia.gpu, cuda, 11.8) / linux-job (gh)
• macos (macos-12) / macos-job (gh)
• python-source-and-configs / linux-job (gh)
• unittests-linux (3.9, linux.12xlarge, cpu) / linux-job (gh)
• unittests-macos (3.10, macos-12) / macos-job (gh)
• unittests-macos (3.11, macos-12) / macos-job (gh)
• unittests-macos (3.8, macos-12) / macos-job (gh)
• unittests-macos (3.9, macos-12) / macos-job (gh)
• unittests-prototype (3.10, linux.12xlarge, cpu) / linux-job (gh)
• unittests-prototype (3.11, linux.12xlarge, cpu) / linux-job (gh)
• unittests-prototype (3.8, linux.g5.4xlarge.nvidia.gpu, cuda, 11.8) / linux-job (gh)
• unittests-prototype (3.9, linux.12xlarge, cpu) / linux-job (gh)
• unittests-windows (3.10, windows.4xlarge, cpu) / windows-job (gh)
• unittests-windows (3.11, windows.4xlarge, cpu) / windows-job (gh)
• unittests-windows (3.8, windows.4xlarge, cpu) / windows-job (gh)
• unittests-windows (3.9, windows.4xlarge, cpu) / windows-job (gh)
• windows (windows.g5.4xlarge.nvidia.gpu, cuda, 11.8) / windows-job (gh)

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[NicolasHug]

Is this supposed to be 5 since height is 20?

I'm not sure I understand the "left, right, top, bottom side" and "left, right, top, bottom corner" notations though. Nor why we need so many configurations.

Suggested change

	dict(top=-5, left=-5, height=20, width=20),
	dict(top=5, left=-5, height=20, width=20),

[pmeier]

F.crop implicitly pads when we have a negative anchor or the crop size is larger than the input size

vision/torchvision/transforms/v2/functional/_geometry.py

Lines 1360 to 1368 in e928537

	if left < 0 or top < 0 or right > w or bottom > h:
	image = image[..., max(top, 0) : bottom, max(left, 0) : right]
	torch_padding = [
	max(min(right, 0) - left, 0),
	max(right - max(w, left), 0),
	max(min(bottom, 0) - top, 0),
	max(bottom - max(h, top), 0),
	]
	return _pad_with_scalar_fill(image, torch_padding, fill=0, padding_mode="constant")

Although I dislike it, we need to keep BC for it.

I tried to enumerate all possible cases here:

• valid crop, i.e. cropped region is fully on the inside
• cropped region is strictly larger than image, i.e. we need padding on all sides
• cropped region only contains one side of the image (4 times for each side)
• cropped region only contains a corner of the image (4 times for each corner)

[pmeier]

Here is a small snippet to verify the configurations listed here are actually correct:

import torch
from torchvision.transforms.v2 import functional as F

INPUT_SIZE = (21, 11)

CORRECTNESS_CROP_KWARGS = [
    # center
    dict(top=5, left=5, height=10, width=5),
    # larger than input, i.e. pad
    dict(top=-5, left=-5, height=30, width=20),
    # sides: left, right, top, bottom
    dict(top=-5, left=-5, height=30, width=10),
    dict(top=-5, left=5, height=30, width=10),
    dict(top=-5, left=-5, height=20, width=20),
    dict(top=5, left=-5, height=20, width=20),
    # corners: top-left, top-right, bottom-left, bottom-right
    dict(top=-5, left=-5, height=20, width=10),
    dict(top=-5, left=5, height=20, width=10),
    dict(top=5, left=-5, height=20, width=10),
    dict(top=5, left=5, height=20, width=10),
]

TITLE_WITH_KWARGS = zip(
    [
        "center",
        "pad",
        *[f"{side}_side" for side in ["left", "right", "top", "bottom"]],
        *[f"{corner}_corner" for corner in ["top_left", "top_right", "bottom_left", "bottom_right"]],
    ],
    CORRECTNESS_CROP_KWARGS,
)


image = torch.full((3, *INPUT_SIZE), 255, dtype=torch.uint8)

for title, kwargs in TITLE_WITH_KWARGS:
    F.to_pil_image(F.crop_image(image, **kwargs)).save(f"{title}.png")

[pmeier]

We should probably add support for this. Note that this is not a regression:

vision/torchvision/transforms/v2/functional/_geometry.py

Line 1260 in b9447fd

_pad_image_pil = _register_kernel_internal(pad, PIL.Image.Image)(_FP.pad)

[pmeier]

Driveby

[NicolasHug]

Thanks Philip, LGTM if green