`A.Compose`, `A.Sequential` components run multiple times inside of `A.OneOf` #1051

ProtossDragoon · 2021-11-03T08:07:05Z

🐛 Bug

Hi, I used A.OneOf method on my A.Compose objects,
but when I run entire pipeline, all components of A.Compose objects are called altogether.

To Reproduce

Steps to reproduce the behavior:

define augmentation

class Augmentation():

    def __init__(self, 
                 dataset_w:int=None, 
                 dataset_h:int=None, 
                 final_return_w:int=None, 
                 final_return_h:int=None):
        
        self.train_augmentation = self.set_train_augmentation(
            dataset_w=dataset_w,
            dataset_h=dataset_h,
            final_return_w=final_return_w,
            final_return_h=final_return_h,
        )

    def get_train_augmentation(self):
        return lambda image, mask: list(self.train_augmentation(image=image, mask=mask).values())
    
    def set_train_augmentation(self, **kwargs):
        ret = self.augmentation_method(**kwargs)
        return ret

    def _slightly_smaller(self,
        target_h:int,
        target_w:int, 
        original_h:int,
        original_w:int,
        ratio:float=0.5
    )->tuple:
        assert 0. < ratio and ratio < 1, f'0 < ratio {repr(ratio)} < 1'
        dw = int((original_w - target_w) * ratio)
        dh = int((original_h - target_h) * ratio)
        return (target_h + dh, target_w + dw)

    def _get_crop(self,
        target_h:int,
        target_w:int,
    )->list:
        tf = []
        tf += [A.RandomCrop(target_h, target_w, p=1.0)]
        return tf

    def _get_resize(self,
        target_h:int,
        target_w:int
    )->list:
        tf = []
        tf += [A.Resize(target_h, target_w, p=1.0)]
        return tf

    def _get_pad(self,
        target_h:int,
        target_w:int
    )->list:
        longer_side = target_h if target_h > target_w else target_w
        tf = []
        tf += [A.PadIfNeeded(
            min_height=longer_side,
            min_width=longer_side,
            p=1.0,
            border_mode=4)]
        return tf

    def _get_resize_crop(self,
        target_h:int,
        target_w:int,
        original_h:int,
        original_w:int,
    )->list:
        tf = []
        tf += self._get_resize(*self._slightly_smaller(
            target_h, target_w, 
            original_h, original_w)
        )
        tf += self._get_crop(target_h, target_w)
        return tf

    def _get_crop_resize(self,
        target_h:int,
        target_w:int,
        original_h:int,
        original_w:int,
    )->list:
        tf = []
        tf += self._get_crop(*self._slightly_smaller(
            target_h, target_w, 
            original_h, original_w)
        )
        tf += self._get_resize(target_h, target_w)
        return tf

    def _get_pad_resize(self,
        target_h:int,
        target_w:int,
        original_h:int,
        original_w:int,
    )->list:
        tf = []
        tf += self._get_pad(original_h, original_w)
        tf += self._get_reszie(target_h, target_w)
        return tf

    def _get_pad_crop(self,
        target_h:int,
        target_w:int,
        original_h:int,
        original_w:int,
    )->list:
        tf = []
        tf += self._get_pad(original_h, original_w)
        tf += self._get_crop(target_h, target_w)
        return tf

    def _get_pad_resize_crop(self,
        target_h:int,
        target_w:int,
        original_h:int,
        original_w:int,
    )->list:
        tf = []
        tf += self._get_pad(original_h, original_w)
        tf += self._get_resize(*self._slightly_smaller(
            target_h, target_w, 
            original_h, original_w)
        )
        tf += self._get_crop(target_h, target_w)
        return tf

    def _get_pad_crop_resize(self,
        target_h:int,
        target_w:int,
        original_h:int,
        original_w:int,
    )->list:
        tf = []
        tf += self._get_pad(original_h, original_w)
        tf += self._get_crop(*self._slightly_smaller(
            target_h, target_w, 
            original_h, original_w)
        )
        tf += self._get_resize(target_h, target_w)
        return tf

    def augmentation_method(self,
        dataset_w=None, 
        dataset_h=None,
        final_return_w=None,
        final_return_h=None,
        ):
        kwargs = {'target_h':final_return_h,
                  'target_w':final_return_w,
                  'original_h':dataset_h,
                  'original_w':dataset_w}

        print(self._get_resize_crop(**kwargs))
        print(self._get_crop_resize(**kwargs))
        print(self._get_pad_crop_resize(**kwargs))
        print(self._get_pad_resize_crop(**kwargs))
        tf = A.OneOf([
                 #A.Compose(self._get_resize_crop(**kwargs), p=1.0), # <-- this line makes error either.
                 A.Compose(self._get_crop_resize(**kwargs), p=1.0),
                 A.Compose(self._get_pad_crop_resize(**kwargs), p=1.0),
                 A.Compose(self._get_pad_resize_crop(**kwargs), p=1.0),
                ], p=1.0)
        return A.Compose(tf, p=1.0)

new instance

aug = Augmentation(DATASET_WIDTH, 
                   DATASET_HEIGHT,
                   DATASET_TRAINING_TARGET_W,
                   DATASET_TRAINING_TARGET_H)    
aug_fn_for_train = aug.get_train_augmentation()

run pipeline

image_train_debug, mask_train_debug = aug_fn_for_train(image, mask)

Expected behavior

Doing augmentation from randomly selected one of the composed pipeline.

tf = A.OneOf([
         A.Compose(self._get_resize_crop(**kwargs), p=1.0), # 25%
         A.Compose(self._get_crop_resize(**kwargs), p=1.0), # 25%
         A.Compose(self._get_pad_crop_resize(**kwargs), p=1.0), # 25%
         A.Compose(self._get_pad_resize_crop(**kwargs), p=1.0), # 25%
        ], p=1.0)

Environment

Albumentations version (e.g., 0.1.8): 0.1.12
Python version (e.g., 3.7): python3.8
OS (e.g., Linux): Google COLAB COLAB
How you installed albumentations (conda, pip, source): colab default installed
Any other relevant information:

Additional context

The text was updated successfully, but these errors were encountered:

BloodAxe · 2021-11-03T08:16:42Z

A.Compose designed to be top-level augmentation and not intended for nested use. You may want to change nested augmentations to A.Sequential

ProtossDragoon · 2021-11-03T09:41:19Z

@BloodAxe Thank you for advice! I couldn't search proper keyword like "nested" which wasn't familiar with me. :)
I updated albumentation from colab built-in version to 1.1.0 and removed A.Compose inside of A.OneOf block.
But another problem has occurred.

1.

tf = A.OneOf([
         #A.Sequential(self._get_resize_crop(**kwargs), p=1.0), # <-- this line still makes error.
         A.Sequential(self._get_crop_resize(**kwargs), p=1.0),
         A.Sequential(self._get_pad_crop_resize(**kwargs), p=1.0),
         A.Sequential(self._get_pad_resize_crop(**kwargs), p=1.0),
        ], p=1.0)
return A.Compose(tf, p=1.0)

When print inside of transform components,

print(tf)

nothing seemed to matter,

output

OneOf([
  Sequential([
    RandomCrop(always_apply=False, p=1.0, height=780, width=1280),
    Resize(always_apply=False, p=1.0, height=480, width=640, interpolation=1),
  ], p=1.0),
  Sequential([
    PadIfNeeded(always_apply=False, p=1.0, min_height=1920, min_width=1920, pad_height_divisor=None, pad_width_divisor=None, border_mode=4, value=None, mask_value=None),
    RandomCrop(always_apply=False, p=1.0, height=780, width=1280),
    Resize(always_apply=False, p=1.0, height=480, width=640, interpolation=1),
  ], p=1.0),
  Sequential([
    PadIfNeeded(always_apply=False, p=1.0, min_height=1920, min_width=1920, pad_height_divisor=None, pad_width_divisor=None, border_mode=4, value=None, mask_value=None),
    Resize(always_apply=False, p=1.0, height=780, width=1280, interpolation=1),
    RandomCrop(always_apply=False, p=1.0, height=480, width=640),
  ], p=1.0),
], p=1.0)

but result images are same.

2.

There is something even strange. When run this code without annotation,

tf = A.OneOf([
         A.Sequential(self._get_resize_crop(**kwargs), p=1.0),
         A.Sequential(self._get_crop_resize(**kwargs), p=1.0),
         A.Sequential(self._get_pad_crop_resize(**kwargs), p=1.0),
         A.Sequential(self._get_pad_resize_crop(**kwargs), p=1.0),
        ], p=1.0)
return A.Compose(tf, p=1.0)

print(tf)

output

OneOf([
  Sequential([
    Resize(always_apply=False, p=1.0, height=780, width=1280, interpolation=1),
    RandomCrop(always_apply=False, p=1.0, height=480, width=640),
  ], p=1.0),
  Sequential([
    RandomCrop(always_apply=False, p=1.0, height=780, width=1280),
    Resize(always_apply=False, p=1.0, height=480, width=640, interpolation=1),
  ], p=1.0),
  Sequential([
    PadIfNeeded(always_apply=False, p=1.0, min_height=1920, min_width=1920, pad_height_divisor=None, pad_width_divisor=None, border_mode=4, value=None, mask_value=None),
    RandomCrop(always_apply=False, p=1.0, height=780, width=1280),
    Resize(always_apply=False, p=1.0, height=480, width=640, interpolation=1),
  ], p=1.0),
  Sequential([
    PadIfNeeded(always_apply=False, p=1.0, min_height=1920, min_width=1920, pad_height_divisor=None, pad_width_divisor=None, border_mode=4, value=None, mask_value=None),
    Resize(always_apply=False, p=1.0, height=780, width=1280, interpolation=1),
    RandomCrop(always_apply=False, p=1.0, height=480, width=640),
  ], p=1.0),
], p=1.0)

Note that always_apply argument always False, every Sequential group gradually decreases image width and height

but error raises

ValueError: Requested crop size (780, 1280) is larger than the image size (480, 640)

Entire error description

output

ValueError                                Traceback (most recent call last)
<ipython-input-17-4725b0ed649d> in <module>()
     50 
     51         start = time.time()
---> 52         image_train_debug, mask_train_debug = aug_fn_for_train(image, mask)
     53         end = time.time()
     54         print('training augmentation time cost (per image):', end - start)

6 frames
<ipython-input-16-90611d3e3759> in <lambda>(image, mask)
     23 
     24     def get_train_augmentation(self):
---> 25         return lambda image, mask: list(self.train_augmentation(image=image, mask=mask).values())
     26 
     27     def get_valid_augmentation(self):

/content/gdrive/MyDrive/ColabWorkspace/albumentations/albumentations/core/composition.py in __call__(self, force_apply, *args, **data)
    213 
    214         for idx, t in enumerate(transforms):
--> 215             data = t(force_apply=force_apply, **data)
    216 
    217             if check_each_transform:

/content/gdrive/MyDrive/ColabWorkspace/albumentations/albumentations/core/composition.py in __call__(self, **data)
    604     def __call__(self, **data):
    605         for t in self.transforms:
--> 606             data = t(**data)
    607         return data

/content/gdrive/MyDrive/ColabWorkspace/albumentations/albumentations/core/transforms_interface.py in __call__(self, force_apply, *args, **kwargs)
     95                     )
     96                 kwargs[self.save_key][id(self)] = deepcopy(params)
---> 97             return self.apply_with_params(params, **kwargs)
     98 
     99         return kwargs

/content/gdrive/MyDrive/ColabWorkspace/albumentations/albumentations/core/transforms_interface.py in apply_with_params(self, params, force_apply, **kwargs)
    110                 target_function = self._get_target_function(key)
    111                 target_dependencies = {k: kwargs[k] for k in self.target_dependence.get(key, [])}
--> 112                 res[key] = target_function(arg, **dict(params, **target_dependencies))
    113             else:
    114                 res[key] = None

/content/gdrive/MyDrive/ColabWorkspace/albumentations/albumentations/augmentations/crops/transforms.py in apply(self, img, h_start, w_start, **params)
     45 
     46     def apply(self, img, h_start=0, w_start=0, **params):
---> 47         return F.random_crop(img, self.height, self.width, h_start, w_start)
     48 
     49     def get_params(self):

/content/gdrive/MyDrive/ColabWorkspace/albumentations/albumentations/augmentations/crops/functional.py in random_crop(img, crop_height, crop_width, h_start, w_start)
     26             "Requested crop size ({crop_height}, {crop_width}) is "
     27             "larger than the image size ({height}, {width})".format(
---> 28                 crop_height=crop_height, crop_width=crop_width, height=height, width=width
     29             )
     30         )

ValueError: Requested crop size (780, 1280) is larger than the image size (480, 640)

BloodAxe · 2021-11-03T10:11:50Z

The second sequential operation could cause this:

 Sequential([
    RandomCrop(always_apply=False, p=1.0, height=780, width=1280),
    Resize(always_apply=False, p=1.0, height=480, width=640, interpolation=1),
  ], p=1.0),

Here you trying to get a random crop of size 1280x780. Depending on your dataset this may or may be not guaranteed that all images has sufficient size. If some image comes in smaller size, this operation will raise an exception (Like you see above). Could be your case.

ProtossDragoon · 2021-11-03T10:35:21Z

If problem was came from dataset as you said @BloodAxe, problem should occur when I run a single Sequential pipeline, but every time, every single Sequential pipelines used same .png file and works fine.

1.

tf = A.OneOf([
         A.Sequential(self._get_resize_crop(**kwargs), p=1.0),
         #A.Sequential(self._get_crop_resize(**kwargs), p=1.0),
         #A.Sequential(self._get_pad_crop_resize(**kwargs), p=1.0),
         #A.Sequential(self._get_pad_resize_crop(**kwargs), p=1.0),
        ], p=1.0)

['/content/gdrive/MyDrive/data/pedestrian/surface_masking/Surface_021/MP_SEL_SUR_002440.jpg', '/content/gdrive/MyDrive/data/pedestrian/surface_masking/Surface_021/MASK/MP_SEL_SUR_002440.png']

2.

tf = A.OneOf([
         #A.Sequential(self._get_resize_crop(**kwargs), p=1.0),
         A.Sequential(self._get_crop_resize(**kwargs), p=1.0),
         #A.Sequential(self._get_pad_crop_resize(**kwargs), p=1.0),
         #A.Sequential(self._get_pad_resize_crop(**kwargs), p=1.0),
        ], p=1.0)

['/content/gdrive/MyDrive/data/pedestrian/surface_masking/Surface_021/MP_SEL_SUR_002440.jpg', '/content/gdrive/MyDrive/data/pedestrian/surface_masking/Surface_021/MASK/MP_SEL_SUR_002440.png']

3.

tf = A.OneOf([
         #A.Sequential(self._get_resize_crop(**kwargs), p=1.0),
         #A.Sequential(self._get_crop_resize(**kwargs), p=1.0),
         A.Sequential(self._get_pad_crop_resize(**kwargs), p=1.0),
         #A.Sequential(self._get_pad_resize_crop(**kwargs), p=1.0),
        ], p=1.0)

['/content/gdrive/MyDrive/data/pedestrian/surface_masking/Surface_021/MP_SEL_SUR_002440.jpg', '/content/gdrive/MyDrive/data/pedestrian/surface_masking/Surface_021/MASK/MP_SEL_SUR_002440.png']

4.

tf = A.OneOf([
         #A.Sequential(self._get_resize_crop(**kwargs), p=1.0),
         #A.Sequential(self._get_crop_resize(**kwargs), p=1.0),
         #A.Sequential(self._get_pad_crop_resize(**kwargs), p=1.0),
         A.Sequential(self._get_pad_resize_crop(**kwargs), p=1.0),
        ], p=1.0)

['/content/gdrive/MyDrive/data/pedestrian/surface_masking/Surface_021/MP_SEL_SUR_002440.jpg', '/content/gdrive/MyDrive/data/pedestrian/surface_masking/Surface_021/MASK/MP_SEL_SUR_002440.png']

But when A.OneOf() gets double, triple Sequential pipeline, generates strange result.

BloodAxe · 2021-11-03T11:03:15Z

Hm, actually I was able to reproduce this issue. We will investigate this. Thanks for reporting!

Dipet · 2021-11-03T12:04:06Z

I think problem in augmentation_method.
Change A.Compose(tf, p=1.0) to A.Compose([tf], p=1.0)

ProtossDragoon · 2021-11-04T13:23:55Z

Thanks, @Dipet's reply helped to fix solve the issue.

albumentations/albumentations/core/composition.py

Lines 37 to 58 in 6c733c4

    
           class Transforms: 
        
               def __init__(self, transforms): 
        
                   self.transforms = transforms 
        
                   self.start_end = self._find_dual_start_end(transforms) 
        
               def _find_dual_start_end(self, transforms): 
        
                   dual_start_end = None 
        
                   last_dual = None 
        
                   for idx, transform in enumerate(transforms): 
        
                       if isinstance(transform, DualTransform): 
        
                           last_dual = idx 
        
                           if dual_start_end is None: 
        
                               dual_start_end = [idx] 
        
                       if isinstance(transform, BaseCompose): 
        
                           inside = self._find_dual_start_end(transform) 
        
                           if inside is not None: 
        
                               last_dual = idx 
        
                               if dual_start_end is None: 
        
                                   dual_start_end = [idx] 
        
                   if dual_start_end is not None: 
        
                       dual_start_end.append(last_dual) 
        
                   return dual_start_end

Do you have a plan to make transforms argument get list only?

Dipet · 2021-11-04T13:53:03Z

Yes, I am working on this problem.

ProtossDragoon changed the title ~~A.Compose components run multiple times inside of A.OneOf~~ A.Compose, A.Sequential components run multiple times inside of A.OneOf Nov 3, 2021

BloodAxe added the bug Something isn't working label Nov 3, 2021

Dipet mentioned this issue Nov 5, 2021

Warning if Compose got single transform, wrap transform into list #1055

Merged

Dipet closed this as completed in #1055 Nov 6, 2021

FlorentMeyer mentioned this issue Feb 22, 2023

Behaviour of Compose and Sequential #1408

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

`A.Compose`, `A.Sequential` components run multiple times inside of `A.OneOf` #1051

`A.Compose`, `A.Sequential` components run multiple times inside of `A.OneOf` #1051

ProtossDragoon commented Nov 3, 2021

BloodAxe commented Nov 3, 2021

ProtossDragoon commented Nov 3, 2021 •

edited

Loading

BloodAxe commented Nov 3, 2021

ProtossDragoon commented Nov 3, 2021 •

edited

Loading

BloodAxe commented Nov 3, 2021

Dipet commented Nov 3, 2021

ProtossDragoon commented Nov 4, 2021

Dipet commented Nov 4, 2021

A.Compose, A.Sequential components run multiple times inside of A.OneOf #1051

A.Compose, A.Sequential components run multiple times inside of A.OneOf #1051

Comments

ProtossDragoon commented Nov 3, 2021

🐛 Bug

To Reproduce

Expected behavior

Environment

Additional context

BloodAxe commented Nov 3, 2021

ProtossDragoon commented Nov 3, 2021 • edited Loading

1.

2.

Entire error description

BloodAxe commented Nov 3, 2021

ProtossDragoon commented Nov 3, 2021 • edited Loading

1.

2.

3.

4.

BloodAxe commented Nov 3, 2021

Dipet commented Nov 3, 2021

ProtossDragoon commented Nov 4, 2021

Dipet commented Nov 4, 2021

`A.Compose`, `A.Sequential` components run multiple times inside of `A.OneOf` #1051

`A.Compose`, `A.Sequential` components run multiple times inside of `A.OneOf` #1051

ProtossDragoon commented Nov 3, 2021 •

edited

Loading

ProtossDragoon commented Nov 3, 2021 •

edited

Loading