Error occurred when using "RandomMosaic"

[Ranhuiryan]

Describe the bug

An assertion error: assert 'mix_results' in results occurred when using "RandomMosaic"

Reproduction

1. What command or script did you run?

   python tools/train.py custom_configs/deeplabv3plus/deeplabv3plus_r50-d8_256x256_80k_irfissure_fusion_60.py
   

2. Did you make any modifications on the code or config? Did you understand what you have modified?

I'm modified the config in order to use my own dataset configration.

Here's my dataset config

dataset_type = 'IRFissure'
...
crop_size = (256, 256) 
img_scale = (320, 256)
train_pipeline = [
    dict(type='LoadImageFromFile'),
    dict(type='LoadAnnotations'),
    dict(type='Resize', img_scale=img_scale, ratio_range=(1., 2.)), 

    dict(type='RandomMosaic', prob=0.2, img_scale=tuple(size * factor for size, factor in zip(img_scale, (2, 2)))),
    dict(type='RandomCrop', crop_size=crop_size, cat_max_ratio=0.75),
    dict(type='RandomFlip', prob=0.5),
    dict(type='RandomRotate', prob=0.5, degree=(0, 360)),
    dict(type='PhotoMetricDistortion'),
    dict(type='RandomCutOut', prob=0.5, n_holes=5, cutout_shape=(50, 50)),
    dict(type='Pad', size=crop_size, pad_val=0, seg_pad_val=255),
    dict(type='Normalize', **img_norm_cfg),

    dict(type='DefaultFormatBundle'),
    dict(type='Collect', keys=['img', 'gt_semantic_seg']),
]
test_pipeline = [
    dict(type='LoadImageFromFile'),
    dict(
        type='MultiScaleFlipAug',
        img_scale=img_scale,
        # img_ratios=[0.5, 0.75, 1.0, 1.25, 1.5, 1.75],
        flip=False,
        transforms=[
            dict(type='Resize', keep_ratio=True),
            dict(type='RandomFlip'),
            dict(type='Normalize', **img_norm_cfg),
            dict(type='ImageToTensor', keys=['img']),
            dict(type='Collect', keys=['img']),
        ])
]
data = dict(
    ...
    train=dict(
        type='RepeatDataset',
        times=5,
        dataset=dict(
            type=dataset_type,
            data_root=data_root,
            img_suffix=img_suffix,
            img_dir='images',
            ann_dir='annotes',
            split='splits/train.txt',
            pipeline=train_pipeline)),
    val=dict(
        ...),
    test=dict(
        ...))

3. What dataset did you use?

A CustomDataset type, it's definition is given below.

@DATASETS.register_module()
class IRFissure(CustomDataset):

    CLASSES = ('ground', 'fissure')

    PALETTE = [[128, 128, 128], [38, 127, 129]]

    def __init__(self, split, **kwargs):
        super().__init__(seg_map_suffix='.png', split=split, **kwargs)
        assert osp.exists(self.img_dir) and self.split is not None

Environment

1. Please run python mmseg/utils/collect_env.py to collect necessary environment information and paste it here.

sys.platform: linux
Python: 3.10.4 (main, Mar 31 2022, 08:41:55) [GCC 7.5.0]
CUDA available: True
GPU 0: NVIDIA GeForce RTX 3080 Ti
CUDA_HOME: None
GCC: gcc (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0
PyTorch: 1.11.0
PyTorch compiling details: PyTorch built with:
  - GCC 7.3
  - C++ Version: 201402
  - Intel(R) oneAPI Math Kernel Library Version 2021.4-Product Build 20210904 for Intel(R) 64 architecture applications
  - Intel(R) MKL-DNN v2.5.2 (Git Hash a9302535553c73243c632ad3c4c80beec3d19a1e)
  - OpenMP 201511 (a.k.a. OpenMP 4.5)
  - LAPACK is enabled (usually provided by MKL)
  - NNPACK is enabled
  - CPU capability usage: AVX2
  - CUDA Runtime 11.3
  - NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_61,code=sm_61;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86;-gencode;arch=compute_37,code=compute_37
  - CuDNN 8.2
  - Magma 2.5.2
  - Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=11.3, CUDNN_VERSION=8.2.0, CXX_COMPILER=/opt/rh/devtoolset-7/root/usr/bin/c++, CXX_FLAGS= -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -DNDEBUG -DUSE_KINETO -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -DEDGE_PROFILER_USE_KINETO -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-sign-compare -Wno-unused-parameter -Wno-unused-function -Wno-unused-result -Wno-unused-local-typedefs -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=1.11.0, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=OFF, USE_MPI=OFF, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON, USE_ROCM=OFF, 

TorchVision: 0.12.0
OpenCV: 4.5.5
MMCV: 1.4.8
MMCV Compiler: GCC 7.3
MMCV CUDA Compiler: 11.3
MMSegmentation: 0.25.0+033f182

Error traceback

If applicable, paste the error trackback here.

Original Traceback (most recent call last):
  File "/home/ranhui/Software/miniconda3/envs/mmseg/lib/python3.10/site-packages/torch/utils/data/_utils/worker.py", line 287, in _worker_loop
    data = fetcher.fetch(index)
  File "/home/ranhui/Software/miniconda3/envs/mmseg/lib/python3.10/site-packages/torch/utils/data/_utils/fetch.py", line 49, in fetch
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/home/ranhui/Software/miniconda3/envs/mmseg/lib/python3.10/site-packages/torch/utils/data/_utils/fetch.py", line 49, in <listcomp>
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/home/ranhui/Workspace/mmsegmentation/mmseg/datasets/custom.py", line 215, in __getitem__
    return self.prepare_train_img(idx)
  File "/home/ranhui/Workspace/mmsegmentation/mmseg/datasets/custom.py", line 232, in prepare_train_img
    return self.pipeline(results)
  File "/home/ranhui/Workspace/mmsegmentation/mmseg/datasets/pipelines/compose.py", line 41, in __call__
    data = t(data)
  File "/home/ranhui/Workspace/mmsegmentation/mmseg/datasets/pipelines/transforms.py", line 1139, in __call__
    results = self._mosaic_transform_img(results)
  File "/home/ranhui/Workspace/mmsegmentation/mmseg/datasets/pipelines/transforms.py", line 1166, in _mosaic_transform_img
    assert 'mix_results' in results
AssertionError

[xiexinch]

Hi @Ranhuiryan
The dataset should be wrapped with the MultiImageMixDataset wrapper when using mosaic augmentation.
https://mmsegmentation.readthedocs.io/en/latest/tutorials/customize_datasets.html#multi-image-mix-dataset

[Ranhuiryan]

Hi @Ranhuiryan The dataset should be wrapped with the MultiImageMixDataset wrapper when using mosaic augmentation. https://mmsegmentation.readthedocs.io/en/latest/tutorials/customize_datasets.html#multi-image-mix-dataset

That really helped. Thanks a lot.