ade20k test

[ak9250]

when I try to run test.py with ade20k i get this error

Traceback (most recent call last):
File "test.py", line 32, in
for i, data in enumerate(dataloader):
File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/dataloader.py", line 615, in next
batch = self.collate_fn([self.dataset[i] for i in indices])
File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/dataloader.py", line 615, in
batch = self.collate_fn([self.dataset[i] for i in indices])
File "/content/SPADE/data/pix2pix_dataset.py", line 82, in getitem
(label_path, image_path)
AssertionError: The label_path /content/SPADE/ADE20K_2016_07_26/images/validation/a/abbey/ADE_val_00000001_parts_1.png and image_path /content/SPADE/ADE20K_2016_07_26/images/validation/a/abbey/ADE_val_00000001.jpg don't match.

I got the zip
wget http://groups.csail.mit.edu/vision/datasets/ADE20K/ADE20K_2016_07_26.zip
unzip
unzip ADE20K_2016_07_26.zip
set path of dataset
python test.py --name ade20k_pretrained --dataset_mode ade20k --dataroot /content/SPADE/ADE20K_2016_07_26/

[wasd96040501]

@ak9250
This is a bug in ade20k_dataset.py.
Although this bug is resolved, it will come up a new bug during running ade20k model.

Have a look at how to resolve this bug.

In ade20k_datset.py, first it recursively iterate all image file, and split them into two list: image and label.

SPADE/data/ade20k_dataset.py

Lines 39 to 45 in c3e6dad

	for p in all_images:
	if '_%s_' % phase not in p:
	continue
	if p.endswith('.jpg'):
	image_paths.append(p)
	elif p.endswith('.png'):
	label_paths.append(p)

But have a look at ade20k dataset directory structure:

➜  abbey ls
ADE_val_00000001_atr.txt      ADE_val_00000002_parts_1.png
ADE_val_00000001.jpg          ADE_val_00000002_seg.png
ADE_val_00000001_parts_1.png  ADE_val_00001001_atr.txt
ADE_val_00000001_seg.png      ADE_val_00001001.jpg
ADE_val_00000002_atr.txt      ADE_val_00001001_parts_1.png
ADE_val_00000002.jpg          ADE_val_00001001_seg.png

For every .jpg image, it will be two or higher corresponding label image (eg. ADE_val_00000001_seg.png and ADE_val_00000001_parts_1.png).
So the size of image list and label list is not identical.

Just modify these code with:

        for p in all_images:
            if '_%s_' % phase not in p:
                continue
            if p.endswith('.jpg'):
                image_paths.append(p)
            elif p.endswith('_seg.png'):
                label_paths.append(p)

will fix that.

There are another bug associated with this

SPADE/data/pix2pix_dataset.py

Lines 63 to 66 in c3e6dad

	def paths_match(self, path1, path2):
	filename1_without_ext = os.path.splitext(os.path.basename(path1))[0]
	filename2_without_ext = os.path.splitext(os.path.basename(path2))[0]
	return filename1_without_ext == filename2_without_ext

This code examine whether image and label are paired.
Because (eg.) ADE_val_00000001_seg != ADE_val_00000001,
we should add another function paths_match under Class ade20k_dataset.py

    def paths_match(self, path1, path2):
        filename1_without_ext = os.path.splitext(os.path.basename(path1))[0]
        filename2_without_ext = os.path.splitext(os.path.basename(path2))[0]
        filename1_without_ext = filename1_without_ext.replace('_seg', '')
        filename2_without_ext = filename2_without_ext.replace('_seg', '')
        return  filename1_without_ext == filename2_without_ext

After this, I think bug associated with ade20k datset will be resolved.
But you will come up with another bug with cudnn.

...
...
/opt/conda/conda-bld/pytorch_1544174967633/work/aten/src/THC/THCTensorScatterGather.cu:176: void THCudaTensor_scatterFillKernel(TensorInfo<Real, IndexType>, TensorInfo<long, IndexType>, Real, int, IndexType) [with IndexType = unsigned int, Real = float, Dims = -1]: block: [292,0,0], thread: [95,0,0] Assertion `indexValue >= 0 && indexValue < tensor.sizes[dim]` failed.
Traceback (most recent call last):
  File "test.py", line 36, in <module>
    generated = model(data, mode='inference')
  File "/home/lyz/anaconda3/lib/python3.6/site-packages/torch/nn/modules/module.py", line 489, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/lyz/workdir/SPADE/models/pix2pix_model.py", line 58, in forward
    fake_image, _ = self.generate_fake(input_semantics, real_image)
  File "/home/lyz/workdir/SPADE/models/pix2pix_model.py", line 197, in generate_fake
    fake_image = self.netG(input_semantics, z=z)
  File "/home/lyz/anaconda3/lib/python3.6/site-packages/torch/nn/modules/module.py", line 489, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/lyz/workdir/SPADE/models/networks/generator.py", line 89, in forward
    x = self.fc(x)
  File "/home/lyz/anaconda3/lib/python3.6/site-packages/torch/nn/modules/module.py", line 489, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/lyz/anaconda3/lib/python3.6/site-packages/torch/nn/modules/conv.py", line 320, in forward
    self.padding, self.dilation, self.groups)
RuntimeError: cuDNN error: CUDNN_STATUS_NOT_INITIALIZED

Then we can locate the error in:

SPADE/models/pix2pix_model.py

Line 124 in c3e6dad

input_semantics = input_label.scatter_(1, label_map, 1.0)

This bug comes up when the range of mask value is negative or greater than the input dim:

Assertion `indexValue >= 0 && indexValue < tensor.sizes[dim]` failed.

Then we need to modify this file:

SPADE/data/ade20k_dataset.py

Lines 53 to 56 in c3e6dad

	def postprocess(self, input_dict):
	label = input_dict['label']
	label = label - 1
	label[label == -1] = self.opt.label_nc

to assure segmentation mask value is in range(0, label_nc)

    def postprocess(self, input_dict):
        label = input_dict['label']
        label = label - 1
        label[label == -1] = self.opt.label_nc
        input_dict['label'] = label.clamp(0, self.opt.label_nc)

After fix that, it will not raise runtime error, But the result synthesized image looks weird.

[hhsinping]

I will try the dataset here:
http://sceneparsing.csail.mit.edu/

[banyet1]

The synthesized image from coco datasets also looks weird.

[taesungp]

For label map of ADE20k, we used the scene parsing dataset in this link: http://sceneparsing.csail.mit.edu/. Direct dataset link: http://data.csail.mit.edu/places/ADEchallenge/ADEChallengeData2016.zip. Put the images files ADEChallengeData2016/images/validation/*.jpg and label files ADEChallengeData2016/annotations/validatoin/*.png in the same directory, and use the command below

python test.py --name ade20k_pretrained --dataset_mode ade20k --dataroot [path_to_dataset_dir]

In the result directory, the first two outputs should look like this: