How to test for custom Images

[sbharadwajj]

I'm trying to run inference by modifying test.py to run on custom images. I got the following error with respect to the model checkpoint. I have downloaded the pre-trained model and have added it to danet/cityscapes/model path. Here is my modified code; its referencing the correct path of the checkpoint directory.

args = Options().parse()
model = get_segmentation_model(args.model, dataset=args.dataset,backbone=args.backbone, aux=args.aux,se_loss=args.se_loss, norm_layer=BatchNorm2d, base_size=args.base_size, crop_size=args.crop_size,multi_grid=args.multi_grid, multi_dilation=args.multi_dilation)

if args.resume_dir is None or not os.path.isdir(args.resume_dir):
        raise RuntimeError("=> no checkpoint found at '{}'".format(args.resume_dir))
for resume_file in os.listdir(args.resume_dir):
    if os.path.splitext(resume_file)[1] == '.tar':
        args.resume = os.path.join(args.resume_dir, resume_file)
        assert os.path.exists(args.resume)
        print(args.resume)

checkpoint = torch.load(args.resume) # strict=False, so that it is compatible with old pytorch saved models
model.load_state_dict(checkpoint['state_dict'], strict=False)
print(model)

I'm running this from my terminal

CUDA_VISIBLE_DEVICES=0 python test_on_custom.py --model danet --resume-dir cityscapes/model/ --base-size 2048 --crop-size 768 --workers 1 --backbone resnet101 --multi-grid --multi-dilation 4 8 16 --eval

This is the error:

Traceback (most recent call last):
  File "test_on_custom.py", line 34, in <module>
    model.load_state_dict(checkpoint['state_dict'], strict=False)
  File "/datadrive/virtualenvs/torch3.5/lib/python3.5/site-packages/torch/nn/modules/module.py", line 719, in load_state_dict
    self.__class__.__name__, "\n\t".join(error_msgs)))
RuntimeError: Error(s) in loading state_dict for DANet:
	size mismatch for head.conv6.1.weight: copying a param of torch.Size([150, 512, 1, 1]) from checkpoint, where the shape is torch.Size([19, 512, 1, 1]) in current model.
	size mismatch for head.conv6.1.bias: copying a param of torch.Size([150]) from checkpoint, where the shape is torch.Size([19]) in current model.
	size mismatch for head.conv7.1.weight: copying a param of torch.Size([150, 512, 1, 1]) from checkpoint, where the shape is torch.Size([19, 512, 1, 1]) in current model.
	size mismatch for head.conv7.1.bias: copying a param of torch.Size([150]) from checkpoint, where the shape is torch.Size([19]) in current model.
	size mismatch for head.conv8.1.weight: copying a param of torch.Size([150, 512, 1, 1]) from checkpoint, where the shape is torch.Size([19, 512, 1, 1]) in current model.
	size mismatch for head.conv8.1.bias: copying a param of torch.Size([150]) from checkpoint, where the shape is torch.Size([19]) in current model.

Line 34 is model.load_state_dict(checkpoint['state_dict'], strict=False)

[junfu1115]

You need add --dataset cityscapes in your command

[sbharadwajj]

But I'm trying to test the model for custom images. Is there any way I can do that?
I tried running inference previously with cityscapes, that gave me another error saying image mask id not found.
But now I want to test it for my own images.

[junfu1115]

You need updata val_fine.txt according to your custom images, and move --eval if you have no groundtruth of your custom image. Please refer to the code in encoding/datasets/cityscapes.py

[sbharadwajj]

I updated the val_fine.txt according to my images. I'm still not sure what to do about the --eval(i do not have ground truth).

This is the error:

Traceback (most recent call last):
  File "test_on_custom.py", line 33, in <module>
    testset = get_segmentation_dataset(args.dataset,transform=input_transform)
  File "/datadrive/virtualenvs/torch3.5/lib/python3.5/site-packages/encoding/datasets/__init__.py", line 17, in get_segmentation_dataset
    return datasets[name.lower()](**kwargs)
  File "/datadrive/virtualenvs/torch3.5/lib/python3.5/site-packages/encoding/datasets/cityscapes.py", line 37, in __init__
    " + root + "\n"))
RuntimeError: Found 0 images in subfolders of:                 ../datasets/cityscapes

My images are in the /cityscapes folder and the val_fine.txt is updated accordingly.
Code I'm running:

args = Options().parse()
model = get_segmentation_model(args.model, dataset=args.dataset,backbone=args.backbone, aux=args.aux,se_loss=args.se_loss, norm_layer=BatchNorm2d, base_size=args.base_size, crop_size=args.crop_size,multi_grid=args.multi_grid, multi_dilation=args.multi_dilation)

input_transform = transform.Compose([
        transform.ToTensor(),
        transform.Normalize([.485, .456, .406], [.229, .224, .225])])
testset = get_segmentation_dataset(args.dataset,transform=input_transform)

if args.resume_dir is None or not os.path.isdir(args.resume_dir):
        raise RuntimeError("=> no checkpoint found at '{}'".format(args.resume_dir))
for resume_file in os.listdir(args.resume_dir):
    if os.path.splitext(resume_file)[1] == '.tar':
        args.resume = os.path.join(args.resume_dir, resume_file)
        assert os.path.exists(args.resume)
        print(args.resume)

checkpoint = torch.load(args.resume) # strict=False, so that it is compatible with old pytorch saved models
model.load_state_dict(checkpoint['state_dict'], strict=False)
#print(model)

evaluator = MultiEvalModule(model, testset.num_class, multi_scales=args.multi_scales).cuda()
evaluator.eval()

On my command line:
CUDA_VISIBLE_DEVICES=0 python test_on_custom.py --model danet --dataset cityscapes --resume-dir cityscapes/model/ --base-size 2048 --crop-size 768 --workers 1 --backbone resnet101 --multi-grid --multi-dilation 4 8 16

[junfu1115]

You could print the imgpath in encoding/datasets/cityscapes.py to make sure the path is correct. --eval is used to calculate mean iou when ground truth is existed

[sbharadwajj]

What about the mask images? Are they required when I dont have ground truth?
I changed the paths accordingly.

[junfu1115]

If u want to test your own data, u can just modify the val_fine.txt in folder datasets/cityscapes/ according to your data. If u dont have mask images, u can replace mask path with ur img path(for visualize,no GT is ok), then u can run the visualize command:
CUDA_VISIBLE_DEVICES=0,1,2,3 python test.py --dataset cityscapes --model danet --resume-dir cityscapes/model --base-size 2048 --crop-size 768 --workers 1 --backbone resnet101 --multi-grid --multi-dilation 4 8 16

[sbharadwajj]

@junfu1115 Thank you so much!
It ran successfully for my custom images. The steps that I followed:

• Replaced all the names in val_fine.txt found in datasets/cityscapes/ according to my images. Kept the same names for both the columns. Remember to add tab spaces.
• The pixel accuracy and IoU did not work, but the visualizations were saved in danet/cityscapes/danet_vis.