VGG19 for grayscale images

[mustaeenqazi]

Dear Hasnain,
Thanks for a well developed code for SRGAN.
I want ot use it for grayscale images. but the problem is with the content loss, that VGG19 is trained on imagenet dataset which is not grayscale and vgg19 requires the sr and hr images to have 3 colour channels when used in SRGAN. Is there a way out for this? maybe i can use vgg19 but train it first with my own grayscale images. any help would be appreciated. also i would also use SRGAN for 3D images in near future, any help in that regard would be appreciated also.
Cheers,

[HasnainRaz]

Hey Mustaeen,

A simple workaround for the grayscale problem would be to convert your grayscale images to RGB (using PIL or opencv). Then you can use everything as is. I'm not sure if it would have an impact on how the vgg model outputs features as like you mentioned, it wasn't trained on grayscale images. You can look at fine tuning an existing vgg model on grayscale images too (download a dataset for classification, convert images to grayscale using PIL or OpenCV and then train that) and then you can use that VGG model for super resolution. I would assume this would boost the performance of SR on grayscale images.

For your second question, I haven't done much work with super resolution on 3D volumes. But assuming your volumetric data can be sliced up into 2D images, you can use the 2D slices to train existing SR models. If you want to go further, you'll have to introduce 3D convolutions into the code and model the problem correctly.