Multiple inputs network

[Grenadee]

Hi,
Thanks for sharing this amazing work!
I have tried to apply BicycleGAN into MRI image translation tasks, and it works well!
Now, I am trying to change the network as a multiple inputs network. My idea is by given multiple corresponding inputs, the output will be more realistic and accurate.
Do you think this is possible to achieve based on bicycleGAN?

[junyanz]

It's possible. You can write your own dataloader to read multiple inputs from the disk (you can modify this line for example), concatenate them and feed it to the network. (also change --input_nc accordingly)

[Grenadee]

Thanks for your suggestion.
My idea is to make the data loader load input_1, input_2 and groundtruth_output at the same time(side by side). Then concatenate two input images. The modified code is like this:

When I trying to run this code, I got an error at the transforms.Normalize() process said: TypeError: tensor is not a torch image.

Could you give me some suggestion about how to fix this issue and make this modified code works.

Thanks!

[Grenadee]

I have worked out the last issue. By successfully applying two corresponding images as input, the generated output does not have better performance by comparing with original BicycleGAN.

I have another question:
The generator used in bicycleGAN is U-net.As far as I know, U-net is widely used in biomedical image segmentation. What is the advantage to use U-net as generator in BicycleGAN other than other generators?

[junyanz]

We adopt it from pix2pix [Isola et al. 2017]. Please refer to the pix2pix paper. See Sec 3.2.1 Generator with skips and Sec 4.3 Analysis of the generator architecture for more details.

[Grenadee]

Thanks for your suggestion!