Cycle GAN

[puhach]

The preprocessing step in the Cycle GAN proect is supposed to resize input images to be 128x128 in size. However, as long as only one value (not a tuple) is passed to transforms.Resize(), the code resizes only the smaller side of the input image to 128. The larger side will be resized to keep the aspect ratio of the original image. See docs.

# resize and normalize the images
    transform = transforms.Compose([transforms.Resize(image_size), # resize to 128x128
                                    transforms.ToTensor()])

The issue has probably been unnoticed as long as input images are square.

The output of the last discriminator layer should be 1x1x1, right? In the code we apply a kernel of size 4 to 8x8 feature map. That will be a 7x7 matrix, not a single number. Was it intended?

class Discriminator(nn.Module):
    
    def __init__(self, conv_dim=64):
        super(Discriminator, self).__init__()

        # Define all convolutional layers
        # Should accept an RGB image as input and output a single value

        # Convolutional layers, increasing in depth
        ...
        
        # Classification layer
        self.conv5 = conv(conv_dim*8, 1, 4, stride=1, batch_norm=False)

The original code seems to work due to the torch.mean() function, which returns one value for a multidimensional array, but I doubt if that is correct.

It is not quite clear why sigmoid can't be used in the output layer of the discriminator along with MSE loss. Sigmoid would squeeze the descriminator outputs to the (0, 1) range, hence the squared distance between a ground truth label (1 or 0) and the output would be more meaningful.

What is the reason to use one optimizer for both generators, but separately optimize the descriminators?

The training loop: as far as I understand the code, what is called "epoch" here, in fact is a step processing one batch of data and not the whole training set.

[apullin]

I just picked on this possible issue with the Discriminator architecture in the CycleGAN solution.

Thinking through it:
If the conv5 layer was a kernel size of 8, stride=1, padding=0, then we would get a 1x1x1 output.
This would imply a weights matrix of 8x8x512, and if I remember CNN kernel fundamentals correctly, it implies that each of the 88512 values in the intermediate result will be applied by their own weight and summed up to a single value.

Comparing to the solution that exists, you could possible consider the torch.mean to be doing something similar, where all the values are summed, so the result would not be fundamentally different than a 1/(7*7) scaling on the weights for the case above.

However: given that the solution as given is actually a convolution that takes an 8x8x512 result to 7x7x1, the values are impinging on their spatially neighboring values. So things are not exactly equivalent.