UNet help (seems to work only on fake images)?

[branimir29]

It seems that UWGAN generated properly fake water images, it also seems that UNet trained properly on those images as I can get back original image from fake when I run test. However, if I try to feed test with actual water image I get out garbage. I am including sample fake image generated by UWGAN. The same image reconstructed by UNet (seems properly) and what I get when I try to feed Unet one of Type1 water images.
(fake water image generated by UWGAN)
(the same image fed through test loop of UNet (after training it of course))
(sample Type1 water image after going through UNet)

Any idea, what might be going wrong?
Any help would be appreciated.

[infrontofme]

I think there may be some following problems:

1. The fake water images you generated through UWGAN seems to mismatch the image you tested in the color style, which leads to that UNet learned wrong mapping functions. You should adjust UWGAN parameters or retrain to synthesize images with a similar style.

(fake water image generated by UWGAN)
(real water image named YDXJ0001_4199.jpg)

(real water image named YDXJ0001_9156.jpg and its enhanced version)
We randomly selected some images for testing, we didn't test on YDXJ0001_4199.jpg, but we observed that the color-style of the two images (YDXJ0001_4199.jpg & YDXJ0001_9156.jpg) were the same.

2. UNet doesn't seem to be trained well, I suggest you try another loss function.

3. If you want UNet to be suitable for more kinds of water qualities, you should use UWGAN to synthesize more images of different water qualities, and then retrain UNet.

4. We found that the discriminator of UWGAN does not seem smart enough to distinguish between different water qualities, which is one of the directions we will improve in our future work.

I hope these replies can help you.

[infrontofme]

TensorFlow 1.8，but I don't think it has much to do with the version of TF.
The following code in wugan_model.py, which determined light attenuation and haze blur in fake water images. You can try to adjust the initial values of these parameters in TF to fit your training.

# water-based attenuation and backscatter
            with tf.variable_scope("g_atten", reuse=True):
                eta_r = tf.get_variable(name='g_eta_r', shape=[1, 1, 1], dtype=tf.float32,
                                        initializer=tf.random_normal_initializer(mean=0.35, stddev=0.01))
                eta_g = tf.get_variable(name='g_eta_g', shape=[1, 1, 1], dtype=tf.float32,
                                        initializer=tf.random_normal_initializer(mean=0.015, stddev=0.01))
                eta_b = tf.get_variable(name='g_eta_b', shape=[1, 1, 1], dtype=tf.float32,
                                        initializer=tf.random_normal_initializer(mean=0.036, stddev=0.01))

# Haze effect
            eta_rr = tf.constant(value=0.75, dtype=tf.float32, shape=[1, 1, 1], name='g_eta_rr')
            eta_gg = tf.constant(value=0.75, dtype=tf.float32, shape=[1, 1, 1], name='g_eta_gg')
            eta_bb = tf.constant(value=0.75, dtype=tf.float32, shape=[1, 1, 1], name='g_eta_bb')
            eta_haze = tf.stack([eta_rr, eta_gg, eta_bb], axis=3)
            tm_haze = tf.exp(tf.multiply(-1.0, tf.multiply(eta_haze, depth)))
            image_haze = tf.multiply(tf.multiply(255.0 * A, tf.subtract(1.0, tm_haze)), eta_d)

I also suggest you take a closer look at underwater imaging model.