Config Day2Night | Split Datasets

[solesensei]

Hi, there, i'm trying to train dataset and stucked with some learning problems

Example of 10000 iterations day 2 night (train)

Example of 10000 iterations night 2 day (train)

I think that it's unit_day2night.yaml configuration problem. Can you share yours config file?

---

Another option, that it's dataset's split issue.
I used 50/50 for train and test. And both divided into two equal domains A and B.

[mingyuliutw]

Could you provide the link to the config file?

[solesensei]

@mingyuliutw yup

# logger options
image_save_iter: 1000           # How often do you want to save output images during training
image_display_iter: 50000       # How often do you want to display output images during training
display_size: 8                 # How many images do you want to display each time
snapshot_save_iter: 1000        # How often do you want to save trained models
snapshot_smart_override: True   # Use to smart cleaning old snapshots while saving  
log_iter: 10                    # How often do you want to log the training stats

# optimization options
max_iter: 1000000             # maximum number of training iterations
batch_size: 1                 # batch size
weight_decay: 0.0001          # weight decay
beta1: 0.5                    # Adam parameter
beta2: 0.999                  # Adam parameter
init: kaiming                 # initialization [gaussian/kaiming/xavier/orthogonal]
lr: 0.0001                    # initial learning rate
lr_policy: step               # learning rate scheduler
step_size: 100000             # how often to decay learning rate
gamma: 0.5                    # how much to decay learning rate
gan_w: 1                      # weight of adversarial loss
recon_x_w: 10                 # weight of image reconstruction loss
recon_h_w: 0                  # weight of hidden reconstruction loss
recon_kl_w: 0.01              # weight of KL loss for reconstruction
recon_x_cyc_w: 10             # weight of cycle consistency loss
recon_kl_cyc_w: 0.01          # weight of KL loss for cycle consistency
vgg_w: 0                      # weight of domain-invariant perceptual loss

# model options
gen:
  dim: 64                     # number of filters in the bottommost layer
  mlp_dim: 256                # number of filters in MLP
  style_dim: 8                # length of style code
  activ: relu                 # activation function [relu/lrelu/prelu/selu/tanh]
  n_downsample: 2             # number of downsampling layers in content encoder
  n_res: 4                    # number of residual blocks in content encoder/decoder
  pad_type: reflect           # padding type [zero/reflect]
dis: 
  dim: 64                     # number of filters in the bottommost layer
  norm: none                  # normalization layer [none/bn/in/ln]
  activ: lrelu                # activation function [relu/lrelu/prelu/selu/tanh]
  n_layer: 4                  # number of layers in D
  gan_type: lsgan             # GAN loss [lsgan/nsgan]
  num_scales: 3               # number of scales
  pad_type: reflect           # padding type [zero/reflect]

# data options
input_dim_a: 3                              # number of image channels [1/3]
input_dim_b: 3                              # number of image channels [1/3]
num_workers: 8                              # number of data loading threads
new_size: 256                               # first resize the shortest image side to this size
crop_image_height: 256                      # random crop image of this height
crop_image_width: 256                       # random crop image of this width
data_root: /mnt/w/prj/data/nexet/nexet_2017_1   # dataset folder location

Also loss:

[solesensei]

@mingyuliutw i think it is dataset problem, it is NEXET dataset.
All images are labeled: Day, Night, Twilight. I splitted into domains like this:

trainA: day / 4               testA: day / 4
trainB: night + twilight / 4  testB: night + twilight / 4

Also i found some labels mismatches after processing dataset with simple python script detecting time of day by color intensity. here

Twilight images be like

And like this

---

Some doubtful image's labels:

This is Day (label) (and it's correct, because it is tunnel)

This is Night (and it's obvious incorrect)

[solesensei]

So in further training attempts i took into account all problems above.

• I splitted 80/20
• I remove all doubtfull images from training sample

Here loss

Results

Day2Night (200000 iterations)

Night2Day (200000 iterations)

I think it's good enough without using VGG?

[solesensei]

@mingyuliutw one more question. Can i add accuracy to tensorboardX somehow? I found you calculating loss in UNIT. But i didn't find anything about calculation accuracy between domain-translated images and source.

In paper you describe how you done experiments:

Do you have any code of this? It'll be cool if you share that.
I need to compare test/train loss or accuracy.

For example here:
green - without vgg
purple - with vgg
I have better visual results with VGG model

I understand that

loss_gen_total = sum_of_all_losses

but how can i determine that model trained better?
Is loss_dis_total is good metric for this?

[mingyuliutw]

The results look nice. I obtained my original results without using the domain invariant VGG loss too. As I mentioned, the VGG loss helps for some datasets, particularly for synthetic to real.

Yes, you can add accuracy to tensorboard.

[solesensei]

@mingyuliutw The results look nice.

with VGG looks better for me: