What is the fail_percentage? Why supervised's fail_percentage less than unsupervised?

[Agchai52]

Hi,

Thank you so much for your code!
I successfully generate the synthetic datasets and test both supervised and unsupervised homography network by homography_CNN_synthetic.py. One of its outputs is fail_percentage. I checked where it is from. In the train(), it should be "apply_grad_opt". But I don't really understand what it is. Can you explain it to me?

Another related question is why supervised's fail_percentage less than unsupervised?
Both l1_loss and h_loss are very closed for two networks. But big difference shows in fail_percentage.

Thanks!

[tyunist]

Hi,

As you can see in homography_model.py, a failure is considered when the h_loss of the estimated homography > h_loss of an identity matrix (meaning the estimate is too bad).
Being said that, fail_percentage tells the percentage of estimates worst then just returning an identity matrix.

The fail_percentage can be different but should not much. Can you show your values here?

[Agchai52]

Thank you for your reply! I find the def of num_fail in homography_model.py line 278-279.

You mention that the difference should not much. I think the difference between your results and mine could be the different synthetic dataset, since it is generated randomly. I'll generate new synthetic dataset to see if the fail_percentage shows the same difference.

Here are my test results for supervised and unsupervised network on synthetic data:
Supervised
python homography_CNN_synthetic.py --mode test --lr 5e-4 --loss_type h_loss
<==================== Loading data ===================>

===> There are totally 5000 test files
===> Test: There are totally 5000 Test files
The gt_file is :

/Users/Nora/Desktop/ImageProcessing/HomographyGAN/unsupervisedDeepHomographyRAL2018-master/syntheticdata/synthetic/45/test_gt.txt
--Shape of A_mat: [64, 8, 8]
--shape of b: [64, 8, 1]
--shape of H_8el Tensor("MatrixSolve:0", shape=(64, 8, 1), dtype=float32, device=/device:GPU:0)
('--Inter- scale_h:', True)
--Shape of A_mat: [64, 8, 8]
--shape of b: [64, 8, 1]
--shape of H_8el Tensor("MatrixSolve_1:0", shape=(64, 8, 1), dtype=float32, device=/device:GPU:1)
('--Inter- scale_h:', True)
2018-05-02 16:22:51.967937: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.2 instructions, but these are available on your machine and could speed up CPU computations.
2018-05-02 16:22:51.967958: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX instructions, but these are available on your machine and could speed up CPU computations.
2018-05-02 16:22:51.967966: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX2 instructions, but these are available on your machine and could speed up CPU computations.
2018-05-02 16:22:51.967972: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use FMA instructions, but these are available on your machine and could speed up CPU computations.
../models/synthetic_models/h_loss_normalize
('Wrote file %s', '../results/synthetic/report/0_h_loss_loss_13.017536.jpg')
===> This iteration num Fail: 8

('Wrote file %s', '../results/synthetic/report/128_h_loss_loss_11.815462.jpg')s241ms | Tot: 0ms | Test, h_loss 13.018, rec_loss 1.043, ssim_loss 0.406, l1_loss 0.678, fail_percent 0.0625
('Wrote file %s', '../results/synthetic/report/256_h_loss_loss_12.169714.jpg')
('Wrote file %s', '../results/synthetic/report/384_h_loss_loss_12.473728.jpg')
('Wrote file %s', '../results/synthetic/report/512_h_loss_loss_12.33558.jpg')
('Wrote file %s', '../results/synthetic/report/640_h_loss_loss_11.917628.jpg')
('Wrote file %s', '../results/synthetic/report/768_h_loss_loss_11.659668.jpg')
('Wrote file %s', '../results/synthetic/report/896_h_loss_loss_14.228176.jpg')
('Wrote file %s', '../results/synthetic/report/1024_h_loss_loss_12.058962.jpg')
('Wrote file %s', '../results/synthetic/report/1152_h_loss_loss_11.835266.jpg')
('Wrote file %s', '../results/synthetic/report/1280_h_loss_loss_12.494346.jpg')
===> This iteration num Fail: 8

('Wrote file %s', '../results/synthetic/report/1408_h_loss_loss_10.353984.jpg')s | Tot: 4m2s | Test, h_loss 12.364, rec_loss 1.043, ssim_loss 0.422, l1_loss 0.691, fail_percent 0.0597
('Wrote file %s', '../results/synthetic/report/1536_h_loss_loss_11.792387.jpg')
('Wrote file %s', '../results/synthetic/report/1664_h_loss_loss_12.710796.jpg')
('Wrote file %s', '../results/synthetic/report/1792_h_loss_loss_12.431883.jpg')
('Wrote file %s', '../results/synthetic/report/1920_h_loss_loss_12.626864.jpg')
('Wrote file %s', '../results/synthetic/report/2048_h_loss_loss_12.170179.jpg')
('Wrote file %s', '../results/synthetic/report/2176_h_loss_loss_11.321284.jpg')
('Wrote file %s', '../results/synthetic/report/2304_h_loss_loss_11.493059.jpg')
('Wrote file %s', '../results/synthetic/report/2432_h_loss_loss_11.626288.jpg')
('Wrote file %s', '../results/synthetic/report/2560_h_loss_loss_12.437614.jpg')
===> This iteration num Fail: 9

('Wrote file %s', '../results/synthetic/report/2688_h_loss_loss_12.811972.jpg')s | Tot: 8m5s | Test, h_loss 12.141, rec_loss 1.040, ssim_loss 0.424, l1_loss 0.688, fail_percent 0.0577
('Wrote file %s', '../results/synthetic/report/2816_h_loss_loss_12.988997.jpg')
('Wrote file %s', '../results/synthetic/report/2944_h_loss_loss_13.112974.jpg')
('Wrote file %s', '../results/synthetic/report/3072_h_loss_loss_13.148828.jpg')
('Wrote file %s', '../results/synthetic/report/3200_h_loss_loss_11.664024.jpg')
('Wrote file %s', '../results/synthetic/report/3328_h_loss_loss_13.24741.jpg')
('Wrote file %s', '../results/synthetic/report/3456_h_loss_loss_11.375984.jpg')
('Wrote file %s', '../results/synthetic/report/3584_h_loss_loss_12.076891.jpg')
('Wrote file %s', '../results/synthetic/report/3712_h_loss_loss_12.487751.jpg')
('Wrote file %s', '../results/synthetic/report/3840_h_loss_loss_11.786917.jpg')
===> This iteration num Fail: 6

('Wrote file %s', '../results/synthetic/report/3968_h_loss_loss_10.991777.jpg')3s | Tot: 11m58s | Test, h_loss 12.247, rec_loss 1.040, ssim_loss 0.424, l1_loss 0.688, fail_percent 0.0617
('Wrote file %s', '../results/synthetic/report/4096_h_loss_loss_12.250401.jpg')
('Wrote file %s', '../results/synthetic/report/4224_h_loss_loss_12.349435.jpg')
('Wrote file %s', '../results/synthetic/report/4352_h_loss_loss_13.203747.jpg')
('Wrote file %s', '../results/synthetic/report/4480_h_loss_loss_13.175642.jpg')
('Wrote file %s', '../results/synthetic/report/4608_h_loss_loss_11.554409.jpg')
('Wrote file %s', '../results/synthetic/report/4736_h_loss_loss_12.036743.jpg')
('Wrote file %s', '../results/synthetic/report/4864_h_loss_loss_11.928762.jpg')
('Wrote file %s', '../results/synthetic/report/4992_h_loss_loss_11.526682.jpg')
('Wrote file %s', '../results/synthetic/report/5120_h_loss_loss_11.449365.jpg')
===> This iteration num Fail: 9

('Wrote file %s', '../results/synthetic/report/5248_h_loss_loss_12.481041.jpg')9s | Tot: 15m57s | Test, h_loss 12.199, rec_loss 1.043, ssim_loss 0.424, l1_loss 0.690, fail_percent 0.0617
('Wrote file %s', '../results/synthetic/report/5376_h_loss_loss_11.224356.jpg')
('Wrote file %s', '../results/synthetic/report/5504_h_loss_loss_12.67424.jpg')
('Wrote file %s', '../results/synthetic/report/5632_h_loss_loss_13.11825.jpg')
('Wrote file %s', '../results/synthetic/report/5760_h_loss_loss_12.218927.jpg')
('Wrote file %s', '../results/synthetic/report/5888_h_loss_loss_13.178122.jpg')
('Wrote file %s', '../results/synthetic/report/6016_h_loss_loss_11.72691.jpg')
('Wrote file %s', '../results/synthetic/report/6144_h_loss_loss_12.782761.jpg')
('Wrote file %s', '../results/synthetic/report/6272_h_loss_loss_12.374836.jpg')
('Wrote file %s', '../results/synthetic/report/6400_h_loss_loss_12.324858.jpg')
===> This iteration num Fail: 8

('Wrote file %s', '../results/synthetic/report/6528_h_loss_loss_11.90332.jpg')56s | Tot: 19m54s | Test, h_loss 12.240, rec_loss 1.045, ssim_loss 0.424, l1_loss 0.692, fail_percent 0.0616
('Wrote file %s', '../results/synthetic/report/6656_h_loss_loss_12.137791.jpg')
('Wrote file %s', '../results/synthetic/report/6784_h_loss_loss_11.677058.jpg')
('Wrote file %s', '../results/synthetic/report/6912_h_loss_loss_12.425444.jpg')
('Wrote file %s', '../results/synthetic/report/7040_h_loss_loss_11.929211.jpg')
('Wrote file %s', '../results/synthetic/report/7168_h_loss_loss_12.182209.jpg')
('Wrote file %s', '../results/synthetic/report/7296_h_loss_loss_11.142374.jpg')
('Wrote file %s', '../results/synthetic/report/7424_h_loss_loss_11.812819.jpg')
('Wrote file %s', '../results/synthetic/report/7552_h_loss_loss_12.445438.jpg')
('Wrote file %s', '../results/synthetic/report/7680_h_loss_loss_11.977521.jpg')
===> This iteration num Fail: 11

('Wrote file %s', '../results/synthetic/report/7808_h_loss_loss_12.233584.jpg')54ms | Tot: 23m54s | Test, h_loss 12.195, rec_loss 1.045, ssim_loss 0.424, l1_loss 0.692, fail_percent 0.0619
('Wrote file %s', '../results/synthetic/report/7936_h_loss_loss_11.729382.jpg')
('Wrote file %s', '../results/synthetic/report/8064_h_loss_loss_12.342363.jpg')
('Wrote file %s', '../results/synthetic/report/8192_h_loss_loss_11.923828.jpg')
('Wrote file %s', '../results/synthetic/report/8320_h_loss_loss_12.388173.jpg')
('Wrote file %s', '../results/synthetic/report/8448_h_loss_loss_11.431114.jpg')
('Wrote file %s', '../results/synthetic/report/8576_h_loss_loss_12.552185.jpg')
('Wrote file %s', '../results/synthetic/report/8704_h_loss_loss_11.516036.jpg')
('Wrote file %s', '../results/synthetic/report/8832_h_loss_loss_11.465176.jpg')
('Wrote file %s', '../results/synthetic/report/8960_h_loss_loss_12.520947.jpg')
===> This iteration num Fail: 14

('Wrote file %s', '../results/synthetic/report/9088_h_loss_loss_11.753185.jpg')2s | Tot: 28m17s | Test, h_loss 12.169, rec_loss 1.041, ssim_loss 0.423, l1_loss 0.688, fail_percent 0.0619
('Wrote file %s', '../results/synthetic/report/9216_h_loss_loss_12.873783.jpg')
('Wrote file %s', '../results/synthetic/report/9344_h_loss_loss_12.541215.jpg')
('Wrote file %s', '../results/synthetic/report/9472_h_loss_loss_12.835957.jpg')
('Wrote file %s', '../results/synthetic/report/9600_h_loss_loss_11.863873.jpg')
('Wrote file %s', '../results/synthetic/report/9728_h_loss_loss_11.813689.jpg')
('Wrote file %s', '../results/synthetic/report/9856_h_loss_loss_11.269012.jpg')
('Wrote file %s', '../results/synthetic/report/9984_h_loss_loss_12.410902.jpg')
('Wrote file %s', '../results/synthetic/report/10112_h_loss_loss_12.483238.jpg')
('Wrote file %s', '../results/synthetic/report/10240_h_loss_loss_11.746168.jpg')
===> This iteration num Fail: 5

('Wrote file %s', '../results/synthetic/report/10368_h_loss_loss_11.462982.jpg')s | Tot: 33m2s | Test, h_loss 12.168, rec_loss 1.042, ssim_loss 0.424, l1_loss 0.689, fail_percent 0.0621
('Wrote file %s', '../results/synthetic/report/10496_h_loss_loss_12.449156.jpg')
('Wrote file %s', '../results/synthetic/report/10624_h_loss_loss_12.801935.jpg')
('Wrote file %s', '../results/synthetic/report/10752_h_loss_loss_12.804964.jpg')
('Wrote file %s', '../results/synthetic/report/10880_h_loss_loss_12.624654.jpg')
('Wrote file %s', '../results/synthetic/report/11008_h_loss_loss_12.686079.jpg')
('Wrote file %s', '../results/synthetic/report/11136_h_loss_loss_12.270837.jpg')
('Wrote file %s', '../results/synthetic/report/11264_h_loss_loss_12.401556.jpg')
('Wrote file %s', '../results/synthetic/report/11392_h_loss_loss_11.496135.jpg')
('Wrote file %s', '../results/synthetic/report/11520_h_loss_loss_13.515879.jpg')
===> This iteration num Fail: 11

('Wrote file %s', '../results/synthetic/report/11648_h_loss_loss_12.536741.jpg')s | Tot: 37m12s | Test, h_loss 12.199, rec_loss 1.044, ssim_loss 0.424, l1_loss 0.691, fail_percent 0.0607
('Wrote file %s', '../results/synthetic/report/11776_h_loss_loss_12.497176.jpg')
('Wrote file %s', '../results/synthetic/report/11904_h_loss_loss_12.546736.jpg')
('Wrote file %s', '../results/synthetic/report/12032_h_loss_loss_11.127098.jpg')
('Wrote file %s', '../results/synthetic/report/12160_h_loss_loss_10.591163.jpg')
('Wrote file %s', '../results/synthetic/report/12288_h_loss_loss_13.355326.jpg')
('Wrote file %s', '../results/synthetic/report/12416_h_loss_loss_11.378929.jpg')
('Wrote file %s', '../results/synthetic/report/12544_h_loss_loss_11.827927.jpg')
('Wrote file %s', '../results/synthetic/report/12672_h_loss_loss_12.14179.jpg')
('Wrote file %s', '../results/synthetic/report/12800_h_loss_loss_11.095583.jpg')
===> This iteration num Fail: 0

('Wrote file %s', '../results/synthetic/report/12928_h_loss_loss_12.287821.jpg') | Tot: 41m20s | Test, h_loss 12.170, rec_loss 1.042, ssim_loss 0.424, l1_loss 0.689, fail_percent 0.0599
('Wrote file %s', '../results/synthetic/report/13056_h_loss_loss_12.265129.jpg')
('Wrote file %s', '../results/synthetic/report/13184_h_loss_loss_12.82636.jpg')
('Wrote file %s', '../results/synthetic/report/13312_h_loss_loss_10.906004.jpg')
('Wrote file %s', '../results/synthetic/report/13440_h_loss_loss_11.515938.jpg')
('Wrote file %s', '../results/synthetic/report/13568_h_loss_loss_12.303676.jpg')
('Wrote file %s', '../results/synthetic/report/13696_h_loss_loss_11.371214.jpg')
('Wrote file %s', '../results/synthetic/report/13824_h_loss_loss_11.309425.jpg')
('Wrote file %s', '../results/synthetic/report/13952_h_loss_loss_11.093759.jpg')
('Wrote file %s', '../results/synthetic/report/14080_h_loss_loss_12.308014.jpg')
===> This iteration num Fail: 6

('Wrote file %s', '../results/synthetic/report/14208_h_loss_loss_11.624669.jpg')s | Tot: 45m37s | Test, h_loss 12.138, rec_loss 1.039, ssim_loss 0.424, l1_loss 0.687, fail_percent 0.0598
('Wrote file %s', '../results/synthetic/report/14336_h_loss_loss_11.554046.jpg')
('Wrote file %s', '../results/synthetic/report/14464_h_loss_loss_12.299379.jpg')
('Wrote file %s', '../results/synthetic/report/14592_h_loss_loss_11.668701.jpg')
('Wrote file %s', '../results/synthetic/report/14720_h_loss_loss_11.63073.jpg')
('Wrote file %s', '../results/synthetic/report/14848_h_loss_loss_11.776663.jpg')
('Wrote file %s', '../results/synthetic/report/14976_h_loss_loss_11.665909.jpg')
('Wrote file %s', '../results/synthetic/report/15104_h_loss_loss_11.812042.jpg')
('Wrote file %s', '../results/synthetic/report/15232_h_loss_loss_12.038973.jpg')
[============================ 120/120 ===========================>] Step: 3m29s | Tot: 49m6s | Test, h_loss 12.112, rec_loss 1.038, ssim_loss 0.424, l1_loss 0.685, fail_percent 0.0592
====> Result for RHO: 45 loss h_loss noise 0.5
|Steps | h_loss | l1_loss | Fail percent |
119 12.112012569109599 0.6851609165469805 5.924479166666667

Top 0 - 30 %
Top 30 - 60 %
Top 60 - 100 %
===> Percentile Values: (20, 50, 80, 100):
[[11.380283 0.3065826 ]
[12.018547 0.19068556]
[12.730908 0.3788473 ]]
======> End! ====================================

Unsupervised
python homography_CNN_synthetic.py --mode test --lr 1e-4 --loss_type l1_loss

<==================== Loading data ===================>

===> There are totally 5000 test files
===> Test: There are totally 5000 Test files
The gt_file is :

/Users/Nora/Desktop/ImageProcessing/HomographyGAN/unsupervisedDeepHomographyRAL2018-master/syntheticdata/synthetic/45/test_gt.txt
--Shape of A_mat: [64, 8, 8]
--shape of b: [64, 8, 1]
--shape of H_8el Tensor("MatrixSolve:0", shape=(64, 8, 1), dtype=float32, device=/device:GPU:0)
('--Inter- scale_h:', True)
--Shape of A_mat: [64, 8, 8]
--shape of b: [64, 8, 1]
--shape of H_8el Tensor("MatrixSolve_1:0", shape=(64, 8, 1), dtype=float32, device=/device:GPU:1)
('--Inter- scale_h:', True)
../models/synthetic_models/l1_loss_normalize
('Wrote file %s', '../results/synthetic/report/0_l1_loss_loss_13.686736.jpg')
===> This iteration num Fail: 26

[>................................................................] Step: 56s553ms | Tot: 0ms | Test, h_loss 13.687, rec_loss 0.999, ssim_loss 0.394, l1_loss 0.615, fail_percent 0.2031 ����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������� 1/120
('Wrote file %s', '../results/synthetic/report/128_l1_loss_loss_14.163922.jpg')
('Wrote file %s', '../results/synthetic/report/256_l1_loss_loss_14.702034.jpg')
('Wrote file %s', '../results/synthetic/report/384_l1_loss_loss_13.983331.jpg')
('Wrote file %s', '../results/synthetic/report/512_l1_loss_loss_15.06563.jpg')
('Wrote file %s', '../results/synthetic/report/640_l1_loss_loss_14.135531.jpg')
('Wrote file %s', '../results/synthetic/report/768_l1_loss_loss_14.166774.jpg')
('Wrote file %s', '../results/synthetic/report/896_l1_loss_loss_14.924227.jpg')
('Wrote file %s', '../results/synthetic/report/1024_l1_loss_loss_13.560821.jpg')
('Wrote file %s', '../results/synthetic/report/1152_l1_loss_loss_13.733147.jpg')
('Wrote file %s', '../results/synthetic/report/1280_l1_loss_loss_13.131209.jpg')
===> This iteration num Fail: 24

[=====>...........................................................] Step: 4m28s | Tot: 4m28s | Test, h_loss 14.114, rec_loss 0.989, ssim_loss 0.405, l1_loss 0.626, fail_percent 0.2010 ����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������� 11/120
('Wrote file %s', '../results/synthetic/report/1408_l1_loss_loss_14.827921.jpg')
('Wrote file %s', '../results/synthetic/report/1536_l1_loss_loss_13.817992.jpg')
('Wrote file %s', '../results/synthetic/report/1664_l1_loss_loss_14.402395.jpg')
('Wrote file %s', '../results/synthetic/report/1792_l1_loss_loss_13.379984.jpg')
('Wrote file %s', '../results/synthetic/report/1920_l1_loss_loss_14.572965.jpg')
('Wrote file %s', '../results/synthetic/report/2048_l1_loss_loss_15.379339.jpg')
('Wrote file %s', '../results/synthetic/report/2176_l1_loss_loss_14.112824.jpg')
('Wrote file %s', '../results/synthetic/report/2304_l1_loss_loss_13.099989.jpg')
('Wrote file %s', '../results/synthetic/report/2432_l1_loss_loss_14.449642.jpg')
('Wrote file %s', '../results/synthetic/report/2560_l1_loss_loss_13.731074.jpg')
===> This iteration num Fail: 26

[==========>......................................................] Step: 4m19s | Tot: 8m47s | Test, h_loss 14.144, rec_loss 0.987, ssim_loss 0.406, l1_loss 0.627, fail_percent 0.2046 ����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������� 21/120
('Wrote file %s', '../results/synthetic/report/2688_l1_loss_loss_13.776274.jpg')
('Wrote file %s', '../results/synthetic/report/2816_l1_loss_loss_13.049566.jpg')
('Wrote file %s', '../results/synthetic/report/2944_l1_loss_loss_13.609674.jpg')
('Wrote file %s', '../results/synthetic/report/3072_l1_loss_loss_14.626366.jpg')
('Wrote file %s', '../results/synthetic/report/3200_l1_loss_loss_13.784372.jpg')
('Wrote file %s', '../results/synthetic/report/3328_l1_loss_loss_14.105943.jpg')
('Wrote file %s', '../results/synthetic/report/3456_l1_loss_loss_13.803667.jpg')
('Wrote file %s', '../results/synthetic/report/3584_l1_loss_loss_13.523324.jpg')
('Wrote file %s', '../results/synthetic/report/3712_l1_loss_loss_14.043745.jpg')
('Wrote file %s', '../results/synthetic/report/3840_l1_loss_loss_13.69809.jpg')
===> This iteration num Fail: 29

[================>................................................] Step: 4m13s | Tot: 13m1s | Test, h_loss 14.034, rec_loss 0.982, ssim_loss 0.405, l1_loss 0.622, fail_percent 0.2014 ����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������� 31/120
('Wrote file %s', '../results/synthetic/report/3968_l1_loss_loss_13.575291.jpg')
('Wrote file %s', '../results/synthetic/report/4096_l1_loss_loss_13.842568.jpg')
('Wrote file %s', '../results/synthetic/report/4224_l1_loss_loss_13.800649.jpg')
('Wrote file %s', '../results/synthetic/report/4352_l1_loss_loss_14.291029.jpg')
('Wrote file %s', '../results/synthetic/report/4480_l1_loss_loss_12.584939.jpg')
('Wrote file %s', '../results/synthetic/report/4608_l1_loss_loss_14.538559.jpg')
('Wrote file %s', '../results/synthetic/report/4736_l1_loss_loss_14.654457.jpg')
('Wrote file %s', '../results/synthetic/report/4864_l1_loss_loss_12.686545.jpg')
('Wrote file %s', '../results/synthetic/report/4992_l1_loss_loss_14.722412.jpg')
('Wrote file %s', '../results/synthetic/report/5120_l1_loss_loss_14.786278.jpg')
===> This iteration num Fail: 35

[=====================>...........................................] Step: 3m52s | Tot: 16m53s | Test, h_loss 14.013, rec_loss 0.976, ssim_loss 0.403, l1_loss 0.617, fail_percent 0.2008 ����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������� 41/120
('Wrote file %s', '../results/synthetic/report/5248_l1_loss_loss_13.818134.jpg')
('Wrote file %s', '../results/synthetic/report/5376_l1_loss_loss_14.336334.jpg')
('Wrote file %s', '../results/synthetic/report/5504_l1_loss_loss_14.482048.jpg')
('Wrote file %s', '../results/synthetic/report/5632_l1_loss_loss_12.952009.jpg')
('Wrote file %s', '../results/synthetic/report/5760_l1_loss_loss_13.144179.jpg')
('Wrote file %s', '../results/synthetic/report/5888_l1_loss_loss_14.513166.jpg')
('Wrote file %s', '../results/synthetic/report/6016_l1_loss_loss_13.971729.jpg')
('Wrote file %s', '../results/synthetic/report/6144_l1_loss_loss_14.225855.jpg')
('Wrote file %s', '../results/synthetic/report/6272_l1_loss_loss_13.795819.jpg')
('Wrote file %s', '../results/synthetic/report/6400_l1_loss_loss_14.576513.jpg')
===> This iteration num Fail: 31

[===========================>.....................................] Step: 3m55s | Tot: 20m49s | Test, h_loss 14.007, rec_loss 0.976, ssim_loss 0.402, l1_loss 0.615, fail_percent 0.2005 ����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������� 51/120
('Wrote file %s', '../results/synthetic/report/6528_l1_loss_loss_13.526925.jpg')
('Wrote file %s', '../results/synthetic/report/6656_l1_loss_loss_13.615358.jpg')
('Wrote file %s', '../results/synthetic/report/6784_l1_loss_loss_14.591264.jpg')
('Wrote file %s', '../results/synthetic/report/6912_l1_loss_loss_13.479589.jpg')
('Wrote file %s', '../results/synthetic/report/7040_l1_loss_loss_14.786627.jpg')
('Wrote file %s', '../results/synthetic/report/7168_l1_loss_loss_14.069378.jpg')
('Wrote file %s', '../results/synthetic/report/7296_l1_loss_loss_14.263207.jpg')
('Wrote file %s', '../results/synthetic/report/7424_l1_loss_loss_14.292161.jpg')
('Wrote file %s', '../results/synthetic/report/7552_l1_loss_loss_12.199593.jpg')
('Wrote file %s', '../results/synthetic/report/7680_l1_loss_loss_14.22786.jpg')
===> This iteration num Fail: 26

[================================>................................] Step: 3m57s | Tot: 24m47s | Test, h_loss 13.990, rec_loss 0.975, ssim_loss 0.403, l1_loss 0.615, fail_percent 0.1989 ����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������� 61/120
('Wrote file %s', '../results/synthetic/report/7808_l1_loss_loss_14.755758.jpg')
('Wrote file %s', '../results/synthetic/report/7936_l1_loss_loss_13.461891.jpg')
('Wrote file %s', '../results/synthetic/report/8064_l1_loss_loss_13.085265.jpg')
('Wrote file %s', '../results/synthetic/report/8192_l1_loss_loss_14.546217.jpg')
('Wrote file %s', '../results/synthetic/report/8320_l1_loss_loss_13.523918.jpg')
('Wrote file %s', '../results/synthetic/report/8448_l1_loss_loss_15.545043.jpg')
('Wrote file %s', '../results/synthetic/report/8576_l1_loss_loss_14.639109.jpg')
('Wrote file %s', '../results/synthetic/report/8704_l1_loss_loss_13.330393.jpg')
('Wrote file %s', '../results/synthetic/report/8832_l1_loss_loss_13.4049225.jpg')
('Wrote file %s', '../results/synthetic/report/8960_l1_loss_loss_14.17681.jpg')
===> This iteration num Fail: 32

[=====================================>...........................] Step: 3m57s | Tot: 28m44s | Test, h_loss 13.998, rec_loss 0.978, ssim_loss 0.403, l1_loss 0.616, fail_percent 0.1987 ����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������� 71/120
('Wrote file %s', '../results/synthetic/report/9088_l1_loss_loss_14.233114.jpg')
('Wrote file %s', '../results/synthetic/report/9216_l1_loss_loss_13.952438.jpg')
('Wrote file %s', '../results/synthetic/report/9344_l1_loss_loss_14.043037.jpg')
('Wrote file %s', '../results/synthetic/report/9472_l1_loss_loss_14.043768.jpg')
('Wrote file %s', '../results/synthetic/report/9600_l1_loss_loss_13.770577.jpg')
('Wrote file %s', '../results/synthetic/report/9728_l1_loss_loss_13.771662.jpg')
('Wrote file %s', '../results/synthetic/report/9856_l1_loss_loss_15.071829.jpg')
('Wrote file %s', '../results/synthetic/report/9984_l1_loss_loss_13.9093275.jpg')
('Wrote file %s', '../results/synthetic/report/10112_l1_loss_loss_13.684342.jpg')
('Wrote file %s', '../results/synthetic/report/10240_l1_loss_loss_12.606498.jpg')
===> This iteration num Fail: 18

[===========================================>.....................] Step: 4m5s | Tot: 32m50s | Test, h_loss 13.987, rec_loss 0.976, ssim_loss 0.403, l1_loss 0.615, fail_percent 0.1992 ����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������� 81/120
('Wrote file %s', '../results/synthetic/report/10368_l1_loss_loss_15.637377.jpg')
('Wrote file %s', '../results/synthetic/report/10496_l1_loss_loss_12.76598.jpg')
('Wrote file %s', '../results/synthetic/report/10624_l1_loss_loss_13.411814.jpg')
('Wrote file %s', '../results/synthetic/report/10752_l1_loss_loss_14.293726.jpg')
('Wrote file %s', '../results/synthetic/report/10880_l1_loss_loss_14.909916.jpg')
('Wrote file %s', '../results/synthetic/report/11008_l1_loss_loss_14.05713.jpg')
('Wrote file %s', '../results/synthetic/report/11136_l1_loss_loss_15.6908455.jpg')
('Wrote file %s', '../results/synthetic/report/11264_l1_loss_loss_14.185396.jpg')
('Wrote file %s', '../results/synthetic/report/11392_l1_loss_loss_12.823639.jpg')
('Wrote file %s', '../results/synthetic/report/11520_l1_loss_loss_13.671327.jpg')
===> This iteration num Fail: 28

[================================================>................] Step: 4m6s | Tot: 36m56s | Test, h_loss 14.004, rec_loss 0.978, ssim_loss 0.403, l1_loss 0.616, fail_percent 0.1994 ����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������� 91/120
('Wrote file %s', '../results/synthetic/report/11648_l1_loss_loss_14.460916.jpg')
('Wrote file %s', '../results/synthetic/report/11776_l1_loss_loss_12.690928.jpg')
('Wrote file %s', '../results/synthetic/report/11904_l1_loss_loss_13.8640785.jpg')
('Wrote file %s', '../results/synthetic/report/12032_l1_loss_loss_13.397703.jpg')
('Wrote file %s', '../results/synthetic/report/12160_l1_loss_loss_13.540346.jpg')
('Wrote file %s', '../results/synthetic/report/12288_l1_loss_loss_13.540372.jpg')
('Wrote file %s', '../results/synthetic/report/12416_l1_loss_loss_13.476835.jpg')
('Wrote file %s', '../results/synthetic/report/12544_l1_loss_loss_14.777797.jpg')
('Wrote file %s', '../results/synthetic/report/12672_l1_loss_loss_13.559889.jpg')
('Wrote file %s', '../results/synthetic/report/12800_l1_loss_loss_13.380949.jpg')
===> This iteration num Fail: 21

[======================================================>..........] Step: 4m5s | Tot: 41m2s | Test, h_loss 13.971, rec_loss 0.974, ssim_loss 0.403, l1_loss 0.613, fail_percent 0.1968 ����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������� 101/120
('Wrote file %s', '../results/synthetic/report/12928_l1_loss_loss_14.21394.jpg')
('Wrote file %s', '../results/synthetic/report/13056_l1_loss_loss_12.892838.jpg')
('Wrote file %s', '../results/synthetic/report/13184_l1_loss_loss_14.048085.jpg')
('Wrote file %s', '../results/synthetic/report/13312_l1_loss_loss_13.682612.jpg')
('Wrote file %s', '../results/synthetic/report/13440_l1_loss_loss_13.093262.jpg')
('Wrote file %s', '../results/synthetic/report/13568_l1_loss_loss_13.298544.jpg')
('Wrote file %s', '../results/synthetic/report/13696_l1_loss_loss_13.102976.jpg')
('Wrote file %s', '../results/synthetic/report/13824_l1_loss_loss_13.591106.jpg')
('Wrote file %s', '../results/synthetic/report/13952_l1_loss_loss_15.095733.jpg')
('Wrote file %s', '../results/synthetic/report/14080_l1_loss_loss_15.056795.jpg')
===> This iteration num Fail: 29

[===========================================================>.....] Step: 3m55s | Tot: 44m58s | Test, h_loss 13.956, rec_loss 0.976, ssim_loss 0.403, l1_loss 0.614, fail_percent 0.1966 ����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������� 111/120
('Wrote file %s', '../results/synthetic/report/14208_l1_loss_loss_13.3238125.jpg')
('Wrote file %s', '../results/synthetic/report/14336_l1_loss_loss_15.144823.jpg')
('Wrote file %s', '../results/synthetic/report/14464_l1_loss_loss_12.719545.jpg')
('Wrote file %s', '../results/synthetic/report/14592_l1_loss_loss_13.823379.jpg')
('Wrote file %s', '../results/synthetic/report/14720_l1_loss_loss_13.675428.jpg')
('Wrote file %s', '../results/synthetic/report/14848_l1_loss_loss_14.918747.jpg')
('Wrote file %s', '../results/synthetic/report/14976_l1_loss_loss_14.758549.jpg')
('Wrote file %s', '../results/synthetic/report/15104_l1_loss_loss_13.989742.jpg')
('Wrote file %s', '../results/synthetic/report/15232_l1_loss_loss_14.478268.jpg')
[================================================================>] Step: 3m30s | Tot: 48m28s | Test, h_loss 13.967, rec_loss 0.974, ssim_loss 0.403, l1_loss 0.613, fail_percent 0.1965 ����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������� 120/120
====> Result for RHO: 45 loss l1_loss noise 0.5
|Steps | h_loss | l1_loss | Fail percent |
119 13.966667119661967 0.6131097227334976 19.654947916666668

Top 0 - 30 %
Top 30 - 60 %
Top 60 - 100 %
===> Percentile Values: (20, 50, 80, 100):
[[13.17573 0.3507613 ]
[13.843848 0.15365075]
[14.651982 0.39429116]]
======> End! ====================================

[MrSorrow]

I also met this problem， and supervised method‘s performance is better than unsupervised method’s， did i miss something important？