Reproducing of the paper results

[leekanggeun]

Hello, I have one issue for reproducing the quantitative results of paper.
Even though I set up same parameter with paper, but I couldn’t get paper result for BSD68 with sigma=25 additive white gaussian noise.
So Could you give me the original code to get paper result? Because I want to cite your paper but I couldn’t reproduce exactly..
I mean original code would be training and testing code for BSD400 and BSD68 include dataset.

[alex-krull]

Dear @leekanggeun,
Due to the stochastic nature of neural network training, results can vary slightly between individual runs.
So you should not expect to see the precise same numbers.
However, the numbers should definitely not be systematically worse.
Could you please share the numbers you obtained with us?
Thanks!

[leekanggeun]

Thank you for response.
I set epochs and iteration/epoch as like 100, 300. Accuracy was 26.92 average PSNR. I thought it converges enough to lower bound. I know it's almost impossible to reproduce the same result exactly.
So could you share the avg. PSNR(numerical) for Noise2Void, as shown in Figure 7?

Thanks!!

[alex-krull]

This sounds definitely too low, compared to what we got.
We have unfortunately currently a deadline coming up and cannot look the the problem immediately.
I will get back to you in approx. a week and assemble all the data and code to reproduce the results.
I hope this is still in time for your paper submission?

[leekanggeun]

I'd appreciate it if you would. To verify my test data set which was generated by gaussian noise(sigma 25), I used bm3d method and got 28.29 while papers bm3d resulr is 28.59. I think that random seed to generate noise will affect to final psnr. Fortunately, I got 27.14 avg PSNR by another parameter of noise2void..but it looks still a little row.. I hope assemble could be completed until November 1. Thank you!!!

[tibuch]

Hello @leekanggeun

Thank you for you patience.

Here is the training/validation/test data which we used in our paper and a reproducibility jupyter notebook.

Unfortunately the computation of number of blind-spots in our current release is different from the one used during our paper-experiment time. This is a bug and is fixed on n2v/fix_numPix_computation. If you want to rerun the notebook immediately you would have to install n2v from this branch.

The branch will be merged soon, but we would have to rerun all examples. Additionally we probably add the reproducibility notebook from the zip-file above. In the future reproducing the PSNR numbers should only be a jupyter notebook execution away.

[tibuch]

Released: https://github.com/juglab/n2v/releases/tag/v0.1.10