Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The enlarged result is different from waifu2x-caffe. #4

Closed
f11894 opened this issue Apr 7, 2019 · 21 comments
Closed

The enlarged result is different from waifu2x-caffe. #4

f11894 opened this issue Apr 7, 2019 · 21 comments
Labels
bug Something isn't working

Comments

@f11894
Copy link

f11894 commented Apr 7, 2019

Ryzen 2400G
[0 AMD Radeon(TM) RX Vega 11 Graphics] queueC=1 queueT=2 memU=2 memDL=0 memHV=1
[0 AMD Radeon(TM) RX Vega 11 Graphics] fp16s=1 fp16a=1 int8s=1 int8a=1

input file
test

waifu2x-ncnn-vulkan ver 20190407

waifu2x.exe "%~1" "%~dpn1-ncnn-vulkan-2x.png" -1 2
waifu2x.exe "%~dpn1-ncnn-vulkan-2x.png" "%~dpn1-ncnn-vulkan-4x.png" -1 2

test-ncnn-vulkan-4x

waifu2x-caffe ver 1.2.0.2

waifu2x-caffe-cui.exe -t 0 -b 1 -c 128 -p cpu --model_dir ./models/cunet -s 2 -m scale -i "%~1" -o "%~dpn1-caffe-2x.png"
waifu2x-caffe-cui.exe -t 0 -b 1 -c 128 -p cpu --model_dir ./models/cunet -s 2 -m scale -i "%~dpn1-caffe-2x.png" -o "%~dpn1-caffe-4x.png"

test-caffe-4x

@Dearmw
Copy link

Dearmw commented Apr 7, 2019

Input
1

Running code waifu2x 1.jpg 2.jpg 2 2
[0 Radeon RX Vega] queueC=1 queueT=2 memU=2 memDL=0 memHV=1
[0 Radeon RX Vega] fp16s=1 fp16a=1 int8s=1 int8a=1

Output
2

@f11894
Copy link
Author

f11894 commented Apr 7, 2019

In addition, Benchmark score seems to be worse than waifu2x-caffe.
Average score of 55 images.

waifu2x-caffe RGB
35.492dB

waifu2x-caffe CUnet
37.619dB

waifu2x-ncnn-vulkan CUnet
34.469dB

This is the bat file I used for benchmarking.
https://github.com/f11894/SR_benchmark

I use Google Translate because I am not good at English.

@nihui
Copy link
Owner

nihui commented Apr 7, 2019

Input
1

Running code waifu2x 1.jpg 2.jpg 2 2
[0 Radeon RX Vega] queueC=1 queueT=2 memU=2 memDL=0 memHV=1
[0 Radeon RX Vega] fp16s=1 fp16a=1 int8s=1 int8a=1

Output
2

the incorrect output caused by fp16 arithmetic on gtx20/rtx20/vega issue is confirmed. A workaround will be applied for these gpu soon...

@nihui
Copy link
Owner

nihui commented Apr 7, 2019

thanks for your report
will investigate ...

@Dearmw
Copy link

Dearmw commented Apr 7, 2019

感谢您的报告
将进行调查......
Are you from Shanghai?

@nihui
Copy link
Owner

nihui commented Apr 7, 2019

deconvolution operator produce wrong result on fp16 storage/arithmetic mode.
I will fix it soon ...

@nihui
Copy link
Owner

nihui commented Apr 7, 2019

the underlying ncnn library get fixed in Tencent/ncnn@9643916
new binary will be rebuild and uploaded about an hour later, stay tuned ...

@nihui
Copy link
Owner

nihui commented Apr 7, 2019

20190407 package updated, please try the new binary

@f11894
Copy link
Author

f11894 commented Apr 7, 2019

There is no change in the output result using the new binary.
MD5 is also consistent with the previous.

@nihui
Copy link
Owner

nihui commented Apr 8, 2019

There is no change in the output result using the new binary.
MD5 is also consistent with the previous.

hi, I just tried the new binary with your original image on AMD r9-m290x machine and my Intel UHD 620 laptop, the output image looks the same with caffe one.

waifu2x.exe
SHA1: 6892F871041F4DAE1BF74FF9692547317A99DFBD

@f11894
Copy link
Author

f11894 commented Apr 8, 2019

I'm sorry I didn't have enough words.
MD5 is from the output PNG image, not the executable file.

When I converted the sample described in README.md, I noticed that I could not get the same result as you.
Apparently there are things that do not work well depending on the environment.
image

@nihui
Copy link
Owner

nihui commented Apr 8, 2019

I'm sorry I didn't have enough words.
MD5 is from the output PNG image, not the executable file.

When I converted the sample described in README.md, I noticed that I could not get the same result as you.
Apparently there are things that do not work well depending on the environment.
image

another issue which affects modern gpu with fp16a capability is confirmed.
because of lack of these specific devices, I may try to figure it out with some workarounds.

@nihui
Copy link
Owner

nihui commented Apr 8, 2019

new binary 20190408 has been uploaded with a workaround for gtx20 and vega series.
please try this new version and see if this issue still exists.

@kalcohol
Copy link

kalcohol commented Apr 8, 2019

@nihui @f11894 @Dearmw
i tested version 20190408, it seems this issue was fixed @rtx2060 and RTX2080.

@f11894
Copy link
Author

f11894 commented Apr 8, 2019

Even in my environment, I confirmed that this problem was solved.
The output result is equivalent to waifu2x-caffe.

@f11894 f11894 closed this as completed Apr 8, 2019
@f11894
Copy link
Author

f11894 commented Apr 8, 2019

In the case of visual inspection, I do not feel much difference, but there seems to be a little difference in the score of Benchmark.

waifu2x-caffe RGB
35.492dB

waifu2x-caffe CUnet
37.619dB

waifu2x-ncnn-vulkan(20190408) CUnet
36.942dB

@nihui nihui added the bug Something isn't working label Apr 9, 2019
@f11894
Copy link
Author

f11894 commented Apr 14, 2019

https://i.imgur.com/SFs1Ws8.gif

If you look closely and compare it, the contrast of the image seems different.
Will the issue be solved if the bug related to fp16a is fixed?

よく見比べてみると画像のコントラストが異なるようです。
fp16aに関する不具合が修正されるとこの問題は解決しますか?

@f11894 f11894 reopened this Apr 14, 2019
@nihui
Copy link
Owner

nihui commented Apr 14, 2019

https://i.imgur.com/SFs1Ws8.gif

If you look closely and compare it, the contrast of the image seems different.
Will the issue be solved if the bug related to fp16a is fixed?

よく見比べてみると画像のコントラストが異なるようです。
fp16aに関する不具合が修正されるとこの問題は解決しますか?

yeah, I found the little difference too.
It is not because of fp16s or fp16a, but because of postprocess.
waifu2x-caffe add a small eps value to all pixels during postprocess, while waifu2x-ncnn-vulkan do not.
I may port this behavior to let the result pixel-wise identity to the caffe one.

@nihui
Copy link
Owner

nihui commented Apr 14, 2019

20190414 package updated, please try the new binary
the result is still not pixel-wise identity to the caffe output due to loss-of-precision during floating point arithmetic and fp16 storage conversion
but it is close enough I think ^^:)

@f11894
Copy link
Author

f11894 commented Apr 14, 2019

Yes, ver 20190414 is almost the same as waif2x-caffe.
You will not notice the difference unless you compare them fairly carefully.

はい、ver 20190414ではwaif2x-caffeと殆ど変わらなくなりました。
かなり注意深く比較しない限り違いに気づくことはありません。

Ryzen 2400G

waifu2x-caffe ver 1.2.0.2
waifu2x-caffe-cui.exe -i input.png -t 0 --crop_size 128 -m scale --scale_ratio 2 --model_dir models\CUnet -p cpu -o output.png

waifu2x-ncnn-vulkan ver 20190407 and 20190408
waifu2x.exe input.png output.png -1 2

waifu2x-ncnn-vulkan ver 20190414
waifu2x.exe input.png output.png -1 2 300

PSNR(RGB) SSIM(RGB) Average time(s)
waifu2x-caffe 37.619 0.987 20.250
waifu2x-ncnn-vulkan(20190407) 34.469 0.975 8.557
waifu2x-ncnn-vulkan(20190408) 36.942 0.986 8.664
waifu2x-ncnn-vulkan(20190414) 37.334 0.986 3.641

@nihui nihui closed this as completed Apr 17, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

5 participants
@nihui @kalcohol @Dearmw @f11894 and others