Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Benchmarking #1

Open
IanButterworth opened this issue Feb 15, 2020 · 2 comments
Open

Benchmarking #1

IanButterworth opened this issue Feb 15, 2020 · 2 comments

Comments

@IanButterworth
Copy link
Member

IanButterworth commented Feb 15, 2020

I just added a benchmark script at https://github.com/JuliaIO/PNG.jl/blob/master/benchmarking/benchmark.jl

Looks like QuartzImageIO >> PNG > ImageMagick overall, but PNG is fastest to load

All of this is before @Drvi's IanButterworth/ImageIODevelopment.jl#3 which looks like it will improve things 👍🏻

Small image

[ Info: Testing with rand(RGB{N0f8}, 244, 244) image -----------------------------

[ Info: Testing FileIO with ImageMagick (dedicated environment, new instance)
Time to first save (including save): 2.336 seconds
Save @btime:  27.209 ms (3562 allocations: 710.64 KiB)
Load @btime:  5.420 ms (3722 allocations: 717.97 KiB)

[ Info: Testing FileIO with QuartzImageIO (dedicated environment, new instance)
Time to first save (including save): 2.445 seconds
Save @btime:  8.773 ms (1837 allocations: 794.13 KiB)
Load @btime:  3.477 ms (2062 allocations: 574.75 KiB)

[ Info: Testing FileIO with PNG (dedicated environment, new instance)
Time to first save (including save): 1.776 seconds
Save @btime:  20.581 ms (279 allocations: 370.69 KiB)
Load @btime:  2.569 ms (184 allocations: 183.33 KiB)

Large image

[ Info: Testing with rand(RGB{N0f8}, 20000, 4000) image -----------------------------

[ Info: Testing FileIO with ImageMagick (dedicated environment, new instance)
Time to first save (including save): 32.806 seconds
Save @btime:  30.127 s (3562 allocations: 686.83 MiB)
Load @btime:  4.962 s (3729 allocations: 686.84 MiB)

[ Info: Testing FileIO with QuartzImageIO (dedicated environment, new instance)
Time to first save (including save): 14.143 seconds
Save @btime:  11.721 s (1837 allocations: 915.62 MiB)
Load @btime:  3.934 s (2064 allocations: 610.46 MiB)

[ Info: Testing FileIO with PNG (dedicated environment, new instance)
Time to first save (including save): 30.107 seconds
Save @btime:  28.491 s (20035 allocations: 460.82 MiB)
Load @btime:  4.107 s (186 allocations: 228.89 MiB)
@Drvi Drvi mentioned this issue Feb 16, 2020
@IanButterworth
Copy link
Member Author

IanButterworth commented Feb 16, 2020

As expected, we now win with #2 merged

[ Info: Testing with rand(RGB{N0f8}, 244, 244) image -----------------------------
[ Info: Testing FileIO with ImageMagick (dedicated environment, new instance)
Time to first save (including save): 2.303 seconds
Save @btime:  27.416 ms (3562 allocations: 710.64 KiB)
Load @btime:  5.335 ms (3722 allocations: 717.97 KiB)

[ Info: Testing FileIO with QuartzImageIO (dedicated environment, new instance)
Time to first save (including save): 2.37 seconds
Save @btime:  8.847 ms (1837 allocations: 794.13 KiB)
Load @btime:  3.456 ms (2062 allocations: 574.75 KiB)

[ Info: Testing FileIO with PNGFiles (dedicated environment, new instance)
Time to first save (including save): 2.05 seconds
Save @btime:  4.608 ms (46 allocations: 178.64 KiB)
Load @btime:  3.145 ms (182 allocations: 359.59 KiB)
[ Info: Testing with rand(RGB{N0f8}, 20000, 4000) image -----------------------------
[ Info: Testing FileIO with ImageMagick (dedicated environment, new instance)
Time to first save (including save): 33.791 seconds
Save @btime:  29.370 s (3562 allocations: 686.83 MiB)
Load @btime:  5.350 s (3729 allocations: 686.84 MiB)

[ Info: Testing FileIO with QuartzImageIO (dedicated environment, new instance)
Time to first save (including save): 15.481 seconds
Save @btime:  13.516 s (1837 allocations: 915.62 MiB)
Load @btime:  4.120 s (2064 allocations: 610.46 MiB)

[ Info: Testing FileIO with PNGFiles (dedicated environment, new instance)
Time to first save (including save): 9.059 seconds
Save @btime:  6.418 s (49 allocations: 229.04 MiB)
Load @btime:  4.638 s (185 allocations: 457.92 MiB)

@jaakkor2
Copy link
Contributor

jaakkor2 commented May 26, 2020

Wondering if ImageMagick and PNGFiles numbers are improving after Zlib_jll got -O3 flag in JuliaPackaging/Yggdrasil#1051.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants