CV Benchmark

Who is fast, and who is the fastest.

Results

• Read a .jpeg color image as an numpy array.

./mmbench benchmarks/run_resize.py
# resize cv2  50 loops, best of 5: 5.13 msec per loop
# resize np   20 loops, best of 5: 9.05 msec per loop

./mmbench jpg2np_color
# jpg2np opencv     200 loops, best of 5: 1.51 msec per loop
# jpg2np turbojpeg  500 loops, best of 5: 624 usec per loop

• Read a .jpeg gray scale image as an numpy array.

./mmbench jpg2np_gray
# jpg2np_gray opencv     500 loops, best of 5: 658 usec per loop
# jpg2np_gray turbojpeg  500 loops, best of 5: 451 usec per loop

• Read a .jpeg color image from bytes to an numpy array.

./mmbench jpgbytes2np_color
# jpgbytes2np_color opencv     200 loops, best of 5: 1.39 msec per loop
# jpgbytes2np_color turbojpeg  500 loops, best of 5: 608 usec per loop

• Read a .jpeg gray scale image from bytes to an numpy array.

./mmbench jpgbytes2np_gray
# jpgbytes2np_gray opencv     500 loops, best of 5: 629 usec per loop
# jpgbytes2np_gray turbojpeg  500 loops, best of 5: 438 usec per loop

• Image array normalization

./mmbench img_array_normalize
# img_array_normalize cv2_div    200 loops, best of 5: 1.81 msec per loop
# img_array_normalize cv2_mult   200 loops, best of 5: 1.68 msec per loop
# img_array_normalize cv2_mult2  200 loops, best of 5: 1.33 msec per loop
# img_array_normalize cv2_mult3  200 loops, best of 5: 1.25 msec per loop
# img_array_normalize cv2_mult4  500 loops, best of 5: 594 usec per loop
# img_array_normalize np         100 loops, best of 5: 2.43 msec per loop

• Convert numpy array from unit8 to float32

./mmbench np_uint8_to_float32
# np_uint8_to_float32 astype       5000 loops, best of 5: 69 usec per loop
# np_uint8_to_float32 constructor  5000 loops, best of 5: 67.8 usec per loop

• Stack a list of np array (non-contiguous) along axis 0. This test is unstable. Not sure why, yet

./mmbench img_list_stack
# img_list_stack constructor  100 loops, best of 5: 2.12 msec per loop
# img_list_stack stack        200 loops, best of 5: 1.96 msec per loop

• Clip bbox

./mmbench bbox_clip
# bbox_clip clip  50000 loops, best of 5: 4.09 usec per loop
# bbox_clip fast  100000 loops, best of 5: 3.49 usec per loop
# bbox_clip slow  50000 loops, best of 5: 5.71 usec per loop

• Permute channel dim before spatial at cpu or gpu.

./mmbench -n 500 benchmarks/run_permute_at.py
# permute_at cpu  500 loops, best of 5: 1.22 msec per loop
# permute_at gpu  500 loops, best of 5: 478 usec per loop

• Flip image array using np or cv2.

./mmbench benchmarks/run_flip.py
# flip cv2  500 loops, best of 5: 915 usec per loop
# flip np   50 loops, best of 5: 7.66 msec per loop

• Pipeline: load -> resize -> crop -> flip -> normalize -> transpose

## color

# color 400x300 -> x1.41 -> 224x224
./mmbench -n 200 pipeline_color
# pipeline_color opencv          200 loops, best of 5: 4.85 msec per loop
# pipeline_color opencv_fast     200 loops, best of 5: 2.81 msec per loop
# pipeline_color opencv_fastest  200 loops, best of 5: 1.8 msec per loop
# pipeline_color pil             200 loops, best of 5: 2.6 msec per loop
# pipeline_color pil_fast        200 loops, best of 5: 1.85 msec per loop

# color 400x300 -> x0.80 -> 224x224
./mmbench -n 200 pipeline_color_shrink
# pipeline_color_shrink opencv          200 loops, best of 5: 3.26 msec per loop
# pipeline_color_shrink opencv_fast     200 loops, best of 5: 2.13 msec per loop
# pipeline_color_shrink opencv_fastest  200 loops, best of 5: 1.89 msec per loop
# pipeline_color_shrink pil             200 loops, best of 5: 1.81 msec per loop
# pipeline_color_shrink pil_fast        200 loops, best of 5: 1.86 msec per loop

# color 800x600 -> x0.426 -> 224x224
./mmbench -n 200 pipeline_color_shrink_big
# pipeline_color_shrink_big opencv          200 loops, best of 5: 7.22 msec per loop
# pipeline_color_shrink_big opencv_fast     200 loops, best of 5: 3.1 msec per loop
# pipeline_color_shrink_big opencv_fastest  200 loops, best of 5: 2.65 msec per loop
# pipeline_color_shrink_big pil             200 loops, best of 5: 2.7 msec per loop
# pipeline_color_shrink_big pil_fast        200 loops, best of 5: 2.87 msec per loop

# color 133x100 -> x2.56 -> 224x224
./mmbench -n 200 pipeline_color_small
# pipeline_color_small opencv          200 loops, best of 5: 1.91 msec per loop
# pipeline_color_small opencv_fast     200 loops, best of 5: 1.4 msec per loop
# pipeline_color_small opencv_fastest  200 loops, best of 5: 1.14 msec per loop
# pipeline_color_small pil             200 loops, best of 5: 1.23 msec per loop
# pipeline_color_small pil_fast        200 loops, best of 5: 1.18 msec per loop


## gray

# gray 400x300 -> x1.41 -> 224x224
./mmbench -n 200 pipeline_gray
# pipeline_gray opencv          200 loops, best of 5: 1.63 msec per loop
# pipeline_gray opencv_fast     200 loops, best of 5: 1.33 msec per loop
# pipeline_gray opencv_fastest  200 loops, best of 5: 865 usec per loop
# pipeline_gray pil             200 loops, best of 5: 2.31 msec per loop
# pipeline_gray pil_fast        200 loops, best of 5: 1.19 msec per loop

# gray 400x300 -> x0.80 -> 224x224
./mmbench -n 200 pipeline_gray_shrink
# pipeline_gray_shrink opencv          200 loops, best of 5: 1.59 msec per loop
# pipeline_gray_shrink opencv_fast     200 loops, best of 5: 1.28 msec per loop
# pipeline_gray_shrink opencv_fastest  200 loops, best of 5: 898 usec per loop
# pipeline_gray_shrink pil             200 loops, best of 5: 2.3 msec per loop
# pipeline_gray_shrink pil_fast        200 loops, best of 5: 1.22 msec per loop

# gray 133x100 -> x2.56 -> 224x224
./mmbench -n 200 pipeline_gray_small
# pipeline_gray_small opencv          200 loops, best of 5: 445 usec per loop
# pipeline_gray_small opencv_fast     200 loops, best of 5: 434 usec per loop
# pipeline_gray_small opencv_fastest  200 loops, best of 5: 428 usec per loop
# pipeline_gray_small pil             200 loops, best of 5: 942 usec per loop
# pipeline_gray_small pil_fast        200 loops, best of 5: 727 usec per loop


## batch

./mmbench -n 10 pipeline_color_batch
# pipeline_color_batch opencv          10 loops, best of 5: 265 msec per loop
# pipeline_color_batch opencv_fast     10 loops, best of 5: 145 msec per loop
# pipeline_color_batch opencv_fastest  10 loops, best of 5: 88.4 msec per loop
# pipeline_color_batch pil             10 loops, best of 5: 99.8 msec per loop
# pipeline_color_batch pil_fast        10 loops, best of 5: 96 msec per loop

./mmbench -n 10 pipeline_color_shrink_batch
# pipeline_color_shrink_batch opencv          10 loops, best of 5: 187 msec per loop
# pipeline_color_shrink_batch opencv_fast     10 loops, best of 5: 119 msec per loop
# pipeline_color_shrink_batch opencv_fastest  10 loops, best of 5: 100 msec per loop
# pipeline_color_shrink_batch pil             10 loops, best of 5: 90.7 msec per loop
# pipeline_color_shrink_batch pil_fast        10 loops, best of 5: 101 msec per loop

./mmbench -n 10 pipeline_color_small_batch
# pipeline_color_small_batch opencv          10 loops, best of 5: 103 msec per loop
# pipeline_color_small_batch opencv_fast     10 loops, best of 5: 67.1 msec per loop
# pipeline_color_small_batch opencv_fastest  10 loops, best of 5: 46.2 msec per loop
# pipeline_color_small_batch pil             10 loops, best of 5: 52.4 msec per loop
# pipeline_color_small_batch pil_fast        10 loops, best of 5: 49.7 msec per loop


## gpu

./mmbench -n 100 img_array_normalize_at
# img_array_normalize_at cpu  100 loops, best of 5: 2.57 msec per loop
# img_array_normalize_at gpu  100 loops, best of 5: 486 usec per loop

Setup

conda uninstall -y --force pillow pil jpeg libtiff libjpeg-turbo
pip   uninstall -y         pillow pil jpeg libtiff libjpeg-turbo
conda install -yc conda-forge libjpeg-turbo
CFLAGS="${CFLAGS} -mavx2" pip install --upgrade --no-cache-dir --force-reinstall --no-binary :all: --compile pillow-simd

pip install -U git+git://github.com/lilohuang/PyTurboJPEG.git