Array serialization to string too slow (caffe.io.array_to_datum, SerializeToString) #5740

Open
tpet opened this Issue Jul 5, 2017 · 1 comment

Comments

Projects
None yet
1 participant

tpet commented Jul 5, 2017

Issue summary

Serialization of float array to string is very slow.

Steps to reproduce

from __future__ import print_function
import caffe
import numpy as np
from timeit import default_timer as timer

for i in range(10):
    arr = np.zeros((5, 1000, 1000), dtype=np.float32)
    t0 = timer()
    item_dat = caffe.io.array_to_datum(arr.astype(float))
    t1 = timer()
    item_str = item_dat.SerializeToString()
    t2 = timer()
    print(t1 - t0, t2 - t1)

gives around 1.7 s and 2.1 s on my i5.
Are there any reasons for it to be that slow? It seems to be rather suspicious, for serializing 5M floats.

Your system configuration

Operating system: Ubuntu 16.04
Compiler: gcc
BLAS: openblas
Python or MATLAB version (for pycaffe and matcaffe respectively): 2.7.12

tpet commented Jul 5, 2017

Maybe related to #4489?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment