object_detection_tutorial test runtime

[89douner]

System information

• What is the top-level directory of the model you are using: ssd_inception_v2_coco_2017_11_17
• Have I written custom code (as opposed to using a stock example script provided in TensorFlow):
• OS Platform and Distribution (e.g., Linux Ubuntu 16.04):Linux Ubuntu 16.04
• TensorFlow installed from (source or binary): installing on Python and Pip
  (https://www.tensorflow.org/versions/r1.5/install/install_linux#InstallingNativePip)
• TensorFlow version (use command below): 1.5.0
• Bazel version (if compiling from source): there is no bazel command in my computer, i just install tensorflow 1.5.0 through Python and Pip method as i mentioned above
• CUDA/cuDNN version:9.0
• GPU model and memory: GeForce GTX 1070
• Exact command to reproduce: ??

###You can collect some of this information using our environment capture script:

2018-03-06 22:58:39.700094: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
2018-03-06 22:58:39.816687: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:895] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2018-03-06 22:58:39.816965: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1105] Found device 0 with properties:
name: GeForce GTX 1070 major: 6 minor: 1 memoryClockRate(GHz): 1.683
pciBusID: 0000:01:00.0
totalMemory: 7.93GiB freeMemory: 7.48GiB
2018-03-06 22:58:39.816981: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1195] Creating TensorFlow device (/device:GPU:0) -> (device: 0, name: GeForce GTX 1070, pci bus id: 0000:01:00.0, compute capability: 6.1)
Wrote environment to tf_env.txt. You can review the contents of that file.
and use it to populate the fields in the github issue template.

Describe the problem

1. Test runtime too long.

• I executed object_detection_tutorial after changing 'ipynb' into 'py' file. As you can see below code,
  It took about 3 second to detect one image. Also, It took about 130 second to detect 130 images, 253 second 400 frames. But, I think that, at least, it take 200 frames 10 second since this model has real-time performance.

2. Possible cause I expected...

1. GPU doesn't work properly.
   --> if in this case, you can give me answer through above environment capture, system specification.

2. Wrong measurement test runtime method (you can see the method below picture)
   --> Please, let me know how to exactly measure detection time.

Please answer me detail.. I wanna give your team help to upgrade Google Object Detection models !

Source code / logs

with detection_graph.as_default():

with tf.Session(graph=detection_graph) as sess:
    start = time.time()
    image_tensor = detection_graph.get_tensor_by_name('image_tensor:0')
    # Each box represents a part of the image where a particular object was detected.
    detection_boxes = detection_graph.get_tensor_by_name('detection_boxes:0')
    # Each score represent how level of confidence for each of the objects.
    # Score is shown on the result image, together with the class label.
    detection_scores = detection_graph.get_tensor_by_name('detection_scores:0')
    detection_classes = detection_graph.get_tensor_by_name('detection_classes:0')
    num_detections = detection_graph.get_tensor_by_name('num_detections:0')
   
    for image_path in TEST_IMAGE_PATHS:
        image = Image.open(image_path)
        image_np = load_image_into_numpy_array(image)
        # Expand dimensions since the model expects images to have shape: [1, None, None, 3]
        image_np_expanded = np.expand_dims(image_np, axis=0)
        # Actual detection.
        (boxes, scores, classes, num) = sess.run(
            [detection_boxes, detection_scores, detection_classes, num_detections],
            feed_dict={image_tensor: image_np_expanded})

        # Visualization of the results of a detection.
        vis_util.visualize_boxes_and_labels_on_image_array(
            image_np,
            np.squeeze(boxes),
            np.squeeze(classes).astype(np.int32),
            np.squeeze(scores),
            category_index,
            use_normalized_coordinates=True,
            line_thickness=2)

    print("--- %s seconds ---" % (time.time() - start))

    '''
    except for 'start = time.time()' and 'print("--- %s seconds ---" % (time.time() - start))' code,
    It's the same as 'object_detection_tutorial.ipynb' code. I used  ssd_inception_v2_coco_2017_11_17
    ''' 

################################################################################

#################################################################################

P.S
If you need more information, please tell me. Thanks!

[pkulzc]

Your running time is "image loading + inference + visualization", which isn't inference at all.
Please also refer to the following section in model zoo:

model speed --- we report running time in ms per 600x600 image (including all pre and post-processing), but please be aware that these timings depend highly on one's specific hardware configuration (these timings were performed using an Nvidia GeForce GTX TITAN X card) and should be treated more as relative timings in many cases.

[89douner]

Thanks for you reply
Could you let me know how to measure detection time (except for above them you mentioned such as image loading, inference, visualization and so on) ??
It is also good method to let me know any reference sites or paper.
Thanks!

[pkulzc]

Typically the inference time refers to the time spent on the "session.run()" block. It's the time of a forward pass(from input layer to output layer).

[89douner]

Thanks for your answer. Could I question one more?
When I detect some images (framse), I applied below code to measure test time
`
TEST_IMAGE_PATHS = [ os.path.join(PATH_TO_TEST_IMAGES_DIR, 'frame{}.jpg'.format(i)) for i in range(2, 10) ]

%%%%....%%%%

start = time.time()
(boxes, scores, classes, num) = sess.run(
[detection_boxes, detection_scores, detection_classes, num_detections],
feed_dict={image_tensor: image_np_expanded})
stop = time.time()
print(stop-start)
`
frame 1 takes 3.4 second, but frame 2, frame 3, ... frame n take 0.04 second.
I think that 0.04 is proper test time. I want to know why frame 1 takes long time.
Because of memory loading time? or other reasons?
please answer. Thanks again

[cpmgs]

I am guessing there is a lot of overhead for the first image e.g. loading the graph, transferring it to the GPU and so on. Your findings seem to suggest that: once the graph is loaded and data is coming in fast, the GPU is reducing the processing time a lot. Did you find out anything else related to this? Curious as I am having the exact same question/experience and run a nearly identical setup to you.

[tensorflowbutler]

Hi There,
We are checking to see if you still need help on this, as this seems to be considerably old issue. Please update this issue with the latest information, code snippet to reproduce your issue and error you are seeing.
If we don't hear from you in the next 7 days, this issue will be closed automatically. If you don't need help on this issue any more, please consider closing this.