is my model using GPU or not?

[IamExperimenting]

Prerequisites

Please answer the following questions for yourself before submitting an issue.

• I am using the latest TensorFlow Model Garden release and TensorFlow 2.
• I am reporting the issue to the correct repository. (Model Garden official or research directory)
• I checked to make sure that this issue has not been filed already.

1. The entire URL of the file you are using

https://github.com/tensorflow/models/tree/master/official/research/object_detection/models/ssd_mobilenet_v2_fpn_feature_extractor

2. Describe the bug

I’m performing object detection using TensorFlow Object Detection API. I have got a couple of doubts.

1. Whether my object detection training utilise GPU or not?
2. Why it is displaying CPU devices and ran out of memory? But then when I run the training script it shows that “Successfully opened dynamic library cudart64_110.dll”.

3. Steps to reproduce

python Tensorflow\models\research\object_detection\model_main_tf2.py --model_dir=Tensorflow\workspace\models\my_ssd_mobnet --pipeline_config_path=Tensorflow\workspace\models\my_ssd_mobnet\pipeline.config --num_train_steps=500

4. Expected behavior

should use GPU

5. Additional context

Include any logs that would be helpful to diagnose the problem.

6. System information

• OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Windows 10
• Mobile device name if the issue happens on a mobile device:
• TensorFlow installed from (source or binary): source
• TensorFlow version (use command below): 2.5.0
• Python version: 3.8.11
• Bazel version (if compiling from source): Bazel 3.7.2
• GCC/Compiler version (if compiling from source): MSVC 2019
• CUDA/cuDNN version: 11.2/8.1
• GPU model and memory: GPU: GEFORCE RTX 2060 and 8GB

[IamExperimenting]

[ymodak]

“Successfully opened dynamic library cudart64_110.dll”.

Indicates that gpu setup has been acknowledged.

Have you tried limiting gpu memory growth to overcome oom error;

gpus = tf.config.list_physical_devices('GPU')
tf.config.set_visible_devices(gpus[1], 'GPU') #1 assigns to  nvidia gpu
tf.config.experimental.set_memory_growth(gpus[1], True)

[IamExperimenting]

@ymodak, I'm afraid I haven't tried this.

Do you want me to mention the below three lines in the model_main_tf2.py script?

gpus = tf.config.list_physical_devices('GPU')
tf.config.set_visible_devices(gpus[1], 'GPU') #1 assigns to  nvidia gpu
tf.config.experimental.set_memory_growth(gpus[1], True)

because I faced this error whilst training the object detection model. It would be helpful if you point to some relevant article.
Thanks in advance.

[ymodak]

Yes. Can you try to putting those lines on top of the script after importing tf?

[google-ml-butler]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you.