Tensorflow 2 Export Training Data Result is Not working

[lukeNguyen0202]

I tried to train my custom Data to detect 2 objects

https://tensorflow-object-detection-api-tutorial.readthedocs.io/en/latest/training.html#configure-the-training-pipeline

Here is my Pre-train model : ssd_resnet50_v1_fpn_640x640_coco17_tpu

Here is my model pipefile:

model {
  ssd {
    num_classes: 2
    image_resizer {
      fixed_shape_resizer {
        height: 640
        width: 640
      }
    }
    feature_extractor {
      type: "ssd_resnet50_v1_fpn_keras"
      depth_multiplier: 1.0
      min_depth: 16
      conv_hyperparams {
        regularizer {
          l2_regularizer {
            weight: 0.0004
          }
        }
        initializer {
          truncated_normal_initializer {
            mean: 0.0
            stddev: 0.03
          }
        }
        activation: RELU_6
        batch_norm {
          decay: 0.997
          scale: true
          epsilon: 0.001
        }
      }
      override_base_feature_extractor_hyperparams: true
      fpn {
        min_level: 3
        max_level: 7
      }
    }
    box_coder {
      faster_rcnn_box_coder {
        y_scale: 10.0
        x_scale: 10.0
        height_scale: 5.0
        width_scale: 5.0
      }
    }
    matcher {
      argmax_matcher {
        matched_threshold: 0.5
        unmatched_threshold: 0.5
        ignore_thresholds: false
        negatives_lower_than_unmatched: true
        force_match_for_each_row: true
        use_matmul_gather: true
      }
    }
    similarity_calculator {
      iou_similarity {
      }
    }
    box_predictor {
      weight_shared_convolutional_box_predictor {
        conv_hyperparams {
          regularizer {
            l2_regularizer {
              weight: 0.0004
            }
          }
          initializer {
            random_normal_initializer {
              mean: 0.0
              stddev: 0.01
            }
          }
          activation: RELU_6
          batch_norm {
            decay: 0.997
            scale: true
            epsilon: 0.001
          }
        }
        depth: 256
        num_layers_before_predictor: 4
        kernel_size: 3
        class_prediction_bias_init: -4.6
      }
    }
    anchor_generator {
      multiscale_anchor_generator {
        min_level: 3
        max_level: 7
        anchor_scale: 4.0
        aspect_ratios: 1.0
        aspect_ratios: 2.0
        aspect_ratios: 0.5
        scales_per_octave: 2
      }
    }
    post_processing {
      batch_non_max_suppression {
        score_threshold: 1e-08
        iou_threshold: 0.6
        max_detections_per_class: 100
        max_total_detections: 100
        use_static_shapes: false
      }
      score_converter: SIGMOID
    }
    normalize_loss_by_num_matches: true
    loss {
      localization_loss {
        weighted_smooth_l1 {
        }
      }
      classification_loss {
        weighted_sigmoid_focal {
          gamma: 2.0
          alpha: 0.25
        }
      }
      classification_weight: 1.0
      localization_weight: 1.0
    }
    encode_background_as_zeros: true
    normalize_loc_loss_by_codesize: true
    inplace_batchnorm_update: true
    freeze_batchnorm: false
  }
}
train_config {
  batch_size: 2
  data_augmentation_options {
    random_horizontal_flip {
    }
  }
  data_augmentation_options {
    random_crop_image {
      min_object_covered: 0.0
      min_aspect_ratio: 0.75
      max_aspect_ratio: 3.0
      min_area: 0.75
      max_area: 1.0
      overlap_thresh: 0.0
    }
  }
  sync_replicas: true
  optimizer {
    momentum_optimizer {
      learning_rate {
        cosine_decay_learning_rate {
          learning_rate_base: 0.04
          total_steps: 25000
          warmup_learning_rate: 0.013333
          warmup_steps: 2000
        }
      }
      momentum_optimizer_value: 0.9
    }
    use_moving_average: false
  }
  fine_tune_checkpoint: "/home/lnguyen/Apps/Tensorflow_2/workspace/pre-trained-models/ssd_resnet50_v1_fpn_640x640_coco17_tpu-8/checkpoint/ckpt-25"
  num_steps: 25000
  startup_delay_steps: 0.0
  replicas_to_aggregate: 8
  max_number_of_boxes: 100
  unpad_groundtruth_tensors: false
  fine_tune_checkpoint_type: "detection"
  use_bfloat16: false
  fine_tune_checkpoint_version: V2
}
train_input_reader {
  label_map_path: "/home/lnguyen/Apps/Tensorflow_2/workspace/training/label_map.pbtxt"
  tf_record_input_reader {
    input_path: "/home/lnguyen/Apps/Tensorflow_2/workspace/data/train.record"
  }
}
eval_config {
  metrics_set: "coco_detection_metrics"
  use_moving_averages: false
}
eval_input_reader {
  label_map_path: "/home/lnguyen/Apps/Tensorflow_2/workspace/training/label_map.pbtxt"
  shuffle: false
  num_epochs: 1
  tf_record_input_reader {
    input_path: "/home/lnguyen/Apps/Tensorflow_2/workspace/data/train.record"
  }
}

I use model_main_tf2.py to start the training
then I use exporter_main_v2.py to export the result

I got everything as expect, but I cant use the save_model.pb for the actual detection process.

in TF1, after the training, I have files like
model.ckpt ????? - ?????
model.ckpt.meta
model.ckpt.index
.
.

but why in TF2 I got:
ckpt-0.data ?????? - ?????
ckpt-0.index

Please help

[lukeNguyen0202]

I got this Error when I use the save_model.pb 👍

2020-10-14 16:03:51.891769: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudart.so.10.1
Traceback (most recent call last):
  File "detection.py", line 52, in <module>
    od_graph_def.ParseFromString(serialized_graph)
google.protobuf.message.DecodeError: Error parsing message

[sglvladi]

@lukeNguyen0202 did you follow the provided examples (see here)?

[sglvladi]

You can use above example to load the saved_model.pb. To do so, set PATH_TO_MODEL_DIR equal to the path to the folder that contains the exported files, e.g. C:/Users/sglvladi/Documents/Tensorflow/workspace/training_demo/exported-models/my_model

[lukeNguyen0202]

@lukeNguyen0202 did you follow the provided examples (see here)?

I found this instruction confusing, because I already have my training data, why would I need to download an extra source of data for detection.

So I follow an different instruction (here) which is similar and kinda same with what I need to do for detection. It need to connect to the video stream via Opencv, then break down each frame for detection

[lukeNguyen0202]

You can use above example to load the saved_model.pb. To do so, set PATH_TO_MODEL_DIR equal to the path to the folder that contains the exported files, e.g. C:/Users/sglvladi/Documents/Tensorflow/workspace/training_demo/exported-models/my_model

I have following what you refer on the load model part here is my code:

import time
from .models.research.object_detection.utils import label_map_util
from .models.research.object_detection.utils import visualization_utils as viz_utils
import tensorflow as tf


# Download and extract model
def download_model(model_name, model_date):
    base_url = 'http://download.tensorflow.org/models/object_detection/tf2/'
    model_file = model_name + '.tar.gz'
    model_dir = tf.keras.utils.get_file(fname=model_name,
                                        origin=base_url + model_date + '/' + model_file,
                                        untar=True)
    return str(model_dir)


MODEL_DATE = '20200711'
MODEL_NAME = 'centernet_hg104_1024x1024_coco17_tpu-32'

# PATH_TO_MODEL_DIR = download_model(MODEL_NAME, MODEL_DATE)
PATH_TO_MODEL_DIR = "/home/lnguyen/Apps/Tensorflow_2/workspace/exported_model_1"

PATH_TO_SAVED_MODEL = PATH_TO_MODEL_DIR + "/saved_model"

print('Loading model...', end='')
start_time = time.time()

# Load saved model and build the detection function
detect_fn = tf.saved_model.load(PATH_TO_SAVED_MODEL)

end_time = time.time()
elapsed_time = end_time - start_time
print('Done! Took {} seconds'.format(elapsed_time))

Here is the result after run:

Traceback (most recent call last):
  File "load_model.py", line 2, in <module>
    from .models.research.object_detection.utils import label_map_util
ImportError: attempted relative import with no known parent package

It seem like complaining about import tensorflow.compat.v1 as tf under label_map_util
do bypass this i have to disable TF2. Do you have any information about why TF2 cant do detection like TF1? but it have to be disable so that the detection work? or TF2 is not mead to be used for detection ?

[sglvladi]

@lukeNguyen0202 I have not included specific instructions on how to use your own data, since it should be straightforward to do so based on the provided examples.

Considering the example I linked above: There is not need to download the data. If you already have you own data, simply replace IMAGE_PATH with the path to the folder containing your data and comment out the downloading.

[sglvladi]

You can use above example to load the saved_model.pb. To do so, set PATH_TO_MODEL_DIR equal to the path to the folder that contains the exported files, e.g. C:/Users/sglvladi/Documents/Tensorflow/workspace/training_demo/exported-models/my_model

I have following what you refer on the load model part here is my code:

import time
from .models.research.object_detection.utils import label_map_util
from .models.research.object_detection.utils import visualization_utils as viz_utils
import tensorflow as tf


# Download and extract model
def download_model(model_name, model_date):
    base_url = 'http://download.tensorflow.org/models/object_detection/tf2/'
    model_file = model_name + '.tar.gz'
    model_dir = tf.keras.utils.get_file(fname=model_name,
                                        origin=base_url + model_date + '/' + model_file,
                                        untar=True)
    return str(model_dir)


MODEL_DATE = '20200711'
MODEL_NAME = 'centernet_hg104_1024x1024_coco17_tpu-32'

# PATH_TO_MODEL_DIR = download_model(MODEL_NAME, MODEL_DATE)
PATH_TO_MODEL_DIR = "/home/lnguyen/Apps/Tensorflow_2/workspace/exported_model_1"

PATH_TO_SAVED_MODEL = PATH_TO_MODEL_DIR + "/saved_model"

print('Loading model...', end='')
start_time = time.time()

# Load saved model and build the detection function
detect_fn = tf.saved_model.load(PATH_TO_SAVED_MODEL)

end_time = time.time()
elapsed_time = end_time - start_time
print('Done! Took {} seconds'.format(elapsed_time))

Here is the result after run:

Traceback (most recent call last):
  File "load_model.py", line 2, in <module>
    from .models.research.object_detection.utils import label_map_util
ImportError: attempted relative import with no known parent package

It seem like complaining about import tensorflow.compat.v1 as tf under label_map_util
do bypass this i have to disable TF2. Do you have any information about why TF2 cant do detection like TF1? but it have to be disable so that the detection work? or TF2 is not mead to be used for detection ?

Please provide the full traceback. You shouldn't have to disable TF2.

TF2 object detection API is different from TF1 due to design changes. Models in TF2 use Keras, which means that a common API can be followed as done for other Keras models.

P.S. I am not in any way affiliated with Google or Tensoflow, so cannot answer any questions regarding the reasons behind TF2 design choices.

[sglvladi]

It need to connect to the video stream via Opencv, then break down each frame for detection

Just FYI, there is an example on how to use your webcam in the tutorial. See here

[lukeNguyen0202]

You can use above example to load the saved_model.pb. To do so, set PATH_TO_MODEL_DIR equal to the path to the folder that contains the exported files, e.g. C:/Users/sglvladi/Documents/Tensorflow/workspace/training_demo/exported-models/my_model

I have following what you refer on the load model part here is my code:

import time
from .models.research.object_detection.utils import label_map_util
from .models.research.object_detection.utils import visualization_utils as viz_utils
import tensorflow as tf


# Download and extract model
def download_model(model_name, model_date):
    base_url = 'http://download.tensorflow.org/models/object_detection/tf2/'
    model_file = model_name + '.tar.gz'
    model_dir = tf.keras.utils.get_file(fname=model_name,
                                        origin=base_url + model_date + '/' + model_file,
                                        untar=True)
    return str(model_dir)


MODEL_DATE = '20200711'
MODEL_NAME = 'centernet_hg104_1024x1024_coco17_tpu-32'

# PATH_TO_MODEL_DIR = download_model(MODEL_NAME, MODEL_DATE)
PATH_TO_MODEL_DIR = "/home/lnguyen/Apps/Tensorflow_2/workspace/exported_model_1"

PATH_TO_SAVED_MODEL = PATH_TO_MODEL_DIR + "/saved_model"

print('Loading model...', end='')
start_time = time.time()

# Load saved model and build the detection function
detect_fn = tf.saved_model.load(PATH_TO_SAVED_MODEL)

end_time = time.time()
elapsed_time = end_time - start_time
print('Done! Took {} seconds'.format(elapsed_time))

Here is the result after run:

Traceback (most recent call last):
  File "load_model.py", line 2, in <module>
    from .models.research.object_detection.utils import label_map_util
ImportError: attempted relative import with no known parent package

It seem like complaining about import tensorflow.compat.v1 as tf under label_map_util
do bypass this i have to disable TF2. Do you have any information about why TF2 cant do detection like TF1? but it have to be disable so that the detection work? or TF2 is not mead to be used for detection ?

Please provide the full traceback. You shouldn't have to disable TF2.

TF2 object detection API is different from TF1 due to design changes. Models in TF2 use Keras, which means that a common API can be followed as done for other Keras models.

P.S. I am not in any way affiliated with Google or Tensoflow, so cannot answer any questions regarding the reasons behind TF2 design choices.

Here is the full full traceback
models/research/object_detection/utils/label_map_util.py <-- then im not sure why it complain about the import tensorflow.compat.v1 as tf

Im checking out your webcame code right now.

Thank you for all your time, assistant, and the information your share :)

[lukeNguyen0202]

@sglvladi

It still complain about the label_map_util.py

(Tensorflow_Opencv) lnguyen@pop-luke:~/PycharmProjects/Tensorflow_Opencv$ python3 object_detection_camera.py 
Downloading model. This may take a while... Done
Downloading label file... Done
Traceback (most recent call last):
  File "object_detection_camera.py", line 88, in <module>
    from .models.research.object_detection.utils import label_map_util
ImportError: attempted relative import with no known parent package

full traceback
/home/lnguyen/PycharmProjects/Tensorflow_Opencv/models/research/object_detection/utils/label_map_util.py

The Error in the label_map_util

[sglvladi]

@lukeNguyen0202 hmmm... That's odd.. I'll try to reproduce it locally and will get back to you.

In the meantime, can you confirm the version of Tensorflow you have installed? Also, how did you download tensorflow/models? If you used git, can you specify the latest commit on your local clone?

[lukeNguyen0202]

@sglvladi

tensorflow==2.3.1
I think i downloaded model using git, downloaded as a zip file. I dont know how to check on the "latest commit" on my local clone, but I download it in Oct 8

[sglvladi]

@lukeNguyen0202 I am afraid I cannot reproduce the issue. I've performed a clean install of Tensorflow as per the instructions of the tutorial, and then installed the latest version of tensorflow/models. The object_detection_camera.py script seems to work just fine.

The traceback you have provided seems to suggest that the code is tripping over at line 88 of object_detection_camera.py. If you have not modified the file, this is the line where import tensorflow as tf should be, which is very bizarre. If you are getting this issue when importing tensorflow, then you should not have been able to train your model.

[sglvladi]

(Tensorflow_Opencv) lnguyen@pop-luke:~/PycharmProjects/Tensorflow_Opencv$ python3 object_detection_camera.py 
Downloading model. This may take a while... Done
Downloading label file... Done
Traceback (most recent call last):
  File "object_detection_camera.py", line 88, in <module>
    from .models.research.object_detection.utils import label_map_util
ImportError: attempted relative import with no known parent package

Just noticed you have replaced from object_detection.utils import label_map_util with from .models.research.object_detection.utils import label_map_util. Why have you done this?

[lukeNguyen0202]

@sglvladi
I was able to do the training outside of the project.
after the training, I created a new project, copied the tensorflow/models and the export_model in I assumed everything should be working

If I didnt do it, the project would not know where to look for the label_map_util.