Ways to freeze RetinaNet to a .pb file?

[xiaoyongzhu]

Is there a way to freeze RetinaNet checkpoint to a .pb file for further referencing, after it got trained? From my limited knowledge, there are two ways to convert a checkpoint to a .pb file in TF, which are all impossible to convert the trained RetinaNet model to .pb file.

1. use the freeze_graph tool by TensorFlow, as described here (https://github.com/tensorflow/tensorflow/blob/master/tensorflow/python/tools/freeze_graph.py). However this command requires to specify output_node_names parameter which is hard to get for RetinaNet by analyzing its graph or using the summarize_graph provided (https://github.com/tensorflow/tensorflow/tree/master/tensorflow/tools/graph_transforms#inspecting-graphs). The summarize_graph tool will give over 1,000 possible names.

2. Use the export_inference_graph tool provided by the Object Detection API (https://github.com/tensorflow/models/blob/master/research/object_detection/export_inference_graph.py), which requires the model definition, which does not exist yet for RetinaNet.

So my question is - what's the best way to freeze the trained RetinaNet model to a .pb file for further inference?

[roitmaster]

same issue

how to make inference

[andr0idsensei]

I also want to run inference with one of the trained models and I added the following code in the retinanet_main.py main function after the train and eval if checks:

if FLAGS.mode == 'infer':
      gpu_options = tf.GPUOptions(
          allow_growth=True)

      cfg_proto = tf.ConfigProto(
          allow_soft_placement=True, log_device_placement=False, gpu_options=gpu_options)

      if FLAGS.use_xla and not FLAGS.use_tpu:
          cfg_proto.graph_options.optimizer_options.global_jit_level = (
              tf.OptimizerOptions.ON_1)

      run_cfg = run_config.RunConfig(
          model_dir=FLAGS.model_dir,
          log_step_count_steps=FLAGS.iterations_per_loop,
          session_config=cfg_proto
      )

      tf.logging.info('running predictions on: %s', FLAGS.predict_file_pattern)

      predict_params = dict(
          params,
          resnet_checkpoint=None,
          input_rand_hflip=False,
          is_training_bn=False)

      tf.logging.info('running predictions with params: %s', predict_params)

      predict_estimator = estimator.Estimator(
          model_fn=retinanet_model.retinanet_model_fn,
          config=run_cfg,
          params=predict_params)
      predictions = 
          list(predict_estimator.predict(input_fn=dataloader.InputReader(FLAGS.predict_file_pattern,
                                                                                   is_training=False)))
      tf.logging.info("predictions: %s", predictions)

I am trying to run this on a GeForce 1080 with 12 Gb or RAM and using a single tf records file of about 150 MB but afte running for about a minute or so I get a CUDA_ERROR_OUT_OF_MEMORY error and I can't figure out what I am doing wrong or what params should I pass to the inference method of the Estimator in order to work...

[andr0idsensei]

The actual problem seems to be that in the model function retinanet adds the image in the prediction dictionary, so if you run the above code over a large number of images, it will store all of them in memory, thus resulting in the out of memory error. The solution was to run predictions over a smaller number of images and/or to remove the image from the predicted output. So, if you are looking for a way to run predictions with trained checkpoints from the tpu retinanet, the code above should allow you to do that without the need to convert the checkpoints to pb files.

[ernstgoyer]

same issue
how can i get the output nodes from retinanet

[andr0idsensei]

@ernstgoyer a starting point would be to look at how that is done in their evaluation code.

[mgoundge11]

Hi, @xiaoyongzhu I am facing the same issue. I have Retinanet implementation using tensorflow eager, I'm able to do the training after that it is saving the .ckpt files but I'm not finding the way how to use the model for inference? need sugetions on the same thank you.

[xiaoyongzhu]

I ended up using the TensorFlow Object Detection API. They recently added RetinaNet and it’s easy to freeze the pb graph from there. Thanks! Xiaoyong From: mgoundge11 <notifications@github.com> Sent: Tuesday, August 28, 2018 2:03 AM To: tensorflow/tpu <tpu@noreply.github.com> Cc: Xiaoyong Zhu <xiaoyzhu@microsoft.com>; Mention <mention@noreply.github.com> Subject: Re: [tensorflow/tpu] Ways to freeze RetinaNet to a .pb file? (#125) Hi, @xiaoyongzhu<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fxiaoyongzhu&data=02%7C01%7Cxiaoyzhu%40microsoft.com%7Cfecab75105ff44b22c6108d60cc50d8c%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636710437804264696&sdata=S7tQhWG1mAyzW46X5Kd7GtKsh3CQ8DW3j2PIYFc270Y%3D&reserved=0> I am facing the same issue. I have Retinanet implementation using tensorflow eager, I'm able to do the training after it is saving the .ckpt files but I'm not finding the way how to use the model for inference? need sugetions on the same thank you. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftensorflow%2Ftpu%2Fissues%2F125%23issuecomment-416507345&data=02%7C01%7Cxiaoyzhu%40microsoft.com%7Cfecab75105ff44b22c6108d60cc50d8c%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636710437804274713&sdata=tb%2BtWW7BU%2B42XJOSP0y5OY5htfLoDMr39IYCuHqOuSw%3D&reserved=0>, or mute the thread<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FABIVQpbqYJMXRNK6jJNBUcC52RSfZY6zks5uVQdDgaJpZM4T-WXQ&data=02%7C01%7Cxiaoyzhu%40microsoft.com%7Cfecab75105ff44b22c6108d60cc50d8c%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636710437804284717&sdata=UBxxW5XlVN84bhgN2SRamnsuI3u%2FPrnx54gU%2FL1pE9I%3D&reserved=0>.

[mgoundge11]

Hi @xiaoyongzhu thank you for replay. can you pls send me the link for TensorFlow Object Detection API with Retinanet added ? it will be helpful. thanks.

[aman2930]

Closing the issue as it seems this has been resolved.
Please re-open this ticket if need some more help. Thanks!