Adding scripts for TensorRT #3808
Conversation
research/tensorrt/README.md
Outdated
| already, and add the Official Models to your Python path: | ||
|
|
||
| ``` | ||
| cd /path/to |
There was a problem hiding this comment.
nit: remove cd /path/to
Might just be me, but I a bit confused by the missing noun at the end. The /path/to/models in the third line is clear on its own.
|
|
||
| ### Step 3: Get an image to test | ||
|
|
||
| The script can accept a JPEG image file to use for predictions. If none is |
There was a problem hiding this comment.
Are there any requirements for the image, other than the JPEG format?
There was a problem hiding this comment.
I don't think so, assuming that we all assume that the picture should have something identifiable and worth labeling in it.
research/tensorrt/README.md
Outdated
|
|
||
| 1. [Install TensorFlow.](https://www.tensorflow.org/install/) | ||
| 2. [Install TensorRT.](http://docs.nvidia.com/deeplearning/sdk/tensorrt-install-guide/index.html) | ||
| 3. We use the [Official version of ResNet image preprocessing](/official/resnet/imagenet_preprocessing.py). Please checkout the Models repository if you haven't |
There was a problem hiding this comment.
nit: Link text is quite long, I would recommend changing "We use the Official version of ResNet image preprocessing" to "We use image preprocessing functions from the Official version of Resnet"
| ### Step 4: Run the model | ||
|
|
||
| You have TensorFlow, TensorRT, a graph def, and a picture. | ||
| Now it's time to time. |
research/tensorrt/tensorrt.py
Outdated
| input_map={input_node: iterator.get_next()}, | ||
| return_elements=output_nodes | ||
| ) | ||
| output = outputs[0].outputs[0] |
There was a problem hiding this comment.
That's a lot of 'output' in one line... what does this do?
There was a problem hiding this comment.
Gooood question. As it turns out, it removes the return values of all the passed "output_nodes" except for the first. That seemed unclear, so I fixed the script to accept only one output node, and added comments here, rather than silently dropping the others.
research/tensorrt/tensorrt.py
Outdated
|
|
||
| graph = tf.Graph() | ||
| with tf.Session(graph=graph) as sess: | ||
|
|
|
Thanks-- first set of comments much appreciated and addressed. |
|
I'm going to merge this now so that we have it ready for tomorrow, but don't hesitate to add more comments; I can fix in separate PRs. |
| # Run the graph in various precision modes. | ||
| ################################################################################ | ||
| def get_gpu_config(): | ||
| """Share GPU memory between image preprocessing and inference.""" |
There was a problem hiding this comment.
I might be wrong, but the image preprocessing and inference sessions don't run at the same time. Is there a reason per_process_gpu_memory_fraction is set to .5?
There was a problem hiding this comment.
I think that it is because we reserve the memory for each session, and that must not get cleared. I am not sure, but I tried without that part, and got CUDA malloc errors.
| used for statistic calculation | ||
| batch_size: int, number of examples per batch | ||
|
|
||
| Returns: |
There was a problem hiding this comment.
Does log_stats need to return anything?
|
Thanks for the review on short notice-- frozen graphs, as I understand the story, are not generally encouraged other than a few special use-cases, so probably best not to include by default for now; I think TFLite is planning of having a replacement format at some point in the future as well. |
* Adding scripts for TensorRT * Fixing README link * Responding to CR
This script is a work in progress, but we want to have some examples available for the Summit of usin g TensorRT with the official models.
@tfboyd - Take a look and make sure it meets your needs.
@k-w-w - Can you read over for readability and general clarity? Deadline is tight, but we can always do follow-up changes in another PR.