optimize_for_inference.py should remove Dropout operations

[mattrajca]

When I first tried using an exported MNIST model with TensorFlow on iOS, I got the following error:

Invalid argument: No OpKernel was registered to support Op 'RandomUniform' with these attrs.  Registered devices: [CPU], Registered kernels:
  <no registered kernels>

     [[Node: dropout/random_uniform/RandomUniform = RandomUniform[T=DT_INT32, dtype=DT_FLOAT, seed=0, seed2=0](dropout/Shape)]]

Since Dropout operations are no-ops during inference (we pass in a keep probability of 1), it would be nice if they were removed (or turned into no-ops of some kind that can be parsed and ignored by the iOS library).

While I was able to work around this by explicitly exporting a separate graph that does not contain Dropout, it was pretty tedious and it would be nice if the optimize_for_inference.py script did this automatically.

Environment info

Operating System: macOS 10.12

Installed version of CUDA and cuDNN:
None

Source:
This week's tip-of-tree (around d93d526)

[nikashitsa]

Yes, it would be nice if the optimize_for_inference.py script did this automatically.
My current manual solution https://dato.ml/drop-dropout-from-frozen-model/

[cancan101]

related / dupe: #6124

[cancan101]

There is also this SO (attempt at a) solution: http://stackoverflow.com/questions/40358892/wipe-out-dropout-operations-from-tensorflow-graph
and my attempt at this:

for node in temp_graph_def.node:
    for idx, i in enumerate(node.input):
        input_clean = node_name_from_input(i)
        if input_clean.endswith('/cond/Merge') and input_clean.split('/')[-3].startswith('dropout'):
            identity = node_from_map(input_node_map, i).input[0]
            assert identity.split('/')[-1] == 'Identity'
            parent = node_from_map(input_node_map, node_from_map(input_node_map, identity).input[0])
            pred_id = parent.input[1]
            assert pred_id.split('/')[-1] == 'pred_id'            
            good = parent.input[0]
            node.input[idx] = good

[swarathesh]

i'am getting the following error in android
ERROR: Analysis of target '//tensorflow/contrib/util:convert_graphdef_memmapped_format' failed; build aborted.

[haydenth]

Is there anyway this can be accomplished using the transform_graph tool and remove_nodes?

[kmonachopoulos]

Is there a way to get rid of the dropout in a model? I am currently trying to remove it but I haven't found a way yet :

Operation Name : dropout_1/keep_prob
Tensor Stats : (<tf.Tensor 'dropout_1/keep_prob:0' shape=() dtype=float32>,)
Operation Name : dropout_1/Shape
Tensor Stats : (<tf.Tensor 'dropout_1/Shape:0' shape=(4,) dtype=int32>,)
Operation Name : dropout_1/random_uniform/min
Tensor Stats : (<tf.Tensor 'dropout_1/random_uniform/min:0' shape=() dtype=float32>,)
Operation Name : dropout_1/random_uniform/max
Tensor Stats : (<tf.Tensor 'dropout_1/random_uniform/max:0' shape=() dtype=float32>,)
Operation Name : dropout_1/random_uniform/RandomUniform
Tensor Stats : (<tf.Tensor 'dropout_1/random_uniform/RandomUniform:0' shape=(?, ?, ?, 4096) dtype=float32>,)
Operation Name : dropout_1/random_uniform/sub
Tensor Stats : (<tf.Tensor 'dropout_1/random_uniform/sub:0' shape=() dtype=float32>,)
Operation Name : dropout_1/random_uniform/mul
Tensor Stats : (<tf.Tensor 'dropout_1/random_uniform/mul:0' shape=(?, ?, ?, 4096) dtype=float32>,)
Operation Name : dropout_1/random_uniform
Tensor Stats : (<tf.Tensor 'dropout_1/random_uniform:0' shape=(?, ?, ?, 4096) dtype=float32>,)
Operation Name : dropout_1/add
Tensor Stats : (<tf.Tensor 'dropout_1/add:0' shape=(?, ?, ?, 4096) dtype=float32>,)
Operation Name : dropout_1/Floor
Tensor Stats : (<tf.Tensor 'dropout_1/Floor:0' shape=(?, ?, ?, 4096) dtype=float32>,)
Operation Name : dropout_1/Inv
Tensor Stats : (<tf.Tensor 'dropout_1/Inv:0' shape=() dtype=float32>,)
Operation Name : dropout_1/mul
Tensor Stats : (<tf.Tensor 'dropout_1/mul:0' shape=(?, ?, ?, 4096) dtype=float32>,)
Operation Name : dropout_1/mul_1
Tensor Stats : (<tf.Tensor 'dropout_1/mul_1:0' shape=(?, ?, ?, 4096) dtype=float32>,)

Any updates on this ?

[alextp]

Is using a tensor for keep_prob which is set to 1 during inference not enough?

[ngeorgis]

Any updates on this? Just tried optimize_for_inference.py but dropout still there confusing NVIDIA TensorRT and Jetson TX2 inference.

[kmonachopoulos]

There is a way to create a new graph and copy all the nodes you want (except dropout) from the old graph to the new one, fixing the input/output names of the nodes before and after the dropout layer. This way you can create a new graph without the dropout layer.