TF_ImportGraphDef crashes for the following graph

[ajayaraman]

What related GitHub issues or StackOverflow threads have you found by searching the web for your problem?

None

Environment info

Operating System: Ubuntu 16.04

Installed version of CUDA and cuDNN:
(please attach the output of ls -l /path/to/cuda/lib/libcud*):

CUDA

-rw-r--r-- 1 root root 558720 Nov 30 13:53 libcudadevrt.a
lrwxrwxrwx 1 root root 16 Nov 30 13:53 libcudart.so -> libcudart.so.8.0
lrwxrwxrwx 1 root root 19 Nov 30 13:53 libcudart.so.8.0 -> libcudart.so.8.0.44
-rwxr-xr-x 1 root root 415432 Nov 30 13:53 libcudart.so.8.0.44
-rw-r--r-- 1 root root 775162 Nov 30 13:53 libcudart_static.a

CUDNN

lrwxrwxrwx 1 ajayaram ajayaram 13 Nov 30 14:02 libcudnn.so -> libcudnn.so.5
lrwxrwxrwx 1 ajayaram ajayaram 17 Nov 30 14:02 libcudnn.so.5 -> libcudnn.so.5.1.5
-rwxrwxr-x 1 ajayaram ajayaram 79337624 Jul 27 2016 libcudnn.so.5.1.5
-rw-rw-r-- 1 ajayaram ajayaram 69756172 Jul 27 2016 libcudnn_static.a

If installed from binary pip package, provide:

1. A link to the pip package you installed:
2. The output from python -c "import tensorflow; print(tensorflow.__version__)".

If installed from source, provide

1. The commit hash (git rev-parse HEAD)
   29a6b46

2. The output of bazel version
   .............................
   Build label: 0.4.4
   Build target: bazel-out/local-fastbuild/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar
   Build time: Wed Feb 1 18:54:21 2017 (1485975261)
   Build timestamp: 1485975261
   Build timestamp as int: 1485975261

If possible, provide a minimal reproducible example (We usually don't have time to read hundreds of lines of your code)

lstm_issue.zip

#lstmkt.py : Write a keras model with LSTM layer to tensorflow graph

from keras.models import Sequential
from keras.layers import Input, Dense, Reshape
from keras.layers.recurrent import LSTM
from keras import backend as K

import tensorflow as tf

def build_model():
  model = Sequential()	
  model.add(LSTM(4, input_shape= (1,1) ))  
  model.add(Dense(1))
  model.compile(optimizer='adam', loss='mse')
  return model

model = build_model()
sess = K.get_session()
graph = sess.graph
sess.run('init')

tf.train.write_graph(sess.graph, './', 'lstmkt.pb', as_text=False)

void free_buffer(void* data, size_t length);
TF_Buffer* read_file(const char* file);

int main()
{

    const char filename[] = "lstmkt.pb";
    
    TF_Status* s = TF_NewStatus();
    TF_Graph* graph = TF_NewGraph();
    TF_Buffer* graph_def = read_file(filename);
    
    // Import graph
    TF_ImportGraphDefOptions* opts = TF_NewImportGraphDefOptions();
 
    // This line of code crashes if the keras model contains an LSTM layer
    TF_GraphImportGraphDef(graph, graph_def, opts, s);

    return 0;
}

What other attempted solutions have you tried?

None. Stuck here. Works in python but I need to load this graph in C.

Logs or other output that would be helpful

(If logs are large, please upload as attachment or provide link).

Stack Trace (from fault):
[  0] 0x00007f91b7a87338 /home/ajayaram/tensorflow/bazel-bin/tensorflow/libtensorflow.so+63492920 _ZN10tensorflow15shape_inference16InferenceContext8WithRankENS0_11ShapeHandleEiPS2_+00000024
[  1] 0x00007f91b66b8c88 /home/ajayaram/tensorflow/bazel-bin/tensorflow/libtensorflow.so+42724488
[  2] 0x00007f91b7a65a44 /home/ajayaram/tensorflow/bazel-bin/tensorflow/libtensorflow.so+63355460 _ZNSt17_Function_handlerIFN10tensorflow6StatusEPNS0_15shape_inference16InferenceContextEEPS5_E9_M_invokeERKSt9_Any_dataOS4_+00000020
[  3] 0x00007f91b79e2094 /home/ajayaram/tensorflow/bazel-bin/tensorflow/libtensorflow.so+62816404 _ZN10tensorflow12ShapeRefiner7AddNodeEPKNS_4NodeE+00002996
[  4] 0x00007f91b5bd42e5 /home/ajayaram/tensorflow/bazel-bin/tensorflow/libtensorflow.so+31302373
[  5] 0x00007f91b7a0dc2c /home/ajayaram/tensorflow/bazel-bin/tensorflow/libtensorflow.so+62995500
[  6] 0x00007f91b7a0e6d3 /home/ajayaram/tensorflow/bazel-bin/tensorflow/libtensorflow.so+62998227 _ZN10tensorflow14ImportGraphDefERKNS_21ImportGraphDefOptionsERKNS_8GraphDefEPNS_5GraphEPNS_12ShapeRefinerE+00000499
[  7] 0x00007f91b5cb2434 /home/ajayaram/tensorflow/bazel-bin/tensorflow/libtensorflow.so+32212020
[  8] 0x00007f91b5cb52fc /home/ajayaram/tensorflow/bazel-bin/tensorflow/libtensorflow.so+32223996 TF_GraphImportGraphDef+00000204

[asimshankar]

@ajayaraman : Thanks for the very detailed report and the instructions to reproduce it. This is very helpful. There seems to be some issue triggered in the shape inference code, I'm looking into it.