-
Notifications
You must be signed in to change notification settings - Fork 782
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Failed to run optimizer ArithmeticOptimizer, stage RemoveStackStridedSliceSameAxis. #7
Comments
How are you running the tests? What environment? What commands? They all pass in my dev environment. I ran the tests as follows: Clone and enter repo git clone https://github.com/deepmind/graph_nets.git
cd gatph_nets/ I use docker images with all the dependencies included so I don't have to worry about system incompatibilities or version conflicts. If you have docker, you can try the Graph Nets images I'm currently hosting to see if it's an issue with your local dev environment. # CPU Image
docker run --rm -u $(id -u):$(id -g) -p 8888:8888 -v $(pwd):/my-devel -it imburbank/graph_nets bash -l
# GPU image
docker run --rm --runtime=nvidia --user $(id -u):$(id -g) -p 8888:8888 -v $(pwd):/my-devel -it imburbank/graph_nets:latest-gpu bash -l Then I ran each test python graph_nets/tests/blocks_test.py
python graph_nets/tests/modules_test.py
python graph_nets/tests/utils_tf_test.py
...ect |
Hi @IMBurbank thank you for commenting. I was just Realized I had Running with my own compiled binaries (no docker, no conda env, just Ubuntu 16.04) gave me something similar to your docker image.
It looks like it couldn't find BLAS with your docker image and in my local environment I'm having trouble getting data from the GPU to the CPU because of a misbehaving stream. |
As long as Docker is working, your local installations of python, conda, bazel, tensorflow, etc won't matter. Everything needed to run the tests is already in the container environments. Let's start with CPU (I'm not sure if you have GPU configured).
git clone https://github.com/deepmind/graph_nets.git
cd graph_nets/
docker run --rm -u $(id -u):$(id -g) -p 8888:8888 -v $(pwd):/my-devel -it imburbank/graph_nets bash -l
python graph_nets/tests/blocks_test.py
python graph_nets/tests/modules_test.py
python graph_nets/tests/utils_tf_test.py This will not use your locally-compiled tensorflow. The tests should pass. From there, you may be able to work on isolating the problem in your local dev environment. I would recommend trying the tests on your local dev system with a standard tensorflow package and seeing if they pass. If they do, move to the next link in the chain with your compiled tensorflow. |
I'll try out your CPU version, but I have the GPU configured. My locally installed tensorflow-r1.10 build works on the GPU. All tests passing. Lots of code run with it. If it's causing the problem, I'm only seeing it when trying to run the tests in Looks like your CPU binaries work. Doesn't really do me a bit of good, but they work. Kudos. The thing you recommend, trying the tests on local dev system with standard tf (no GPU) installed with pip is what produced the |
Okay I figured it out it's related to this issue. Merci! |
To run the GPU version, follow the exact same steps again, but swap in the GPU image in step 3. docker run --rm --runtime=nvidia --user $(id -u):$(id -g) -p 8888:8888 -v $(pwd):/my-devel -it imburbank/graph_nets:latest-gpu bash -l That should duplicate a standard environment running tensorflow_gpu, tensorflow_probability_gpu, graph_nets and the standard dependencies. I see you got it worked out. Cheers! |
I'm still using |
I am facing the similar error while running my object detection python file.I have completed all the above steps given by @IMBurBank.But still the error is same. What is the top-level directory of the model you are using: Please help me @IMBurbank |
I am facing the similar error while running my object detection python file. Could you please tell me how to solve it ? Thanks a lot! @abh2424 @tomas-wood |
Getting this error when I run
blocks_test.py
,modules_test.py
, andutils_tf_test.py
.2018-10-22 14:07:06.293160: W ./tensorflow/core/grappler/optimizers/graph_optimizer_stage.h:241] Failed to run optimizer ArithmeticOptimizer, stage RemoveStackStridedSliceSameAxis. Error: Pack node (data_dicts_to_graphs_tuple/stack) axis attribute is out of bounds: 0
Was using tensorflow version
1.13.0-dev20181022
.The text was updated successfully, but these errors were encountered: