Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DCE script for caffe2 nets #2419

Merged
merged 1 commit into from Feb 21, 2019

Conversation

Projects
None yet
3 participants
@jackm321
Copy link
Contributor

jackm321 commented Feb 21, 2019

Description:
Basic script that takes caffe2 predict_net and init_net protobuf files and removes all dead nodes (nodes that aren't used to calculate the external_output of the predict_net then outputs this back to protobuf files.

The motivation behind this script is that sometimes we get models that have a bunch of extra nodes in them that we don't need for computing the prediction output and some of them we don't support in Glow (for example the Python op) so this can clean out these nodes so that we can then import them using the model loader.

Testing:
Added a test init_net and predict_net and manually checked the output of running the script on those files.
Ran the script on a large graph with lots of dead nodes and eyeballed the output in a graph visualizing tool.

Documentation:
Comments in code

@beicy

This comment has been minimized.

Copy link
Contributor

beicy commented Feb 21, 2019

Thanks a lot! Btw, for the inin_net.pbtxt file, is that OK to add one more test : two tensors with the same name and type? If this happens, just report an error/warning. In our model loader, we assumes that the tensors have a unique name. However, I've met an case before that there were duplicated tensors (same name, but different values) which caused an issue in our side. (I think this should be viewed as some kind of dead code? )

@jackm321

This comment has been minimized.

Copy link
Contributor Author

jackm321 commented Feb 21, 2019

@beicy that's an interesting case. In this case, I think our loader should be throwing an error because how can we know which is the correct value. The same would apply to this script, it's not clear how we would resolve which is the correct value and which should be eliminated. Also just to clarify, this script is not meant to do general error checking on c2 graphs, it's just to remove stuff that's not needed to compute the outputs of the predict_net

@beicy

beicy approved these changes Feb 21, 2019

@jackm321 jackm321 merged commit ac7ee90 into pytorch:master Feb 21, 2019

6 checks passed

ci/circleci: ASAN Your tests passed on CircleCI!
Details
ci/circleci: DEBUG Your tests passed on CircleCI!
Details
ci/circleci: RELEASE_WITH_EXPENSIVE_TESTS Your tests passed on CircleCI!
Details
ci/circleci: SHARED Your tests passed on CircleCI!
Details
ci/circleci: TSAN Your tests passed on CircleCI!
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.