This repository was archived by the owner on Jul 1, 2025. It is now read-only.
Enable network sharing in torch_glow #4235
Closed
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
This pull request was exported from Phabricator. Differential Revision: D20174479 |
yinghai
force-pushed
the
export-D20174479
branch
from
5543d29 to
f3a4fc9
Compare
|
This pull request was exported from Phabricator. Differential Revision: D20174479 |
yinghai
force-pushed
the
export-D20174479
branch
from
f3a4fc9 to
2b8a955
Compare
|
This pull request was exported from Phabricator. Differential Revision: D20174479 |
yinghai
force-pushed
the
export-D20174479
branch
from
2b8a955 to
2c59940
Compare
|
This pull request was exported from Phabricator. Differential Revision: D20174479 |
facebook-github-bot
pushed a commit
to pytorch/pytorch
that referenced
this pull request
Mar 1, 2020
Summary: HashNode and CompareNode are useful functions for hanlding jit::Node. This is to unblock pytorch/glow#4235. Pull Request resolved: #34045 Reviewed By: houseroad Differential Revision: D20184733 Pulled By: yinghai fbshipit-source-id: 6c829f2f111a490fd2d85017475c1731cd97fb20
yinghai
force-pushed
the
export-D20174479
branch
from
2c59940 to
e52b271
Compare
|
This pull request was exported from Phabricator. Differential Revision: D20174479 |
yinghai
force-pushed
the
export-D20174479
branch
from
e52b271 to
05afd0a
Compare
|
This pull request was exported from Phabricator. Differential Revision: D20174479 |
yinghai
force-pushed
the
export-D20174479
branch
from
05afd0a to
c0a50ca
Compare
|
This pull request was exported from Phabricator. Differential Revision: D20174479 |
yinghai
force-pushed
the
export-D20174479
branch
from
c0a50ca to
48ea2d2
Compare
|
This pull request was exported from Phabricator. Differential Revision: D20174479 |
yinghai
force-pushed
the
export-D20174479
branch
from
48ea2d2 to
7a3454c
Compare
|
This pull request was exported from Phabricator. Differential Revision: D20174479 |
Summary: Pull Request resolved: pytorch#4235 We will compile and cache the glow fused graph only once, shared across all the module/function from different inference threads. We have a global singleton of CachingGraphRunner keyed on the hash of jit::Graph. For each unique jit::Graph, we have one CachingGraphRunner. Then for each shape specifalization of CachingGraphRunner, we have a unique Glow function. Strategy on caching CachingGraphRunner: We will first query by hashing the fused subgraph. If not found, we will query by hashing the node symbol string, which suits the AOT case. Currently, we have to leak the CachingGraphRunner. This is fine for the forseeable future but we probably need more application level (say, predictor) signal to help with this. For example, if the application framework can save an instance of the CachingGraphRunner as a shared_ptr in AOT case and free it when the model is unloaded, we can then use weak_ptr for the graph runner map and hence will make sure everything will release except for an null weak_ptr. Reviewed By: jackm321 Differential Revision: D20174479 fbshipit-source-id: aa0a564700ded2a02c988b2cad535d5a10c6cfb8
yinghai
force-pushed
the
export-D20174479
branch
from
7a3454c to
dc5e36d
Compare
|
This pull request was exported from Phabricator. Differential Revision: D20174479 |
|
This pull request has been merged in dadf6f1. |
ttumiel
pushed a commit
to ttumiel/pytorch
that referenced
this pull request
Mar 4, 2020
Summary: HashNode and CompareNode are useful functions for hanlding jit::Node. This is to unblock pytorch/glow#4235. Pull Request resolved: pytorch#34045 Reviewed By: houseroad Differential Revision: D20184733 Pulled By: yinghai fbshipit-source-id: 6c829f2f111a490fd2d85017475c1731cd97fb20
vdantu
pushed a commit
to vdantu/glow
that referenced
this pull request
Jul 12, 2020
Summary: Pull Request resolved: pytorch#4235 We will compile and cache the glow fused graph only once, shared across all the module/function from different inference threads. We have a global singleton of CachingGraphRunner keyed on the hash of jit::Graph. For each unique jit::Graph, we have one CachingGraphRunner. Then for each shape specifalization of CachingGraphRunner, we have a unique Glow function. Strategy on caching CachingGraphRunner: We will first query by hashing the fused subgraph. If not found, we will query by hashing the node symbol string, which suits the AOT case. Currently, we have to leak the CachingGraphRunner. This is fine for the forseeable future but we probably need more application level (say, predictor) signal to help with this. For example, if the application framework can save an instance of the CachingGraphRunner as a shared_ptr in AOT case and free it when the model is unloaded, we can then use weak_ptr for the graph runner map and hence will make sure everything will release except for an null weak_ptr. Reviewed By: jackm321 Differential Revision: D20174479 fbshipit-source-id: 0c18880fe80c3ad72c9cad3010479fd7a8c2b9d6
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary:
We will compile and cache the glow fused graph only once, shared across all the module/function from different inference threads. We have a global singleton of CachingGraphRunner keyed on the hash of jit::Graph. For each unique jit::Graph, we have one CachingGraphRunner. Then for each shape specifalization of CachingGraphRunner, we have a unique Glow function.
Strategy on caching CachingGraphRunner:
We will first query by hashing the fused subgraph. If not found, we will query by hashing the node symbol string, which suits the AOT case. Currently, we have to leak the CachingGraphRunner as I don't have a good idea of where the glow op is going to be destroyed and what hook I can add. This is fine for the forseeable future but we probably need more application level (say, predictor) signal to help with this.
Differential Revision: D20174479