Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Extended PyTorchFileLoader class. #3391

Closed
wants to merge 1 commit into from

Conversation

@putivsky
Copy link
Contributor

commented Aug 6, 2019

Summary: Added Fusion JIT Graph loading into Glow Function.

Differential Revision: D16678267

@putivsky putivsky requested review from bwasti and jackm321 Aug 6, 2019

@putivsky putivsky force-pushed the putivsky:export-D16678267 branch from 4218860 to 71623b2 Aug 6, 2019

putivsky added a commit to putivsky/glow that referenced this pull request Aug 6, 2019

Extended PyTorchFileLoader class. (pytorch#3391)
Summary:
Pull Request resolved: pytorch#3391

Added Fusion JIT Graph loading into Glow Function.

Differential Revision: D16678267

fbshipit-source-id: 0e7fb95fd72f9b1ae669d42a734c093e5b1a538b

@putivsky putivsky force-pushed the putivsky:export-D16678267 branch from 71623b2 to f819a87 Aug 7, 2019

putivsky added a commit to putivsky/glow that referenced this pull request Aug 7, 2019

Extended PyTorchFileLoader class. (pytorch#3391)
Summary:
Pull Request resolved: pytorch#3391

Added Fusion JIT Graph loading into Glow Function.

Differential Revision: D16678267

fbshipit-source-id: 8b3fe2af94ead7c4a1033051378de0e073b63af2

@putivsky putivsky force-pushed the putivsky:export-D16678267 branch from f819a87 to 55f8ab7 Aug 7, 2019

putivsky added a commit to putivsky/glow that referenced this pull request Aug 7, 2019

Extended PyTorchFileLoader class. (pytorch#3391)
Summary:
Pull Request resolved: pytorch#3391

Added Fusion JIT Graph loading into Glow Function.

Differential Revision: D16678267

fbshipit-source-id: 981ba2b448a52f1974cae0643fb62f415ce20331

@putivsky putivsky force-pushed the putivsky:export-D16678267 branch from 55f8ab7 to f6ebc21 Aug 7, 2019

putivsky added a commit to putivsky/glow that referenced this pull request Aug 7, 2019

Extended PyTorchFileLoader class. (pytorch#3391)
Summary:
Pull Request resolved: pytorch#3391

Added Fusion JIT Graph loading into Glow Function.

Differential Revision: D16678267

fbshipit-source-id: 968f236c006b6b7885ef2a0a644da2b0e371b741

@putivsky putivsky force-pushed the putivsky:export-D16678267 branch from f6ebc21 to 5c8af19 Aug 7, 2019

putivsky added a commit to putivsky/glow that referenced this pull request Aug 7, 2019

Extended PyTorchFileLoader class. (pytorch#3391)
Summary:
Pull Request resolved: pytorch#3391

Added Fusion JIT Graph loading into Glow Function.

Differential Revision: D16678267

fbshipit-source-id: 40981af6487da301598754f552418a592fdc79e3

@putivsky putivsky force-pushed the putivsky:export-D16678267 branch from 5c8af19 to fbe7017 Aug 7, 2019

putivsky added a commit to putivsky/glow that referenced this pull request Aug 7, 2019

Extended PyTorchFileLoader class. (pytorch#3391)
Summary:
Pull Request resolved: pytorch#3391

Added Fusion JIT Graph loading into Glow Function.

Differential Revision: D16678267

fbshipit-source-id: ba697aee297dce6b1830afa9d58894876642e6bb

@putivsky putivsky force-pushed the putivsky:export-D16678267 branch from fbe7017 to b5abd0f Aug 7, 2019

putivsky added a commit to putivsky/glow that referenced this pull request Aug 7, 2019

Extended PyTorchFileLoader class. (pytorch#3391)
Summary:
Pull Request resolved: pytorch#3391

Added Fusion JIT Graph loading into Glow Function.

Differential Revision: D16678267

fbshipit-source-id: ed30e32e96b9fc997cef5d1f55793ff4beac4f32

@putivsky putivsky force-pushed the putivsky:export-D16678267 branch from b5abd0f to fea7f4f Aug 7, 2019

putivsky added a commit to putivsky/glow that referenced this pull request Aug 7, 2019

Extended PyTorchFileLoader class. (pytorch#3391)
Summary:
Pull Request resolved: pytorch#3391

Added Fusion JIT Graph loading into Glow Function.

Differential Revision: D16678267

fbshipit-source-id: 18db38b2783eaf4a8891bc3b288e561ac47f8653

@putivsky putivsky force-pushed the putivsky:export-D16678267 branch from fea7f4f to 82f8985 Aug 7, 2019

putivsky added a commit to putivsky/glow that referenced this pull request Aug 7, 2019

Extended PyTorchFileLoader class. (pytorch#3391)
Summary:
Pull Request resolved: pytorch#3391

Added Fusion JIT Graph loading into Glow Function.

Differential Revision: D16678267

fbshipit-source-id: 0e413f6a4167fe2a8b93f78886947ac5ede6f8d5

@putivsky putivsky force-pushed the putivsky:export-D16678267 branch from 82f8985 to 0185123 Aug 7, 2019

putivsky added a commit to putivsky/glow that referenced this pull request Aug 7, 2019

Extended PyTorchFileLoader class. (pytorch#3391)
Summary:
Pull Request resolved: pytorch#3391

Added Fusion JIT Graph loading into Glow Function.

Differential Revision: D16678267

fbshipit-source-id: 7ff8cab2052e22ecaaf6434e9e5f28d17ffc378d
torch_glow/src/PyTorchFileLoader.cpp Outdated Show resolved Hide resolved
torch_glow/src/PyTorchFileLoader.cpp Outdated Show resolved Hide resolved
torch_glow/src/PyTorchFileLoader.cpp Outdated Show resolved Hide resolved
torch_glow/src/PyTorchFileLoader.cpp Outdated Show resolved Hide resolved

@putivsky putivsky force-pushed the putivsky:export-D16678267 branch from 0185123 to b0adfa3 Aug 8, 2019

putivsky added a commit to putivsky/glow that referenced this pull request Aug 8, 2019

Extended PyTorchFileLoader class. (pytorch#3391)
Summary:
Pull Request resolved: pytorch#3391

Added Fusion JIT Graph loading into Glow Function.

Differential Revision: D16678267

fbshipit-source-id: c7ab77d869a4013e50164f4b14ae2794fca338c6
torch_glow/src/PyTorchLoaderTest.cpp Outdated Show resolved Hide resolved
torch_glow/src/PyTorchLoaderTest.cpp Outdated Show resolved Hide resolved
torch_glow/src/FusingOptimizer.cpp Outdated Show resolved Hide resolved
torch_glow/src/PyTorchFileLoader.cpp Show resolved Hide resolved
torch_glow/src/PyTorchFileLoader.cpp Outdated Show resolved Hide resolved
@jackm321
Copy link
Contributor

left a comment

@putivsky looks mostly good to me, just a few questions and comments

torch_glow/src/PyTorchFileLoader.h Outdated Show resolved Hide resolved

@putivsky putivsky force-pushed the putivsky:export-D16678267 branch from b0adfa3 to f6d0702 Aug 9, 2019

putivsky added a commit to putivsky/glow that referenced this pull request Aug 9, 2019

Extended PyTorchFileLoader class. (pytorch#3391)
Summary:
Pull Request resolved: pytorch#3391

Added Fusion JIT Graph loading into Glow Function.

Differential Revision: D16678267

fbshipit-source-id: 134a1d211b8dc9fdbf03c56a968ab048a8645ce2
torch_glow/src/PyTorchLoaderTest.cpp Outdated Show resolved Hide resolved
torch_glow/src/PyTorchLoaderTest.cpp Outdated Show resolved Hide resolved
torch_glow/src/PyTorchModelLoader.h Outdated Show resolved Hide resolved
torch_glow/src/PyTorchFileLoader.h Show resolved Hide resolved
torch_glow/src/PyTorchFileLoader.cpp Show resolved Hide resolved
torch_glow/src/PyTorchFileLoader.cpp Show resolved Hide resolved
// Disable gradient nodes generation.
at::NoGradGuard guard;

// Set thread local pointer, not null values activate custom fusion pass.

This comment has been minimized.

Copy link
@jackm321

jackm321 Aug 9, 2019

Contributor

doxygen

This comment has been minimized.

Copy link
@putivsky

putivsky Aug 10, 2019

Author Contributor

can you explain, what should be changed to comply with doxygen style?.

torch_glow/src/PyTorchCommon.h Show resolved Hide resolved
torch_glow/src/PyTorchFileLoader.cpp Show resolved Hide resolved
torch_glow/src/PyTorchFileLoader.cpp Show resolved Hide resolved

@putivsky putivsky force-pushed the putivsky:export-D16678267 branch from f6d0702 to 4278ab7 Aug 10, 2019

putivsky added a commit to putivsky/glow that referenced this pull request Aug 10, 2019

Extended PyTorchFileLoader class. (pytorch#3391)
Summary:
Pull Request resolved: pytorch#3391

Added Fusion JIT Graph loading into Glow Function.

Differential Revision: D16678267

fbshipit-source-id: fa5ad20eb0015813daa2f1db33d4afef5a9ef51e
@jackm321
Copy link
Contributor

left a comment

Accepting to keep things moving, thanks for making changes. Some things to revisit later are:

  • Do we really want to keep all original nodes in the graph? (maybe the answer to this is yes)
  • Run C++ pytorch-glow tests in CI and using gtest
torch_glow/src/FusingOptimizer.cpp Outdated Show resolved Hide resolved
torch_glow/src/PyTorchCommon.h Show resolved Hide resolved
torch_glow/src/PyTorchFileLoader.cpp Show resolved Hide resolved
@@ -124,6 +125,7 @@ def _run_cmake(self):
# PyTorch cmake args
'-DPYTORCH_DIR={}'.format(
os.path.dirname(os.path.realpath(torch.__file__))),
'-DTORCH_GLOW={}'.format(FILE_DIR),

This comment has been minimized.

Copy link
@jackm321

jackm321 Aug 10, 2019

Contributor

What is this?

This comment has been minimized.

Copy link
@putivsky

putivsky Aug 11, 2019

Author Contributor

Normally we should use include paths relative to "glow" folder, otherwise we would need (as in this case) path to the include file from different folder. Because I have moved test file into a separate folder, test makefile needs include path to other torch_glow files.

@putivsky putivsky force-pushed the putivsky:export-D16678267 branch from 4278ab7 to 01990ff Aug 11, 2019

putivsky added a commit to putivsky/glow that referenced this pull request Aug 11, 2019

Extended PyTorchFileLoader class. (pytorch#3391)
Summary:
Pull Request resolved: pytorch#3391

Added Fusion JIT Graph loading into Glow Function.

Differential Revision: D16678267

fbshipit-source-id: b942287d96fabd7872224c19f320eab84f6f845a

@putivsky putivsky force-pushed the putivsky:export-D16678267 branch from 01990ff to c66a6ed Aug 11, 2019

putivsky added a commit to putivsky/glow that referenced this pull request Aug 11, 2019

Extended PyTorchFileLoader class. (pytorch#3391)
Summary:
Pull Request resolved: pytorch#3391

Added Fusion JIT Graph loading into Glow Function.

Differential Revision: D16678267

fbshipit-source-id: 3523215d3f72081f32eb969ccc3caca92fbc5f4a
@facebook-github-bot
Copy link

left a comment

@putivsky is landing this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

@putivsky putivsky force-pushed the putivsky:export-D16678267 branch from c66a6ed to 529ce3e Aug 11, 2019

putivsky added a commit to putivsky/glow that referenced this pull request Aug 11, 2019

Extended PyTorchFileLoader class. (pytorch#3391)
Summary:
Added Fusion JIT Graph loading into Glow Function.
Pull Request resolved: pytorch#3391

Test Plan: run unit tests

Differential Revision: D16678267

fbshipit-source-id: c3eaf24ce46ab5befbf51d0f6b23b69f24890a7e

@putivsky putivsky force-pushed the putivsky:export-D16678267 branch from 529ce3e to 1bee76d Aug 11, 2019

putivsky added a commit to putivsky/glow that referenced this pull request Aug 11, 2019

Extended PyTorchFileLoader class. (pytorch#3391)
Summary:
Added Fusion JIT Graph loading into Glow Function.
Pull Request resolved: pytorch#3391

Test Plan: run unit tests

Differential Revision: D16678267

Pulled By: putivsky

fbshipit-source-id: 8281a797e85a56e0b0b7a16b291971cd3be30f43
Extended PyTorchFileLoader class. (#3391)
Summary:
Added Fusion JIT Graph loading into Glow Function.
Pull Request resolved: #3391

Test Plan: run unit tests

Differential Revision: D16678267

Pulled By: putivsky

fbshipit-source-id: 4ebd791e9a53e34109a479b1e1d33981a8d4b81c

@putivsky putivsky force-pushed the putivsky:export-D16678267 branch from 1bee76d to 196c911 Aug 11, 2019

@putivsky putivsky reopened this Aug 11, 2019

@putivsky putivsky closed this Aug 11, 2019

@putivsky putivsky deleted the putivsky:export-D16678267 branch Aug 11, 2019

@facebook-github-bot

This comment has been minimized.

Copy link

commented Aug 12, 2019

This pull request has been merged in 38d00aa.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants
You can’t perform that action at this time.