[EXECUTOR] Experimental Mirror option to squeez mem #884

Merged
merged 4 commits into from Dec 19, 2015

Conversation

Projects
None yet
3 participants
@tqchen
Member

tqchen commented Dec 9, 2015

as discussed with @antinucleon and @mavenlin . Can you two help review this PR? I want to make sure code is correct before merge in

src/symbol/static_graph.cc
@@ -62,7 +66,7 @@ std::vector<uint32_t> StaticGraph::TopoSort() const {
head_nodes.push_back(static_cast<uint32_t>(i));
}
}
- return PostDFSOrder(head_nodes);
+ return PostDFSOrder(head_nodes, {});

This comment has been minimized.

@antinucleon

antinucleon Dec 11, 2015

Contributor

change {} to {{}} to make it compile in gcc 4.9. Need to change several places for same reason.

@antinucleon

antinucleon Dec 11, 2015

Contributor

change {} to {{}} to make it compile in gcc 4.9. Need to change several places for same reason.

@antinucleon

This comment has been minimized.

Show comment
Hide comment
@antinucleon

antinucleon Dec 12, 2015

Contributor

CIFAR-10 No BN:

MXNET_BACKWARD_DO_MIRROR Time Memory Val Result
0 35.4 sec 921 MB 35.8%
1 36.5 sec 765 MB 33.9%
Contributor

antinucleon commented Dec 12, 2015

CIFAR-10 No BN:

MXNET_BACKWARD_DO_MIRROR Time Memory Val Result
0 35.4 sec 921 MB 35.8%
1 36.5 sec 765 MB 33.9%
@tqchen

This comment has been minimized.

Show comment
Hide comment
@tqchen

tqchen Dec 18, 2015

Member

@antinucleon Please do a review again

Member

tqchen commented Dec 18, 2015

@antinucleon Please do a review again

@antinucleon

This comment has been minimized.

Show comment
Hide comment
@antinucleon

antinucleon Dec 19, 2015

Contributor

CIFAR-10, without CuDNN, with BN

MXNET_BACKWARD_DO_MIRROR Time Memory Train Result Val Result
0 84.7 sec 1255 MB 55.6% 60.6%
1 92.4 sec 960 MB 55.6% 60.6

CIFAR-10, with CuDNN, with BN

MXNET_BACKWARD_DO_MIRROR Time Memory Train Result Val Result
0 46.2 sec 1286 MB 55.7% 66.7%
1 53.5 sec 832 MB 55.0% 66.6%
Contributor

antinucleon commented Dec 19, 2015

CIFAR-10, without CuDNN, with BN

MXNET_BACKWARD_DO_MIRROR Time Memory Train Result Val Result
0 84.7 sec 1255 MB 55.6% 60.6%
1 92.4 sec 960 MB 55.6% 60.6

CIFAR-10, with CuDNN, with BN

MXNET_BACKWARD_DO_MIRROR Time Memory Train Result Val Result
0 46.2 sec 1286 MB 55.7% 66.7%
1 53.5 sec 832 MB 55.0% 66.6%
@antinucleon

This comment has been minimized.

Show comment
Hide comment
@antinucleon

antinucleon Dec 19, 2015

Contributor

LGTM, merge first, I will try to fix BN fix_gamma today.

Contributor

antinucleon commented Dec 19, 2015

LGTM, merge first, I will try to fix BN fix_gamma today.

antinucleon added a commit that referenced this pull request Dec 19, 2015

Merge pull request #884 from tqchen/master
[EXECUTOR] Experimental Mirror option to squeez mem

@antinucleon antinucleon merged commit 087c6e6 into apache:master Dec 19, 2015

1 check passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details
@linrio

This comment has been minimized.

Show comment
Hide comment
@linrio

linrio Apr 15, 2017

mark! I'll try to use maxnet

linrio commented Apr 15, 2017

mark! I'll try to use maxnet

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment