Skip to content
This repository has been archived by the owner on Nov 17, 2023. It is now read-only.

[EXECUTOR] Experimental Mirror option to squeez mem #884

Merged
merged 4 commits into from
Dec 19, 2015

Conversation

tqchen
Copy link
Member

@tqchen tqchen commented Dec 9, 2015

as discussed with @antinucleon and @mavenlin . Can you two help review this PR? I want to make sure code is correct before merge in

@@ -62,7 +66,7 @@ std::vector<uint32_t> StaticGraph::TopoSort() const {
head_nodes.push_back(static_cast<uint32_t>(i));
}
}
return PostDFSOrder(head_nodes);
return PostDFSOrder(head_nodes, {});
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

change {} to {{}} to make it compile in gcc 4.9. Need to change several places for same reason.

@antinucleon
Copy link
Contributor

CIFAR-10 No BN:

MXNET_BACKWARD_DO_MIRROR Time Memory Val Result
0 35.4 sec 921 MB 35.8%
1 36.5 sec 765 MB 33.9%

@tqchen
Copy link
Member Author

tqchen commented Dec 18, 2015

@antinucleon Please do a review again

@antinucleon
Copy link
Contributor

CIFAR-10, without CuDNN, with BN

MXNET_BACKWARD_DO_MIRROR Time Memory Train Result Val Result
0 84.7 sec 1255 MB 55.6% 60.6%
1 92.4 sec 960 MB 55.6% 60.6

CIFAR-10, with CuDNN, with BN

MXNET_BACKWARD_DO_MIRROR Time Memory Train Result Val Result
0 46.2 sec 1286 MB 55.7% 66.7%
1 53.5 sec 832 MB 55.0% 66.6%

@antinucleon
Copy link
Contributor

LGTM, merge first, I will try to fix BN fix_gamma today.

antinucleon added a commit that referenced this pull request Dec 19, 2015
[EXECUTOR] Experimental Mirror option to squeez mem
@antinucleon antinucleon merged commit 087c6e6 into apache:master Dec 19, 2015
@linrio
Copy link

linrio commented Apr 15, 2017

mark! I'll try to use maxnet

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants