Skip to content
This repository has been archived by the owner on Nov 17, 2023. It is now read-only.

Model Parallelism #5807

Closed
hondaathma opened this issue Apr 12, 2017 · 3 comments
Closed

Model Parallelism #5807

hondaathma opened this issue Apr 12, 2017 · 3 comments

Comments

@hondaathma
Copy link

hondaathma commented Apr 12, 2017 •
edited

Does MXNET support model parallelism?

What I mean is I have 2 GPUs (12GB each).A single Caffe model (Resnet152 with some changes and additions) exceeds 12GB memory and does not fit in a single GPU for training. How can I solve that problem with 2 GPUs?
Can I split this huge model to both the GPUs and make sure they communicate gradients with each other?
If so what all should I change in the solver/train files in MXNET?
@hondaathma hondaathma changed the title MultiModel Parallelism Model Parallelism Apr 12, 2017
@eric-haibin-lin
Copy link
Member

eric-haibin-lin commented Apr 12, 2017 •
edited

http://mxnet.io/how_to/model_parallel_lstm.html

@KeyKy
Copy link
Contributor

KeyKy commented Aug 18, 2017 •
edited

Is there any other easier examples about model parallel? I want to put resnet in gpu2 and lenet in gpu0

@tqchen tqchen closed this as completed Oct 19, 2017
@kaonashi-tyc
Copy link

The example in the tutorial is no longer available

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@tqchen @KeyKy @eric-haibin-lin @kaonashi-tyc @hondaathma