Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Simplify mnnist example using Gluon API and update README #886

Merged
merged 3 commits into from Mar 8, 2019

Conversation

4 participants
@apeforest
Copy link
Contributor

apeforest commented Mar 5, 2019

Current mxnet examples uses an old module API which has been gradually replaced by a more user-friendly Gluon API. Replacing the example and README using Gluon API for mxnet.

Simplify mnnist example using Gluon API and update README
Signed-off-by: Lin Yuan <apeforest@gmail.com>

@apeforest apeforest force-pushed the apeforest:mxnet/example branch from 8392ef5 to 8302da7 Mar 5, 2019

@apeforest

This comment has been minimized.

Copy link
Contributor Author

apeforest commented Mar 5, 2019

@yuxihu @eric-haibin-lin Please help to review. Thanks!

Show resolved Hide resolved examples/mxnet_mnist.py Outdated
replace no-cuda argument by -use-gpu
Signed-off-by: Lin Yuan <apeforest@gmail.com>

@apeforest apeforest force-pushed the apeforest:mxnet/example branch from ec77bdc to 3e64b27 Mar 5, 2019

@alsrgv

This comment has been minimized.

Copy link
Collaborator

alsrgv commented Mar 6, 2019

change argument name
Signed-off-by: Lin Yuan <apeforest@gmail.com>

@apeforest apeforest force-pushed the apeforest:mxnet/example branch from 0f005bf to 77e5e17 Mar 6, 2019

@eric-haibin-lin
Copy link
Contributor

eric-haibin-lin left a comment

Thanks for the update! Two questions:

import mxnet as mx
import horovod.mxnet as hvd
from mxnet import autograd, gluon, nd
from mxnet.test_utils import download

# Training settings
parser = argparse.ArgumentParser(description='MXNet MNIST Example')

This comment has been minimized.

@eric-haibin-lin

eric-haibin-lin Mar 6, 2019

Contributor

This may be a general question for all horovod examples:
There are a lot of examples for ResNet on ImageNet. This is great. But I do not find instructions on how to run those example (e.g. with x learning rate and y batch size on z GPUs this example converges to 75.5% accuracy). This will help people validate that everything is working when they try out the example.

This comment has been minimized.

@apeforest

apeforest Mar 7, 2019

Author Contributor

@eric-haibin-lin The default values are the ones we used to achieve 75.5% reported. Users can reproduce this using the defalt argument values.

import mxnet as mx
import horovod.mxnet as hvd
from mxnet import autograd, gluon
# Initialize Horovod
hvd.init()

This comment has been minimized.

@eric-haibin-lin

eric-haibin-lin Mar 6, 2019

Contributor

Is the parameter hook for gluon already done? As we discussed last time, if the shape of a parameter has to be inferred after a batch of data is seen, the parameter will not be broadcast. Therefore, the correctness of the program depends on whether all workers are initialized with the same random seed. Maybe we should call it out as a known issue if it is not yet fixed.

This comment has been minimized.

@yuxihu

yuxihu Mar 6, 2019

Contributor

Not yet. I will look into it soon.

This comment has been minimized.

@eric-haibin-lin

eric-haibin-lin Mar 8, 2019

Contributor

I created an issue in case others it affects others: #895

This comment has been minimized.

@yuxihu

yuxihu Mar 8, 2019

Contributor

@eric-haibin-lin I think you should mention Gluon in the title of the issue to avoid confusion. It does not affect Module API.

@yuxihu

yuxihu approved these changes Mar 8, 2019

Copy link
Contributor

yuxihu left a comment

LGTM.

@alsrgv alsrgv merged commit 91d5ae8 into horovod:master Mar 8, 2019

1 check passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.