New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Integrate ChainerMN to Chainer repository #5226

Merged
merged 7 commits into from Sep 6, 2018

Conversation

Projects
None yet
3 participants
@keisukefukuda
Contributor

keisukefukuda commented Aug 14, 2018

This PR integrates ChainerMN, an add-on package to support multinode training.
Background:

  • Distributed deep learning is getting more popular, thus Chainer should natively support it
  • As both of Chainer and ChainerMN are continuously updated, it is getting harder to support all version combinations of the softwares.

What this PR does:

  • Move ChainerMN to Chainer's repository and synchronize the release cycle.
  • Integrate .travis.yml setup

What this PR doesn't

  • It does not change the library usage. The chainermn package remains as is and existing ChainerMN programs work without modification.
  • It doesn't am to integrate various components of Chainer and ChainerMN. There are several duplicate building blocks between Chainer and ChainerMN, but we focus on integrating the repositories and do not step further for now.

Major TODOs:

  • Integrate .travis.yml
  • Integrate documents.
  • Check coding styles of both projects.
  • Improve error messages when mpi4py is not installed
@kmaehashi

This comment has been minimized.

Show comment
Hide comment
@kmaehashi

kmaehashi Aug 14, 2018

Member

In addition, ChainerMN tests should be excluded from chainer-test execution. (I'll do this later)
https://github.com/chainer/chainer-test/blob/4afcfb73bee19cd4fbd4daff4fb7f872870a0ce6/test.sh#L38

Member

kmaehashi commented Aug 14, 2018

In addition, ChainerMN tests should be excluded from chainer-test execution. (I'll do this later)
https://github.com/chainer/chainer-test/blob/4afcfb73bee19cd4fbd4daff4fb7f872870a0ce6/test.sh#L38

@keisukefukuda

This comment has been minimized.

Show comment
Hide comment
@keisukefukuda

keisukefukuda Aug 15, 2018

Contributor

@kmaehashi Please let us talk offline later on the test issue.

Contributor

keisukefukuda commented Aug 15, 2018

@kmaehashi Please let us talk offline later on the test issue.

@kmaehashi kmaehashi added this to the v5 milestone Aug 20, 2018

@kmaehashi kmaehashi added the feature label Aug 20, 2018

@kmaehashi kmaehashi self-assigned this Aug 20, 2018

@kmaehashi

This comment has been minimized.

Show comment
Hide comment
@kmaehashi

kmaehashi Sep 4, 2018

Member

In addition, ChainerMN tests should be excluded from chainer-test execution. (I'll do this later)

chainer/chainer-test#448

Member

kmaehashi commented Sep 4, 2018

In addition, ChainerMN tests should be excluded from chainer-test execution. (I'll do this later)

chainer/chainer-test#448

@keisukefukuda keisukefukuda changed the title from [WIP] Integrate ChainerMN to Chainer repository to Integrate ChainerMN to Chainer repository Sep 5, 2018

@keisukefukuda

This comment has been minimized.

Show comment
Hide comment
@keisukefukuda

keisukefukuda Sep 5, 2018

Contributor

This PR is based on ChainerMN
f66d35e56810d7e67c509ffb05db3e3a816e6e7b
chainer/chainermn@f66d35e

Contributor

keisukefukuda commented Sep 5, 2018

This PR is based on ChainerMN
f66d35e56810d7e67c509ffb05db3e3a816e6e7b
chainer/chainermn@f66d35e

@kmaehashi

This comment has been minimized.

Show comment
Hide comment
@kmaehashi

kmaehashi Sep 5, 2018

Member

Jenkins, test this please.

Member

kmaehashi commented Sep 5, 2018

Jenkins, test this please.

@chainer-ci

This comment has been minimized.

Show comment
Hide comment
@chainer-ci

chainer-ci Sep 5, 2018

Jenkins CI test (for commit 9a0e1b7, target branch master) succeeded!

chainer-ci commented Sep 5, 2018

Jenkins CI test (for commit 9a0e1b7, target branch master) succeeded!

@kmaehashi kmaehashi merged commit f0ff7f8 into master Sep 6, 2018

5 checks passed

continuous-integration/appveyor/branch AppVeyor build succeeded
Details
continuous-integration/appveyor/pr AppVeyor build succeeded
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
continuous-integration/travis-ci/push The Travis CI build passed
Details
coverage/coveralls Coverage decreased (-1.9%) to 88.748%
Details
@kmaehashi

This comment has been minimized.

Show comment
Hide comment
@kmaehashi

kmaehashi Sep 6, 2018

Member

LGTM!

Member

kmaehashi commented Sep 6, 2018

LGTM!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment