-
Notifications
You must be signed in to change notification settings - Fork 57
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
MultiNodeIterator #186
MultiNodeIterator #186
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
How about adding tests to check the compatibility of the iterator.
https://github.com/chainer/chainer/blob/master/tests/chainer_tests/iterators_tests/test_iterator_compatibility.py
chainermn/communicators/_base.py
Outdated
|
||
msgtype = self.mpi_comm.bcast(msgtype, root) | ||
shape = msgtype.shapes[0] | ||
buf = _memory_utility.array_to_buffer_object(x) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Since the data type of x
assumes float32, I think that it is better to add an assertion to check x.dtype.
chainermn/communicators/_base.py
Outdated
'gather cannot handle different shapes of data') | ||
|
||
# Gather data. | ||
sbuf = _memory_utility.array_to_buffer_object(x) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Same as above.
chainermn/communicators/_base.py
Outdated
if is_master: | ||
msgtype = _MessageType(x) | ||
if msgtype.is_tuple: | ||
raise ValueError('cannot broadcast tuple data') |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What about TypeError?
This PR adds
MultiNodeIterator
, where master process broadcasts shuffled batches to slave processes.In addition to
MultiNodeIterator
, this PR also adds the following features, needed for implementing it:bcast
andgather
for ChainerMN communicatorchainermn.functions.bcast
-> not needed anymore