Skip to content

Fix float 16 for cuda collectives #414

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
nouiz merged 2 commits into from
May 2, 2017
Merged

Fix float 16 for cuda collectives #414

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
bb24874
Convert GA_FLOAT16 to ncclHalf in cuda collectives
tsirif Apr 20, 2017
c11b0b3
Remove ifdef as all supported version have it.
nouiz May 2, 2017
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion src/gpuarray_collectives_cuda_nccl.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -175,11 +175,12 @@ static inline ncclDataType_t convert_data_type(int typecode) {
switch (typecode) {
case GA_BYTE: return ncclChar;
case GA_INT: return ncclInt;
case GA_HALF: return ncclHalf;
case GA_FLOAT: return ncclFloat;
case GA_DOUBLE: return ncclDouble;
case GA_LONG: return ncclInt64;
case GA_ULONG: return ncclUint64;
case GA_HALF: return ncclHalf;
case GA_FLOAT16: return ncclHalf;
Copy link
Copy Markdown
Member

@abergeron abergeron May 23, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

GA_FLOAT16 means a vector of 16 floats. This is horribly inaccurate.

Copy link
Copy Markdown
Contributor Author

@tsirif tsirif May 23, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had asked on #411 about the types. Excuse me for the inconvenience.

}
return nccl_NUM_TYPES;
}
Expand Down