Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] DL4J MKL-DNN Support #7151

Merged
merged 19 commits into from Mar 12, 2019

Conversation

@AlexDBlack
Copy link
Contributor

commented Feb 12, 2019

Adds MKL-DNN support for DL4J layers.

MKL-DNN support for:

  • Subsampling layer. Implemented and passing tests.
  • ConvolutionLayer: Implemented and passing tests.
  • Batch norm: forward pass only implemented (passing tests). No backprop on account of this: #7166

Note that conv3d is already using conv3dnew op, hence already supports MKL-DNN by default.

Previous performance issues with MKL-DNN have been resolved.

LRN is implemented but is disabled on account of this: #7272

LSTM is not implemneted here - we don't yet have MKL-DNN support at libnd4j blockLSTM op (or similar) level.

@saudet

This comment has been minimized.

Copy link
Member

commented Feb 13, 2019

If I'm not mistaken, that's not limited to MKL-DNN, but whatever gets used by ND4J, right?

@AlexDBlack

This comment has been minimized.

Copy link
Contributor Author

commented Feb 13, 2019

@saudet so far, yes - the plan is to check if mkl-dnn is actually available and will be used by libnd4j when executing these ops. I should also check the environment to ensure mkl-dnn isn't explicitly disabled too...

@saudet

This comment has been minimized.

Copy link
Member

commented Feb 13, 2019

I mean, your code looks generic, so it would pick up, for example, cuDNN when running for GPU, assuming it's implemented in libnd4j, right?

@AlexDBlack

This comment has been minimized.

Copy link
Contributor Author

commented Feb 13, 2019

so it would pick up, for example, cuDNN when running for GPU

TBH I didn't actually think about that, but yes - if cuDNN is implemented in the same manner as mkl-dnn, then it should do so.

@AlexDBlack AlexDBlack force-pushed the ab_dl4j_mkldnn branch 2 times, most recently from 0b6b25e to acb3e5e Feb 14, 2019

@AlexDBlack AlexDBlack force-pushed the ab_dl4j_mkldnn branch 2 times, most recently from 6fd2573 to 8399e87 Feb 23, 2019

@AlexDBlack AlexDBlack force-pushed the ab_dl4j_mkldnn branch from 8399e87 to c51e165 Mar 7, 2019

@AlexDBlack AlexDBlack force-pushed the ab_dl4j_mkldnn branch from c51e165 to 455466f Mar 9, 2019

@AlexDBlack AlexDBlack merged commit 61bba35 into master Mar 12, 2019

0 of 2 checks passed

codeclimate Code Climate encountered an error attempting to analyze this pull request.
Details
Codacy/PR Quality Review Not up to standards. This pull request quality could be better.
Details

@AlexDBlack AlexDBlack deleted the ab_dl4j_mkldnn branch Mar 12, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants
You can’t perform that action at this time.