Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DL4J Spark fixes (Gradient Sharing) #5858

merged 9 commits into from Jul 9, 2018


Copy link

commented Jul 9, 2018

Fixes/improvements for gradinet sharing.


  • Bug fix in repartitioning for gradient sharing
  • Makes repartitioning more configurable (adds Repartitioner interface)
  • Improves repartitioning (new default algorithm allows spark scheduler to do proper load balancing)
  • Post training, logs number of minibatches fit for each JVM

@AlexDBlack AlexDBlack changed the title [WIP] DL4J Spark fixes (Gradient Sharing) DL4J Spark fixes (Gradient Sharing) Jul 9, 2018

@AlexDBlack AlexDBlack merged commit 2ead48e into master Jul 9, 2018

0 of 2 checks passed

codeclimate Code Climate is analyzing this code.
continuous-integration/jenkins/pr-merge This commit is being built

@AlexDBlack AlexDBlack deleted the ab_sparkdebug branch Jul 9, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
None yet
1 participant
You can’t perform that action at this time.