Skip to content

Nbatches parameter#1365

Closed
atefeh-asayesh wants to merge 9 commits intoapache:masterfrom
atefeh-asayesh:NbatchesParameter
Closed

Nbatches parameter#1365
atefeh-asayesh wants to merge 9 commits intoapache:masterfrom
atefeh-asayesh:NbatchesParameter

Conversation

@atefeh-asayesh
Copy link
Copy Markdown
Contributor

In this PR we add freq=NBATCHES along with the optional parameter nbatches to extend paramserv builtin for NBATCH updates.
In this step, It has been implemented for both local (computeNBatches(dataSize, batchIter)) and federated parameter servers (computeWithNBatchUpdates in FederatedPSControlThread.java).

…tches to extend paramserv builtin for NBATCH updates.

In this step it has been implemented for both local (computeNBatches(dataSize, batchIter)) and federated parameter servers (computeWithNBatchUpdates in FederatedPSControlThread.java).
…tches to extend paramserv builtin for NBATCH updates.

In this step it has been implemented for both local (computeNBatches(dataSize, batchIter)) and federated parameter servers (computeWithNBatchUpdates in FederatedPSControlThread.java).
…tches to extend paramserv builtin for NBATCH updates.

In this step it has been implemented for both local (computeNBatches(dataSize, batchIter)) and federated parameter servers (computeWithNBatchUpdates in FederatedPSControlThread.java).
…tches to extend paramserv builtin for NBATCH updates.

In this step it has been implemented for both local (computeNBatches(dataSize, batchIter)) and federated parameter servers (computeWithNBatchUpdates in FederatedPSControlThread.java).
…tches to extend paramserv builtin for NBATCH updates.

In this step it has been implemented for both local (computeNBatches(dataSize, batchIter)) and federated parameter servers (computeWithNBatchUpdates in FederatedPSControlThread.java).
@atefeh-asayesh
Copy link
Copy Markdown
Contributor Author

Draft- Some small changes would be done.

…tches to extend paramserv builtin for NBATCH updates.

Draft- Some small changes would be done.
…tches to extend paramserv builtin for NBATCH updates.

Draft- Some small changes would be done.
…tches to extend paramserv builtin for NBATCH updates.

In this step, It has been implemented for both local (computeNBatches(dataSize, batchIter)) and federated parameter servers (computeWithNBatchUpdates in FederatedPSControlThread.java).
@mboehm7
Copy link
Copy Markdown
Contributor

mboehm7 commented Sep 10, 2021

LGTM - thanks @atefeh-asayesh for the extension of the parameter server. During the merge I mode the following changes:

  • Fixed the local paramserv worker which pulled and pushed at the same iteration (for all but the first), although it needs to push the iteration before it pulls a new model and then work with that for nbatches.
  • Fixed the federated paramserv implementation which did not update the model correctly under modelAvg (last model update missing, and outside computeGradientsForNBatches the returned model was assumed as gradients in an additional model update). The related changes also simplified some duplicated code.
  • Fixed the computation of numSetsPerEpocNbatches which due to integer division might have lost a few batches.
  • Removed unused nbatch member variables and test methods, fixed test methods which used the constructor name
  • Fixed the federated tests to use a longer wait time after the last worker to ensure everything is ready before the experiments start.
  • Fixed some minor formatting issues and warnings

@mboehm7
Copy link
Copy Markdown
Contributor

mboehm7 commented Sep 10, 2021

Let's close this PR as it was merged with #1382

@asfgit asfgit closed this in 82536c1 Sep 18, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants