Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added functionality to launch PyGrid network and nodes upon creation of a cluster #3896

Merged
merged 7 commits into from
Aug 6, 2020
Merged

Conversation

rimijoker
Copy link
Member

@rimijoker rimijoker commented Jul 25, 2020

Description

I have added startup-scripts and other necessary changes which will enable the autoscale API to automatically launch PyGrid servers(network and nodes). Now using the autoscale API can launch PyGird cluster(1 network + N nodes) with one function call or create a PyGrid network and add PyGrid nodes to it one at a time as per the requirement.

I have also added tests and an example tutorial training script for the demonstration of the same.

@hericlesme
Issue #3099

Affected Dependencies

None (New Feature)

How has this been tested?

  • I have created test.py script and test.ipynb notebook to test the functionality of the features implemented
  • I have also modified the README.md with examples on how to use the functionality.
  • I have also modified a PyGrid tutorial to use this API for demonstration.

Checklist

@rimijoker rimijoker requested a review from a team as a code owner July 25, 2020 15:02
@review-notebook-app
Copy link

Check out this pull request on  ReviewNB

Review Jupyter notebook visual diffs & provide feedback on notebooks.


Powered by ReviewNB

@monuelo monuelo self-requested a review July 25, 2020 15:36
@codecov
Copy link

codecov bot commented Jul 25, 2020

Codecov Report

Merging #3896 into master will decrease coverage by 0.13%.
The diff coverage is 98.59%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #3896      +/-   ##
==========================================
- Coverage   94.94%   94.80%   -0.14%     
==========================================
  Files         197      200       +3     
  Lines       19944    20528     +584     
==========================================
+ Hits        18936    19462     +526     
- Misses       1008     1066      +58     
Impacted Files Coverage Δ
syft/execution/placeholder.py 95.23% <66.66%> (+0.12%) ⬆️
syft/execution/plan.py 94.33% <85.71%> (-0.50%) ⬇️
syft/frameworks/torch/nn/functional.py 89.83% <100.00%> (ø)
...frameworks/torch/tensors/interpreters/precision.py 97.08% <100.00%> (+0.02%) ⬆️
test/execution/test_rnn_plan.py 100.00% <100.00%> (ø)
test/torch/tensors/test_additive_shared.py 100.00% <100.00%> (ø)
syft/frameworks/torch/he/fv/context.py 80.55% <0.00%> (-19.45%) ⬇️
syft/frameworks/torch/he/fv/modulus.py 80.95% <0.00%> (-15.83%) ⬇️
syft/frameworks/torch/mpc/spdz.py 90.36% <0.00%> (-9.64%) ⬇️
syft/frameworks/torch/mpc/fss.py 91.66% <0.00%> (-7.44%) ⬇️
... and 31 more

syft/grid/utils/autoscale/README.md Outdated Show resolved Hide resolved
syft/grid/utils/autoscale/README.md Outdated Show resolved Hide resolved
syft/grid/utils/autoscale/README.md Outdated Show resolved Hide resolved
syft/grid/utils/autoscale/README.md Outdated Show resolved Hide resolved
syft/grid/utils/autoscale/README.md Outdated Show resolved Hide resolved
syft/grid/utils/autoscale/gcloud.py Outdated Show resolved Hide resolved
@rimijoker rimijoker requested a review from monuelo August 6, 2020 00:29
@monuelo monuelo merged commit f4fc91b into OpenMined:master Aug 6, 2020
apt-get -y upgrade
sudo -i bash -c 'pip install notebook==5.7.8'
sudo -i bash -c 'pip install git+https://github.com/OpenMined/PyGridNode.git'
sudo -i bash -c 'echo Starting Node {name} \
Copy link
Contributor

@Nilanshrajput Nilanshrajput Aug 6, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

GridNode Repo is private and archived now, this should be updated to new method for creating nodes from Pygrid

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now as the refactoring has completed, I will update it to the new methods shortly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants