Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

🔥 Hot-reload Syft nodes in your notebooks #8988

Merged
merged 6 commits into from
Jul 3, 2024
Merged

🔥 Hot-reload Syft nodes in your notebooks #8988

merged 6 commits into from
Jul 3, 2024

Conversation

itstauq
Copy link
Member

@itstauq itstauq commented Jun 27, 2024 •
edited
Loading

Description

An alternative approach to #8971 and #8987

Affected Dependencies

List any dependencies that are required for this change.

How has this been tested?

  • Describe the tests that you ran to verify your changes.
  • Provide instructions so we can reproduce.
  • List any relevant details for your test configuration.

Checklist

@itstauq itstauq requested a review from snwagh June 27, 2024 22:48
@itstauq itstauq mentioned this pull request Jun 27, 2024
Closed
4 tasks
@itstauq itstauq requested a review from rasswanth-s June 27, 2024 22:49
@snwagh
Copy link
Contributor

snwagh commented Jun 28, 2024

This is incredible, beyond amazing!

snwagh
snwagh approved these changes Jun 28, 2024
View reviewed changes
Copy link
Contributor

@snwagh snwagh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Works like a charm, this will change the course of Syft's dev team!

@itstauq itstauq changed the title [WIP] Add hot reloading capability using uvicorn app factory pattern Add hot reloading capability using uvicorn app factory pattern Jun 29, 2024
@itstauq
Copy link
Member Author

itstauq commented Jun 29, 2024

@madhavajay there are instances when we run multiple nodes in the same notebook, but are actively developing on just one node. For example, in tauquir/enclave-prototype/03-ds-execute-code.ipynb we are usually actively working on the canada-enclave node but need the other two domain nodes active as well. For such cases, it makes sense to have a separate hot_reload: bool flag in orchestra.launch instead of tying the hot reloading functionality with dev_mode=True. What are your thoughts?

@itstauq itstauq requested a review from madhavajay June 29, 2024 18:59
@itstauq itstauq changed the title Add hot reloading capability using uvicorn app factory pattern Hot-reload python-based Syft nodes in your notebooks Jun 30, 2024
@itstauq itstauq changed the title Hot-reload python-based Syft nodes in your notebooks 🔥 Hot-reload python-based Syft nodes in your notebooks Jun 30, 2024
@itstauq itstauq changed the title 🔥 Hot-reload python-based Syft nodes in your notebooks 🔥 Hot-reload Syft nodes in your notebooks Jun 30, 2024
@itstauq itstauq added the Type: New Feature ➕ Introduction of a completely new addition to the codebase label Jun 30, 2024
@itstauq itstauq self-assigned this Jun 30, 2024
@rasswanth-s rasswanth-s requested a review from yashgorana July 1, 2024 11:12
@rasswanth-s rasswanth-s mentioned this pull request Jul 2, 2024
Closed
@rasswanth-s rasswanth-s requested a review from shubham3121 July 2, 2024 12:36
BrendanSchell
BrendanSchell reviewed Jul 2, 2024
View reviewed changes
packages/syft/src/syft/node/server.py
def app_factory() -> FastAPI:
settings = AppSettings()

worker_classes = {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: we define this worker_classes dict a few times in the codebase, can we reuse?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will address this in a separate PR

BrendanSchell
BrendanSchell approved these changes Jul 2, 2024
View reviewed changes
Copy link
Contributor

@BrendanSchell BrendanSchell left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work on this Tauquir! Will save a ton of time in notebook debugging🔥

rasswanth-s and others added 2 commits July 3, 2024 17:12
@rasswanth-s
Copy link
Collaborator

Tested this functionality by changing

  1. NodeMetadata
  2. UserService Fields
  3. Client Side Fields
    In jupyter lab.
    Screenshot 2024-07-03 at 5 13 50 PM

rasswanth-s
rasswanth-s approved these changes Jul 3, 2024
View reviewed changes
Copy link
Collaborator

@rasswanth-s rasswanth-s left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's go
mcqueen

@rasswanth-s rasswanth-s enabled auto-merge July 3, 2024 11:57
@rasswanth-s rasswanth-s disabled auto-merge July 3, 2024 11:57
@rasswanth-s rasswanth-s merged commit 9426b61 into dev Jul 3, 2024
26 checks passed
@rasswanth-s rasswanth-s deleted the tauquir/uvicorn-hotreload-2 branch July 3, 2024 11:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@snwagh snwagh snwagh approved these changes

@BrendanSchell BrendanSchell BrendanSchell approved these changes

@rasswanth-s rasswanth-s rasswanth-s approved these changes

@madhavajay madhavajay Awaiting requested review from madhavajay

@yashgorana yashgorana Awaiting requested review from yashgorana

@shubham3121 shubham3121 Awaiting requested review from shubham3121

Assignees

@itstauq itstauq

Labels
Purple Team Type: New Feature ➕ Introduction of a completely new addition to the codebase
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@itstauq @snwagh @rasswanth-s @BrendanSchell