Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ADD Thread mode execution for jobs and subjobs. #8309

Merged
merged 14 commits into from
Dec 12, 2023
Merged

Conversation

IonesioJunior
Copy link
Member

Description

For development and debug purposes we want to run our helm job/subjobs in thread mode instead of multiprocessing mode.
This can be very useful in development scenarios where we want to debug things using pdb or similar tools.

Affected Dependencies

  • Hagrid Orchestra.launch
  • Node.named function
  • ZMQQueueConfig parameters
  • Queue APICallMessageHandler

How has this been tested?

  • Notebook testing both threading / multiprocessing strategies and their outcomes (subjob results, logs,etc.).
  • Provide instructions so we can reproduce.
    • Run nested_syft_function notebook
    • Just execute it as it is (default: thread_workers = False) to run jobs in multiprocessing mode.
    • ADD thread_workers=True to run jobs and subjobs in threading mode.
threading_mode

@IonesioJunior IonesioJunior changed the title ADD ADD Thread mode execution for jobs and subjobs. Dec 5, 2023
@koenvanderveen koenvanderveen merged commit 8a12f6b into dev Dec 12, 2023
32 checks passed
@koenvanderveen koenvanderveen deleted the enhance/hb_898 branch December 12, 2023 10:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants