Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Modify launch script for iRkernel to delay SC initialization #124

Closed
ckadner opened this issue Aug 19, 2017 · 0 comments
Closed

Modify launch script for iRkernel to delay SC initialization #124

ckadner opened this issue Aug 19, 2017 · 0 comments

Comments

@ckadner
Copy link
Collaborator

ckadner commented Aug 19, 2017

See issue #64: "Investigate async creation of spark session in launchers"

Modify launch_IRkernel.R script similar to launch_ipykernel.py (commit 6b0c2b7):

  • initialize the Spark variables to generic placeholder object(s)
  • "wrap" the initialize_spark_session() call in an asynchronous thread and start it just before calling IRkernel::main(connection_file)
  • running a notebook cell that references any of the Spark variables will get "intercepted" by the placeholder object(s), which will wait for the initialization thread to complete (blocking) and then delegate the execution to the actual Spark objects
  • after the initialization thread completed, the references to the placeholder object(s) will be replaced by references to the actual Spark objects
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants