Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Only spin up resources when needed #1937

schrockn opened this issue Nov 25, 2019 · 1 comment


Copy link

@schrockn schrockn commented Nov 25, 2019

Now that we are going to start pushing stuff like and also supporting larger pipelines and pushing solid subselection more in the tools, we are going to need to start spinning up resources on-demand only. As written for example a spark context would be brought up in every execution step regardless of whether or not it was used.

@mgasner mgasner added the core label Nov 29, 2019

This comment has been minimized.

Copy link
Member Author

@schrockn schrockn commented Dec 2, 2019

@schrockn schrockn self-assigned this Dec 2, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
None yet
2 participants
You can’t perform that action at this time.