Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add Spark cluster management to jupyter, zeppelin, CLI ("+1, -1" Spark worker widget) #71

Closed
cfregly opened this issue Jul 22, 2016 · 6 comments
Closed

Add Spark cluster management to jupyter, zeppelin, CLI ("+1, -1" Spark worker widget) #71

cfregly opened this issue Jul 22, 2016 · 6 comments
Assignees
@cfregly
Labels
enhancement

Comments

@cfregly
Copy link
Contributor

cfregly commented Jul 22, 2016

No description provided.

@cfregly
Copy link
Contributor Author

cfregly commented Aug 1, 2016

http://blog.kubernetes.io/2016/03/using-Spark-and-Zeppelin-to-process-Big-Data-on-Kubernetes.html

@cfregly
Copy link
Contributor Author

cfregly commented Aug 1, 2016

in the short term, a CLI script to add/remove spark workers would be ideal

longer term, we'll make it a REST endpoint callable from anywhere (zeppelin/jupyter notebooks etc)

@cfregly
Copy link
Contributor Author

cfregly commented Aug 1, 2016

the solution should be cloud-vendor neutral

@cfregly cfregly mentioned this issue Aug 1, 2016
Closed
@cfregly cfregly assigned cfregly and unassigned andyzeli Aug 8, 2016
@cfregly cfregly changed the title Demo Spark cluster management and auto-scaling/dynamic allocation (jupyter, zeppelin, CLI) Add Spark cluster management to jupyter, zeppelin, CLI ("+1, -1" Spark worker widget) Aug 8, 2016
@cfregly cfregly added this to the 20160822 milestone Aug 8, 2016
@cfregly cfregly mentioned this issue Aug 8, 2016
Closed
@cfregly cfregly modified the milestones: 20160919, 20160822 Aug 8, 2016
@cfregly
Copy link
Contributor Author

cfregly commented Sep 23, 2016

http://limdauto.github.io/posts/2015-12-24-spark-ipython-notebook.html

@cfregly
Copy link
Contributor Author

cfregly commented Sep 23, 2016

http://thepowerofdata.io/configuring-jupyteripython-notebook-to-work-with-pyspark-1-4-0/

@cfregly cfregly removed this from the 20160919 milestone Feb 11, 2017
@cfregly
Copy link
Contributor Author

cfregly commented Aug 24, 2017

CLI is currently supported in this repo. Moving Jupyter integration to [PipelineAI Advanced/Enterprise Edition]{http://pipeline.ai/products).

@cfregly cfregly closed this as completed Aug 24, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@cfregly cfregly

Labels
enhancement
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@cfregly @andyzeli