- Apache Spark 2.1.1 compiled for Scala 2.11
- Jupyter Notebook
- Python 3.5+
You can install the spylon-kernel package using
pip install spylon-kernel # or conda install -c conda-forge spylon-kernel
Using it as a Scala Kernel
You can use spylon-kernel as Scala kernel for Jupyter Notebook. Do this when you want to work with Spark in Scala with a bit of Python code mixed in.
Create a kernel spec for Jupyter notebook by running the following command:
python -m spylon_kernel install
jupyter notebook and you should see a
spylon-kernel as an option
in the New dropdown menu.
See the basic example notebook for information about how to intiialize a Spark session and use it both in Scala and Python.
Using it as an IPython Magic
You can also use spylon-kernel as a magic in an IPython notebook. Do this when you want to mix a little bit of Scala into your primarily Python notebook.
from spylon_kernel import register_ipython_magics register_ipython_magics()
%%scala val x = 8 x
Using it as a Library
Finally, you can use spylon-kernel as a Python library. Do this when you want to evaluate a string of Scala code in a Python script or shell.
from spylon_kernel import get_scala_interpreter interp = get_scala_interpreter() # Evaluate the result of a scala code block. interp.interpret(""" val x = 8 x """) interp.last_result()
Push a tag and submit a source dist to PyPI.
git commit -m 'REL: 0.2.1' --allow-empty git tag -a 0.2.1 # and enter the same message as the commit git push origin master # or send a PR # if everything builds / tests cleanly, release to pypi make release