Jupyter kernel for scala and spark
Switch branches/tags
Clone or download
Pull request Compare This branch is 13 commits behind Valassis-Digital-Media:master.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
examples
spylon_kernel
test
.coveragerc
.gitattributes
.gitignore
.travis.yml
CHANGELOG.md
LICENSE
MANIFEST.in Add README.md, LICENSE to manifest Mar 4, 2017
Makefile
README.md
codecov.yml
requirements-test.txt
requirements.txt Automate local dev setup Mar 11, 2017
run_tests.py
setup.cfg
setup.py Automate local dev setup Mar 11, 2017
test_spylon_kernel_jkt.py
versioneer.py

README.md

spylon-kernel

Build Status codecov

A Scala Jupyter kernel that uses metakernel in combination with py4j.

Prerequisites

  • Apache Spark 2.1.1 compiled for Scala 2.11
  • Jupyter Notebook
  • Python 3.5+

Install

You can install the spylon-kernel package using pip or conda.

pip install spylon-kernel
# or
conda install -c conda-forge spylon-kernel

Using it as a Scala Kernel

You can use spylon-kernel as Scala kernel for Jupyter Notebook. Do this when you want to work with Spark in Scala with a bit of Python code mixed in.

Create a kernel spec for Jupyter notebook by running the following command:

python -m spylon_kernel install

Launch jupyter notebook and you should see a spylon-kernel as an option in the New dropdown menu.

See the basic example notebook for information about how to intiialize a Spark session and use it both in Scala and Python.

Using it as an IPython Magic

You can also use spylon-kernel as a magic in an IPython notebook. Do this when you want to mix a little bit of Scala into your primarily Python notebook.

from spylon_kernel import register_ipython_magics
register_ipython_magics()
%%scala
val x = 8
x

Using it as a Library

Finally, you can use spylon-kernel as a Python library. Do this when you want to evaluate a string of Scala code in a Python script or shell.

from spylon_kernel import get_scala_interpreter

interp = get_scala_interpreter()

# Evaluate the result of a scala code block.
interp.interpret("""
    val x = 8
    x
""")

interp.last_result()

Release Process

Push a tag and submit a source dist to PyPI.

git commit -m 'REL: 0.2.1' --allow-empty
git tag -a 0.2.1 # and enter the same message as the commit
git push origin master # or send a PR

# if everything builds / tests cleanly, release to pypi
make release

Then update https://github.com/conda-forge/spylon-kernel-feedstock.