Seahorse is an open-source visual framework allowing you to create Apache Spark applications in a fast, simple and interactive way.
Seahorse is distributed under the Apache 2.0 License.
Building Seahorse from source
- docker 1.30
- docker-compose 1.9
- JDK 8
- sbt 0.13
- python 2.7
- npm 4.6
- jekyll 3.2
This will build all the needed docker images and create a
You can now run it using
docker-compose up. Seahorse will start at http://localhost:33321.
A good place to start using Seahorse is the basic examples section of the documentation.
Note that in order to contribute to Seahorse you have to sign the Contributor License Agreement.
Before submitting a PR, please run the Scala style check:
sbt scalastylebackend && (cd ./seahorse-workflow-executor && sbt scalastyle)
Initialize the submodules before running the tests:
git submodule init git submodule update
End-to-end integration tests:
In order for Seahorse to compile and run correctly on Mac OS, you need to increase memory for Docker engine to at least 6GB.
Bash completion for Python scripts
Some of our Python scripts used by devs support bash autocompletion using argcomplete.
pip install argcomplete activate-global-python-argcomplete --user
See this for global completion support.
Note, that bash 4.2 is required. Installation instruction for Mac users
After the bash upgrade, you may have to rename
.bashrc. And maybe add
/usr/local/bin to $PATH.
Also, check if you're actually running the new bash with
echo $BASH_VERSION - your terminal might still be using the old one.
Developing SDK operations on local repository
To compile and test SDK operations on local repository, you can use
git submodule init git submodule update ./build/prepare_sdk_dependencies.sh
Now it will compile and test against the local Seahorse repository:
cd seahorse-sdk-example sbt test