No description, website, or topics provided.
Clone or download
jaroslaw-osmanski Make better ux reports (#94)
* Reports show description without Some prefix

* Closing button for reports doesn't scroll when reports are scrolled

* Add modal with full content of report cell for large text
Latest commit a3293e6 Jul 17, 2018
Permalink
Failed to load latest commit information.
backendcommons Change all headers according to new scheme Sep 15, 2017
build Run e2etests on yarn (#82) Jun 11, 2018
datasourcemanager Change all headers according to new scheme Sep 15, 2017
deployment Remove hearbeat from rabbitmq so notebooks can use pikas blocking con… May 29, 2018
e2etests Remove mboost from e2etests as it is no longer present in cran (#88) Jun 14, 2018
frontend Make better ux reports (#94) Jul 17, 2018
globalresources SHR-787 Add a global predefined-users config Sep 15, 2017
libraryservice Change all headers according to new scheme Sep 15, 2017
migration_e2etests Change all headers according to new scheme Sep 15, 2017
project Spark 2.2 (#65) May 18, 2018
proxy Change all headers according to new scheme Sep 15, 2017
remote_notebook Pass dataframe schema to python notebook context (#66) Jun 4, 2018
schedulingmanager Change all headers according to new scheme Sep 15, 2017
seahorse-sdk-example @ 5ff0294 Add Evaluator example to SDK (#84) Jun 12, 2018
seahorse-workflow-executor Make better ux reports (#94) Jul 17, 2018
sessionmanager Parallelism can be set for local cluster using preset (#93) Jul 6, 2018
testing Run e2etests on yarn (#82) Jun 11, 2018
utils Fix for #19 Mar 4, 2018
workflowmanager Sdk SPI Registrant (#81) Jun 8, 2018
.editorconfig Add script for building/publishing docker images Sep 15, 2017
.gitignore Sdk SPI Registrant (#81) Jun 8, 2018
.gitmodules Remove seahorse-workflow-executor submodule Sep 15, 2017
.scalafmt.conf Add scalafmt plugin Sep 15, 2017
LICENSE Update licenses to apache 2.0 Sep 15, 2017
README.md Sdk SPI Registrant (#81) Jun 8, 2018
build.sbt Rebrand to deepsense.ai: scala packages Sep 15, 2017
nested-WE-repo-integration.sbt Add scalafmt plugin Sep 15, 2017
scalastyle-config.xml Change all headers according to new scheme Sep 15, 2017
version.sbt Change version to 1.4.4-SNAPSHOT Apr 18, 2018

README.md

Seahorse

Seahorse is an open-source visual framework allowing you to create Apache Spark applications in a fast, simple and interactive way.

Seahorse is distributed under the Apache 2.0 License.

Read more about Seahorse on the documentation page: seahorse.deepsense.ai.

Building Seahorse from source

Prerequisites:

  • docker 1.30
  • docker-compose 1.9
  • JDK 8
  • sbt 0.13
  • python 2.7
    • PyYAML
  • npm 4.6
  • jekyll 3.2
    • pygments.rb
    • jekyll-sass-converter
    • jekyll-watch
  • PhantomJS

Run

./build/build_all.sh

This will build all the needed docker images and create a docker-compose.yml file. You can now run it using docker-compose up. Seahorse will start at http://localhost:33321.

A good place to start using Seahorse is the basic examples section of the documentation.

Development

Note that in order to contribute to Seahorse you have to sign the Contributor License Agreement.

Before submitting a PR, please run the Scala style check:

sbt scalastylebackend && (cd ./seahorse-workflow-executor && sbt scalastyle)

Running tests

Initialize the submodules before running the tests:

git submodule init
git submodule update

Backend tests:

./build/build_and_run_tests.sh

Frontend tests:

./frontend/run_unit_tests.sh

End-to-end integration tests:

./build/e2e_tests.sh -a

In order for Seahorse to compile and run correctly on Mac OS, you need to increase memory for Docker engine to at least 6GB.

Bash completion for Python scripts

Some of our Python scripts used by devs support bash autocompletion using argcomplete.

pip install argcomplete
activate-global-python-argcomplete --user

See this for global completion support.

Mac OS

Note, that bash 4.2 is required. Installation instruction for Mac users

After the bash upgrade, you may have to rename .bash_profile to .bashrc. And maybe add /usr/local/bin to $PATH. Also, check if you're actually running the new bash with echo $BASH_VERSION - your terminal might still be using the old one.

Developing SDK operations on local repository

To compile and test SDK operations on local repository, you can use seahorse-sdk-example submodule

git submodule init
git submodule update
./build/prepare_sdk_dependencies.sh

Now it will compile and test against the local Seahorse repository:

cd seahorse-sdk-example
sbt test

Enterprise options and support

Seahorse was originally created at deepsense.ai. Technical support and customization options are available upon contact.