Skip to content
InsightEdge Core
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
insightedge-cli changed project version to 15.0.0 Jun 30, 2019
insightedge-core
insightedge-examples
insightedge-integration-tests
insightedge-packager change version in gs.home.dir from 14.5 to 15.0 Jul 15, 2019
insightedge-zeppelin changed project version to 15.0.0 Jun 30, 2019
.gitignore IE-205 added release script (#88) Jul 2, 2017
CONTRIBUTING.md
LICENSE.md [IE-182] add Apache 2 license Sep 2, 2016
README.md Fixed Zeppelin port in examples readme Dec 27, 2018
pom.xml changed project version to 15.0.0 Jun 30, 2019

README.md

InsightEdge

Documentation: User Guide
Community: Slack Channel, StackOverflow tag, Email
Contributing: Contribution Guide
Issue Tracker: Jira
License: Apache 2.0

InsightEdge is a Spark distribution on top of in-memory Data Grid. A single platform for analytical and transactional workloads.

Features

  • Exposes Data Grid as Spark RDDs
  • Saves Spark RDDs to Data Grid
  • Full DataFrames and Dataset API support with persistence
  • Geospatial API for RDD and DataFrames. Geospatial indexes.
  • Transparent integration with SparkContext using Scala implicits
  • Data Grid side filtering with ability apply indexes
  • Running SQL queries in Spark over Data Grid
  • Data locality between Spark and Data Grid nodes
  • Storing MLlib models in Data Grid
  • Continuously saving Spark Streaming computation to Data Grid
  • Off-Heap persistence
  • Interactive Web Notebook
  • Python support

Building InsightEdge

InsightEdge is built using Apache Maven.

First, compile and install InsightEdge Core libraries:

# without unit tests
mvn clean install -DskipTests=true

# with unit tests
mvn clean install

To build InsightEdge zip distribution you need the following binary dependencies:

  • insightedge-datagrid 12.3.0: download a copy of the XAP 12.x Open Source Edition
  • insightedge-examples: use the same branch as in this repo, find build instructions in repository readme
  • insightedge-zeppelin: use the same branch as in this repo, run ./dev/change_scala_version.sh 2.11, then build with mvn clean install -DskipTests -P spark-2.1 -P scala-2.11 -P build-distr -Dspark.version=2.1.1
  • Apache Spark 2.3.0: download zip

Package InsightEdge distribution:

mvn clean package -P package-open -DskipTests=true -Ddist.spark=<path to spark.tgz> -Ddist.xap=file:///<path to xap.zip> -Ddist.zeppelin=<path to zeppelin.tar.gz> -Ddist.examples.target=<path to examples target>

The archive is generated under insightedge-packager/target/open directory. The archive content is under insightedge-packager/target/contents-community.

To run integration tests refer to the wiki page

Quick Start

Build the project and start InsightEdge demo mode with

cd insightedge-packager/target/contents-community
./bin/insightedge -demo

It starts Zeppelin at http://127.0.0.1:9090 with InsightEdge tutorial and example notebooks you can play with. The full documentation is available at website.

You can’t perform that action at this time.