Skip to content
A spark package for loading Spark ML models to Redis-ML
Scala Python Shell Dockerfile
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
build
forest-example demo python scripts May 24, 2017
project
scripts fix typo Jul 16, 2017
src/main/scala/com/redislabs/provider/redis/ml added STATS flag to query syntax May 7, 2017
.gitignore
Dockerfile
LICENSE
README.md
build.sbt add stats to leaf node Jan 2, 2017
scalastyle-config.xml

README.md


Notice RedisML is planned to be replaced by RedisAI, adding support for deep learning.


Spark-Redis-ML

A spark package for loading Spark ML models to Redis-ML

Requirments:

Apache Spark 2.0 or later

Redis build from unstable branch

Jedis

Jedis-ml

Installation:

#get and build redis-ml
git clone https://github.com/RedisLabsModules/redis-ml.git
cd redis-ml/src
make 

#get and build jedis
git clone https://github.com/xetorthio/jedis.git
cd jedis
mvn package -Dmaven.test.skip=true

#get and build jedis-ml
cd..
git clone https://github.com/RedisLabs/jedis-ml.git
cd jedis-ml
mkdir lib
cp ../jedis/target/jedis-3.0.0-SNAPSHOT.jar lib/
mvn install 

#get and build spark-jedis-ml
cd.. 
git clone https://github.com/RedisLabs/spark-redis-ml.git
cd spark-redis-ml
cp ../jedis/target/jedis-3.0.0-SNAPSHOT.jar lib/
cp ../jedis-ml/target/jedis-ml-1.0-SNAPSHOT.jar lib/
sbt assembly

Usage:

Run Redis server with redis-ml module:

/path/to/redis-server --loadmodule ./redis-ml.so

From Spark root directory, Run Spark shell with the required jars:

./bin/spark-shell --jars ../spark-redis-ml/target/scala-2.11/spark-redis-ml-assembly-0.1.0.jar,../spark-redis-ml/lib/jedis-3.0.0-SNAPSHOT.jar,../spark-redis-ml/lib/jedis-ml-1.0-SNAPSHOT.jar

On Spark shell:

scala> :load "../spark-redis-ml/scripts/forest-example.scala"
scala> benchmark(10)

You can’t perform that action at this time.