Skip to content
Branch: master
Go to file
Code

Latest commit

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
bin
 
 
 
 
sbt
 
 
 
 
 
 
 
 
 
 

README.md

keystone-example

An example skeleton building applications on top of KeystoneML.

To Run This Example:

Launch a KeystoneML cluster using the provided scripts according to these instructions.

Once the cluster is up, ssh onto the master node and execute these commands:

# Build keystone-example
cd /mnt
git clone https://github.com/amplab/keystone-example.git
cd keystone-example
sbt/sbt assembly
~/spark-ec2/copy-dir target/

# Get the data
wget http://qwone.com/~jason/20Newsgroups/20news-bydate.tar.gz
tar -xvzf 20news-bydate.tar.gz

# Copy to HDFS
/root/ephemeral-hdfs/bin/hadoop fs -copyFromLocal 20news-bydate-train/ /data/
/root/ephemeral-hdfs/bin/hadoop fs -copyFromLocal 20news-bydate-test/ /data/

# Run the pipeline
export SPARK_HOME=/root/spark
export KEYSTONE_HOME=/root/keystone
KEYSTONE_MEM=4g ./bin/run-pipeline.sh \
  pipelines.ExamplePipeline \
  --trainLocation /data/20news-bydate-train \
  --testLocation /data/20news-bydate-test

About

A example skeleton for an application built on top of KeystoneML

Resources

Releases

No releases published
You can’t perform that action at this time.