Skip to content
master
Go to file
Code

Latest commit

 

Git stats

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
 
 
src
 
 
 
 
 
 
 
 
 
 

README.md

Yggdrasil: Faster Decision Trees Using Column Partitioning in Spark

Yggdrasil is a more efficient way in Apache Spark to train decision trees for large depths and datasets with a high number of features. For depths greater than 10, Yggdrasil is an order of magnitude faster than Spark MLlib v1.6.0.

Usage

Add the dependency to your SBT project by adding the following to build.sbt (see the Spark Packages listing for spark-submit and Maven instructions):

resolvers += "Spark Packages Repo" at "http://dl.bintray.com/spark-packages/maven"

libraryDependencies += "fabuzaid21" % "yggdrasil" % "1.0"

Then use Yggdrasil as follows:

import org.apache.spark.ml.tree.impl.YggdrasilClassifier // YgddrasilRegressor

// Identical to the Spark MLlib Decision Tree API
val dt = new YggdrasilClassifier()
      .setFeaturesCol("indexedFeatures")
      .setLabelCol(labelColName)
      .setMaxDepth(params.maxDepth)
      .setMaxBins(params.maxBins)
      .setMinInstancesPerNode(params.minInstancesPerNode)
      .setMinInfoGain(params.minInfoGain)
      .setCacheNodeIds(params.cacheNodeIds)
      .setCheckpointInterval(params.checkpointInterval)

About

Yggdrasil: Faster Decision Trees Using Column Partitioning in Spark

Resources

License

Releases

No releases published

Languages

You can’t perform that action at this time.