A ruby library which implements ID3 (information gain) algorithm for decision tree learning. Currently, continuous and discrete datasets can be learned.
-
Discrete model assumes unique labels & can be graphed and converted into a png for visual analysis
-
Continuous looks at all possible values for a variable and iteratively chooses the best threshold between all possible assignments. This results in a binary tree which is partitioned by the threshold at every step. (e.g. temperate > 20C)
-
ID3 algorithms for continuous and discrete cases, with support for incosistent datasets.
-
Graphviz component to visualize the learned tree (rockit.sourceforge.net/subprojects/graphr/)
-
Support for multiple, and symbolic outputs and graphing of continuos trees.
-
Returns default value when no branches are suitable for input
-
Ruleset is a class that trains an ID3Tree with 2/3 of the training data, converts it into a set of rules and prunes the rules with the remaining 1/3 of the training data (in a C4.5 way).
-
Bagging is a bagging-based trainer (quite obvious), which trains 10 Ruleset trainers and when predicting chooses the best output based on voting.
Blog post with explanation & examples: www.igvita.com/2007/04/16/decision-tree-learning-in-ruby/
require 'decisiontree'
attributes = ['Temperature']
training = [
[36.6, 'healthy'],
[37, 'sick'],
[38, 'sick'],
[36.7, 'healthy'],
[40, 'sick'],
[50, 'really sick'],
]
# Instantiate the tree, and train it based on the data (set default to '1')
dec_tree = DecisionTree::ID3Tree.new(attributes, training, 'sick', :continuous)
dec_tree.train
test = [37, 'sick']
decision = dec_tree.predict(test)
puts "Predicted: #{decision} ... True decision: #{test.last}";
=> Predicted: sick ... True decision: sick