forked from pengsun/MultiClassLogitBoost
-
Notifications
You must be signed in to change notification settings - Fork 0
/
notes.txt
109 lines (77 loc) · 4.66 KB
/
notes.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
batch_vb_samp_boost
num_it, abs_grad, F, P, Trees, nr_wts (number remained after Weight Trimming for samples), nr_wtc (... after ... classes)
batch_vb2_samp_boost
num_it, abs_grad, F, P, Trees, nr_wts, nr_wtc, tree_node_cc
batch_vb3_samp_boost
num_it, abs_grad, F, P, Trees, nr_wts, nr_wtc, tree_node_cc, tree_node_sc (cc: class count, sc: sample count)
batch_vb4_samp_boost
output: num_it, abs_grad, F, P, Trees, tree_node_cc, tree_node_sc (nr_xxx can be infered from tree_node_xxx), tree_is_leaf, GradCls, LossCls
input: rs,wrs, rf, rc,wrc (rs: ratio sample, wrs: weight ratio sample, rf: ratio feature, rc: ratio classes, wrc: weight ratio classes )
batch_vb5_samp_boost
output: num_it, abs_grad, F, P, Trees, tree_node_cc, tree_node_sc (nr_xxx can be infered from tree_node_xxx), tree_is_leaf
input: rs,wrs, rf, rc,wrc (rs: ratio sample, wrs: weight ratio sample, rf: ratio feature, rc: ratio classes, wrc: weight ratio classes )
batch_CoSampboost_basic
output: num_it, abs_grad, F, P, Trees, tree_node_cc, tree_node_sc (nr_xxx can be infered from tree_node_xxx), tree_is_leaf
input: rb, wrb, rf (rb: ratio budget, wrb: weight ratio budget, rf: ratio feature)
batch_vb6_samp_boost
output: num_it, abs_grad, F, P, Trees, tree_node_cc, ...
tree_node_sc (nr_xxx can be infered from tree_node_xxx), tree_node_all_sc,...
tree_is_leaf,...
tree_leaf_gain, tree_leaf_allsample_gain
input: rs,wrs, rf, rc,wrc (rs: ratio sample, wrs: weight ratio sample, rf: ratio feature, rc: ratio classes, wrc: weight ratio classes )
batch_avgsamp_boost
output: num_it, abs_grad,
tree_node_cc, ...
tree_node_sc (nr_xxx can be infered from tree_node_xxx), tree_node_all_sc,...
tree_is_leaf,...
tree_leaf_gain, tree_leaf_allsample_gain
input: Tdot, rs,wrs, rf, rc,wrc (rs: ratio sample, wrs: weight ratio sample, rf: ratio feature, rc: ratio classes, wrc: weight ratio classes )
pSampVTLogitBoost
VTLogitBoost with uniform sampling for samples, features and classes
pExtSampVTLogitBoost
VTLogitBoost, but does Extreme Sampling for features: each node uniformly samples features
pExtSamp2VTLogitBoost
pExtSampVTLogitBoost, but use weight trimming for samples, where weight is simply the absolute value of gradient. rs is weight ratio
pExtSamp3VTLogitBoost
pExtSampVTLogitBoost, but use Friedman's weight trimming for samples, i.e. p*(1-p)
pExtSamp4VTLogitBoost
pExtSampVTLogitBoost, but use weight trimming for samples, where the weight is the Newton decrement = (r-p)^2 / (p*(1-p))
pVbExtSamp5VTLogitBoost
pExtSampVTLogitBoost, but use weight trimming for both samples and classes, where the weight is simply the absolute value of gradient. Verbose version
pVbExtSamp6VTLogitBoost
pVbExtSamp5VTLogitBoost, but the parameter "rs" means ratio of number of examples, instead of weights.
pVbExtSamp7VTLogitBoost
pVbExtSamp3VTLogitBoost (i.e. Friedman's weight trimming where weight is p*(1-p) ), but the parameter "rs" means ratio of number of examples,
instead of weights.
pVbExtSamp8VTLogitBoost
pVbExtSamp6VTLogitBoost, but do the Class Weight Trimming for each node. rc is the class count ratio.
pVbExtSamp9VTLogitBoost
pVbExtSamp8VTLogitBoost, but use all classes when fitting node values.
pVbExtSamp10VTLogitBoost
pVbExtSamp9VTLogitBoost, but rc is the weight sum ratio (instead of class count ratio). The selected class count is recorded for each node.
pVbExtSamp11VTLogitBoost
pVbExtSamp10VTLogitBoost, but record also #examples for each node (to see the actual computation needed); set minimum #classes after sampling to 2.
parameters: rs,wrs, rf, rc,wrc
pVbExtSamp12VTLogitBoost (Stable, verbose)
see batch_vb4_samp_boost for its parameters
pVbExtSamp13VTLogitBoost (Stable, verbose)
pVbExtSamp12VTLogitBoost, but record the loss & the gradients of each class. See batch_vb5_samp_boost.
pCoSampVTLogitBoost
Sample-Class co-sampling. see batch_CoSampboost_basic.
pVbExtSamp12SkimVTLogitBoost (Stable)
Skim version of pVbExtSamp12VTLogitBoost. tight memory.
pVbExtSamp14VTLogitBoost
pVbExtSamp13VTLogitBoost, but record the the node gain and use all the classes & examples to update node values.
For parameters see batch_vb6_samp_boost.
pAvgSampVTLogitBoost
pVbExtSamp14VTLogitBoost. Estimate w matrix by averaging. For parameters see batch_avgsamp_boost
pAOSOLogitBoostV2 (stable)
pAOSOLogitBoost with weight pruning and feature subsampling
pAOSOGradBoost
AOSO Boosting with gradient descent
pAOSOLogitBoostV2Vb
pAOSOLogitBoostV2, verbose. Record leaf-to-instance mapping.
pAOSOGradBoostVb
pAOSOGradBoost, verbose. Record leaf-to-instance mapping.
pAOSOMARTVb
pAOSOLogitBoostV2Vb, but using Least Square Fitting of gradient when growing a tree (one step Newton when fitting leaf value).