Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Updated cross validation examples to use new API (as far as I know) #4309

Merged
merged 6 commits into from Jun 1, 2018
Merged

Updated cross validation examples to use new API (as far as I know) #4309

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
81c2369
Updated cross validation examples to use new API (as far as I know)
FaroukY May 26, 2018
7d0fcff
Chagned more files to new API
FaroukY May 29, 2018
dcf83b8
user getter + factory instead of obtain from generic
FaroukY May 29, 2018
0fa5e20
fixed the mini review
FaroukY May 29, 2018
da01ef0
added arguments in kwargs instread of using .put(), and changed label…
FaroukY May 30, 2018
8e1d21d
changes labels to labs to get rid of the thrown error
FaroukY May 30, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
37 changes: 18 additions & 19 deletions examples/meta/src/evaluation/cross_validation_multiple_kernel_learning_weights_storage.sg
View file
Open in desktop
@@ -1,36 +1,34 @@
CSVFile f_feats("../../data/fm_train_real.dat")
CSVFile f_labels("../../data/label_train_twoclass.dat")
File f_feats = csv_file("../../data/fm_train_real.dat")
File f_labels = csv_file("../../data/label_train_twoclass.dat")

#![create_features]
Features feats = features(f_feats)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you do the CombinedFeatures and BinaryLabels below with the factories?

CombinedFeatures combined_features()
combined_features.append_feature_obj(feats)
combined_features.append_feature_obj(feats)
combined_features.append_feature_obj(feats)
BinaryLabels labels(f_labels)
Labels labels_train = labels(f_labels)
#![create_features]

#![create_kernels]
CombinedKernel kernel()
GaussianKernel k_1(10, 0.1)
GaussianKernel k_2(10, 1)
GaussianKernel k_3(10, 2)
kernel.append_kernel(k_1)
kernel.append_kernel(k_2)
kernel.append_kernel(k_3)
Kernel combined_kernel = kernel("CombinedKernel")
Kernel k_1 = kernel("GaussianKernel", log_width=0.1)
Kernel k_2 = kernel("GaussianKernel", log_width=1.0)
Kernel k_3 = kernel("GaussianKernel", log_width=2.0)
combined_kernel.add("kernel_array", k_1)
combined_kernel.add("kernel_array", k_2)
combined_kernel.add("kernel_array", k_3)
#![create_kernels]

#![create_classifier]
LibSVM libsvm()
MKLClassification svm(libsvm)
svm.set_interleaved_optimization_enabled(False)
svm.set_kernel(kernel)
Machine libsvm = machine("LibSVM")
Machine svm = machine("MKLClassification", kernel=combined_kernel, interleaved_optimization=False, svm=libsvm)
#![create_classifier]

#![create_cross_validation]
StratifiedCrossValidationSplitting splitting_strategy(labels, 5)
AccuracyMeasure evaluation_criterium()
CrossValidation cross(svm, combined_features, labels, splitting_strategy, evaluation_criterium)
StratifiedCrossValidationSplitting splitting_strategy(labels_train, 5)
Evaluation evaluation_criterium = evaluation("AccuracyMeasure")
CrossValidation cross(svm, combined_features, labels_train, splitting_strategy, evaluation_criterium)
cross.set_autolock(False)
cross.set_num_runs(2)
#![create_cross_validation]
Expand All @@ -52,10 +50,11 @@ real stddev = result.get_real("std_dev")
#![get_fold_machine]
CrossValidationStorage obs = mkl_obs.get_observation(0)
CrossValidationFoldStorage fold = obs.get_fold(0)
MKLClassification machine = MKLClassification:obtain_from_generic(fold.get_trained_machine())
Machine machine_mkl = machine(fold.get_trained_machine())

#![get_fold_machine]

#![get_weights]
CombinedKernel k = CombinedKernel:obtain_from_generic(machine.get_kernel())
Kernel k = kernel(machine_mkl.get("kernel"))
RealVector w = k.get_subkernel_weights()
#![get_weights]
10 changes: 5 additions & 5 deletions examples/meta/src/evaluation/cross_validation_support_vector_machine.sg
View file
Open in desktop
@@ -1,5 +1,5 @@
CSVFile f_feats("../../data/fm_train_real.dat")
CSVFile f_labels("../../data/label_train_twoclass.dat")
File f_feats = csv_file("../../data/fm_train_real.dat")
File f_labels = csv_file("../../data/label_train_twoclass.dat")

Features feats = features(f_feats)
Labels labs = labels(f_labels)
Expand All @@ -8,11 +8,11 @@ Machine svm = machine("LibLinear")
svm.put("liblinear_solver_type", enum LIBLINEAR_SOLVER_TYPE.L2R_L2LOSS_SVC)

StratifiedCrossValidationSplitting splitting_strategy(labs, 5)
AccuracyMeasure evaluation_criterion()
Evaluation evaluation_criterion = evaluation("AccuracyMeasure")
CrossValidation cross(svm, feats, labs, splitting_strategy, evaluation_criterion)
cross.set_num_runs(2)

CrossValidationResult result = CrossValidationResult:obtain_from_generic(cross.evaluate())
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

EvaluationResult result = cross.evaluate() didnt work?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

my mistake, just testing the patch and pushing once done.


real mean = result.get_mean()
real stddev = result.get_std_dev()
real mean = result.get_real("mean")
real stddev = result.get_real("std_dev")
8 changes: 4 additions & 4 deletions examples/meta/src/evaluation/cross_validation_timeseries_split.sg
View file
Open in desktop
@@ -1,13 +1,13 @@
CSVFile f_labels("../../data/label_train_regression.dat")
File f_labels = csv_file("../../data/label_train_regression.dat")

#![create labels]
RegressionLabels labels(f_labels)
Labels reg_labels = labels(f_labels)
#![create labels]

#![set parameters]
int num_subsets = 5
int min_subset_size = 4
TimeSeriesSplitting splitting(labels, num_subsets)
TimeSeriesSplitting splitting(reg_labels, num_subsets)
splitting.set_min_subset_size(min_subset_size)
#![set parameters]

Expand All @@ -18,4 +18,4 @@ splitting.build_subsets()
#![generate subsets and inverse (aka test labels and train labels)]
IntVector test_labels_indices = splitting.generate_subset_indices(1)
IntVector train_labels_indices = splitting.generate_subset_inverse(1)
#![generate subsets and inverse (aka test labels and train labels)]
#![generate subsets and inverse (aka test labels and train labels)]
1 change: 0 additions & 1 deletion examples/meta/src/regression/least_angle_regression.sg
View file
Open in desktop
Expand Up @@ -40,4 +40,3 @@ real mse = eval.evaluate(labels_predict, labels_test)

# integration testing variables
RealVector output = labels_predict.get_real_vector("labels")

12 changes: 6 additions & 6 deletions examples/meta/src/regression/random_forest_regression.sg
View file
Open in desktop
@@ -1,7 +1,7 @@
CSVFile f_feats_train("../../data/regression_1d_linear_features_train.dat")
CSVFile f_feats_test("../../data/regression_1d_linear_features_test.dat")
CSVFile f_labels_train("../../data/regression_1d_linear_labels_train.dat")
CSVFile f_labels_test("../../data/regression_1d_linear_labels_test.dat")
File f_feats_train = csv_file("../../data/regression_1d_linear_features_train.dat")
File f_feats_test = csv_file("../../data/regression_1d_linear_features_test.dat")
File f_labels_train = csv_file("../../data/regression_1d_linear_labels_train.dat")
File f_labels_test = csv_file("../../data/regression_1d_linear_labels_test.dat")
Math:init_random(1)

#![create_features]
Expand All @@ -26,10 +26,10 @@ RegressionLabels labels_predict = rand_forest.apply_regression(features_test)
#![train_and_apply]

#![evaluate_error]
MeanSquaredError mse()
Evaluation mse = evaluation("MeanSquaredError")
real oob = rand_forest.get_oob_error(mse)
real mserror = mse.evaluate(labels_predict, labels_test)
#![evaluate_error]

# additional integration testing variables
RealVector output = labels_predict.get_labels()
RealVector output = labels_predict.get_real_vector("labels")