Add transformer base class and adapt preprocessor / converter to fit + apply #4285

vinx13 · May 14, 2018

add transformer base class and make converter / preprocessor its direct subclasses
adapt CPreprocessor::init(CFeatures *) to fit, possibly call cleanup in fit if already fitted to support re-fit
cleanup type check in init of preprocessors, use CFeatures::as instead in apply phase
add parameter 'inplace' (default true) to preprocessor / converter :: apply, in which case transform in place if possible
drop CDimensionReductionPreprocessor since it is only a wrapper of converters and provides irrelevant things (kernel, distance, ..) for subtypes.
subclasses of CDimensionReductionPreprocessor inherit directly from `CDensePreprocessor<float64_t>
Make KernelPCA inherit from CPreprocessor since it can work with other types of features
refactor ICA converters into fit + apply

vigsterkr · May 15, 2018

vigsterkr requested changes May 15, 2018

View changes

great start!

vigsterkr · May 15, 2018

src/shogun/transformer/Transformer.h

+		}
+
+		/** Fit transformer to features */
+		virtual void fit(CFeatures* features)


this could be a pure virtual function in order to be an abstract class, or?

some transformers actually do not need fitting, so i leave this a no-op

vigsterkr · May 15, 2018

src/shogun/preprocessor/HomogeneousKernelMap.cpp

@@ -163,7 +156,7 @@ void CHomogeneousKernelMap::init()

 SGMatrix<float64_t> CHomogeneousKernelMap::apply_to_feature_matrix (CFeatures* features)
 {
-	CDenseFeatures<float64_t>* simple_features = (CDenseFeatures<float64_t>*)features;
+	auto simple_features = features->as<CDenseFeatures<float64_t>>();


needs a quickfix to check whether it has been trained or not... :)

vigsterkr · May 15, 2018

src/shogun/preprocessor/KernelPCA.cpp

-		    kernel_matrix, eigenvalues, eigenvectors, m_target_dim);
+	SGVector<float64_t> bias_tmp = linalg::rowwise_sum(kernel_matrix);
+	linalg::scale(bias_tmp, bias_tmp, -1.0 / n);
+	float64_t s = linalg::sum(bias_tmp) / n;


vigsterkr · May 15, 2018

src/shogun/preprocessor/KernelPCA.cpp

-		SGMatrix<float64_t> eigenvectors(kernel_matrix.num_rows, m_target_dim);
-		linalg::eigen_solver_symmetric(
-		    kernel_matrix, eigenvalues, eigenvectors, m_target_dim);
+	SGVector<float64_t> bias_tmp = linalg::rowwise_sum(kernel_matrix);


vigsterkr · May 15, 2018

src/shogun/preprocessor/PruneVarSubMean.cpp

-		float64_t* var=SG_MALLOC(float64_t, num_features);
-		int32_t i,j;
+	m_mean.resize_vector(num_features);
+	float64_t* var = SG_MALLOC(float64_t, num_features);


sgvector could be used, or?

vigsterkr · May 15, 2018

src/shogun/preprocessor/PruneVarSubMean.cpp

-		int32_t num_ok=0;
-		int32_t* idx_ok=SG_MALLOC(int32_t, num_features);
+	int32_t num_ok = 0;
+	int32_t* idx_ok = SG_MALLOC(int32_t, num_features);


vigsterkr · May 15, 2018

src/shogun/preprocessor/SortUlongString.cpp


 	for (i=0; i<num_vec; i++)
 	{
 		int32_t len=0;
 		bool free_vec;
-		uint64_t* vec=((CStringFeatures<uint64_t>*)f)->
-			get_feature_vector(i, len, free_vec);
+		uint64_t* vec = sf->get_feature_vector(i, len, free_vec);


vigsterkr · May 15, 2018

src/shogun/preprocessor/SortUlongString.cpp

@@ -56,14 +47,14 @@ bool CSortUlongString::save(FILE* f)
 bool CSortUlongString::apply_to_string_features(CFeatures* f)
 {
 	int32_t i;
-	int32_t num_vec=((CStringFeatures<uint64_t>*)f)->get_num_vectors();
+	auto sf = f->as<CStringFeatures<uint64_t>>();
+	int32_t num_vec = sf->get_num_vectors();


vigsterkr · May 15, 2018

src/shogun/preprocessor/SortWordString.cpp

@@ -56,13 +47,14 @@ bool CSortWordString::save(FILE* f)
 bool CSortWordString::apply_to_string_features(CFeatures* f)
 {
 	int32_t i;
-	int32_t num_vec=((CStringFeatures<uint16_t>*)f)->get_num_vectors() ;
+	auto sf = f->as<CStringFeatures<uint16_t>>();
+	int32_t num_vec = sf->get_num_vectors();


vigsterkr · May 15, 2018

src/shogun/preprocessor/SortWordString.cpp


 	for (i=0; i<num_vec; i++)
 	{
 		int32_t len = 0 ;
 		bool free_vec;
-		uint16_t* vec = ((CStringFeatures<uint16_t>*)f)->get_feature_vector(i, len, free_vec);
+		uint16_t* vec = sf->get_feature_vector(i, len, free_vec);


vigsterkr · May 16, 2018

note the indenting!

karlnapf · May 21, 2018

whitespace

karlnapf · May 21, 2018

This is a cool push!

vigsterkr · May 29, 2018

vigsterkr requested changes May 29, 2018

View changes

initial review

vigsterkr · May 29, 2018

examples/meta/src/regression/least_angle_regression.sg

-Normalize.apply_to_feature_matrix(features_train)
-Normalize.apply_to_feature_matrix(features_test)	
+SubMean.fit(features_train)
+Features features_train1 = SubMean.apply(features_train)


a more 'representative' naming would be more desirable :P than just features_train1

vigsterkr · May 29, 2018

examples/meta/src/regression/least_angle_regression.sg

+Features features_train1 = SubMean.apply(features_train)
+Features features_test1 = SubMean.apply(features_test)
+Normalize.fit(features_train)
+Features features_train2 = Normalize.apply(features_train1)


maybe a better name would be normalized_features_train/test

vigsterkr · May 29, 2018

src/shogun/converter/ica/FFSep.cpp

 {
 	ASSERT(features);
 	SG_REF(features);

-	SGMatrix<float64_t> X = ((CDenseFeatures<float64_t>*)features)->get_feature_matrix();
+	auto X = features->as<CDenseFeatures<float64_t>>()->get_feature_matrix();


we could do before this an assertion for get_feature_class() == C_DENSE just to have a sane error message?

although .as has a quite good message as well...

vigsterkr · May 29, 2018

src/shogun/converter/ica/ICAConverter.cpp

@@ -59,3 +61,26 @@ float64_t CICAConverter::get_tol() const
 	return tol;
 }

+CFeatures* CICAConverter::apply(CFeatures* features, bool inplace)
+{
+	REQUIRE(m_mixing_matrix.matrix, "ICAConverter not fitted.");


ICAConverter has not been fitted.

vigsterkr · May 29, 2018

src/shogun/converter/ica/ICAConverter.h

 		 */
-		virtual CFeatures* apply(CFeatures* features) = 0;
+		virtual void fit(CFeatures* features) = 0;


if we know that this class is an abstract class for other classes that does conversion over CDenseFeatures<float64_t> then here it would be good to actually do the type mapping, meaning not to have this function pure virtual but implement something like this:

virtual void fit(CFeatures* features) { REQUIRE(features->get_feature_class() == C_DENSE, "ICA converters only work with dense features") fit((CDenseFeatures<float64_t>*)features); }

and have a pure virtual protected member function that has a declaration:

virtual void fit(CDenseFeatures<float64_t>*) = 0;

this way the casting thing you need to do it in one place, all the other places you can already assume CDenseFeatures<float64_t> as input.

vigsterkr · May 29, 2018

src/shogun/preprocessor/StringPreprocessor.cpp

+namespace shogun
+{
+
+	template <>


why move this into implementation ?

the header is mixed with declaration and implementation, so i split into .h and .cpp as they are done in other template classes, e.g. DensePreprocessor

vigsterkr · May 29, 2018

src/shogun/preprocessor/StringPreprocessor.h

+		[[deprecated]]
+#endif
+		    virtual bool
+		    apply_to_string_features(CFeatures* f);


whitespace problem.

vigsterkr · May 29, 2018

src/shogun/transformer/Transformer.h

+		 *	@param inplace whether transform in place
+		 *	@return the result feature object after applying the transformer
+		 */
+		virtual CFeatures* apply(CFeatures* features, bool inplace = true) = 0;


if this is pure virtual, then surely fit could be as well?
i'm not sure if i follow the logic here why fit has a default implementation and why the apply does not or vice versa.

there are many transformers that do not need fitting, we don't need to put a empty fit in every subclasses, but only one place here
but as for apply, it should always be implemented (except those abstract classes)

ah yeah i remember now... sorry short memory :(

mmm actually i've realised... dont we actually wanna follow actual transformer api, meaning
.fit and .transform ? :)

vigsterkr · May 29, 2018

examples/undocumented/python/preprocessor_sortwordstring.py

-	feats_train.apply_preprocessor()
+	preproc = SortWordString()
+	preproc.fit(feats_train)
+        feats_train = preproc.apply(feats_train)


vigsterkr · May 29, 2018

src/interfaces/swig/Preprocessor.i

@@ -16,7 +16,6 @@
 %rename(PNorm) CPNorm;
 %rename(RescaleFeatures) CRescaleFeatures;

-%rename(DimensionReductionPreprocessor) CDimensionReductionPreprocessor;


why do we drop CDimensionReductionPreprocessor? :)

CDimensionReductionPreprocessor is dropped from both cpp and interface.
It has fields like m_converter, m_kernel, m_distance but they are not used by subclasses it most cases (e.g PCA). So it is not very helpful as a superclass. And we don't need to wrap a converter with it now

vigsterkr · May 29, 2018

vigsterkr approved these changes May 29, 2018

View changes

vigsterkr · May 30, 2018

have u seen these @vinx13

The following tests FAILED:
	 84 - python_legacy-distance_canberraword (SEGFAULT)
	 86 - python_legacy-distance_hammingword (SEGFAULT)
	 87 - python_legacy-distance_manhattenword (OTHER_FAULT)
	137 - python_legacy-kernel_comm_ulong_string (SEGFAULT)
	138 - python_legacy-kernel_comm_word_string (SEGFAULT)
	180 - python_legacy-kernel_weighted_comm_word_string (Failed)
	207 - python_legacy-preprocessor_sortulongstring (SEGFAULT)
	208 - python_legacy-preprocessor_sortwordstring (SEGFAULT)
	214 - python_legacy-serialization_string_kernels (SEGFAULT)
	227 - python_legacy-tests_check_commwordkernel_memleak (SEGFAULT)
Errors while running CTest

karlnapf · May 30, 2018

karlnapf reviewed May 30, 2018

View changes

karlnapf · May 30, 2018

examples/meta/src/converter/independent_component_analysis_jade.sg

@@ -13,7 +13,8 @@ ica.set_tol(0.00001)
 #![set_parameters]

 #![apply_convert]
-Features converted = ica.apply(feats)
+ica.fit(feats)


I like the new API here!

vinx13 added some commits May 14, 2018

Add transformer base class

c4d77f9

Make converter and preprocessor subclasses of transformer

bf42a2b

vinx13 force-pushed the vinx13:feature/transformers branch from 187ce2c to e813594 May 14, 2018

vinx13 force-pushed the vinx13:feature/transformers branch from e813594 to 73bc7ec May 15, 2018

vinx13 added some commits May 21, 2018

Implement apply api in string preproc, deprecate apply_to_string_feat…
…ures

5ecf291

Implement apply in sparse preproc

8cd24e3

vinx13 changed the title from [WIP] Add transformer base class and adapt preprocessor / converter to fit + apply to Add transformer base class and adapt preprocessor / converter to fit + apply May 27, 2018

vinx13 added some commits May 28, 2018

Fix ica converters

a83ee08

Don't ref the result features in preproc

3dc409e

Some-ize lars unittests to fix ref count

Loading status checks…

2bfc4b5

vinx13 added some commits May 29, 2018

Fix style

b927fc1

Rename transformer::apply -> transform

Loading status checks…

26e5224

vinx13 added some commits May 30, 2018

Fix transformer rename in swig

2937f88

Add %newobject to transformer in swig

Loading status checks…

8493511

vigsterkr merged commit d2baac3 into shogun-toolbox:feature/transformers May 30, 2018
1 of 2 checks passed

1 of 2 checks passed

continuous-integration/appveyor/pr Waiting for AppVeyor build to complete
Details

continuous-integration/travis-ci/pr The Travis CI build passed
Details

shogun-toolbox/shogun

Join GitHub today

Add transformer base class and adapt preprocessor / converter to fit + apply #4285

Conversation

vinx13 commented May 14, 2018 • edited

vinx13 added some commits May 14, 2018

vinx13 force-pushed the vinx13:feature/transformers branch from 187ce2c to e813594 May 14, 2018

vinx13 added some commits May 14, 2018

vinx13 force-pushed the vinx13:feature/transformers branch from e813594 to 73bc7ec May 15, 2018

vigsterkr requested changes May 15, 2018 View changes

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

vinx13 added some commits May 15, 2018

This comment has been minimized.

vigsterkr commented on examples/undocumented/python/preprocessor_sortulongstring.py in 5cdd269 May 16, 2018

vinx13 added some commits May 16, 2018

This comment has been minimized.

karlnapf commented on examples/undocumented/python/preprocessor_sortwordstring.py in 5cdd269 May 21, 2018

This comment has been minimized.

karlnapf commented May 21, 2018

vinx13 added some commits May 21, 2018

vinx13 added some commits May 25, 2018

vinx13 changed the title from [WIP] Add transformer base class and adapt preprocessor / converter to fit + apply to Add transformer base class and adapt preprocessor / converter to fit + apply May 27, 2018

vinx13 added some commits May 28, 2018

vigsterkr requested changes May 29, 2018 View changes

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

vinx13 added some commits May 29, 2018

vigsterkr approved these changes May 29, 2018 View changes

vinx13 added some commits May 29, 2018

This comment has been minimized.

vigsterkr commented May 30, 2018

vinx13 added some commits May 30, 2018

Hide details View details vigsterkr merged commit d2baac3 into shogun-toolbox:feature/transformers May 30, 2018 1 of 2 checks passed

1 of 2 checks passed

karlnapf reviewed May 30, 2018 View changes

This comment has been minimized.

vinx13 commented May 14, 2018 •

edited

vinx13 force-pushed the vinx13:feature/transformers branch from `187ce2c` to `e813594` May 14, 2018

vinx13 force-pushed the vinx13:feature/transformers branch from `e813594` to `73bc7ec` May 15, 2018

vigsterkr requested changes May 15, 2018

View changes

vigsterkr commented on `examples/undocumented/python/preprocessor_sortulongstring.py` in `5cdd269` May 16, 2018

karlnapf commented on `examples/undocumented/python/preprocessor_sortwordstring.py` in `5cdd269` May 21, 2018

vigsterkr requested changes May 29, 2018

View changes

vigsterkr approved these changes May 29, 2018

View changes

vigsterkr merged commit `d2baac3` into shogun-toolbox:feature/transformers May 30, 2018
1 of 2 checks passed

karlnapf reviewed May 30, 2018

View changes