Add transformer base class and adapt preprocessor / converter to fit + apply #4285

vinx13 · 2018-05-14T14:49:24Z

add transformer base class and make converter / preprocessor its direct subclasses
adapt CPreprocessor::init(CFeatures *) to fit, possibly call cleanup in fit if already fitted to support re-fit
cleanup type check in init of preprocessors, use CFeatures::as instead in apply phase
add parameter 'inplace' (default true) to preprocessor / converter :: apply, in which case transform in place if possible
drop CDimensionReductionPreprocessor since it is only a wrapper of converters and provides irrelevant things (kernel, distance, ..) for subtypes.
subclasses of CDimensionReductionPreprocessor inherit directly from `CDensePreprocessor<float64_t>
Make KernelPCA inherit from CPreprocessor since it can work with other types of features
refactor ICA converters into fit + apply

vigsterkr

great start!

vigsterkr · 2018-05-14T14:53:38Z

src/shogun/transformer/Transformer.h

+		}
+
+		/** Fit transformer to features */
+		virtual void fit(CFeatures* features)


this could be a pure virtual function in order to be an abstract class, or?

some transformers actually do not need fitting, so i leave this a no-op

vigsterkr · 2018-05-15T06:02:13Z

src/shogun/preprocessor/HomogeneousKernelMap.cpp

@@ -163,7 +156,7 @@ void CHomogeneousKernelMap::init()

 SGMatrix<float64_t> CHomogeneousKernelMap::apply_to_feature_matrix (CFeatures* features)
 {
-	CDenseFeatures<float64_t>* simple_features = (CDenseFeatures<float64_t>*)features;
+	auto simple_features = features->as<CDenseFeatures<float64_t>>();


needs a quickfix to check whether it has been trained or not... :)

vigsterkr · 2018-05-15T06:02:37Z

src/shogun/preprocessor/KernelPCA.cpp

-		    kernel_matrix, eigenvalues, eigenvectors, m_target_dim);
+	SGVector<float64_t> bias_tmp = linalg::rowwise_sum(kernel_matrix);
+	linalg::scale(bias_tmp, bias_tmp, -1.0 / n);
+	float64_t s = linalg::sum(bias_tmp) / n;


vigsterkr · 2018-05-15T06:02:53Z

src/shogun/preprocessor/KernelPCA.cpp

-		SGMatrix<float64_t> eigenvectors(kernel_matrix.num_rows, m_target_dim);
-		linalg::eigen_solver_symmetric(
-		    kernel_matrix, eigenvalues, eigenvectors, m_target_dim);
+	SGVector<float64_t> bias_tmp = linalg::rowwise_sum(kernel_matrix);


vigsterkr · 2018-05-15T06:04:43Z

src/shogun/preprocessor/PruneVarSubMean.cpp

-		float64_t* var=SG_MALLOC(float64_t, num_features);
-		int32_t i,j;
+	m_mean.resize_vector(num_features);
+	float64_t* var = SG_MALLOC(float64_t, num_features);


sgvector could be used, or?

vigsterkr · 2018-05-15T06:07:42Z

src/shogun/preprocessor/PruneVarSubMean.cpp

-		int32_t num_ok=0;
-		int32_t* idx_ok=SG_MALLOC(int32_t, num_features);
+	int32_t num_ok = 0;
+	int32_t* idx_ok = SG_MALLOC(int32_t, num_features);


vigsterkr · 2018-05-15T06:08:49Z

src/shogun/preprocessor/SortUlongString.cpp


 	for (i=0; i<num_vec; i++)
 	{
 		int32_t len=0;
 		bool free_vec;
-		uint64_t* vec=((CStringFeatures<uint64_t>*)f)->
-			get_feature_vector(i, len, free_vec);
+		uint64_t* vec = sf->get_feature_vector(i, len, free_vec);


vigsterkr · 2018-05-15T06:08:58Z

src/shogun/preprocessor/SortUlongString.cpp

@@ -56,14 +47,14 @@ bool CSortUlongString::save(FILE* f)
 bool CSortUlongString::apply_to_string_features(CFeatures* f)
 {
 	int32_t i;
-	int32_t num_vec=((CStringFeatures<uint64_t>*)f)->get_num_vectors();
+	auto sf = f->as<CStringFeatures<uint64_t>>();
+	int32_t num_vec = sf->get_num_vectors();


vigsterkr · 2018-05-15T06:09:09Z

src/shogun/preprocessor/SortWordString.cpp

@@ -56,13 +47,14 @@ bool CSortWordString::save(FILE* f)
 bool CSortWordString::apply_to_string_features(CFeatures* f)
 {
 	int32_t i;
-	int32_t num_vec=((CStringFeatures<uint16_t>*)f)->get_num_vectors() ;
+	auto sf = f->as<CStringFeatures<uint16_t>>();
+	int32_t num_vec = sf->get_num_vectors();


vigsterkr · 2018-05-15T06:09:16Z

src/shogun/preprocessor/SortWordString.cpp


 	for (i=0; i<num_vec; i++)
 	{
 		int32_t len = 0 ;
 		bool free_vec;
-		uint16_t* vec = ((CStringFeatures<uint16_t>*)f)->get_feature_vector(i, len, free_vec);
+		uint16_t* vec = sf->get_feature_vector(i, len, free_vec);


Implement apply in PCA and FisherLDA Inherit directly from dense preproc

Drop DimensionReductionPreprocessor in swig Guard deprecated in swig Remove deprecated flag for apply_to_vector

karlnapf · 2018-05-21T09:48:38Z

This is a cool push!

…ures

vigsterkr

initial review

vigsterkr · 2018-05-29T07:50:58Z

examples/meta/src/regression/least_angle_regression.sg

-Normalize.apply_to_feature_matrix(features_train)
-Normalize.apply_to_feature_matrix(features_test)	
+SubMean.fit(features_train)
+Features features_train1 = SubMean.apply(features_train)


a more 'representative' naming would be more desirable :P than just features_train1

vigsterkr · 2018-05-29T07:51:12Z

examples/meta/src/regression/least_angle_regression.sg

+Features features_train1 = SubMean.apply(features_train)
+Features features_test1 = SubMean.apply(features_test)
+Normalize.fit(features_train)
+Features features_train2 = Normalize.apply(features_train1)


maybe a better name would be normalized_features_train/test

vigsterkr · 2018-05-29T07:54:46Z

src/shogun/converter/ica/FFSep.cpp

 {
 	ASSERT(features);
 	SG_REF(features);

-	SGMatrix<float64_t> X = ((CDenseFeatures<float64_t>*)features)->get_feature_matrix();
+	auto X = features->as<CDenseFeatures<float64_t>>()->get_feature_matrix();


we could do before this an assertion for get_feature_class() == C_DENSE just to have a sane error message?

although .as has a quite good message as well...

vigsterkr · 2018-05-29T07:56:33Z

src/shogun/converter/ica/ICAConverter.cpp

@@ -59,3 +61,26 @@ float64_t CICAConverter::get_tol() const
 	return tol;
 }

+CFeatures* CICAConverter::apply(CFeatures* features, bool inplace)
+{
+	REQUIRE(m_mixing_matrix.matrix, "ICAConverter not fitted.");


ICAConverter has not been fitted.

vigsterkr · 2018-05-29T08:04:29Z

src/shogun/converter/ica/ICAConverter.h

 		 */
-		virtual CFeatures* apply(CFeatures* features) = 0;
+		virtual void fit(CFeatures* features) = 0;


if we know that this class is an abstract class for other classes that does conversion over CDenseFeatures<float64_t> then here it would be good to actually do the type mapping, meaning not to have this function pure virtual but implement something like this:

virtual void fit(CFeatures* features) { REQUIRE(features->get_feature_class() == C_DENSE, "ICA converters only work with dense features") fit((CDenseFeatures<float64_t>*)features); }

and have a pure virtual protected member function that has a declaration:

virtual void fit(CDenseFeatures<float64_t>*) = 0;

this way the casting thing you need to do it in one place, all the other places you can already assume CDenseFeatures<float64_t> as input.

vigsterkr · 2018-05-29T08:21:04Z

src/shogun/preprocessor/StringPreprocessor.cpp

+namespace shogun
+{
+
+	template <>


why move this into implementation ?

the header is mixed with declaration and implementation, so i split into .h and .cpp as they are done in other template classes, e.g. DensePreprocessor

vigsterkr · 2018-05-29T08:22:21Z

src/shogun/preprocessor/StringPreprocessor.h

+		[[deprecated]]
+#endif
+		    virtual bool
+		    apply_to_string_features(CFeatures* f);


whitespace problem.

vigsterkr · 2018-05-29T08:25:06Z

src/shogun/transformer/Transformer.h

+		 *	@param inplace whether transform in place
+		 *	@return the result feature object after applying the transformer
+		 */
+		virtual CFeatures* apply(CFeatures* features, bool inplace = true) = 0;


if this is pure virtual, then surely fit could be as well?
i'm not sure if i follow the logic here why fit has a default implementation and why the apply does not or vice versa.

there are many transformers that do not need fitting, we don't need to put a empty fit in every subclasses, but only one place here
but as for apply, it should always be implemented (except those abstract classes)

ah yeah i remember now... sorry short memory :(

mmm actually i've realised... dont we actually wanna follow actual transformer api, meaning
.fit and .transform ? :)

vigsterkr · 2018-05-29T08:26:46Z

examples/undocumented/python/preprocessor_sortwordstring.py

-	feats_train.apply_preprocessor()
+	preproc = SortWordString()
+	preproc.fit(feats_train)
+        feats_train = preproc.apply(feats_train)


vigsterkr · 2018-05-29T08:28:00Z

src/interfaces/swig/Preprocessor.i

@@ -16,7 +16,6 @@
 %rename(PNorm) CPNorm;
 %rename(RescaleFeatures) CRescaleFeatures;

-%rename(DimensionReductionPreprocessor) CDimensionReductionPreprocessor;


why do we drop CDimensionReductionPreprocessor? :)

CDimensionReductionPreprocessor is dropped from both cpp and interface.
It has fields like m_converter, m_kernel, m_distance but they are not used by subclasses it most cases (e.g PCA). So it is not very helpful as a superclass. And we don't need to wrap a converter with it now

vigsterkr · 2018-05-30T04:45:43Z

have u seen these @vinx13

The following tests FAILED:
	 84 - python_legacy-distance_canberraword (SEGFAULT)
	 86 - python_legacy-distance_hammingword (SEGFAULT)
	 87 - python_legacy-distance_manhattenword (OTHER_FAULT)
	137 - python_legacy-kernel_comm_ulong_string (SEGFAULT)
	138 - python_legacy-kernel_comm_word_string (SEGFAULT)
	180 - python_legacy-kernel_weighted_comm_word_string (Failed)
	207 - python_legacy-preprocessor_sortulongstring (SEGFAULT)
	208 - python_legacy-preprocessor_sortwordstring (SEGFAULT)
	214 - python_legacy-serialization_string_kernels (SEGFAULT)
	227 - python_legacy-tests_check_commwordkernel_memleak (SEGFAULT)
Errors while running CTest

karlnapf · 2018-05-30T13:45:43Z

examples/meta/src/converter/independent_component_analysis_jade.sg

@@ -13,7 +13,8 @@ ica.set_tol(0.00001)
 #![set_parameters]

 #![apply_convert]
-Features converted = ica.apply(feats)
+ica.fit(feats)


I like the new API here!

vinx13 added 2 commits May 14, 2018 21:39

Add transformer base class

c4d77f9

Make converter and preprocessor subclasses of transformer

bf42a2b

vinx13 force-pushed the feature/transformers branch from 187ce2c to e813594 Compare May 14, 2018 15:03

vinx13 added 5 commits May 15, 2018 11:11

Cleanup dense preprocessors and rename init to fit

5f838b6

Cleanup string preprocessors and rename init to fit

76eb698

Cleanup pca preprocessors and rename init to fit

5ff86f3

Cleanup and rename api in DependenceMaximization

c898878

Update meta examples using 'fit' api

73bc7ec

vinx13 force-pushed the feature/transformers branch from e813594 to 73bc7ec Compare May 15, 2018 03:12

vigsterkr requested changes May 15, 2018

View reviewed changes

vinx13 added 15 commits May 15, 2018 14:18

Update FisherLDA api

3168500

Update doc of FisherLDA

140c8c2

Cleanup preprocessors

8c0cb93

Cleanup preprocessors

5b48af8

Add transformer to swig

03b6061

Add transformer::apply(features, inplace) api

521d12c

Adapt python examples to new transformer api

5cdd269

Fix indent in python examples

cd738f9

Deprecate densepreproc old api add apply_to_matrix

4c57338

Cleanup and implement apply_to_matrix in dense preproc subclasses

7c4ace0

Cleanup and refactor PCA and FisherLDA

62dc621

Implement apply in PCA and FisherLDA Inherit directly from dense preproc

Remove DimensionReductionPreprocessor

ff8fd5f

Implement apply in kernel pca

a423241

Fix swig

04c0884

Drop DimensionReductionPreprocessor in swig Guard deprecated in swig Remove deprecated flag for apply_to_vector

Split template implementation of string preproc

49e9f25

vinx13 added 4 commits May 22, 2018 00:56

Implement apply api in string preproc, deprecate apply_to_string_feat…

5ecf291

…ures

Implement apply in sparse preproc

8cd24e3

Use fwd declaration

715bdb7

Drop DimensionReductionPreproc in python example

3d72162

vinx13 added 7 commits May 25, 2018 21:02

Refactor ICA converters into fit + apply

4f486a6

Some-ize and use fit / apply api in tests and meta examples of ica

cf2904c

Use transformer::apply in unit tests

2c4b067

Some-ize PCA test

34a7958

Don't create new features instance in StringPreproc

7f9ca1a

Some-ize unittests

069df5d

Apply formatter

c30e7e6

vinx13 changed the title ~~[WIP] Add transformer base class and adapt preprocessor / converter to fit + apply~~ Add transformer base class and adapt preprocessor / converter to fit + apply May 27, 2018

vinx13 added 3 commits May 28, 2018 10:39

Fix ica converters

a83ee08

Don't ref the result features in preproc

3dc409e

Some-ize lars unittests to fix ref count

2bfc4b5

vigsterkr requested changes May 29, 2018

View reviewed changes

vinx13 added 9 commits May 29, 2018 17:54

Fix indent

e7ed858

Use std min/max instead of cmath

e52712e

Use better var name in meta example

2e7a488

Minor codestyle improvement

560cc99

Fix PruneVarSubMean

88aa5a5

Convert to dense feats in ica base class

4793cf4

Use 'override' in preproc

aaeea9b

Use std::copy

fc49cd2

Remove 'override' as inconsistent override cause many warnings

f0a29c1

vigsterkr approved these changes May 29, 2018

View reviewed changes

vinx13 added 2 commits May 30, 2018 00:26

Fix style

b927fc1

Rename transformer::apply -> transform

26e5224

vinx13 added 2 commits May 30, 2018 15:25

Fix transformer rename in swig

2937f88

Add %newobject to transformer in swig

8493511

vigsterkr merged commit d2baac3 into shogun-toolbox:feature/transformers May 30, 2018

karlnapf reviewed May 30, 2018

View reviewed changes

samdbrice added the Tag: GSoC label Sep 6, 2019

Add transformer base class and adapt preprocessor / converter to fit + apply #4285

Add transformer base class and adapt preprocessor / converter to fit + apply #4285

vinx13 commented May 14, 2018 •

edited

vigsterkr left a comment

vigsterkr May 14, 2018

vinx13 May 15, 2018

vigsterkr May 15, 2018

vigsterkr May 15, 2018

vigsterkr May 15, 2018

vigsterkr May 15, 2018

vigsterkr May 15, 2018

vigsterkr May 15, 2018

vigsterkr May 15, 2018

vigsterkr May 15, 2018

vigsterkr May 15, 2018

karlnapf commented May 21, 2018

vigsterkr left a comment

vigsterkr May 29, 2018

vigsterkr May 29, 2018

vigsterkr May 29, 2018

vigsterkr May 29, 2018

vigsterkr May 29, 2018

vigsterkr May 29, 2018

vigsterkr May 29, 2018

vinx13 May 29, 2018

vigsterkr May 29, 2018

vigsterkr May 29, 2018

vigsterkr May 29, 2018

vinx13 May 29, 2018

vigsterkr May 29, 2018

vigsterkr May 29, 2018

vigsterkr May 29, 2018

vigsterkr May 29, 2018

vinx13 May 29, 2018

vigsterkr May 29, 2018

vigsterkr commented May 30, 2018

karlnapf May 30, 2018

Add transformer base class and adapt preprocessor / converter to fit + apply #4285

Add transformer base class and adapt preprocessor / converter to fit + apply #4285

Conversation

vinx13 commented May 14, 2018 • edited

vigsterkr left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

karlnapf commented May 21, 2018

vigsterkr left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

vigsterkr commented May 30, 2018

Choose a reason for hiding this comment

vinx13 commented May 14, 2018 •

edited