Ref-counting / memory error in CombinedKernel

[karlnapf]

The following listing illustrates the problem. If the last statement is remove, Python segfaults on exit as it tries to destroy the already de-allocated f object

import shogun as sg

import numpy as np
x_tr = np.random.randn(10, 2)

f = sg.RealFeatures(x_tr)
feats_train = sg.CombinedFeatures()
feats_train.append_feature_obj( f)

kernel = sg.CombinedKernel()
kernel.append_kernel(sg.GaussianKernel(3.0))
kernel.append_kernel(sg.PolyKernel(10, 1))

kernel.init(feats_train, feats_train)

sg.get_global_io().set_loglevel(0)
del kernel
print f # segfaults as object's refcounter decreases to zero when destroying the kernel

See also #4410 for the first report

[karlnapf]

Interestingly, the above listing doesnt segfault when only one kernel is added to the combined kernel

[zym-wade]

Interestingly, the above listing doesnt segfault when only one kernel is added to the combined kernel
I am sorry, I just came back to school. This problem is still not solved. I tried your demo and it will display the following error.

[GCDEBUG] unref() refcount 3 obj DenseFeatures (0x16034f0) decreased
[GCDEBUG] unref() refcount 0, obj CombinedKernel (0x1bea950) destroying
[GCDEBUG] ref() refcount 2 obj GaussianKernel (0x1bee9c0) increased
[GCDEBUG] unref() refcount 2 obj DenseFeatures (0x16034f0) decreased
[GCDEBUG] unref() refcount 1 obj GaussianKernel (0x1bee9c0) decreased
[GCDEBUG] ref() refcount 2 obj PolyKernel (0x1bf1980) increased
[GCDEBUG] unref() refcount 1 obj DenseFeatures (0x16034f0) decreased
[GCDEBUG] unref() refcount 1 obj PolyKernel (0x1bf1980) decreased
[GCDEBUG] ref() refcount 2 obj GaussianKernel (0x1bee9c0) increased
[GCDEBUG] unref() refcount 1 obj GaussianKernel (0x1bee9c0) decreased
[GCDEBUG] ref() refcount 2 obj PolyKernel (0x1bf1980) increased
[GCDEBUG] unref() refcount 1 obj PolyKernel (0x1bf1980) decreased
[GCDEBUG] ref() refcount 2 obj GaussianKernel (0x1bee9c0) increased
[GCDEBUG] unref() refcount 1 obj GaussianKernel (0x1bee9c0) decreased
[GCDEBUG] ref() refcount 2 obj PolyKernel (0x1bf1980) increased
[GCDEBUG] unref() refcount 1 obj PolyKernel (0x1bf1980) decreased
[GCDEBUG] ref() refcount 2 obj GaussianKernel (0x1bee9c0) increased
[GCDEBUG] unref() refcount 1 obj GaussianKernel (0x1bee9c0) decreased
[GCDEBUG] ref() refcount 2 obj PolyKernel (0x1bf1980) increased
[GCDEBUG] unref() refcount 1 obj PolyKernel (0x1bf1980) decreased
[GCDEBUG] unref() refcount 1 obj CombinedFeatures (0x1bc7ca0) decreased
[GCDEBUG] unref() refcount 0, obj DynamicObjectArray (0x1bec7e0) destroying
[GCDEBUG] unref() refcount 0, obj GaussianKernel (0x1bee9c0) destroying
[GCDEBUG] unref() refcount 0, obj EuclideanDistance (0x1bf09d0) destroying
[GCDEBUG] unref() refcount 0, obj DenseFeatures (0x16034f0) destroying
[GCDEBUG] unref() refcount 0, obj SubsetStack (0x1b19ac0) destroying
[GCDEBUG] unref() refcount 0, obj DynamicObjectArray (0x1b4c5c0) destroying
[GCDEBUG] SGObject destroyed (0x1b4c5c0)
[GCDEBUG] SGObject destroyed (0x1b19ac0)
[GCDEBUG] unref() refcount 0, obj DynamicObjectArray (0x1b2bc70) destroying
[GCDEBUG] SGObject destroyed (0x1b2bc70)
[GCDEBUG] SGObject destroyed (0x16034f0)
段错误 (核心已转储)

[karlnapf]

Yes the bug is still open.

[zym-wade]

Yes the bug is still open.

So now the two-category multi-core learning can't be used, is there any other way to use it?

[karlnapf]

This example doesnt work?
http://www.shogun-toolbox.org/examples/latest/examples/regression/multiple_kernel_learning.html
It might be excluded of our test build due to license issues...but it should still work

And then there is:
http://www.shogun-toolbox.org/notebook/latest/MKL.html

[rrkarim]

I can work on this. Maybe you already have some ideas where the problem comes from, @karlnapf @zym-wade ?

[karlnapf]

You are more than welcome to do.... the above snippet will reproduce the error. We don't know more atm

[rrkarim]

@karlnapf ok, tho I've checked that for C snippet like:

int main() {
	init_shogun_with_defaults();

	auto f_feats_train = some<CCSVFile>("../../data/classifier_4class_2d_linear_features_train.dat");

	auto f = some<CDenseFeatures<float64_t>>(f_feats_train); // real_features
	auto features_train = new CCombinedFeatures(); 
	features_train->append_feature_obj(f);

	auto poly_kernel = some<CPolyKernel>(10, 1);
	auto gauss_kernel = some<CGaussianKernel>(3.0);

	auto combined_kernel = some<CCombinedKernel>();
	combined_kernel->append_kernel(gauss_kernel);
	combined_kernel->append_kernel(poly_kernel);

	combined_kernel->init(features_train, features_train);
	delete combined_kernel;

	return 0;
}

error is not raised. It is obvious that C++ internal object is destroyed and python just don't see it. I don't know the internals quite good for now so I need more time to analyze it.

[karlnapf]

no need to delete when you use some

[karlnapf]

But yeah, this will be interface related probably, not c++ lib related