ARD and Gradient Search Vector Learning

[puffin444]

This patch supports Gradient Search on Vector elements. An application is the Linear and
Gaussian ARD Kernels. Results seem to be similar to that of GPML

1. In GaussianARD and LinearARD, when I calculate the kernel gradient, should I constantly
   check if the left and right hand features are NULL, or has that been done already in the superclass?
2. What do you think of these two classes? Do you think I can consolidate code through inheritance? They are very similar with the exception of compute() and the gradient calculation.
3. Right now ModelSelectionParameters can generate ParameterCombination through
   get_single_combination. However, it doesn't yet through get_combinations, mostly due to point (4) below. Heiko, do you have any suggestions on integrating vectors/sg_vectors for the get_combinations function?
4. Right now the ModelSelectionParameter framework generates vectors for ParameterCombinations by using a pointer to a vector allocated somewhere else. The problem with this approach is if I wanted to call get_single_combination twice to get two trees. These two trees would have exactly the same vector, and it would be impossible to generate two trees with vectors with different values. Should I try to allocate new vectors in get_single_combination?
5. The Sigma gradient seems to have a large effect on the gradient at small values. This throws off the check_gradient() function at times. Is it possible to adjust the delta and error_tol values based on the actual values of the parameters?

[karlnapf]

btw, if you init shogun with init_with_defaults() you dont need this method, take a look at other examples

[karlnapf]

1.) I think these checks hurt nobody
2.) Yes I think inheritance might be good here, reduces code and therefore error possibilities
3.) not yet, I will think of it
4.) same, lets talk about this soon
5.) dont really get what you are asking here :)

[puffin444]

Okay. Cleaned some stuff up. What do you think?

[karlnapf]

I think its nice!