LASSO Implementation: Condition bug, missing normalization documentation, and intercept fitting option

[georeth]

Bug: Incorrect condition for dimension check

smartcore/src/linear/lasso.rs

Lines 249 to 253 in 0caa830

	if n <= p {
	return Err(Failed::fit(
	"Number of rows in X should be >= number of columns in X",
	));
	}

The condition n <= p is inconsistent with the error message, it should be n < p. (my case is full-rank)

L1 penalty normalization not documented

smartcore/src/linear/lasso.rs

Line 12 in 0caa830

//! \\[\underset{\beta}{minimize} \space \space \sum_{i=1}^n \left( y_i - \beta_0 - \sum_{j=1}^p \beta_jx_{ij} \right)^2 + \alpha \sum_{j=1}^p \lVert \beta_j \rVert_1\\]

the penalty in docs is alpha * sum(abs(beta)), but I think it's alpha * n * sum(abs(beta)) in the code, where n is the number of observations.

Feature Request: Option to disable intercept fitting

smartcore/src/linear/lasso_optimizer.rs

Line 64 in 0caa830

let y = y.sub_scalar(T::from_f64(y.mean_by()).unwrap());

Currently, the mean is always substracted from y. Is it possible to add a parameter (e.g., fit_intercept: bool, or more specific substract_mean_from_y: bool) to allow users to disable intercept fitting? (force the beta_0 parameter in the formula to 0)

[Mec-iS]

thank you for reporting this issue.

[georeth]

sklearn lasso uses (1 / (2 * n_samples)) * ||y - Xw||^2_2 + alpha * ||w||_1 as loss function, and has a fit_intercept option. smartcore uses (1 / n_samples) * ||y - Xw||^2_2 + alpha * ||w||_1.

I need to solve a not well-conditioned, full-rank LASSO without intercept term. sklearn's coordinate descent failed to converge for my problem. I am trying to make smartcore solve LASSO without intercept term, but not familiar with interior point method.

[Mec-iS]

As you are the one using it, you are probably in the best position to fix this.
If you have any question regarding smartcore internals and how to use them for this particular problem, I would be glad to answer.

[georeth]

I have raised a PR for the first two points.
For the third one, I verified by experiments that it converges the desired result if I comment out the code to normalize y. I am not sure whether other changes should be made to the interior point algorithm. (in other words, I doubt if the algorithm depends on the fact that mean of y is zero.)
I will raise this in a PR after the first PR is accepted.

[Mec-iS]

thanks, i will check that