Optimizing lambda

[suryadi-t]

Hi,

What is a good way to optimize lambda here in cases where the true answer is unknown? I tried using the MSE of a validation set as a guide but upon testing it leads to many false detections. Is there a better way to optimize lambda?

Thank you,
Suryadi

[iancovert]

Hi there, that's a good question. This problem comes up whenever you're training a sparse model, whether it's a cMLP/cLSTM for GC discovery or a lasso regularized linear model, and I think it's usually difficult.

One solution would be: if you have a rough idea of how many connections there should be in your system, tune lambda to find the value that returns that level of sparsity. Another solution would be to try a range of lambda values and plot the MSE that is achieved with each sparsity pattern (by retraining after learning the sparsity pattern, and evaluating the MSE on an validation dataset); on this plot, there may be a point at which using larger lambda values results in much worse MSE, so a reasonable choice would be the highest lambda value that still gives good MSE.

Hopefully that makes sense. This is definitely a challenging aspect of the method.

[suryadi-t]

I see, and yes that makes sense. Thank you for the quick response!