The regression learns NaN weights with the example code.

[gursimar]

However, if I try a simple model like y= mx+c, then it gives some weights but never gets the y-intercept right. It's always close to zero. I did try playing around with parameters/ increasing data but it still gives the same fit without y-intercept. Seems like there is some bug in the gradient descent method.

Here is the code I tried

<script src="https://rawgit.com/chen0040/js-regression/master/build/jsregression.min.js" type="application/javascript"></script> <script> // === training data generated from y = 2.0 + 5.0 * x + 2.0 * x^2 === // /* var data = []; for(var x = 1.0; x < 100.0; x += 1.0) { var y = 2.0 + 5.0 * x + 2.0 * x * x + Math.random() * 1.0; data.push([x, x * x, y]); // Note that the last column should be y the output } */ var data = []; for(var x = 1.0; x < 10000.0; x += 1.0) { var y = 6.0 + 5.0 * x*0.1 + Math.random() * 1.0; data.push([x*0.1, y]); // Note that the last column should be y the output } // === Create the linear regression === // var regression = new jsregression.LinearRegression({ alpha: 0.00001, // iterations: 30000, lambda: 0.0 }); // can also use default configuration: var regression = new jsregression.LinearRegression(); // === Train the linear regression === // var model = regression.fit(data); // === Print the trained model === // console.log(model); // === Testing the trained linear regression === // /* var testingData = []; for(var x = 1.0; x < 100.0; x += 1.0) { var actual_y = 2.0 + 5.0 * x + 2.0 * x * x + Math.random() * 1.0; var predicted_y = regression.transform([x, x * x]); console.log("actual: " + actual_y + " predicted: " + predicted_y); } */ </script>

[gursimar]

Sorry for the mess in the above comment.
Here is the code - https://pastebin.com/6BmGhcTV

[chen0040]

Hi @gursimar Many thanks for notifying me the issue and sharing with me your code, I will look into this and get back to you :)

[gursimar]

Thanks, I think the issue is with the gradient function.
You are using a numerical method but I suppose we can use a closed form solution.
Right now, I'm trying to do that but I'm using math library to do this.

[chen0040]

Many thanks @gursimar , i think you are right. i believed the gradient descent method is very sensitive the values of the learning rate alpha and the regularization lambda. I played with a few small values of alpha and lambda using your code sample, it did converge with NaN issue disappear (if you set a sufficiently small alpha value) but behavior is quite sensitive to the value of alpha indeed, i will consider the closed form.

[chen0040]

Hi @gursimar after some close examination on the linear regression source code, i notice that the gradient calculation has a bug which i have now addressed. I have also added a number html demo on the linear regression:

https://rawgit.com/chen0040/js-regression/master/example-regression-3.html
https://rawgit.com/chen0040/js-regression/master/example-regression-2.html
https://rawgit.com/chen0040/js-regression/master/example-regression.html

The y-intercept is still not as effective, subject to the learning rate and regularization.

[pacobalt]

Hi @chen0040, I tried the regression with random data like this;

var jsregression = require('js-regression');
var data = [];
for (var i = 0; i<200; i++) {
data.push([i,Math.random()*10*i+20]);
}
var regression = new jsregression.LinearRegression();
var model = regression.fit(data);
console.log(model);

output is the follwing:

{ theta: [ NaN, NaN ], dim: 2, cost: NaN, config: { alpha: 0.001, lambda: 0, iterations: 1000 } }

when I change the configuration values for alpha and lambda a bit, I do get different results, but nothing remotely similar to the equation used to generate the data.

am I missing something here or is there still a bug present?