Join GitHub today
GitHub is home to over 40 million developers working together to host and review code, manage projects, and build software together.Sign up
Add CLI and python bindngs for linear_svm #1935
Hey @Yashwants19 I totally agree, I have just been busy with other things and haven't really been able to look into this. Basically, we need to figure out what is going on with parallel SGD and why this causes problems with the intercepts. Once we can resolve that then I think it's ready to merge. I hope that maybe I'll have some more time to look into this more comprehensively later this week.
I was able to debug this pretty easily with gdb. Here's the diff to fix the issue:
You can see that the
rcurtin left a comment •
I thought maybe debugging the psgd issue would be really tricky, so I was going to make sure to set aside an entire evening for it, but that turned out to not be necessary---it was pretty straightforward.
I left some comments; I think that some option names have to change, and I found one bug with the accuracy calculation.
I also did some timing and was a little bit surprised by how slow parallel SGD can sometimes be:
as compared to L-BFGS with the same effective number of iterations (5000):
It may be worth investigating further here---such a huge slowdown for parallel SGD seems pretty unreasonable; it shouldn't take an order of magnitude longer to get to a model that's much worse than L-BFGS gives in far less time. However, there are probably two issues here:
Lastly, I ran a quick comparison with the most similar thing I could find that was easy to run (scikit's
Note that that only runs 1000 points with
So, those results are promising in that we've outperformed the scikit implementation pretty resoundingly, but I still think there's a lot of acceleration possible and it's not behaving quite how I would have expected---I definitely would have expected
I know this comment got really long---anyway, most of it is my observations, but if you can handle the couple of comments I left and update