Join GitHub today
GitHub is home to over 31 million developers working together to host and review code, manage projects, and build software together.Sign up
[MRG+1] add multinomial SAG solver for LogisticRegression #5251
I added multinomial sag solver in LogisticRegression.
The benchmark (gist) on RCV1 dataset is very promising. I only took 4 classes of the dataset, and removed samples that have more than one classe.
n_samples_train = 129073 n_samples_test = 23149 n_features = 47236 n_classes = 4
In the same spirit, I could probably add a multinomial LogisticRegression in SGD (previously proposed in #849). Do we want it?
RCV1 dataset, simplified to 4 classes, as in previous benchmark:
Yet it converges to different solutions:
The intercept decay was already used in SGD, and comes from Léon Bottou:
referenced this pull request
Dec 3, 2015
I don't think it makes sense for SAG, or at least it is not straightforward.
I don't remember seing this in the paper, but it makes sense numerically, and it is present in Mark Schmidt's C code.
What do you mean?
Yes it does not. I was mistaken.
On Fri, Dec 11, 2015 at 5:29 AM, Tom Dupré la Tour <email@example.com
If this is inspired by Mark Schmidt's code, we should probably reference that (if we don't already).