memory issue of _class_cov

[bobchennan]

When I was running linear discriminant training, I noticed very high memory usage.
In my case training set consists of 500k samples from 20k classes.
Feature dimension is less than 400.
Memory usage reached more than 100GB.

After debugging I found the problem:

In implementation:

    covs = []
    for group in classes:
        Xg = X[y == group, :]
        covs.append(np.atleast_2d(_cov(Xg, shrinkage)))
    return np.average(covs, axis=0, weights=priors)

The problem is that covariance matrices are stored in a list, which is already not necessary since we only need average of those matrices. After that np.average is called which will copy to convert the list to numpy.array. Both these factors dramatically increase memory usage.

I think we can simply take the sum in the loop if precision is not a problem.

[jnothman]

I think you're right, and don't think that precision was the intention here. _class_means uses a similar idiom in a way that wastes memory (but much less so). PR welcome.

[julietcl]

I would like to claim this issue.

[bobchennan]

@julietcl sorry I just saw your comment. Finish the code already.

[nsorros]

@julietcl @jnothman @bobchennan is that issue taken? If not, I would like to claim it as my first issue.

[qinhanmin2014]

It has been taken, see the referenced PR above (#10904)