Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sparse matrices could allow sparsity-invariant changes to the elements #12254

Open
Didou09 opened this issue May 28, 2020 · 2 comments
Open

sparse matrices could allow sparsity-invariant changes to the elements #12254

Didou09 opened this issue May 28, 2020 · 2 comments

Comments

@Didou09
Copy link
Contributor

Didou09 commented May 28, 2020

Fast element-wise update to a sparse matrix without changing sparsity structure

  • Some algorithms (e.g.: least_squares()) require call a user-defined sparse matrix at every iteration. It happens that, for each iteration, although the matrix is different, its sparsity structure remains constant (i.e.: changing the values of the non-zero floats inside without touching the zeros).
  • Currently, unless I am mistaken, we're forced to re-instanciate the whole matrix at every iteration (which slows down the calling algorithm, like least_square(), which can have a lot of iterations).
  • A more efficient way would be to instanciate the matrix before calling the algorithm, and updating only the non-zero values inside at every iteration, like we could do with a dense numpy array.

Describe the solution you'd like

  • A set_items() method to allow for such element-wise changes with constant sparsity would be very welcome, and since the sparsity is constant, it should be possible to make it quite fast and efficient.
  • I guess this method would have to be sparse-format specific:
    • would allow for row-wise changes for csr_matrix
    • would allow for column-wise changes for csc_matrix
    • ....
  • An optional indices array could be provided to specify which elements in the row / column would be changed (by defaut assume all non-zero elements are changed ?)

Describe alternatives you've considered

  • Alternatively, we could use a different sparse format better at assigning (e.g.: lil_matrix) and perform conversion to csr_matrix (because least_squares is best used with csr_matrix format). But it means there would be a sparse format conversion at every iteration, which would be an unnecessary cpu time usage.
@perimosocordiae
Copy link
Member

The current way that we allow users to make efficient sparsity-preserving updates is by allowing access to the .data member array. It's a power user feature that comes with some caveats, but it's very fast and very flexible.

I'm not so familiar with the least squares usage, but I suspect we could improve it using this technique.

@Didou09
Copy link
Contributor Author

Didou09 commented May 29, 2020

Thanks @perimosocordiae, I didn't know I could access the data attribute, I'll try it and come back here if I see something constructive to propose in relation with least_square

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants