Feature request: default (non-zero) values for sparse matrices

[sbromberger]

Is there any practical way to achieve this? Consider: I have a very large matrix describing edge weights between all pairs of vertices. This is an NxN matrix and is impractical for graphs with orders of 1e7 without using sparse matrices.

If I want the default edge weight of 1.0, I have to code specially around this to make a sparse matrix entry of "0" really be "1" (which is ok in this specific case because an edge weight of 0 is not defined). It would be much easier if I could specify "create this sparse matrix but any unspecified value should be X".

Is there already an easy way to do this that I've missed?

[ViralBShah]

There is an old issue on that somewhere.

[ViralBShah]

I would prefer that the graph package provide this meaning rather than pushing the functionality in sparse matrices.

[sbromberger]

I see how this might complicate things, but I'd appreciate reconsideration: imagine being able to create a full ones-based sparse matrix that took essentially 0 space. The current model (doing this in LightGraphs) only works because 0-weight edges don't mean anything to a simple graph and we can do a simple substitution (several places, which is ugly, but still feasible). This doesn't extend to other types of graphs, though, and can't be used as a general purpose implementation.

[kmsquire]

I'm not an expert on the current sparse matrix implementation, but I believe, e.g., that the current implementation interacts with external C libraries, and that many of the methods are specialized assuming zero as default values. So changing the existing implementation would have both interop and performance implications.

That said, see #7010, especially https://github.com/JuliaLang/julia/issues/7010#issuecomment-44428983. There are certainly good reasons to have non-zero default values in sparse matrices, but it would probably have to be implemented in a new type, rather than on top of the existing one, and that would be best to start in a separate package.

[sbromberger]

OK, so this suggestion doesn't get past the first question ("Is there any practical way to achieve this?") if there are external libraries involved. I'll think about a separate type. Thanks.

[StefanKarpinski]

There is a way to achieve this, but as a separate type wrapping a zero-default sparse matrix. Essentially, you define a non-zero-default sparse matrix type that has a default value and the difference between each entry and the default – i.e. Z + d. Then you can implement various operations on these, e.g.

(Z1 + d1) + (Z2 + d2) = (Z1 + Z2) + (d1 + d2)
(Z1 + d1) * (Z2 + d2) = (Z1*Z2 + d1*Z2 + d2*Z1) + d1*d2

Note that the left-hand-sides only use operations on zero-default sparse matrices.

[StefanKarpinski]

If you want this data structure, you may want to create it in a package and see how it works out.

[nalimilan]

FWIW Wolfram Mathematica support this, "zero" is called "default" there:
https://reference.wolfram.com/language/tutorial/LinearAlgebraSparseArrays.html

[tkelman]

I don't think SparseMatrixCSC should change to do this. But a separate type could be implemented for it, where some operations like the case in #14963 would be more predictable when using the non-zero-default type.

[carnaval]

Yes, we could have a more dynamic type for sparse matrices where the default element is a value. Now that each function has its own type, you could even have map(f, ::SparseMatrixCSC) return one or the other depending on f(0)==0 and still be inferable.