why do we have both full() and dense()?

[JeffBezanson]

Can't remember if this has come up before, but I noticed we have both of these functions. Are they different? Do we need both?

cc @ViralBShah

[jiahao]

julia> methods(dense)
# 4 methods for generic function "dense":
dense{Tv}(S::SparseMatrixCSC{Tv,Ti<:Integer}) at sparse/sparsematrix.jl:165
dense(x::AbstractArray{T,N}) at abstractarray.jl:312
dense(A::CholmodSparse{Tv<:Union(Complex{Float64},Float32,Float64,Complex{Float32}),Ti<:Union(Int64,Int32)}) at linalg/cholmod.jl:1041
dense(A::CholmodDense{T<:Union(Complex{Float64},Float32,Float64,Complex{Float32})}) at linalg/cholmod.jl:1042

In all four methods, dense is defined to be synonymous with full.

[StefanKarpinski]

For what it's worth, I much prefer the name dense to full.

[ViralBShah]

We need a good name too convert all the various diagonal matrix types to Matrix.

[ViralBShah]

We should certainly have just one. Let's do this in 0.3

[andreasnoack]

As @ViralBShah mentions, full is also used for Hermitian, Diagonal, etc. For Hermitian and Triangular I thinkdense is slightly misleading. Alternatively we could use matrix to return a Matrix like float64 returns a Float64.

[ViralBShah]

I was thinking about using matrix myself. It is not a name I like, but it is consistent and guess-able, if you know a little bit about the underlying types. The reality may be that most users will not know the difference between the various *Matrix types.

[johnmyleswhite]

I'd prefer dense over full and both over matrix.

In DataFrames, we use matrix to handle conversion between DataFrames and matrices.

[jiahao]

matrix is potentially misleading though. It would imply that the things we are converting from are not matrices.

full is the Matlab term and I think is the de facto standard.

[johnmyleswhite]

Arguably a DataFrame isn't really a matrix from Julia's perspective. It's not even a subtype of AbstractMatrix.

[ViralBShah]

The main reason I never really moved away from full was because dense really wasn't any more appealing, whereas full was at least familiar.

[jiahao]

@johnmyleswhite Just to clarify, it makes sense to have a matrix(X::DataFrame). My comment was meant for the proposed matrix(X::Tridiagonal), etc.

[johnmyleswhite]

Ok. I think we're in agreement. My point is that I'd matrix to convert things that aren't matrices into matrices. So I'd also not be in favor of having matrix(X::Tridiagonal).

[jiahao]

let's just plan to deprecate dense in favor of full.

[StefanKarpinski]

That's a shame. I know full is the standard term, but it's really vague and not a great word for this. Matrix is worse – anything you're applying this too should already be some kind of matrix. Dense is the only term that describes what this actually does, IMO, which is turn some non-dense matrix representation into your normal dense version.

[andreasnoack]

I think there is a critique of the naming scheme for matrices hidden here. Matrix is a concrete type but from this discussion one would think that it is an abstract type.

[StefanKarpinski]

Definitely. We started out with array, vector and matrix being abstract types and having DenseArray, etc. as the concrete types. But it turns out that you spend a lot more time writing code for the concrete types than abstract ones, so we made the concrete types the shorter names. I don't regret the decision as I think it's still the least confusing choice, but I agree that it's a bit of a disconnect.

[jiahao]

I’m not thrilled by the semantic of full() either, but it practically has a meaning of converting a matrix representation into the “full” one where memory is allocated for every matrix element. One possible breakdown of the dense() semantic would be for packed triangular matrices, where I don’t think it is clear that people would consider triangular matrices as being “sparse”.

[jiahao]

The ambiguity of the dense() semantic becomes more severe when considering more abstract objects that behave like matrices but have in-memory representations that look nothing like Matrix or any of the special matrices. For example, you could have a matrix that is represented as the linear combination of the outer product of a few vectors. (Such low rank matrices are important when using iterative solvers and quasi-Newton optimizers, etc.) For such data structures, there is a disconnect between the representation and its mathematical sparsity/denseness - you could easily have a completely dense matrix that has a very compact, low rank representation. I suppose you could argue that such an object is “not really” a matrix and should be converted with matrix(), but such outer product decompositions are perfectly valid mathematical representations of matrices too.

I guess all I’m really saying is that it’s really hard to construct a robust metaphor between the mathematical object known as a matrix and a convenient representation of it in memory in some suitable basis, and that any semantic that conflates mathematical density/sparsity and the compactness of a data structure could cause problems down the road. Even though in many cases it is fruitful to use representations of matrices that take advantage of sparsity patterns, the most useful representation of a matrix may be in forms where the sparsity is not explicit.