Introduce `RowVector` as the transpose of a vector #19670

andyferris · 2016-12-21T14:16:30Z

This is my take on #4774 (taking vector transposes seriously). The transpose of an AbstractVector becomes a RowVector, which is a single concrete type which is a lightweight wrapper or "view" of the original vector, and is for all purposes outside of linear algebra it is a 1 x n sized AbstractArray. The inner product v' * v is a scalar, v'' returns v, and so-on. For details of my thoughts while developing this, consider the README for the v0.5-compatible companion package here.

I've also taken the liberty of moving most of the definition of transpose and ctranspose to the LinAlg module since ctranspose and transpose are quite arguably specific to linear algebra (as opposed to permutedims which makes sense for an array).

There are a few optimizations left to be made involving map and reduce used in the inner product, but I don't mind if they are done in this PR or separately later (as "feature freeze" is coming soon) (done). Opinions, comments, feedback, etc are very welcome and since I think this is my first serious PR to julia - forgive me if I messed something up :)

andyferris · 2016-12-21T14:24:43Z

OK, there's definitely some fallout in other tests that needs to be addressed.

stevengj · 2016-12-21T15:24:06Z

How does this compare to #6837? cc @andreasnoack

stevengj · 2016-12-21T15:27:28Z

base/linalg/rowvector.jl

+
+# Some overloads from Base
+@inline transpose(r::Range) = RowVector(r)
+@inline ctranspose(r::Range) = RowVector(conj(r)) # is there such a thing as a complex range?


I like the approach in #6837 (which doesn't make a copy for ctranspose) better.

stevengj · 2016-12-21T16:48:17Z

It's also unfortunate that this doesn't handle (2d) matrices.

andyferris · 2016-12-21T22:34:27Z

Hi Steven,

I agree completely with what you are saying - we need to handle conjugation and matrices in a similar manner.

AFAICT #6837 doesn't extend well for conj(::Vector) being a view, or conj(::Array{T,3}), which seems a shame. I should have discussed this in the OP, but I prepared this PR in the context that the future would hold three separate "wrapper" types:

RowVector
TransposedMatrix # non-conjugating, also living in LinAlg
ConjArray   # Any dimension, living in Base

However, the final two are optimizations and not semantically important in the linear algebra sense (the view vs. copy semantic certainly is important for programmers). I think for us as developers it is going to be much easier to have a modular system of simple views rather than type parameters or an interface to ask "are you conjugated?" to a complicated type (dispatch would be difficult).

I was hoping we could deal with this in a process that goes something like this:

A PR for RowVector
A PR for ConjArray (perhaps just Conj)
A PR for TransposedMatrix (perhaps just Transpose)
A PR to remove A_mul_Bc etc from the parser and in LinAlg

However, I don't see all this happening in 9 days! I thought I'd see if there was an appetite to deal with the linear algebra (row vector) semantics in v0.6 or not, with the "software" side of things (optimizations, view vs. copy, etc) to come later.

stevengj · 2016-12-21T23:14:24Z

Why should Transpose be distinct from RowVector, as opposed to @andreasnoack's approach of just Transpose{A<:AbstractArray{T,N}} being a subtype of AbstractArray{T,N} for any N?

It's true that ConjArray extends to higher-dimensional arrays, but I'm not sure how important that is, particular given the fusing conj.(A) for avoiding temporaries. Whereas making the conjugation a field (or type parameter) of Transpose as in #6837 has the advantage of reducing the number of types we need to deal with, and probably reducing code complexity.

andyferris · 2016-12-22T00:51:45Z

Maybe I misunderstood - in the Transpose type I thought the transposition was mandatory and the conjugation was optional. In which case we don't have conjugate views in 1D or 2D either - I feel that would be quite unfortunate.

A transposed matrix is just a matrix, but the transpose of a vector in some senses is not "really" a matrix or a vector (at least, that is what I'm aiming for here). To me, it made sense to separate the two concepts at the (primary) type level.

For example, users might want to dispatch row vectors (or dual vectors) separately, and while Transpose{A} where A <: AbstractVector is possible, it seems a little wordy. I guess a typealias can help here. The other argument for separation is that I can't think of a method that is specialized on Transpose which shouldn't be specialized on both Transpose{A <: AbstractVector} and Transpose{A <: AbstractMatrix} (there might be a small number, I'm not sure, but it seems the concepts would be unrelated).

Transpose as in #6837 has the advantage of reducing the number of types we need to deal with, and probably reducing code complexity.

Hmm, this is where I disagree. I think that a modular system would help deal with each operation atomically, and thus reduce code complexity significantly. We still need just a single definition ctranspose(x) = transpose(conj(x)). With RowVector, TransposedMatrix and ConjArray being fully-function views, for any combination of conj and transpose and ctranspose, you just need five versions of * for generic matrix/vector multiplication (matrix-vector, matrix-matrix, rowvector-matrix, rowvector-vector and vector-rowvector) to replace all the generic versions of A_mul_Bc. We would still need some specializations for sparse and strided structures, but that hardly seems to be worsened in any way, an example being:

*(A::Transpose{Conj{M}} where M<:StridedMatrix, B::StridedVector) = ...
*(A::Transpose{Conj{M}} where M<:StridedMatrix, B::StridedMatrix) = ...
...

(I do realize that BLAS is handled in A_mul_B! but it's an illustration that it's pretty easy to map this directly to which BLAS call and TRANSA = 'C', TRANSB = 'N', or whatever it is). For sparse arrays, it might be just

*(A::Transpose{M} where M<:AbstractSparseMatrix, B::AbstractSparseMatrix)

for a generic Julia implementation. Anyway, it seems prettier to separate the Conj part to me.

As to broadcast fusion (like conj.()), if you could suggest a version that includes fused matrix multiplication I think that would be fantastic (that's a serious request, not tongue-in-cheek - it would be seriously awesome!). But you still have the disadvantage of not being able to split your operations over multiple lines of code (or even multiple functions!), which can be quite powerful.

stevengj · 2016-12-22T01:17:32Z

We have a lot of separate code for Matrix and Vector, but they are still subtypes of the same type.

With a separate Conj type, you would conceivably have to handle Transpose{Conj} and Conj{Transpose} separately, although I guess that conj(A::Transpose) could conceivably canonicalize to Transpose{Conj}.

I guess it's not terrible to have separate Conj and Transpose types, but I'd really strongly prefer that (a) there be only one Transpose type of varying dimensions (not completely separate TransposedMatrix and RowVector types) and (b) that Conj and Transpose be implemented in the same PR to prevent regressions in A'*B and similar operations.

andyferris · 2016-12-22T01:31:04Z

OK - I think I could support that; Transpose and Conj would work fine. Upon further thought, having transpose(vector) as being "the object that when transposed again returns vector" neatly sidesteps many of the subtle semantic issues of RowVector vs DualVector and so-on, and still lets us achieve a scalar inner product.

The question now is, do we work like mad to make this happen this week for v0.6? And does anyone know if jb/subtype will merge soon so we can use the new syntax? (I was really enjoying those signatures I wrote above 😄 )

PS - this PR does already attempt to "canonicalize" conj(transpose(x)) = transpose(conj(x)) since I knew we would want it in the future, but we could go to extremes and have the inner constructor of Conj throw an error if it wraps Transpose, to make it more concrete?

vtjnash · 2016-12-22T04:01:46Z

it should be merged pretty soon. just needs some performance testing and review.

Transpose{M where M<:AbstractSparseMatrix}

This type does not have any subtypes (e.g. does not have where on the "outside"); I don't think that's what you want. What you probably wanted is one of:

(*){M <: AbstractSparseMatrix}(A::Transpose{M}, B::AbstractSparseMatrix)
(*)(A::Transpose{M}, B::AbstractSparseMatrix) where M <: AbstractSparseMatrix

andyferris · 2016-12-22T04:23:16Z

You're right... I'm still in a bit of a headspin with the new syntax :)

andyferris · 2016-12-22T04:26:11Z

I think this is the one I like:

(*)(A::(Transpose{M} where M <: AbstractArray), B::AbstractSparseMatrix)

I don't see that the brackets should be necessary, though. (edit: xref #18457 (comment))

stevengj · 2016-12-22T14:35:02Z

base/linalg/rowvector.jl

@@ -0,0 +1,183 @@
+immutable RowVector{T,V<:AbstractVector} <: AbstractMatrix{T}


@alanedelman and I both tend to think that Transpose{T,N,AbstractArray{T,N}} <: AbstractArray{T,N}, so that a RowVector (AbstractRowVector? = Transpose{T,1,AbstractArray{T,1}}) would be a subtype of AbstractVector, i.e. a "1d" array object (an element of the dual space, with conjugation).

Of course, that necessitates a bit of care, since the assumption AbstractVector == ColumnVector appears in a number of methods, but it doesn't seem too bad. There would need to be specialized +, *, and (most annoying) various broadcast_foo methods.

JeffBezanson · 2016-12-22T20:07:04Z

base/linalg/bidiag.jl

@@ -349,6 +349,15 @@ A_mul_B!(C::AbstractVector, A::BiTri, B::AbstractVector) = A_mul_B_td!(C, A, B)
 A_mul_B!(C::AbstractMatrix, A::BiTri, B::AbstractVecOrMat) = A_mul_B_td!(C, A, B)
 A_mul_B!(C::AbstractVecOrMat, A::BiTri, B::AbstractVecOrMat) = A_mul_B_td!(C, A, B)

+@inline \(::Diagonal, ::RowVector) = error("Cannot left-divide matrix by transposed vector")


I don't think you need to inline a function that just throws an error.

stevengj · 2016-12-22T20:13:17Z

We had a long discussion today with @JeffBezanson, @andreasnoack, and @alanedelman. We initially leaned towards making a RowVector an AbstractVector, but after Alan left we began leaning in the other direction, making a RowVector an AbstractMatrix as in this proposal. Making it an AbstractMatrix seems less likely to cause regressions, and will require less code to be modified, than making it an AbstractVector (since there is lots of linear-algebra code that assumes every AbstractVector is a column vector).

JeffBezanson · 2016-12-22T20:14:44Z

base/linalg/rowvector.jl

+@inline check_tail_indices(i1, i2, i3, is...) = i3 == 1 ? check_tail_indices(i1, i2, is...) :  false
+
+# Some conversions
+convert(::Type{AbstractVector}, rowvec::RowVector) = rowvec.vec


Are we sure we want this? There aren't currently any conversions between arrays with different numbers of dimensions. If there were, I suspect we'd only allow converting (n,)-size arrays to (n,1), not (1,n)-size arrays to (n,).

I agree, this isn't very useful - you can just type .'

stevengj · 2016-12-22T20:27:18Z

base/linalg/rowvector.jl

+@inline *(::RowVector, ::RowVector) = error("Cannot multiply two transposed vectors")
+@inline *(vec::AbstractVector, rowvec::RowVector) = kron(vec, rowvec)
+@inline *(vec::AbstractVector, rowvec::AbstractVector) = error("Cannot multiply two vectors")
+@inline *(mat::AbstractMatrix, rowvec::RowVector) = error("Cannot right-multiply matrix by transposed vector")


Why are all of these error definitions necessary? We already throw errors for multiplying matrices and vectors of incompatible shapes, and since RowVector <: AbstractMatrix the existing code should just work (i.e. continue to throw errors), no?

I wanted users to see that semantically we only allow

matrix * matrix

matrix * vector

rowvector * matrix

rowvector * vector

vector * rowvector

So it's less about the specific shapes and more about the semantics of the type. I'm also deprecating (well, there should be a deprecation warning, not an error, but I haven't got to it yet) the vector * matrix method in favor for the last item on the list.

Ideally, there should only be four nice error message methods (a single one for each of vectormatrix, matrixrowvector, vectorvector, rowvectorrowvector), but they are repeated (a) because of ambiguities and (b) because of all the different A_mul_Bc methods.

@andyferris, but there are already error messages for multiplications that deviate from these. If you want to improve the error messages, can't that be a separate PR?

JeffBezanson · 2016-12-22T20:34:08Z

base/linalg/rowvector.jl

+
+# Multiplication #
+
+@inline *(rowvec::RowVector, vec::AbstractVector) = reduce(+, map(At_mul_B, transpose(rowvec), vec))


sum( rowvec.vec[i].' * vec[i] for i = 1:length(vec) ) to fuse the map and reduce?

Right! I always forget to use generators.

andreasnoack · 2016-12-22T20:35:37Z

base/sparse/sparsevector.jl

-    end
-    A
-end
+@inline transpose(sv::SparseVector) = TransposedVector(sv)


-> RowVector

stevengj · 2016-12-22T20:50:28Z

base/linalg/rowvector.jl

+
+# Multiplication #
+
+@inline *(rowvec::RowVector, vec::AbstractVector) = reduce(+, map(At_mul_B, transpose(rowvec), vec))


I guess you are doing this in order to support vectors of matrices and similar, but it seems like we will also want to define specialized methods for RowVector{<:Number} * AbstractVector{<:Number}. e.g. for BLAS scalars we want to call BLAS.dotu.

(Even in the general case, can't you use mapreduce with zip(rowvec.vec, vec) to avoid the temporary arrays?, or sum with a generator as suggested above?)

Yes, and yes - agreed.

musm · 2016-12-22T20:57:28Z

We had a long discussion today with @JeffBezanson, @andreasnoack, and @alanedelman. We initially leaned towards making a RowVector an AbstractVector, but after Alan left we began leaning in the other direction, making a RowVector an AbstractMatrix as in this proposal. Making it an AbstractMatrix seems less likely to cause regressions, and will require less code to be modified, than making it an AbstractVector (since there is lots of linear-algebra code that assumes every AbstractVector is a column vector).

Care to share the preference for RowVec <: AbstractMatrix over RowVec <: AbstractVector ? Could that not potentially also break future code for functions with x::AbstractMatrix. I don't see why RowVec <: AbstractMatrix should be prefered over RowVec <: AbstractVector ?

stevengj · 2016-12-22T21:53:50Z

@musm, the basic arguments went like this, as I recall:

If you make RowVector <: AbstractVector:

Pro: there is some mathematical elegance, because it is natural to think of a row vector as belonging to the dual space of the column vector, and as such (ala Reisz) it is natural to think of the row vector as a "single-index object". (You can also think of it as raised vs. lowered indices, which doesn't change the number of indices.)
Con: For any code that cares about the "shape" of a vector (row vs. column vector), e.g. linear-algebra code or broadcast, it can't just operate on AbstractVector — it will have to check whether the AbstractVector is a row vector (or define a RowVector method) lest it yield wrong/unexpected results. This is a lot of code, and will be an continuing imposition on more and more code as time passes (as people write new linear-algebra libraries, for example, they will always have to check this on every AbstractVector argument). Worse, unmodified code will silently do the wrong thing.

If you make RowVector <: AbstractMatrix:

Con: not quite as elegant mathematically. There is a natural isomorphism from 1×n matrices (linear operators on n-component column vectors) to elements of the column-vector dual space, but they are not quite the same thing (one produces a scalar, the other a 1×1 matrix). Still, we can define specialized methods to get key properties "right", likeRowVector * AbstractVector = scalar etc. and transpose(transpose(x)) === x.
Pro: Linear-algebra code and broadcast and other shape-dependent code will mostly work as-is, with no additional checks/methods. At worst, if someone doesn't write a specialized RowVector method, then it will continue to do what it does now—might be slightly suboptimal (e.g. somewhere you might get a 1×1 matrix instead of a scalar), but it will still mostly do what you want. And the core operations that need to depend on RowVector, like *(RowVector, AbstractVector), can mostly be modified once at an abstract level (e.g. to call dot) and subsequent AbstractVector types (assuming they implement dot etc.) will mostly inherit the "correct" behavior without having to write specialized RowVector methods. Moreover, failures are loud: if some operation yields a matrix where a scalar or column vector was expected, it will generally die with a MethodError (this is what happens now if you do e.g. y -= y * (y'x) where x and y are vectors).

Put another way, there are very few methods that take an AbstractVector argument where it is useful to be able to pass a RowVector and have it do the same thing. In contexts that don't care about "shape" (row vs column), like string processing of an array of bytes from a file, it is not likely that anyone would want to pass a row vector in the first place. So you don't gain much by that approach, and you pay a huge price by imposing a new requirement on every function that does care about shape.

stevengj · 2016-12-22T22:03:19Z

Another argument in favor of RowVector <: AbstractMatrix, from @JeffBezanson, is that it can be viewed as an incremental step in the right direction. x' is a matrix already, and this just makes it a "better" matrix. If we merge this, and it turns out that it is not enough—that there are lots of places where we would want it to be an AbstractVector, then we can take the next step of changing the dimensionality.

andyferris · 2016-12-22T22:58:04Z

Well said, @stevengj. I agree 100% with the above, but couldn't express it as elegantly.

I also do view this as an incremental step. I have a "feeling" that a 1D dual vector would be a bit easier to deal with in a language with traits, so this could be revisited later if a row-shaped vector seems wrong in the future.

andyferris · 2016-12-22T23:14:18Z

Regarding changing this to a general Transpose type, would people want either of

typealias RowVector{T, V <: AbstractVector} Transpose{V, T}
typealias TransposedMatrix{T, M <: AbstractMatrix} Transpose{M, T}

to be exported?

stevengj · 2016-12-22T23:58:40Z

If RowVector <: AbstractMatrix and TransposedMatrix <: AbstractMatrix, then maybe they should be separate types after all.

andyferris · 2016-12-23T00:15:42Z

Oh dear - now we go full circle! 😄

I think either way can be made to work. For example. if they are separate types, we can even do

typealias Transpose{A,T} Union{RowVector{T,A}, TransposedMatrix{T,A}}

lol

stevengj · 2017-01-13T22:43:08Z

Sounds like we're ready to merge.

stevengj · 2017-01-13T22:43:53Z

Would be great to have a Conj PR to go with this one.

andyferris · 2017-01-13T23:15:39Z

Aim to do so, within a week (hopefully less).

andyferris · 2017-01-15T11:38:08Z

See #20047

stevengj · 2017-01-15T11:50:26Z

base/linalg/rowvector.jl

+    if length(rowvec) != length(vec)
+        throw(DimensionMismatch("A has dimensions $(size(rowvec)) but B has dimensions $(size(vec))"))
+    end
+    sum(@inbounds(return rowvec[i]*vec[i]) for i = 1:length(vec))


Should we be calling dot for two real vectors? This should produce the same result, of course, but is there a performance difference (especially when BLAS is called)?

Yes, good idea, I will address this in the new PR, along with the complex case.

I suppose I should go measure this, but is there much difference between native Julia and BLAS for level-1 operations, and if so, why? (BLAS might be multithreaded?) What about the outer product, which is now broadcast, maybe there is a faster BLAS outer product? (Would it be a rank-1 update on a zero'd array, or a direct op?)

BLAS might be multithreaded, but mainly it is the fact that it is hand-coded for SIMD. @simd can make up for this gap, but not always, and in generic code like you used above you don't have @simd anyway.

For broadcast operations, the advantage of BLAS1 is wiped out by fusion in typical circumstances. Of course, it would be great if broadcast could exploit SIMD well too, but we are dependent on the compiler for this.

tkelman · 2017-01-16T02:20:54Z

test/linalg/rowvector.jl

+    rv = v.'
+
+    @test (rv*d)::RowVector == [2,6,12].'
+    @test_throws Exception d*rv


Please make these test a specific exception type! This test would pass if your implementation had a typo in it.

Sacha0 · 2017-01-16T18:05:36Z

test/sparse/sparse.jl

+# This is broken by the introduction of RowVector... see brittle comment above.
+#@testset "issue #16548" begin
+#    @test which(\, (SparseMatrixCSC, AbstractVecOrMat)).module == Base.SparseArrays
+#end


Rather than simply commenting this test out, replace it with another test or set of tests that capture its spirit?

tkelman · 2017-01-20T05:57:30Z

test/linalg/rowvector.jl

+    @test (cz*mat')::RowVector == [-2im 4 9]
+    @test_throws Exception [1]*reshape([1],(1,1))' # no longer permitted
+    @test cz*cz' === 15 + 0im
+    @test_throws Exception z*vz'


for example, vz is a typo and this test still passed because Exception is far too broad a type to be testing for

Yes, I had to chuckle when I found this after you wrote about this possibility earlier. FYI its being addressed in #20047

StefanKarpinski · 2017-01-25T22:45:53Z

@andyferris: I want to say that this is – despite my waffling about whether we should do this or something else – a pleasure to work with. Thanks for all the hard work on bringing it to fruition!

andyferris · 2017-01-26T00:43:22Z

Thanks Stefan - that means a lot to me :)

This used to be the outer product but was removed by JuliaLang#19670. It makes much more sense to deprecate this than make it a breaking change.

tkelman · 2017-02-07T16:53:17Z

base/linalg/rowvector.jl

+@inline size(rowvec::RowVector, d) = ifelse(d==2, length(rowvec.vec), 1)
+@inline indices(rowvec::RowVector) = (Base.OneTo(1), indices(rowvec.vec)[1])
+@inline indices(rowvec::RowVector, d) = ifelse(d == 2, indices(rowvec.vec)[1], Base.OneTo(1))
+linearindexing{V<:RowVector}(::Union{V,Type{V}}) = LinearFast()


shouldn't this be looking at the linearindexing of the wrapped type?

No, this is what we want. It only wraps vectors, but is itself a two-dimensional object. If it passed this through, then it'd have pessimized performance for LinearSlow types unnecessarily.

then are all AbstractVector types LinearFast? that currently isn't the case for e.g. SparseVector

Right, this is something that'd be nice to change at some point. LinearFast (read: index-with-one-dimension) and LinearSlow (read: index-with-N-dimensions) represent the same thing when N=1. It'd be worth seeing if the performance of SparseVector changes if you make it LinearFast — it may end up hitting simpler methods that way. If so, we could change the default for all vectors. And maybe eventually re-work this to be clearer (#20175 (comment)).

Jutho · 2017-08-04T09:33:51Z

Hi Andy, I don't think I ever thanked you for this. I think we are all happy with this solution.

One final comment that I wanted to make, rather now than after julia v1.0, is that I did come up with one other solution to the issue of (vector) transposes. I didn't want to pollute that infamous issue any further now that it has finally found its peace, so I am posting it here. That solution is to have two parameters N in (Abstract)Array, where N1 represents the number of normal indices, and N2 the number of dual indices.

Linear algebra would than primarily take place with Vector{T} = Array{T,1,0}, Matrix{T} = Array{T,1,1} and RowVector = Array{T,0,1}, whereas most other multidimensional code would probably use Array{T,N,0}, but would actually only care about the total dimensionality N1+N2.

andyferris · 2017-08-04T10:57:43Z

Thank you, @Jutho! :)

Yes, we've certainly have considered more complex combinations of dual and normal dimensions, as type parameters. I remember a chat with @jiahao and @mbauman where this was discussed, and Jiahao's talk at this year's JuliaCon ended by mentioning this.

My understanding is that v1.0 will happen as soon as practically possible (to just implement any foreseeable breaking changes and other "essential" features, bug fixes/correctness/consistency issues and performance bottlenecks). I recommend we play with this stuff in community packages and if we come up with an awesome design for dual spaces and multi linear algebra, we can propose changes for v2.0. For obvious reasons, you'd be a great person to help out with or to discuss such a design :)

For my part, I'm slightly confused about the complex case. If I have a complex matrix, the adjoint (ctranspose?) essentially takes the dual of both indices, no? What is the object (and the elements of the resulting array) where I take the dual of just one space but not the other? I'm sure there is a simple diagram for this, of course...

Jutho · 2017-08-04T11:07:44Z

Actually transpose is the harder one. With the use of two parameters N1 and N2, the convention would be that the first N1 indices are normal, and the next N2 indices are dual.

If you have a matrix / linear map f:A->B between vector spaces A and B, that's an element of B ⊗ dual(A), i.e. N1=N2=1 in that language. Ctranspose maps f to a map f':B->A, i.e. an element of A ⊗ dual(B), so still a matrix with N1=N2=1.

Transpose, on the other hand, maps f to f.':dual(B)->dual(A), an element of dual(A) ⊗ B. That's the harder one to represent if you cannot actually encode which indices are dual and which are not. For real vector spaces, dual(A)=A and therefore there is no distinction, though it is useful to keep it in practice, so that rowvector * vector = scalar and vector * vector = error.

I've been reading a bit about diagrammatic notation and category theory to formalize my understanding: https://arxiv.org/abs/0908.3347v1

andyferris · 2017-08-04T12:00:32Z

Ok - thanks I'll check out that reference.

So the discussion has been that we're more-or-less ditching transpose for linear algebra to replace it with adjoint (formerly ctranspose) and conj(adjoint(mat)) for the "non-conjugating transpose". So this light work out well.

martinholters · 2017-08-04T12:21:18Z

I wonder whether a syntactically nice way of getting a row of a matrix as a RowVector would be worthwhile. Not that I've missed it personally, though. But A[5,:].' to get it could be a mathy context where you actually need transpose instead of adjoint.

StefanKarpinski · 2017-08-04T12:33:04Z

That's a fair point – you just want a row slice there, no conjugate.

andyferris · 2017-08-07T00:27:47Z

We definitely need a way of getting a RowVector straight out of a matrix. The best I've thought up is row(A, 5).

StefanKarpinski · 2017-08-07T00:59:51Z

It's going to be really confusing that A[5,:] and row(A,5) do different things, I'm afraid.

andyferris · 2017-08-07T01:20:21Z

These "taking seriously" issues highlight that getting what we want for arrays and what we want for linear algebra simultaneously will be challenging.

On one hand we want APL indexing. So, if we introduce dual vectors as 1D arrays, then this at least satisfies the AbstractArray interface if the dual vectors just come out of indexing by default. OTOH we might have "data" in the array and we don't want to have recursive adjoint and other linear algebra ideas coming into array indexing, and users would be confused why for this one indexing operation they get a dual vector instead of whatever similar indicates.

So I'm wondering if we should leave the AbstractArray interface alone and introduce the necessary functions to LinAlg that we need to do linear algebra. The row function would be a linear algebra function, like adjoint. Data applications would use indexing (and transpose).

StefanKarpinski · 2017-08-07T02:19:47Z

Ideally we could just have A[5,:].' give us a row vector.

andyferris mentioned this pull request Dec 21, 2016

Taking vector transposes seriously #4774

Closed

andyferris changed the title ~~Introduce RowVector as the transpose of a vector~~ WIP Introduce RowVector as the transpose of a vector Dec 21, 2016

stevengj reviewed Dec 21, 2016

View reviewed changes

stevengj mentioned this pull request Dec 21, 2016

Reconsider uses of promote_op #19669

Closed

stevengj reviewed Dec 22, 2016

View reviewed changes

JeffBezanson reviewed Dec 22, 2016

View reviewed changes

stevengj reviewed Dec 22, 2016

View reviewed changes

JeffBezanson reviewed Dec 22, 2016

View reviewed changes

andreasnoack reviewed Dec 22, 2016

View reviewed changes

stevengj reviewed Dec 22, 2016

View reviewed changes

stevengj merged commit 3d6fa35 into JuliaLang:master Jan 13, 2017

andyferris mentioned this pull request Jan 15, 2017

Added ConjArray wrapper type for conjugate array views #20047

Merged

stevengj reviewed Jan 15, 2017

View reviewed changes

tkelman reviewed Jan 16, 2017

View reviewed changes

Sacha0 reviewed Jan 16, 2017

View reviewed changes

tkelman reviewed Jan 20, 2017

View reviewed changes

tkelman mentioned this pull request Jan 20, 2017

Make rowvector tests check for specific exception type #20147

Merged

stevengj mentioned this pull request Feb 2, 2017

0.6: vec*mat throws "Cannot left-multiply a matrix by a vector" even when mat is 1 x n #20389

Closed

andyferris mentioned this pull request Feb 2, 2017

Added breaking vector*matrix removal to NEWS.md #20394

Closed

andyferris pushed a commit to andyferris/julia that referenced this pull request Feb 3, 2017

Include deprecated methods for vector * matrix

596a3ac

This used to be the outer product but was removed by JuliaLang#19670. It makes much more sense to deprecate this than make it a breaking change.

andyferris mentioned this pull request Feb 3, 2017

Include deprecated methods for vector * matrix #20429

Closed

tkelman reviewed Feb 7, 2017

View reviewed changes

andreasnoack mentioned this pull request Feb 20, 2017

implement Ac_mul_B(Vector, Matrix) to avoid temporary #18056

Closed

dsweber2 mentioned this pull request Oct 17, 2017

updating to v0.6, working with conv2 arsenal9971/Shearlab.jl#14

Merged

		@@ -0,0 +1,183 @@
		immutable RowVector{T,V<:AbstractVector} <: AbstractMatrix{T}


		# Multiplication #

		@inline *(rowvec::RowVector, vec::AbstractVector) = reduce(+, map(At_mul_B, transpose(rowvec), vec))

Introduce RowVector as the transpose of a vector #19670

Introduce RowVector as the transpose of a vector #19670

Conversation

andyferris commented Dec 21, 2016 • edited Loading

andyferris commented Dec 21, 2016

stevengj commented Dec 21, 2016

Choose a reason for hiding this comment

stevengj commented Dec 21, 2016

andyferris commented Dec 21, 2016

stevengj commented Dec 21, 2016

andyferris commented Dec 22, 2016 • edited Loading

stevengj commented Dec 22, 2016

andyferris commented Dec 22, 2016 • edited Loading

vtjnash commented Dec 22, 2016 • edited Loading

andyferris commented Dec 22, 2016

andyferris commented Dec 22, 2016 • edited Loading

stevengj Dec 22, 2016 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

stevengj commented Dec 22, 2016 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

stevengj Dec 22, 2016 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

musm commented Dec 22, 2016

stevengj commented Dec 22, 2016 • edited Loading

stevengj commented Dec 22, 2016 • edited Loading

andyferris commented Dec 22, 2016

andyferris commented Dec 22, 2016 • edited Loading

stevengj commented Dec 22, 2016

andyferris commented Dec 23, 2016

stevengj commented Jan 13, 2017

stevengj commented Jan 13, 2017

andyferris commented Jan 13, 2017

andyferris commented Jan 15, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

StefanKarpinski commented Jan 25, 2017

andyferris commented Jan 26, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Jutho commented Aug 4, 2017

andyferris commented Aug 4, 2017

Jutho commented Aug 4, 2017

andyferris commented Aug 4, 2017

martinholters commented Aug 4, 2017

StefanKarpinski commented Aug 4, 2017

andyferris commented Aug 7, 2017

StefanKarpinski commented Aug 7, 2017 • edited Loading

andyferris commented Aug 7, 2017

StefanKarpinski commented Aug 7, 2017

Introduce `RowVector` as the transpose of a vector #19670

Introduce `RowVector` as the transpose of a vector #19670

andyferris commented Dec 21, 2016 •

edited

Loading

andyferris commented Dec 22, 2016 •

edited

Loading

andyferris commented Dec 22, 2016 •

edited

Loading

vtjnash commented Dec 22, 2016 •

edited

Loading

andyferris commented Dec 22, 2016 •

edited

Loading

stevengj Dec 22, 2016 •

edited

Loading

stevengj commented Dec 22, 2016 •

edited

Loading

stevengj Dec 22, 2016 •

edited

Loading

stevengj commented Dec 22, 2016 •

edited

Loading

stevengj commented Dec 22, 2016 •

edited

Loading

andyferris commented Dec 22, 2016 •

edited

Loading

StefanKarpinski commented Aug 7, 2017 •

edited

Loading