KnetArray transpose and permutedims missing.

[denizyuret]

No description provided.

[denizyuret]

Is permutedims for 4D and 5D practical? Necessary?

[denizyuret]

I am leaving this issue open for the general N-D implementation.

[denizyuret]

@ilkerkesen do we actually have a general N-D implementation (beyond 5-D) that works? If so, let's close this.

[ilkerkesen]

Back then I implemented a general permutedims operation, but couldn't port it to Knet. Yes, we have, but it needs to be ported. On Dec 24, 2017 6:55 PM, "denizyuret" <notifications@github.com> wrote: @ilkerkesen <https://github.com/ilkerkesen> do we actually have a general N-D implementation (beyond 5-D) that works? If so, let's close this. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#39 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ACrjQ4p-NA4lyk5yCrb3QOlRo0rwbegwks5tDnP_gaJpZM4KwX7U> .

[ilkerkesen]

So far so good,

1. Current implementation works correctly.
2. Yes, it takes advantage of parallel programming, but ignores its crucial core concepts.
3. I did have a generic implementation, couldn't have ported it into Knet, but let's discuss it later since it is going to be as slower as the current implementation.

I've looked at PyTorch's source code and I think this is the part where transpose magic happens. As different from Julia, PyTorch doesn't have a permutedims actually, it has transpose for multi-dimensional arrays also. Let's see the differences,

• PyTorch equivalent of permutedims(a, (1, 3, 2)) is a.transpose(1, 2) (let's do not forget Python has indexes starting from zero).
• PyTorch equivalent of permutedims(a, (3, 1, 2)) is a.transpose(0, 2).transpose(1, 2) which is cascaded.
• PyTorch arrays are row majored. So, we need to convert that CUBLAS calls as column majored.

As a first step, I'm going to implement a transposedim operation mimicking PyTorch's transpose method. We'll discuss the remaining features later.

[ilkerkesen]

Caffe2 has a multi-dimensional transpose operation in the way we desire. It takes advantage of cudnn's cudnnTransformTensor kernel. I'm on it.

[denizyuret]

Solved using CuArrays under permutedims branch, please test.