Is there a way to convert a 1x1 2D Tensor to scalar?

[Kublai-Jing]

Hi,
My code looks something like this which is computing the square error of NN.

real_t = (sumall_except_dim<2>(sum_rows(F(outgrad)) / (real_t) pred.shape[1];

however, this seems not working with the following error msg.

error: cannot convert 'mshadow::expr::BinaryMapExp<mshadow::op::div, mshadow::expr::ReduceTo1DExp<mshadow::expr::ReduceTo1DExp<mshadow::expr::UnaryMapExp<square, mshadow::Tensor<mshadow::cpu, 2>, 1>, mshadow::red::sum, 0>, mshadow::red::sum, 2>, mshadow::expr::ScalarExp, 3>' to 'mshadow::real_t {aka float}' in initialization
real_t err = (sumall_except_dim<2>(sum_rows(F(outgrad))) / (real_t)pred.shape[1]);

which basically says the type is different.

I've also tried

(sumall_except_dim<2>(sum_rows(F(outgrad))[0]

but it says there is no operator[] defined on such data type.
It seems that I need to explicit go over the tensor and sum the result(which could be slow compared to optimized vectorized code)?

Thank you.

[tqchen]

You can not direct read scalar from GPU tensor. Currently, one thing you can possibly do is store the result to a Tensor2D and copy to CPU to read it. There was a performance reason why we did not support summing over all the elements in GPU.

If you are implementing neural nets, usually copy to CPU to evaluate the statistics won't harm the overall performance, because this was a cheap operation

[Kublai-Jing]

Hi tq,
Actually I am only going to do this on CPU, so do you suggest I should just do thing like this?

for(unsigned int i = 0 ; i < outgrad.shape[1] ; i ++){
for(unsigned int j = 0 ; j < outgrad.shape[2] ; j ++){
err += outgrad[i][j] * outgrad[i][j];
}
}

Thanks a lot !

[tqchen]

If it was a 2D tensor, remember C++ index starts from 0, so it should be shape[1] and shape[0].. If you use master version of mshadow.

for(unsigned int i = 0 ; i < outgrad.size(0) ; i ++){
    for(unsigned int j = 0 ; j < outgrad.size(1) ; j ++){
        err += outgrad[i][j] * outgrad[i][j];
     }
}

The shape convention in master was reversed from numpy and a bit weird. There is going to be a major refactor to solve the problem

[Kublai-Jing]

so shape[0] is the number of columns in a 2D tensor
and shape[1] is the number of rows, Is this correct ? What if we have a 3D Tensor ?
I am kind of confused by this phrase in the tutorial:
"shape_[0] gives the lowest dimension, shape_[1] gives the second dimension, etc. This is different from numpy."

what does it mean by "the lowest dimension"

Thanks!

[tqchen]

Sorry about the confusion, this is a deprecated convention that will be changed. For temporal solution, you can use tensor.size(i), updated in master, which follows the same convention as numpy. I have updated the example

[Kublai-Jing]

Got it.

Thanks !

[WallSky]

you are welcome