Should Tensor not to have device as a template parameter?

[wangkuiyi]

In mshadow tutorial, it shows that Tensor is defined as

template<typename Device, int dimension, typename DType = float>
struct Tensor {

We know that class template is not a type, but class is a type. To make above class template into a type, we need to specialize it like Tensor<gpu, ...> or Tensor<cpu, ...>. This is equivalent to define a GPU-tensor and a CPU-tensor. However, tensor, as a mathematical concept, means "a multi-dimensional matrix", not "a multi-dimensional matrix on GPU (or CPU)". This design differs from the definition of "tensor".

An alternative design is to make the device, or more accurately, place, a parameter of the constructor of class Tensor:

class Tensor {
 public:
  Tensor(Place place, ...);

This is the idea behind Majel, the Tensor library behind DeepSpeech 2.

We can see that these two designs lead to different usage. Consider that if we are going to define a function move that moves a tensor from one place to another. Following the definition of mshadow, and if we are going to provide a C binding (no C++ template), we need to define at least three variations of move:

void move(Tensor<gpu>* src, Tensor<cpu>* dest);
void move(Tensor<gpu>* src, Tensor<gpu>* dest);
void move(Tensor<cpu>* src, Tensor<gpu>* dest);

But following Majel's definition, we need only one:

void move(Tensor* src, Tensor* dest);

I just noticed this difference when I was comparing some existing Tensor libraries for adoption by PaddlePaddle. Dear authors of mshadow, what do you think about this difference?

[tqchen]

It is mainly about compile and runtime specialization. We are adopting TBlob and https://github.com/dmlc/dlpack for type erased tensor that erases device type and data type into fields.

The templates are used for compilation code specialization

[wangkuiyi]

I see. Template parameter Device here is not for denoting where the data is stored, thus not the same purpose as majel::Place. Thanks! @tqchen