Support PreProcessChannel in PreProcessInfo interface

[huningxin]

It is required to support mean and std values. Please refer to the API doc of PreProcessChannel.

[lionkunonly]

@huningxin Hi, Ningxin. In the native OpenVINO, the user can get and set the PreProcessChannel by overloading [] operator.

It can be seen here https://github.com/openvinotoolkit/openvino/blob/ffe4a74169da049f39a8590199636de1bf6d9c42/inference-engine/include/ie_preprocess.hpp#L75 and https://github.com/openvinotoolkit/openvino/blob/ffe4a74169da049f39a8590199636de1bf6d9c42/inference-engine/include/ie_preprocess.hpp#L93

I think the overloaded function is hard to implement by using N-API, and it needs to return the class PreProcessChannel::Ptr&. So, could I create two different function object getPreProcessChannel(int index) and setPreProcessChannel(int index, object Channel) to get PreProcessChannel and set PreProcessChannel respectively? Do you have any suggestions?

[huningxin]

could I create two different function object getPreProcessChannel(int index) and setPreProcessChannel(int index, object Channel) to get PreProcessChannel and set PreProcessChannel respectively? Do you have any suggestions?

It sounds good. I think we can support the usage like

preprocessinfo.setPreProcessChannel(0, {stdScale: 127.5, meanValue: 127.5});

The API could be

dictionary PreProcessChannel {
  optional float stdScale = 1;
  optional float meanValue = 0;
  optional ArrayBuffer meanData = null;
};

partial interface PreProcessInfo {
  PreProcessChannel getPreProcessChannel(unsigned long channel);
  void setPreProcessChannel(unsigned long channel, PreProcessChannel preProcessChannel);
};

[lionkunonly]

@huningxin . In the test of API getPreProcessChannel(), I find the size of _channelsInfo in class PreProcessInfo in file network.js is 0. It means that I can not take the action _channelsInfo[index], because the channel vector is empty. So I can not test the API successfully.

Defination of _channelsInfo in the class PreProcessInfo: https://github.com/openvinotoolkit/openvino/blob/ffe4a74169da049f39a8590199636de1bf6d9c42/inference-engine/include/ie_preprocess.hpp#L57

Meanwhile, should I add a data structure for PreProcessChannel in utils.h and utils.cc. Or just finish it as a object like version in core.cc and core.h. I am on the second way now.

[huningxin]

@huningxin . In the test of API getPreProcessChannel(), I find the size of _channelsInfo in class PreProcessInfo in file network.js is 0. It means that I can not take the action _channelsInfo[index], because the channel vector is empty. So I can not test the API successfully.

I think you should expose the PreProcessInfo.init that allows the js code to initialize the _channelsInfo.

Meanwhile, should I add a data structure for PreProcessChannel in utils.h and utils.cc. Or just finish it as a object like version in core.cc and core.h. I am on the second way now.

I am fine with the second way.

[lionkunonly]

@huningxin . I meet a issue here in the implementation. I find that the Blob::Ptr meanData in the initialized PreProcessInfo can not be casted to MemoryBlob. It blocks me to get the rmap() and wmap() from the MemoryBlob so that the data in the meanData can not be changed easily by using ArrayBuffer data.

Do you know what causes the casting error? I am not sure about it, but I think it may cause by the defination of the meanData. The code only declares this variable but no value is assigned to it. Do you have any suggestions?

[huningxin]

@huningxin . I meet a issue here in the implementation. I find that the Blob::Ptr meanData in the initialized PreProcessInfo can not be casted to MemoryBlob. It blocks me to get the rmap() and wmap() from the MemoryBlob so that the data in the meanData can not be changed easily by using ArrayBuffer data.

Could you please show me the code? You can create a PR for your work in progress.

Did you check whether meanData is a null ptr? if (meanData.get() == nullptr)

And in your first PR, probably you can skip the meanData support, we can leave it to following PR.

[lionkunonly]

Did you check whether meanData is a null ptr? if (meanData.get() == nullptr)

You are right, Ningxin. The meanData is a null ptr. It is the reason that it can not cast to the MemoryBlob.

[huningxin]

@lionkunonly , during review the PR #30 , I am thinking about we need to allow js code to specify the tensor description of the meanData. The change is to introduce the TensorDesc and MeanData to combine tensor description and data together.

dictionary TensorDesc {
  required Precision precision;
  required sequence<unsigned long long> dims,
  required Layout layout;
};

dictionary MeanData {
  required TensorDesc desc;
  required ArrayBuffer data;
};

dictionary PreProcessChannel {
  optional float stdScale = 1;
  optional float meanValue = 0;
  optional MeanData meanData = null;
};

[lionkunonly]

With the PR Preprocesschannel with meanData merged, this issue should be closed.