-
Notifications
You must be signed in to change notification settings - Fork 6.8k
C++ api executor->Forward(false) is much slower than MXPredForward? #12159
Comments
@XIAOJINGXIE : This is a good question. Could you also post this on discuss.mxnet.io ? The question will receive a larger audience. @mxnet-label-bot [C++, C, Question] |
Hi @XIAOJINGXIE The Executor->Forward() method performs more tasks as compared to MXPredForward(). It invokes following 2 C APIs internally Therefore, we can not compare MXPredForward() with Executor->Forward(). For the correct comparison, we should find out the time required to invoke MXPredGetOutputShape() and MXPredGetOutput() after invoking the MXPredForward() I hope this answers the question. |
Thanks a lot. By the way, the method name Forward() is a bit misleading. |
The running time of C API MXPredForward is much shorter than the running time of MXPredGetOutput:
The result is:
Why? Is it something related to lazy evaluation? |
Description
hi,
I am using the c++ api for prediction based on mobilenet v2. And I found that the codes belows runs 100+ms while I implement the same function using the c api, 10ms only. Both are running on GPU.
c++ version:
executor = net.SimpleBind(global_ctx, args_map, map<string, NDArray>(),
map<string, OpReqType>(), aux_map);
executor->Forward(false); // around 110ms
c version:
MXPredCreate((const char *) json_data.GetBuffer(),
(const char *) param_data.GetBuffer(),
static_cast<size_t>(param_data.GetLength()),
dev_type,
dev_id,
num_input_nodes,
input_keys,
input_shape_indptr,
input_shape_data,
&pred_hnd);
MXPredSetInput(pred_hnd, "data", image_data.data(), image_size);
// Do Predict Forward
MXPredForward(pred_hnd); // 10ms
Is there something that i am doing wrong?
The text was updated successfully, but these errors were encountered: