I cannot make mxnet.ndarray.UpSampling work with bilinear interpolation

[feevos]

Description

Dear all, when I run UpSampling (python, mxnet.version == 1.0.0) with nearest interpolation, everything goes normal:

nfilters = 16
xx = nd.random_normal(shape=[2,nfilters,64,64],ctx=mx.cpu())
print xx.asnumpy().shape
temp = nd.UpSampling(xx,scale=2,sample_type='nearest')
print temp.asnumpy().shape

when I try to perform sample_type = 'bilinear' I get errors:

nfilters = 16
xx = nd.random_normal(shape=[2,nfilters,64,64],ctx=mx.cpu())
print xx.asnumpy().shape
temp = nd.UpSampling(xx,scale=2,sample_type='bilinear')
print temp.asnumpy().shape

any pointers/ideas on what I am doing wrong? I need it to work correctly for both ndarray and mx.sym (but I assume should be the same for both).

Thank you

Error Message:

---------------------------------------------------------------------------
MXNetError                                Traceback (most recent call last)
<ipython-input-57-7b8d60ea54bb> in <module>()
      3 xx = nd.random_normal(shape=[2,nfilters,64,64],ctx=mx.cpu())
      4 print xx.asnumpy().shape
----> 5 temp = mx.nd.UpSampling(xx,scale=2,sample_type='bilinear')
      6 print temp.asnumpy().shape

/home/dia021/anaconda2/lib/python2.7/site-packages/mxnet/ndarray/register.pyc in UpSampling(*data, **kwargs)

/home/dia021/anaconda2/lib/python2.7/site-packages/mxnet/_ctypes/ndarray.pyc in _imperative_invoke(handle, ndargs, keys, vals, out)
     90         c_str_array(keys),
     91         c_str_array([str(s) for s in vals]),
---> 92         ctypes.byref(out_stypes)))
     93 
     94     if original_output is not None:

/home/dia021/anaconda2/lib/python2.7/site-packages/mxnet/base.pyc in check_call(ret)
    144     """
    145     if ret != 0:
--> 146         raise MXNetError(py_str(_LIB.MXGetLastError()))
    147 
    148 

MXNetError: [17:20:11] src/c_api/../imperative/imperative_utils.h:303: Check failed: num_inputs == infered_num_inputs (1 vs. 2) Operator UpSampling expects 2 inputs, but got 1 instead.

Stack trace returned 10 entries:
[bt] (0) /home/dia021/anaconda2/lib/python2.7/site-packages/mxnet/libmxnet.so(+0x289a1c) [0x7fe0ed9d6a1c]
[bt] (1) /home/dia021/anaconda2/lib/python2.7/site-packages/mxnet/libmxnet.so(+0x240538f) [0x7fe0efb5238f]
[bt] (2) /home/dia021/anaconda2/lib/python2.7/site-packages/mxnet/libmxnet.so(+0x24029a2) [0x7fe0efb4f9a2]
[bt] (3) /home/dia021/anaconda2/lib/python2.7/site-packages/mxnet/libmxnet.so(MXImperativeInvokeEx+0x63) [0x7fe0efb4ffb3]
[bt] (4) /home/dia021/anaconda2/lib/python2.7/lib-dynload/_ctypes.so(ffi_call_unix64+0x4c) [0x7fe12e6dd57c]
[bt] (5) /home/dia021/anaconda2/lib/python2.7/lib-dynload/_ctypes.so(ffi_call+0x1f5) [0x7fe12e6dccd5]
[bt] (6) /home/dia021/anaconda2/lib/python2.7/lib-dynload/_ctypes.so(_ctypes_callproc+0x3e6) [0x7fe12e6d4376]
[bt] (7) /home/dia021/anaconda2/lib/python2.7/lib-dynload/_ctypes.so(+0x9db3) [0x7fe12e6cbdb3]
[bt] (8) /home/dia021/anaconda2/bin/../lib/libpython2.7.so.1.0(PyObject_Call+0x53) [0x7fe13375de93]
[bt] (9) /home/dia021/anaconda2/bin/../lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x715d) [0x7fe13381080d]

[chowkamlee81]

you have forgotten to put num_filter

u can use like below

heat_map = mx.symbol.UpSampling(conv4_scale0_relu, num_filter=1, scale=2, sample_type='bilinear', num_args=2,name="upsampling1")

[feevos]

@chowkamlee81 Thank you very much for your answer. I tried this with ndarray instead (the operator UpSampling exists for both ndarray and Symbol) and it doesn't work:

xx = nd.random_normal(shape=[2,16,256,256],ctx=mx.cpu())
temp = nd.UpSampling(xx, num_filter=1, scale=2, sample_type='bilinear', num_args=2,name="upsampling1")

Error message:

MXNetError                                Traceback (most recent call last)
<ipython-input-6-b234addee343> in <module>()
      1 xx = nd.random_normal(shape=[2,16,256,256],ctx=mx.cpu())
----> 2 temp = nd.UpSampling(xx, num_filter=1, scale=2, sample_type='bilinear', num_args=2,name="upsampling1")

/home/dia021/anaconda2/lib/python2.7/site-packages/mxnet/ndarray/register.pyc in UpSampling(*data, **kwargs)

/home/dia021/anaconda2/lib/python2.7/site-packages/mxnet/_ctypes/ndarray.pyc in _imperative_invoke(handle, ndargs, keys, vals, out)
     90         c_str_array(keys),
     91         c_str_array([str(s) for s in vals]),
---> 92         ctypes.byref(out_stypes)))
     93 
     94     if original_output is not None:

/home/dia021/anaconda2/lib/python2.7/site-packages/mxnet/base.pyc in check_call(ret)
    144     """
    145     if ret != 0:
--> 146         raise MXNetError(py_str(_LIB.MXGetLastError()))
    147 
    148 

MXNetError: [13:39:11] src/c_api/../imperative/imperative_utils.h:303: Check failed: num_inputs == infered_num_inputs (1 vs. 2) Operator UpSampling expects 2 inputs, but got 1 instead.

Stack trace returned 10 entries:
[bt] (0) /home/dia021/anaconda2/lib/python2.7/site-packages/mxnet/libmxnet.so(+0x289a1c) [0x7fde399d6a1c]
[bt] (1) /home/dia021/anaconda2/lib/python2.7/site-packages/mxnet/libmxnet.so(+0x240538f) [0x7fde3bb5238f]
[bt] (2) /home/dia021/anaconda2/lib/python2.7/site-packages/mxnet/libmxnet.so(+0x24029a2) [0x7fde3bb4f9a2]
[bt] (3) /home/dia021/anaconda2/lib/python2.7/site-packages/mxnet/libmxnet.so(MXImperativeInvokeEx+0x63) [0x7fde3bb4ffb3]
[bt] (4) /home/dia021/anaconda2/lib/python2.7/lib-dynload/_ctypes.so(ffi_call_unix64+0x4c) [0x7fde7725657c]
[bt] (5) /home/dia021/anaconda2/lib/python2.7/lib-dynload/_ctypes.so(ffi_call+0x1f5) [0x7fde77255cd5]
[bt] (6) /home/dia021/anaconda2/lib/python2.7/lib-dynload/_ctypes.so(_ctypes_callproc+0x3e6) [0x7fde7724d376]
[bt] (7) /home/dia021/anaconda2/lib/python2.7/lib-dynload/_ctypes.so(+0x9db3) [0x7fde77244db3]
[bt] (8) /home/dia021/anaconda2/bin/../lib/libpython2.7.so.1.0(PyObject_Call+0x53) [0x7fde7c2d6e93]
[bt] (9) /home/dia021/anaconda2/bin/../lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x715d) [0x7fde7c38980d]

it seems it expects a different kind of input (also weights of some sort? and if yes, what dimensionality?)

from here:

src/c_api/../imperative/imperative_utils.h:303: 
Check failed: num_inputs == infered_num_inputs (1 vs. 2) 
Operator UpSampling expects 2 inputs, but got 1 instead.

I do not understand how to solve this.

[chowkamlee81]

Your shape is shape =[2,16,256,256]
But number of filter =2 in your code
Also in your code batch size=2

check below code :
xx = nd.random_normal(shape=[1,1,256,256],ctx=mx.cpu()) ###(NCHW)
temp = nd.UpSampling(xx, num_filter=1, scale=2, sample_type='bilinear', num_args=2,name="upsampling1")

[feevos]

@chowkamlee81

I tried the code you've sent me (exact copy/paste), I again obtain the same error - can this be a version related error? In addition, from the documentation of ndarray.UpSampling the num_filter variable is not related (at least to my understanding) to the number of channels of the input, e.g. for input shape = [2,16,256,256] the variable num_filter is not related to the number shape[1]==16, it is somehow related to the bilinear interpolation. By the way, the argument num_args doesn't appear in the official documentation either.

Thanks!

Code:

import mxnet as mx
import mxnet.ndarray as nd

xx = nd.random_normal(shape=[1,1,256,256],ctx=mx.cpu()) ###(NCHW)
temp = nd.UpSampling(xx, num_filter=1, scale=2, sample_type='bilinear',num_args=2,name="upsampling1")

Error

---------------------------------------------------------------------------
MXNetError                                Traceback (most recent call last)
<ipython-input-1-e85163ad8480> in <module>()
      3 
      4 xx = nd.random_normal(shape=[1,1,256,256],ctx=mx.cpu()) ###(NCHW)
----> 5 temp = nd.UpSampling(xx, num_filter=1, scale=2, sample_type='bilinear', num_args=2,name="upsampling1")

/home/dia021/anaconda2/lib/python2.7/site-packages/mxnet/ndarray/register.pyc in UpSampling(*data, **kwargs)

/home/dia021/anaconda2/lib/python2.7/site-packages/mxnet/_ctypes/ndarray.pyc in _imperative_invoke(handle, ndargs, keys, vals, out)
     90         c_str_array(keys),
     91         c_str_array([str(s) for s in vals]),
---> 92         ctypes.byref(out_stypes)))
     93 
     94     if original_output is not None:

/home/dia021/anaconda2/lib/python2.7/site-packages/mxnet/base.pyc in check_call(ret)
    144     """
    145     if ret != 0:
--> 146         raise MXNetError(py_str(_LIB.MXGetLastError()))
    147 
    148 

MXNetError: [17:05:49] src/c_api/../imperative/imperative_utils.h:303: Check failed: num_inputs == infered_num_inputs (1 vs. 2) Operator UpSampling expects 2 inputs, but got 1 instead.

Stack trace returned 10 entries:
[bt] (0) /home/dia021/anaconda2/lib/python2.7/site-packages/mxnet/libmxnet.so(+0x289a1c) [0x7f6e699d6a1c]
[bt] (1) /home/dia021/anaconda2/lib/python2.7/site-packages/mxnet/libmxnet.so(+0x240538f) [0x7f6e6bb5238f]
[bt] (2) /home/dia021/anaconda2/lib/python2.7/site-packages/mxnet/libmxnet.so(+0x24029a2) [0x7f6e6bb4f9a2]
[bt] (3) /home/dia021/anaconda2/lib/python2.7/site-packages/mxnet/libmxnet.so(MXImperativeInvokeEx+0x63) [0x7f6e6bb4ffb3]
[bt] (4) /home/dia021/anaconda2/lib/python2.7/lib-dynload/_ctypes.so(ffi_call_unix64+0x4c) [0x7f6ea919357c]
[bt] (5) /home/dia021/anaconda2/lib/python2.7/lib-dynload/_ctypes.so(ffi_call+0x1f5) [0x7f6ea9192cd5]
[bt] (6) /home/dia021/anaconda2/lib/python2.7/lib-dynload/_ctypes.so(_ctypes_callproc+0x3e6) [0x7f6ea918a376]
[bt] (7) /home/dia021/anaconda2/lib/python2.7/lib-dynload/_ctypes.so(+0x9db3) [0x7f6ea9181db3]
[bt] (8) /home/dia021/anaconda2/bin/../lib/libpython2.7.so.1.0(PyObject_Call+0x53) [0x7f6eae213e93]
[bt] (9) /home/dia021/anaconda2/bin/../lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x715d) [0x7f6eae2c680d]

[zhanghang1989]

I got the same error. Any updates?

[feevos]

I have found this answer helpful, however I don't have a document reference on how to actually create the weights. So I am stack at nearest interpolation.

[zhanghang1989]

Thanks @feevos! A new bilinear upsample operator is coming in the PR #9688
Stay tuned.

[zhanghang1989]

@szha This issue can be closed in favor of #9688

import mxnet as mx
x1 = mx.nd.ones(shape=(2,3,4,4))
y1 = mx.nd.contrib.BilinearResize2D(x1, out_height=5, out_width=5)