InferShape return false not caught in Symbolic mode

[apeforest]

Description

When the inputs to an operator is invalid, the InferShape in the operator returns false. This return value is not caught and treated properly in Symbolic mode, whereas the imperative mode would raise error.

Environment info (Required)

----------Python Info----------
Version      : 3.6.5
Compiler     : GCC 4.2.1 Compatible Apple LLVM 9.0.0 (clang-900.0.39.2)
Build        : ('default', 'Jun 17 2018 12:26:58')
Arch         : ('64bit', '')
------------Pip Info-----------
Version      : 18.0
Directory    : /Users/lnyuan/.virtualenvs/mxnet/lib/python3.6/site-packages/pip
----------MXNet Info-----------
Version      : 1.3.0
Directory    : /Users/lnyuan/work/incubator-mxnet/python/mxnet
Hashtag not found. Not installed from pre-built package.
----------System Info----------
Platform     : Darwin-16.7.0-x86_64-i386-64bit
system       : Darwin
node         : 88e9fe759c49.ant.amazon.com
release      : 16.7.0
version      : Darwin Kernel Version 16.7.0: Thu Jun 21 20:07:39 PDT 2018; root:xnu-3789.73.14~1/RELEASE_X86_64
----------Hardware Info----------
machine      : x86_64
processor    : i386
b'machdep.cpu.extfeatures: SYSCALL XD 1GBPAGE EM64T LAHF LZCNT PREFETCHW RDTSCP TSCI'
b'machdep.cpu.leaf7_features: SMEP ERMS RDWRFSGS TSC_THREAD_OFFSET BMI1 AVX2 BMI2 INVPCID SMAP RDSEED ADX IPT SGX FPU_CSDS MPX CLFSOPT'
b'machdep.cpu.features: FPU VME DE PSE TSC MSR PAE MCE CX8 APIC SEP MTRR PGE MCA CMOV PAT PSE36 CLFSH DS ACPI MMX FXSR SSE SSE2 SS HTT TM PBE SSE3 PCLMULQDQ DTES64 MON DSCPL VMX EST TM2 SSSE3 FMACX16 TPR PDCM SSE4.1 SSE4.2 x2APIC MOVBE POPCNT AES PCID XSAVE OSXSAVE SEGLIM64 TSCTMR AVX1.0 RDRAND F16C'
b'machdep.cpu.brand_string: Intel(R) Core(TM) i7-7700HQ CPU @ 2.80GHz'
----------Network Test----------
Setting timeout: 10
Timing for MXNet: https://github.com/apache/incubator-mxnet, DNS: 0.0155 sec, LOAD: 0.5818 sec.
Timing for Gluon Tutorial(en): http://gluon.mxnet.io, DNS: 0.0546 sec, LOAD: 0.2440 sec.
Timing for Gluon Tutorial(cn): https://zh.gluon.ai, DNS: 0.0239 sec, LOAD: 0.2014 sec.
Timing for FashionMNIST: https://apache-mxnet.s3-accelerate.dualstack.amazonaws.com/gluon/dataset/fashion-mnist/train-labels-idx1-ubyte.gz, DNS: 0.0233 sec, LOAD: 0.1084 sec.
Timing for PYPI: https://pypi.python.org/pypi/pip, DNS: 0.0113 sec, LOAD: 0.4376 sec.
Timing for Conda: https://repo.continuum.io/pkgs/free/, DNS: 0.0187 sec, LOAD: 0.0672 sec.

Package used (Python/R/Scala/Julia):
Python

Minimum reproducible example

import mxnet as mx

cond = mx.sym.Variable('cond')
x = mx.sym.Variable('x')
y = mx.sym.Variable('y')
out = mx.sym.where(cond, x, y)

test = lambda c: out.eval(x=mx.nd.array([[2,3],[4,5],[6,7]]),
                                           y=mx.nd.array([[8,9],[10,11],[12,13]]),
                                           cond=mx.nd.array(c))
## invalid input but no error is raised
print(test([[1, 0, 1],[1,1,0]]))

test = lambda c: mx.nd.where(x=mx.nd.array([[2,3],[4,5],[6,7]]),
                                                   y=mx.nd.array([[8,9],[10,11],[12,13]]),
                                                   cond=mx.nd.array(c))
# error raised in imperative mode
print(test([[1, 0, 1],[1,1,0]]))

[ankkhedia]

@mxnet-label-bot [Python, Bug]

[apeforest]

@mxnet-label-bot [Operator]

[apeforest]

I will work on this issue: https://issues.apache.org/jira/browse/MXNET-865

[piyushghai]

@apeforest What's the progress on this one ?

[apeforest]

@piyushghai I don't have the bandwidth to work on this one now. Please label it [Call for contribution]

[karan6181]

There are two scenarios here in which where operator should pass.

1. When the condition is of 1D array, the below code(snippet from control_flow_op.h in method WhereOpShape()) checks for size equality which is correct given number of dimension is just 1.

Line: 191

 } else if ((*in_attrs)[0].ndim() == 1) {
    CHECK_EQ((*in_attrs)[0].Size(), static_cast<size_t>(tshape[0]));
    return true;

Correct scenario where the test pass:

condition = mx.sym.Variable('condition')
x = mx.sym.Variable('x')
y = mx.sym.Variable('y')
where_sym = mx.sym.where(condition, x, y)

where_sym.eval(x=mx.nd.array([[2,3],[4,5],[6,7]]),
               y=mx.nd.array([[8,9],[10,11],[12,13]]),
               condition=mx.nd.array([1,0,1])) # 1D array

Correct scenario where the test Fails:

condition = mx.sym.Variable('condition')
x = mx.sym.Variable('x')
y = mx.sym.Variable('y')
where_sym = mx.sym.where(condition, x, y)

where_sym.eval(x=mx.nd.array([[2,3],[4,5],[6,7]]),
               y=mx.nd.array([[8,9],[10,11],[12,13]]),
               condition=mx.nd.array([1,0,1,1])) # Incorrect 1D array

The 1D array condition is working perfectly as expected in symbolic mode. It throws error if there is a dimension mismatch.

2. condition should have the same shape as input x. The below code checks for shape_assign between condition and x where it internally checks for dimension compatibility, and it's returning False(Which is expected behaviour if there is a dimension mismatch), but the test is not throwing any error(STRANGE).

Line: 187

if ((*in_attrs)[0].ndim() == tshape.ndim()) {
    if (!shape_assign(&tshape, (*in_attrs)[0])) return false; // This is returning false which is expected behavior when there is dimension mismatch between variable condition and variable x but the  script is not asserting. However, it passes which is strange.
    SHAPE_ASSIGN_CHECK(*in_attrs, 0, tshape);
    return true;

In reality, it should throw error something like:

mxnet.base.MXNetError: Error in operator where0: 
Shape inconsistent, Provided = [1,2], inferred shape=[3,2]

I will look into it further in detail. Please feel free to suggest if I am going in the wrong direction. Thanks!