BatchNorm layer failed with width==height==1

[tensor-tang]

Found the gtest failed with BN layer when manually setting h==w==1

Line 406 at tests/gtests/test_batch_normalization.cpp

INST_TEST_CASE(Simple_NCHW,
    PARAMS(nchw, nchw, 2, 10, 1, 1, EPS)
);

[rsdubtso]

Thanks. Reproduced. Will try to get this fixed this week.

[tensor-tang]

Thanks.

[rsdubtso]

Sorry for a delay. I've not committed a fix, but I have some news. This looks like a test issue. The bnorm kernel precomputes some values as an optimization while the test does not. Also, depending on the compiler, the test may or may not use FMA instructions. I'm looking into finding a way to change the test to allow such behavior while not relaxing the accuracy requirements too much.

[tensor-tang]

I calculated this iw==ih==1 case with ATLAS outside, but got different result with same input data.
While other cases are all equals.

[rsdubtso]

Not that I'm proud of this, but I had to resort to relaxing the precision requirements... I've added some 1x1 cases which now pass. Closing... (note to self: close this via a commit message next time)