Weird LogisticRegressionOutput Bug

[freddycct]

Whenever I swap out SoftmaxOutput for LogisticRegressionOutput,

fc3  = mx.symbol.FullyConnected(data = act2, name='fc3', num_hidden=10)
mlp  = mx.symbol.LogisticRegressionOutput(data = fc3, name = 'softmax')
# mlp  = mx.symbol.SoftmaxOutput(data = fc3, name = 'softmax')

I get this error,

    model.fit(X=image_iter, eval_data=image_test_iter)
  File "C:\Users\chuaf\AppData\Local\Continuum\Miniconda3\lib\site-packages\mxnet-0.5.0-py3.5.egg\mxnet\model.py", line 748, in fit
    self._init_params(dict(data.provide_data+data.provide_label))
  File "C:\Users\chuaf\AppData\Local\Continuum\Miniconda3\lib\site-packages\mxnet-0.5.0-py3.5.egg\mxnet\model.py", line 489, in _init_params
    arg_shapes, _, aux_shapes = self.symbol.infer_shape(**input_shapes)
  File "C:\Users\chuaf\AppData\Local\Continuum\Miniconda3\lib\site-packages\mxnet-0.5.0-py3.5.egg\mxnet\symbol.py", line 412, in infer_shape
    return self._infer_shape_impl(False, *args, **kwargs)
  File "C:\Users\chuaf\AppData\Local\Continuum\Miniconda3\lib\site-packages\mxnet-0.5.0-py3.5.egg\mxnet\symbol.py", line 472, in _infer_shape_impl
    ctypes.byref(complete)))
  File "C:\Users\chuaf\AppData\Local\Continuum\Miniconda3\lib\site-packages\mxnet-0.5.0-py3.5.egg\mxnet\base.py", line 77, in check_call
    raise MXNetError(py_str(_LIB.MXGetLastError()))
mxnet.base.MXNetError: InferShape Error in softmax's label argument
Corresponding keyword of symbol: softmax_label
Shape inconsistent, Provided =(10,), inferred shape=(10,10)

There is not much examples on binary classification using mxnet, is this correct in how I use LogisticRegressionOutput?

[piiswrong]

you need to change num_hidden to 1

[freddycct]

Odd for a number of reasons:

1. I thought that the num_hidden is the number of features given to the logistic regression classifier. In this case if num_hidden is 1, then it would not be a very interesting feature for the classifier.
2. This file https://github.com/dmlc/mxnet/blob/master/example/kaggle-ndsb2/Train.py has num_hidden=600

# first fullc
flatten = mx.symbol.Flatten(net)
flatten = mx.symbol.Dropout(flatten)
fc1 = mx.symbol.FullyConnected(data=flatten, num_hidden=600)
# Name the final layer as softmax so it auto matches the naming of data iterator
# Otherwise we can also change the provide_data in the data iter
return mx.symbol.LogisticRegressionOutput(data=fc1, name='softmax')

What should be the labels for binary classification? Is it 0 and 1, 1 and 2 or -1 and 1 if I use recordio, or the built-in ImageRecordIter

[rz1988]

I have the same issue here. If I change the output activation function to logistic regression and change the num of hidden units ( here they are essentially the number of outputs) to the same as my label, the model is able to run.

When there is only one output, you leave the second dimension empty. For example,

import mxnet as mx
data_set = mx.random.normal(0, 1, (10, 5))
label = mx.nd.ones((10, ))
train_iter = mx.io.NDArrayIter(data_set, label, batch_size = 1)

val_training = mx.random.normal(0, 1, (5, 5))
val_label = mx.nd.ones((5,2))
val_iter = mx.io.NDArrayIter(val_training, val_label, batch_size = 1)

data = mx.symbol.Variable('data')
fc1 = mx.symbol.FullyConnected(data, name='fc1', num_hidden=128)
act1 = mx.symbol.Activation(fc1, name='relu1', act_type='relu')
fc2 = mx.symbol.FullyConnected(act1, name='fc2', num_hidden= 1)
softmax = mx.symbol.LogisticRegressionOutput(fc2, name='softmax')

model = mx.model.FeedForward(
softmax,
num_epoch= 10,
learning_rate=0.01)

model.fit(X = train_iter)