The custom operator not supported for group context?

[mg0880gm]

Description

The group context was used for model parallelism of neural network train. The mx.sym.LogisticRegressionOutput works when it's been used for loss calculation, but not customized operator.

Environment info (Required)

OS: Ubuntu 14.04.5 LTS
Python: 2.7.13
mxnet: 0.12.1, pulled from master branch and built from source

Package used (Python/R/Scala/Julia):
Python: mxnet, numpy, scipy

Build info (Required if built from source)

Bulit from command line by following https://mxnet.incubator.apache.org/get_started/install.html

MXNet commit hash:
2f8c1e8

Build config:
Only change is to enable the profiler.

Error Message:

Traceback (most recent call last):
File "_ctypes/callbacks.c", line 315, in 'calling callback function'
File "/git/mxnet/python/mxnet/operator.py", line 621, in creator
op_prop = prop_cls(**kwargs)
TypeError: init() got an unexpected keyword argument 'ctx_group'

Minimum reproducible example

The script to generate the network, which works:

...
fc2 = mx.symbol.FullyConnected(data=concat, name='fcout_'+str(gpu), num_hidden=out_dim/num_gpus)
loss = mx.sym.LogisticRegressionOutput(data=fc2, label=labs)

The script to generate the network which used customized operator, and didn't work:

...
fc2 = mx.symbol.FullyConnected(data=concat, name='fcout_'+str(gpu), num_hidden=out_dim/num_gpus)
act2 = mx.sym.Activation(data=fc2, name='acout_'+str(gpu), act_type='sigmoid')
loss = mx.sym.Custom(data=act2, label=label, name='ce_'+str(gpu), op_type='CrossEntropyLoss')

Is this because the custom op is not support yet for group context?

[eric-haibin-lin]

@mg0880gm could you try latest version of MXNet and update the __init__ function of your XXCustomOpProp from

def __init__(self):

to

def __init__(self, **kwargs):

And see if it works?

[mbaijal]

Can you please add the labels - "Operator", "Bug", "Question", "Pending Requester Info"

[anirudh2290]

@mg0880gm are you still seeing this issue with the latest version of mxnet ?

[eric-haibin-lin]

Verified on my end that the fix works.