new implementation of prelu

[On-JungWoan]

I implement the operation of PLeLU.

But, I have some limitations.

• Implementation of nn.PReLU

  • I have no idea of implementation of nn.PReLU

• Implementation of alpha (in torch.nn.functional.prelu, torch.prelu, torch.Tensor.prelu)

  • When I trying to receive the weight of prelu by specifying alpha, an unknown error occurs.
  • Below is the error I get

You are passing KerasTensor(type_spec=VariableSpec(shape=(), dtype=tf.float32, trainable=True, alias_id=None),
 name='weight:0', description="created by layer 'weight_layer'"), an intermediate Keras symbolic input/output, to a 
TF API that does not allow registering custom dispatchers, such as `tf.cond`, `tf.function`, gradient tapes, or `tf.map_fn`.
 Keras Functional model construction only supports TF API calls that *do* support dispatching, such as `tf.math.add` or
 `tf.reshape`. Other APIs cannot be called directly on symbolic Kerasinputs/outputs. You can work around this limitation
 by putting the operation in a custom Keras layer `call` and calling that layer on this symbolic input/output.

Can you advise for me?

[AlexanderLutsenko]

@On-JungWoan This one is a bit tricky. Tried to implement the prelu converters in v0.4.4, seem to be working:

nobuco/nobuco/node_converters/activation.py

Line 78 in a8985d3

def prelu_prepare_params(weight_np, input_dim, input_channels, input_channel_order):

One caveat is that a PReLU module cannot be reused in the tensorflow graph, it will be copied. Its parameters depend on the input channel order, so we don't have a good solution here.