【PaddlePaddle Hackathon 4】add paddle softshrink op

[AndPuQing]

Details:

add softshrink op in paddle front end

$$ softshrink(x)=\left{\begin{aligned} x-\text { threshold, } & \text { if } x>\text { threshold } \\ x+\text { threshold, } & \text { if } x<-\text { threshold } \\ 0, & \text { otherwise } \end{aligned}\right. $$

Tickets:

• ticket-id

Reference

https://www.paddlepaddle.org.cn/documentation/docs/en/api/paddle/nn/functional/softshrink_en.html

[OpenVINO-dev-contest]

I just wonder can we use select here ?
https://github.com/openvinotoolkit/openvino/blob/master/docs/ops/condition/Select_1.md

[AndPuQing]

Yes, it can be implemented using select, which looks more concise than the original implementation, but I wonder if matrix multiplication will be more efficient than select ?

[OpenVINO-dev-contest]

My suggestion is to use benchmark_app to test the performance of single node graph, and find out a best implementation ?

[ceciliapeng2011]

Refer to Paddle spec, input x could be float32 and float64 only. How can a test case trigger this else branch?

[liubo-intel]

@ceciliapeng2011 : the other Paddle 'softshrink' api defined here : https://www.paddlepaddle.org.cn/documentation/docs/zh/api/paddle/nn/Softshrink_cn.html#softshrink
looks like there is no such limitation for input x type. To cover this api, I think we may still need the branch to handle '-lambd to unsigned type' cases.

[liubo-intel]

Hi, @AndPuQing : about @ceciliapeng2011 's concern, could you please have a try to add such unsigned type test case by using paddle.nn.Softshrink() api?
if such test case is difficult to generate from paddle side, let's disable such unsigned input support(delete the else branch) this time, and add an input type check PADDLE_OP_CHECK(node, input_element_type.is_signed(), "softshrink only supports singed input data type"); for this op.

[AndPuQing]

I test the unsighed tensor input on the paddlepaddle delelop version, there is the result:

>>>import paddle
>>>a = paddle.to_tensor([1],dtype='uint8')
>>>a
Out[3]:
Tensor(shape=[1], dtype=uint8, place=Place(gpu:0), stop_gradient=True,
       [1])
>>>paddle.nn.functional.softshrink(a)
RuntimeError: (NotFound) The kernel with key (GPU, Undefined(AnyLayout), uint8) of kernel `softshrink` is not registered and fail to fallback to CPU one. Selected wrong DataType `uint8`. Paddle support following DataTypes: float64, float16, float32, bfloat16.
  [Hint: Expected kernel_iter != iter->second.end(), but received kernel_iter == iter->second.end().] (at /paddle/paddle/phi/core/kernel_factory.cc:259)

So, I think just add dtype check currently.