Operators that need to port to new framework

[jacquesqiao]

• paddle/paddle/gserver/layers
  • 108
• paddle/paddle/gserver/activations
• paddle/paddle/gserver/evaluators
  • need to redesign as Operators.

[qingqing01]

• Need a doc about tow to write new layers.
• Op规范：https://github.com/PaddlePaddle/Paddle/pull/3452/files 此规范需要完善
  • 输入/输出命名规范
  • 属性命名规范
  • 公共功能提炼成functor，functor写法规范(operator() or static function)
• cuDNN Wrapper
• 定义ElementWise Op的宏 (class op or functor?)

下面Op排除了operators里已经实现的

• ElementWise:

power
tanh
exp
relu
brelu
softrelu
stanh
abs
square
log
sqrt
reciprocal

• Base on dense tensor.

cnn:

conv2d (cudnn, gemm) (需调研其他框架组织形式)
conv3d
de-conv2d
de-conv3d
img_pool_layer (max, avg)
img_3dpool_layer
spp_layer (pooling组合)
conv_operator

fc_layer
concat_layer
maxout_layer
cross_channel_norm_layer
img_cmrnorm_layer
prelu_layer


math: 
scaling_layer
cos_sim
bilinear_interp_layer
trans_layer
pad_layer
crop_layer
clip_layer
reshape_layer
slice
dropout_layer
dotmul_projection
conv_shift_layer
row_l2_norm_layer
tensor_layer
interpolation_layer
sum_to_one_norm_layer
convex_comb_layer
multiplex_layer
featuremap_expand
out_prod_layer
scale_shift_layer（a*x + b, a,b是学习的参数，该layer需要拆分）

sampling:
sampling_id_layer

• base on sequence info

recurrent_layer
grumemory
lstmemory
gru_step_layer
lstm_step_layer
last_seq
first_seq
context_projection (需增强支持window，stride)
pooling_layer (sequence pooling，需增强支持window，stride，考虑可否与img_pool合并)

seq_concat_layer
seq_reshape_layer
expand_layer

ctc_layer
warp_ctc_layer
crf_layer
crf_decoding_layer

sequence_softmax
block_expand_layer
row_conv_layer
multibox_loss_layer
eos_layer

• Cost

hsigmoid
mse_cost: label包括dense，sparse两类型，需要分成两个Op
nce_layer
cross_entropy_with_selfnorm
smooth_l1_cost
rank_cost
huber_cost
lambda_cost (base on sequence info)

• Base on sparse tensor

selective_fc_layer
multi_binary_label_cross_entropy

• Evaluator

top_k 
classification_error
auc
pnpair
precision_recall
ctc_edit_distance (ctc_error)
chunk_evaluator
detection_map_evaluator

[qingqing01]

Welcome to add new and necessary operators.

[qingqing01]

from @Superjom

感觉， elementmise 类似的应该是作为 helper function 好一些吧
elementwise_check() ...
same_shape_check(inputs) ...

from @zchen0211

大量被各处使用的功能会被定义为functor()

[chengduoZH]

还有 img_pool3D_layer (max, avg)，Stochastic Pooling