Network implement

[dzhwinter]

No description provided.

[Superjomn]

这些 bool 都改成 Error 吧

[reyoung]

We should not use Error now, please use PADDLE_ENFORCE instead.

[Superjomn]

Several interfaces may be needed:

RNN need a proto or string as proto to create a StepNet, so need a method to serialize the net to proto or string:

void Serialize(NetDesc* net_desc) = 0

Forward or backward using Net.Run seems difficult because end_idx is hard to determine, so following methods may be needed:

void Forward() = 0;
void Backward() = 0;

@dzhwinter @reyoung

[Superjomn]

Do AddOptimizerOps and AddBackwardOps must be called in order?
AddBackwardOps first, AddOptimizerOps second?

@dzhwinter @reyoung @jacquesqiao

[jacquesqiao]

AddOptimizerOps should call after AddBackwardOps

[QiJune]

The AddBackwardOps will add all backward related operators. So, we have to find a way to tell the input and output variables of each backward operator.
And the Optimizer will take parameter/gradient/momentum/... as input and update the parameter.
The Optimizer operator will take calculate one pair of parameter and gradient. But the Optimizer will take all.
Since, we also have to make checkpoint of a Net parameters, I suggest we put forward a concept called Model.
The Model has some maps which records the relationship of parameters/gradients. And learning rate, batch_step can also be found in Model.
The Model has a method called CheckPoint, and will serialize its data member to disk.

[Superjomn]

After a discussion, the current decision is as follows:

• add a temporary solution for Net's backward @dzhwinter
• no need to add Serialize currently, use AddOp to build a step net for RNN.
• Model is a user-related API, currently is not needed.

[Superjomn]

LGM

[dzhwinter]

check in for @Superjom 's rnn op dependency.