Discuss possible fluid imperative programming paradigms

[cs2be]

Abhinav, Helin and I have discuss possibilities for implement imperative paradigms. Here are two variants of fit a line example, one implemented using full imperative, the other using a variant of the current API. This proposal will need some discussion.

Note: These examples showcase possible python API/operators that may not currently exist in Paddle.

Fit a line with Imperative

In this scenario, the fluid executor will run after the block exits. Note this will require some methods to change (fluid.optimizer), which may affect backwards compatibility.

def train(place):
    with fluid.Program(place):
        batch_reader = fluid.layers.batch_reader(
            filename = './flowers.recordio', type='recordio',
            batch_size=100, shape=[[13], [1]], dtype=['float32', 'float32'])

        with fluid.While(step=100):
            x, y = fluid.layers.next_batch(batch_reader)
            y_predict = fluid.layers.fc(input=x, size=1, act=None)

            cost = fluid.layers.square_error_cost(input=y_predict, label=y)
            avg_cost = fluid.layers.mean(cost)

            sgd_optimizer = fluid.optimizer.SGD(learning_rate=0.001)
            sgd_optimizer.minimize(avg_cost)

            #fluid.Print(avg_cost)

            # This example shows how to fetch variables from within the scope
            # during execution of the ProgramDesc.  A new fetch operator can
            # be added to the ProgramDesc.  Its job will be to send the data to
            # the python host (using sockets or RPC), and wait for the host to
            # complete the request.  On the python side, the user can implement
            # some logic (like log the data, or send to database, ect).
            fluid.fetch([avg_cost], lambda ac: print(ac))


if __name__ == '__main__':
    train(fluid.CPUPlace)

Fit a line without full imperative, but modified fluid api.

def train(place):
    with fluid.program(place):
        batch_reader = fluid.layers.batch_reader(
            filename = './flowers.recordio', type='recordio',
            batch_size=100, shape=[[13], [1]], dtype=['float32', 'float32'])

        x, y = fluid.layers.next_batch(batch_reader)
        y_predict = fluid.layers.fc(input=x, size=1, act=None)
        cost = fluid.layers.square_error_cost(input=y_predict, label=y)
        avg_cost = fluid.layers.mean(cost)

        sgd_optimizer = fluid.optimizer.SGD(learning_rate=0.001)
        sgd_optimizer.minimize(avg_cost)

        fluid.initialize_variables()

        for pass_id in range(100):
            avg_loss_value, = fluid.run(place=fluid.CPUPlace(), fetch_list=[avg_cost])
            print(avg_loss_value)


if __name__ == '__main__':
    train(fluid.CPUPlace)

[wangkuiyi]

Thanks to @cs2be and @abhinavarora and @helinwang for this issue as a follow-up of #7464.

[wangkuiyi]

In Proposal 2, there is a Python for-loop, which, I am afraid, cannot be an operator call in the ProgramDesc message generated by the above fluid.program(place) call? It looks to me that only Proposal 1 is compatible with the design purpose of Fluid as described in #7464