[RFC] ML pipeline operator spec

[valiantljk]

Describe your feature request

ML pipeline gains more popularity In many machine learning practices. The basic ML pipeline includes data preprocessing, training, and inference. Each step can also be constructed with finer ops. Both Kubeflow and Tensorflow community have been promoting the ML components/operators concept, where operator developers focus on the development of re-usable operators, and users can easily ‘import’ the pre-designed operator to build an end-to-end ml pipeline.

Actor in Ray is a natural fit for the reusable ml operator. Building an operator to run in Ray will be slightly different than a kubeflow operator. The eventual ecosystem will be like a ray-based serverless ‘kubeflow’ (rayflow?). The first step of building such a system is to have a clear and concrete definition of operator. This doc defines the re-usable ray operator.

For more details regarding operator spec and potential usage pattern: [proposal]
(https://docs.google.com/document/d/1V6cCkD3KLVgpJPvWOpirUGvSFETMD2A5JlO5MFON25k/edit?usp=sharing)

[stale]

Hi, I'm a bot from the Ray team :)

To help human contributors to focus on more relevant issues, I will automatically add the stale label to issues that have had no activity for more than 4 months.

If there is no further activity in the 14 days, the issue will be closed!

• If you'd like to keep the issue open, just leave any comment, and the stale label will be removed!
• If you'd like to get more attention to the issue, please tag one of Ray's contributors.

You can always ask for help on our discussion forum or Ray's public slack channel.

[stale]

Hi again! The issue will be closed because there has been no more activity in the 14 days since the last message.

Please feel free to reopen or open a new issue if you'd still like it to be addressed.

Again, you can always ask for help on our discussion forum or Ray's public slack channel.

Thanks again for opening the issue!