[Epic] - DataScientist - MlOperator collaboration flow

[alfsuse]

Following Google definition of the collaboration between:

• Data Engineer (responsible for Data preparation and ingestion)
• Data Scientist (responsible to develop a model to be trained and overview the capability of this model to satisfy project requirements as: precision,accuracy, performance)
• MLOp (responsible of glue together the Data and Model and create an end-to-end workflow that train the model and serve the model to the inference to expose it to the final application)

MlOps Google

We may define the ideal flow to satisfy the following steps:

1. Data extraction: You select and integrate the relevant data from various data sources for the ML task.
2. Data analysis: You perform exploratory data analysis (EDA) to understand the available data for building the ML model. This process leads to the following:
3. Understanding the data schema and characteristics that are expected by the model.
   Identifying the data preparation and feature engineering that are needed for the model.
4. Data preparation: The data is prepared for the ML task. This preparation involves data cleaning, where you split the data into training, validation, and test sets. You also apply data transformations and feature engineering to the model that solves the target task. The output of this step are the data splits in the prepared format.
5. Model training: The data scientist implements different algorithms with the prepared data to train various ML models. In addition, you subject the implemented algorithms to hyperparameter tuning to get the best performing ML model. The output of this step is a trained model.
6. Model evaluation: The model is evaluated on a holdout test set to evaluate the model quality. The output of this step is a set of metrics to assess the quality of the model.
7. Model validation: The model is confirmed to be adequate for deployment—that its predictive performance is better than a certain baseline.
8. Model serving: The validated model is deployed to a target environment to serve predictions. This deployment can be one of the following:
   • Microservices with a REST API to serve online predictions.
   • An embedded model to an edge or mobile device.
   • Part of a batch prediction system.
9. Model monitoring: The model predictive performance is monitored to potentially invoke a new iteration in the ML process.

We may assume that FUSEML in its first releases will have to provide a simple way for those three personas to collaborate, removing, at the same time, the majority of friction and complexity.

To do so as the initial phase we will have to focus on a subset of the above points.

We may describe this subset as:

1. We assume the DE knows how to prepare the data and simply expose the datasets to the DS in some way (i.e.: S3 store, Remote URL,etc..).
2. DS code on his preferred tool/IDE and once ready push its files to FUSEML as a branch of the final repo.
3. We will embed these coding artifacts in a super simple pipeline that has only 2 or 3 steps:
   • data ingestion/preparation
   • training
   • outcome
4. The simple pipeline will deploy everything the DS need (i.e. MLFlow instance for the experimentation phase)
5. Once the Ds is satisfied with the training outcome he will execute a request to the MLOps (from Git logic he does a PR) that notify the MLOp that training code is ready to be pickup.
6. MLOp inject the code into a more complex pipeline (merge) and start the end-to-end workflow