Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add MLflow runtime with basic payload support #160

Closed
adriangonz opened this issue Apr 12, 2021 · 0 comments · Fixed by #175
Closed

Add MLflow runtime with basic payload support #160

adriangonz opened this issue Apr 12, 2021 · 0 comments · Fixed by #175

Comments

@adriangonz
Copy link
Contributor

adriangonz commented Apr 12, 2021

Add a new mlserver-mlflow runtime which allows a user to point to a MLflow Model artifact (or folder) to load a model. As a initial step, the mlserver-mlflow runtime should take care of converting the V2 Dataplane payload to a "dict of tensors", which is one of the formats expected by MLflow models.

To translate this, we could just turn the V2 input into an "index", where the keys would be the inputs[].name fields. That is, an input such as:

{
  "inputs": [
    {
        "name": "a",
        "data": [1, 4],
        "shape": [2],
        "datatype": "INT32"
    },
      {
        "name": "b",
        "data": [2, 5],
        "shape": [2],
        "datatype": "INT32"
    },
      {
        "name": "c",
        "data": [3, 6],
        "shape": [2],
        "datatype": "INT32"
    }
  ]
}

, could be turned to the following MLflow-compatible dictionary of tensors:

{
    "inputs": {
          "a": ["s1", "s2", "s3"], 
          "b": [1, 2, 3], 
          "c": [[1, 2, 3], [4, 5, 6], [7, 8, 9]]
      }
}

Tensors vs Dataframes

While some MLflow models require their inputs to be encoded as dataframes, some others will still need a dictionary of tensors (see #160). To account for this, the scope of this issue includes looking at ways to infer which type of input does a model require, as well as providing a way for the user to choose which input type to use.

The latter could be done through the V2 Protocol's inputs[].parameters field, by setting a "magic key" (e.g. mlflow_encoding: dataframe). This key can then be read by the mlserver-mlflow runtime to choose one encoding or the other.

@adriangonz adriangonz added this to To do in MLOps Sprint 6 (29/04/2021) via automation Apr 19, 2021
@adriangonz adriangonz self-assigned this Apr 19, 2021
@adriangonz adriangonz moved this from To do to In progress in MLOps Sprint 6 (29/04/2021) Apr 28, 2021
@adriangonz adriangonz moved this from To do to In progress in MLflow Inference Runtime Apr 28, 2021
MLflow Inference Runtime automation moved this from In progress to Done May 5, 2021
MLOps Sprint 6 (29/04/2021) automation moved this from In progress to Done May 5, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
No open projects
Development

Successfully merging a pull request may close this issue.

1 participant