docs: add mlflow logging and loading #1641
Conversation
|
Hey @thinkall 👋! |
thinkall
changed the title
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
Codecov Report
@@ Coverage Diff @@
## master #1641 +/- ##
==========================================
+ Coverage 85.74% 85.79% +0.04%
==========================================
Files 272 272
Lines 14230 14230
Branches 739 739
==========================================
+ Hits 12202 12208 +6
+ Misses 2028 2022 -6
Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here. |
| "source": [ | ||
| "# load model back\n", | ||
| "# mlflow will use PipelineModel to wrapper the original model, thus here we extract the original ALSModel from the stages.\n", | ||
| "loaded_model = mlflow.spark.load_model(model_uri, dfs_tmpdir=\"Files/spark\").stages[-1]" |
There was a problem hiding this comment.
I think serena might have fixed this behavior in MLFlow so that it just loads and saves original model, can you check with her to see?
There was a problem hiding this comment.
Hihi, I didn't change this part lol and we can't. Because they need a consistent API to load the model back, which only PipelineModel does, so we have to fetch the stages within it. I was using the similar logic in our dotnet stuff, because you can't know the real type only after you load it back, and sometimes we might just save a PipelineModel so you don't need to fetch the stages.
There was a problem hiding this comment.
Thanks Mark. Just checked with Serena. MLflow still wraps original model with PipelineModel. In most cases, we only use model.transform which also works with PipelineModel. Only in this case, ALSModel has method like recommendForAllUsers which is needed in recommendation scenario. Maybe it's acceptable to extract original model from stages in special cases like this one.
Co-authored-by: Mark Hamilton <mhamilton723@gmail.com>
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
| "with mlflow.start_run() as run:\n", | ||
| " print(\"log model:\")\n", | ||
| " mlflow.spark.log_model(\n", | ||
| " model,\n", | ||
| " f\"{EXPERIMENT_NAME}-alsmodel\",\n", | ||
| " registered_model_name=f\"{EXPERIMENT_NAME}-alsmodel\",\n", | ||
| " dfs_tmpdir=\"Files/spark\",\n", | ||
| " )\n", | ||
| "\n", | ||
| " print(\"log metrics:\")\n", | ||
| " mlflow.log_metrics({\"RMSE\": rmse, \"MAE\": mae, \"R2\": r2, \"Explained variance\": var})\n", | ||
| "\n", | ||
| " print(\"log parameters:\")\n", | ||
| " mlflow.log_params(\n", | ||
| " {\n", | ||
| " \"num_epochs\": num_epochs,\n", | ||
| " \"rank_size_list\": rank_size_list,\n", | ||
| " \"reg_param_list\": reg_param_list,\n", | ||
| " \"model_tuning_method\": model_tuning_method,\n", | ||
| " \"DATA_FOLDER\": DATA_FOLDER,\n", | ||
| " }\n", | ||
| " )\n", | ||
| "\n", | ||
| " model_uri = f\"runs:/{run.info.run_id}/{EXPERIMENT_NAME}-alsmodel\"\n", |
There was a problem hiding this comment.
Can this be simplified with @serena-ruan 's Autologging?
There was a problem hiding this comment.
Currently mlflow.pyspark.ml.autolog doesn't work since the default dfs_tmpdir is /tmp/mlflow, which doesn't work in our platform. Need to wait for another release of mlflow which will include Serena's PR for setting DFS_TMP as an environment variable. Moreover, for ALS model, it's not supported by autolog. I see Serena has two PRs for supporting modifying logModelAllowlistFile in a user friendly way, I guess those will be released in the next version of mlflow too.
Related Issues/PRs
None
What changes are proposed in this pull request?
Updated aisample notebooks.
How is this patch tested?
Does this PR change any dependencies?
Does this PR add a new feature? If so, have you added samples on website?
website/docs/documentationfolder.Make sure you choose the correct class
estimators/transformersand namespace.DocTablepoints to correct API link.yarn run startto make sure the website renders correctly.<!--pytest-codeblocks:cont-->before each python code blocks to enable auto-tests for python samples.WebsiteSamplesTestsjob pass in the pipeline.AB#1958014