-
Notifications
You must be signed in to change notification settings - Fork 832
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
docs: add mlflow logging and loading #1641
Conversation
Hey @thinkall 👋! |
/azp run |
Azure Pipelines successfully started running 1 pipeline(s). |
Codecov Report
@@ Coverage Diff @@
## master #1641 +/- ##
==========================================
+ Coverage 85.74% 85.79% +0.04%
==========================================
Files 272 272
Lines 14230 14230
Branches 739 739
==========================================
+ Hits 12202 12208 +6
+ Misses 2028 2022 -6
Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here. |
"source": [ | ||
"# load model back\n", | ||
"# mlflow will use PipelineModel to wrapper the original model, thus here we extract the original ALSModel from the stages.\n", | ||
"loaded_model = mlflow.spark.load_model(model_uri, dfs_tmpdir=\"Files/spark\").stages[-1]" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think serena might have fixed this behavior in MLFlow so that it just loads and saves original model, can you check with her to see?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hihi, I didn't change this part lol and we can't. Because they need a consistent API to load the model back, which only PipelineModel does, so we have to fetch the stages within it. I was using the similar logic in our dotnet stuff, because you can't know the real type only after you load it back, and sometimes we might just save a PipelineModel so you don't need to fetch the stages.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks Mark. Just checked with Serena. MLflow still wraps original model with PipelineModel. In most cases, we only use model.transform which also works with PipelineModel. Only in this case, ALSModel has method like recommendForAllUsers
which is needed in recommendation scenario. Maybe it's acceptable to extract original model from stages in special cases like this one.
Co-authored-by: Mark Hamilton <mhamilton723@gmail.com>
/azp run |
Azure Pipelines successfully started running 1 pipeline(s). |
"with mlflow.start_run() as run:\n", | ||
" print(\"log model:\")\n", | ||
" mlflow.spark.log_model(\n", | ||
" model,\n", | ||
" f\"{EXPERIMENT_NAME}-alsmodel\",\n", | ||
" registered_model_name=f\"{EXPERIMENT_NAME}-alsmodel\",\n", | ||
" dfs_tmpdir=\"Files/spark\",\n", | ||
" )\n", | ||
"\n", | ||
" print(\"log metrics:\")\n", | ||
" mlflow.log_metrics({\"RMSE\": rmse, \"MAE\": mae, \"R2\": r2, \"Explained variance\": var})\n", | ||
"\n", | ||
" print(\"log parameters:\")\n", | ||
" mlflow.log_params(\n", | ||
" {\n", | ||
" \"num_epochs\": num_epochs,\n", | ||
" \"rank_size_list\": rank_size_list,\n", | ||
" \"reg_param_list\": reg_param_list,\n", | ||
" \"model_tuning_method\": model_tuning_method,\n", | ||
" \"DATA_FOLDER\": DATA_FOLDER,\n", | ||
" }\n", | ||
" )\n", | ||
"\n", | ||
" model_uri = f\"runs:/{run.info.run_id}/{EXPERIMENT_NAME}-alsmodel\"\n", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can this be simplified with @serena-ruan 's Autologging?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Currently mlflow.pyspark.ml.autolog
doesn't work since the default dfs_tmpdir
is /tmp/mlflow
, which doesn't work in our platform. Need to wait for another release of mlflow which will include Serena's PR for setting DFS_TMP
as an environment variable. Moreover, for ALS model, it's not supported by autolog. I see Serena has two PRs for supporting modifying logModelAllowlistFile
in a user friendly way, I guess those will be released in the next version of mlflow too.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It looks good, i would just check with Serena if we can further simplify with her autologging, then the story will be full and very nice
Related Issues/PRs
None
What changes are proposed in this pull request?
Updated aisample notebooks.
How is this patch tested?
Does this PR change any dependencies?
Does this PR add a new feature? If so, have you added samples on website?
website/docs/documentation
folder.Make sure you choose the correct class
estimators/transformers
and namespace.DocTable
points to correct API link.yarn run start
to make sure the website renders correctly.<!--pytest-codeblocks:cont-->
before each python code blocks to enable auto-tests for python samples.WebsiteSamplesTests
job pass in the pipeline.AB#1958014