Add confs for metric prefix and metric dataset info to mlflow.evaluate(), rename accuracy to accuracy_score #6593
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: dbczumar <corey.zumar@databricks.com>
Signed-off-by: dbczumar <corey.zumar@databricks.com>
github-actions
bot
added
area/models
dbczumar
commented
Aug 25, 2022
| @@ -175,7 +171,7 @@ def _get_classifier_global_metrics(is_binomial, y, y_pred, y_probs, labels): | |||
| get classifier metrics which computing over all classes examples. | |||
| """ | |||
| metrics = {} | |||
| metrics["accuracy"] = sk_metrics.accuracy_score(y, y_pred) | |||
| metrics["accuracy_score"] = sk_metrics.accuracy_score(y, y_pred) | |||
There was a problem hiding this comment.
@yxiong @jimmyxu-db Renaming for consistency with mlflow.sklearn.eval_and_log_metrics; let me know if this is unnecessary
There was a problem hiding this comment.
I think it's nice to be consistent :)
There was a problem hiding this comment.
I think it makes sense for it to be accuracy_score. To my understanding, this is the code that logs the metrics, and the existing metric is called accuracy_score
dbczumar
commented
Aug 25, 2022
Comment on lines
+1193
to
+1196
| evaluator_config={ | ||
| "metric_prefix": prefix, | ||
| "log_metrics_with_dataset_info": log_metrics_with_dataset_info, | ||
| }, |
There was a problem hiding this comment.
@yxiong @jimmyxu-db Here's an example of how the new config arguments can be used to set a prefix and disable metric logging with dataset info
yxiong
approved these changes
Aug 25, 2022
| @@ -175,7 +171,7 @@ def _get_classifier_global_metrics(is_binomial, y, y_pred, y_probs, labels): | |||
| get classifier metrics which computing over all classes examples. | |||
| """ | |||
| metrics = {} | |||
| metrics["accuracy"] = sk_metrics.accuracy_score(y, y_pred) | |||
| metrics["accuracy_score"] = sk_metrics.accuracy_score(y, y_pred) | |||
There was a problem hiding this comment.
I think it's nice to be consistent :)
jimmyxu-db
approved these changes
Aug 25, 2022
| @@ -175,7 +171,7 @@ def _get_classifier_global_metrics(is_binomial, y, y_pred, y_probs, labels): | |||
| get classifier metrics which computing over all classes examples. | |||
| """ | |||
| metrics = {} | |||
| metrics["accuracy"] = sk_metrics.accuracy_score(y, y_pred) | |||
| metrics["accuracy_score"] = sk_metrics.accuracy_score(y, y_pred) | |||
There was a problem hiding this comment.
I think it makes sense for it to be accuracy_score. To my understanding, this is the code that logs the metrics, and the existing metric is called accuracy_score
31 tasks
dbczumar
added a commit
that referenced
this pull request
Sep 28, 2022
* Run model registry sqlalchemy tests in database job (#6353) Signed-off-by: harupy <hkawamura0130@gmail.com> * update to run profiles on the first 100 dataframe columns (#6297) * update to run profiles on the first 100 dataframe columns Signed-off-by: Sunish Sheth <sunishsheth2009@gmail.com> * Adding replace=True so we can sample rows twice Signed-off-by: Sunish Sheth <sunishsheth2009@gmail.com> * Removing replace=true and fixing how the max_rows are calculated Signed-off-by: Sunish Sheth <sunishsheth2009@gmail.com> * Fix sklearn autologging input example & signature mutation for models containing transformers (#6230) * Fix with test case Signed-off-by: dbczumar <corey.zumar@databricks.com> * Test fix Signed-off-by: dbczumar <corey.zumar@databricks.com> * Fix and simplification Signed-off-by: dbczumar <corey.zumar@databricks.com> * more fixes & better test coveragE Signed-off-by: dbczumar <corey.zumar@databricks.com> * test Signed-off-by: dbczumar <corey.zumar@databricks.com> * LGBM test Signed-off-by: dbczumar <corey.zumar@databricks.com> * Format Signed-off-by: dbczumar <corey.zumar@databricks.com> * Address comments Signed-off-by: dbczumar <corey.zumar@databricks.com> * Fixes Signed-off-by: dbczumar <corey.zumar@databricks.com> * Use py37-compatible call args syntax Signed-off-by: dbczumar <corey.zumar@databricks.com> * use list (#6363) Signed-off-by: harupy <hkawamura0130@gmail.com> * Log runtime pipeline config to MLflow (#6359) * Log runtime pipeline config to MLflow Signed-off-by: Jin Zhang <jin.zhang@databricks.com> * Fixed lint Signed-off-by: Jin Zhang <jin.zhang@databricks.com> * Add status badge for stale action (#6372) Signed-off-by: harupy <hkawamura0130@gmail.com> * Add stack trace section to bug report template (#6374) Signed-off-by: harupy <hkawamura0130@gmail.com> * Skipping the transform step by using Functional transformer (#6362) Signed-off-by: Sunish Sheth <sunishsheth2009@gmail.com> * Series schema (#6361) * Added in name for ColSpec of pandas datatype infer panda series name for ColSpec Signed-off-by: Ryan Fogle <ryanfogle98@gmail.com> * Added test for Series name inference added one more pytest condition in test_schema:test_schema_inference_on_pandas_series Signed-off-by: Ryan Fogle <ryanfogle98@gmail.com> * Update test_schema.py Added in one more assertion to test schema inference. Signed-off-by: Ryan Fogle <ryanfogle98@gmail.com> * ran black Signed-off-by: Ryan Fogle <ryanfogle98@gmail.com> * added in requested changes in issue #6361 Signed-off-by: Ryan Fogle <ryanfogle98@gmail.com> * fix typo. changed hasattr to getattr Signed-off-by: Ryan Fogle <ryanfogle98@gmail.com> * integrate mlflowdbfs (#6282) Signed-off-by: Brian Barnes <brian.barnes@databricks.com> * Fix step card display breakage in Jupyter notebooks (#6378) * remove Signed-off-by: apurva-koti <apurva.koti@databricks.com> * fix Signed-off-by: apurva-koti <apurva.koti@databricks.com> * Avoid using flask 2.2.0 (#6380) Signed-off-by: harupy <hkawamura0130@gmail.com> * Migrate GCS environment variables to `mlflow.environment_variables` module (#6375) * Migrate GCS environment variables to mlflow.environment_variables Signed-off-by: harupy <hkawamura0130@gmail.com> * add default Signed-off-by: harupy <hkawamura0130@gmail.com> * improve comments Signed-off-by: harupy <hkawamura0130@gmail.com> * Avoid logging model signatures in pyspark ML autologging if model input/output dataframe contains unsupported data types (#6365) * Avoid logging signatures Signed-off-by: harupy <hkawamura0130@gmail.com> * test Signed-off-by: harupy <hkawamura0130@gmail.com> * fix indent Signed-off-by: harupy <hkawamura0130@gmail.com> * check model output Signed-off-by: harupy <hkawamura0130@gmail.com> * fix warning message Signed-off-by: harupy <hkawamura0130@gmail.com> * todo Signed-off-by: harupy <hkawamura0130@gmail.com> * update test and doc Signed-off-by: harupy <hkawamura0130@gmail.com> * fix docstring Signed-off-by: harupy <hkawamura0130@gmail.com> * fix failed tests Signed-off-by: harupy <hkawamura0130@gmail.com> * improve _assert_autolog_infers_model_signature_correctly Signed-off-by: harupy <hkawamura0130@gmail.com> * Refactor `set_matrix.py` (#6373) * refacor set_matrix.py Signed-off-by: harupy <hkawamura0130@gmail.com> * fix Signed-off-by: harupy <hkawamura0130@gmail.com> * fix tests Signed-off-by: harupy <hkawamura0130@gmail.com> * tset for --no-dev Signed-off-by: harupy <hkawamura0130@gmail.com> * install pytest and pytest-cov Signed-off-by: harupy <hkawamura0130@gmail.com> * test for --changed-files Signed-off-by: harupy <hkawamura0130@gmail.com> * add pyyaml to requirements Signed-off-by: harupy <hkawamura0130@gmail.com> * fix is_matrix_empty Signed-off-by: harupy <hkawamura0130@gmail.com> * use raw string and re.match Signed-off-by: harupy <hkawamura0130@gmail.com> * Fix is_matrix_empty (#6385) Signed-off-by: harupy <hkawamura0130@gmail.com> * Allow mlflow-skinny's autolog() to succeed when no scipy is installed (#5897) * allow mlflow-skinny's autolog() to succeed when no scipy is installed Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com> * optional pyfunc scipy dependency Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com> * fix linter-discovered issues Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com> * add test case, autolog() should pass w/o scipy installed Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com> * minor cleanup Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com> * test creating _Example with / without scipy present Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com> * docs: do not warn about sparse matrix classes Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com> * linting Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com> * address review comment Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com> * run black Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com> * allow mlflow-skinny's autolog() to succeed when no scipy is installed Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com> * optional pyfunc scipy dependency Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com> * fix linter-discovered issues Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com> * add test case, autolog() should pass w/o scipy installed Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com> * minor cleanup Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com> * test creating _Example with / without scipy present Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com> * docs: do not warn about sparse matrix classes Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com> * linting Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com> * address review comment Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com> * run black Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com> * remove redundant None check after import Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com> * delete comment Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com> * Revert "remove redundant None check after import" This reverts commit f2e198c944f41d9a96c20ae471d71bff39431b48. Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com> * review comments Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com> * remove redundant test Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com> * Apply suggestions from code review Co-authored-by: Harutaka Kawamura <hkawamura0130@gmail.com> Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com> * Autoformat: https://github.com/mlflow/mlflow/actions/runs/2779791161 Signed-off-by: mlflow-automation <mlflow-automation@users.noreply.github.com> Co-authored-by: Harutaka Kawamura <hkawamura0130@gmail.com> Co-authored-by: dbczumar <corey.zumar@databricks.com> Co-authored-by: mlflow-automation <mlflow-automation@users.noreply.github.com> * Fix extraneous warning about pyspark evaluation truncation (#6389) * Revert #6380 (#6386) * revert Signed-off-by: harupy <hkawamura0130@gmail.com> * workaround for test_prometheus_exporter Signed-off-by: harupy <hkawamura0130@gmail.com> * nit Signed-off-by: harupy <hkawamura0130@gmail.com> * Validate metric value in `FileStore.log_metric` (#6381) * add failing test Signed-off-by: harupy <hkawamura0130@gmail.com> * use _validate_metric Signed-off-by: harupy <hkawamura0130@gmail.com> * fix invalid metric timestamp Signed-off-by: harupy <hkawamura0130@gmail.com> * fix docstring Signed-off-by: harupy <hkawamura0130@gmail.com> * fix java test Signed-off-by: harupy <hkawamura0130@gmail.com> * Make search experiment by tag key comparison case sensitive in MYSQL (#6333) * init Signed-off-by: Weichen Xu <weichen.xu@databricks.com> * add test Signed-off-by: Weichen Xu <weichen.xu@databricks.com> * update Signed-off-by: Weichen Xu <weichen.xu@databricks.com> * update Signed-off-by: Weichen Xu <weichen.xu@databricks.com> * fix BINARY conditions in MySQL Signed-off-by: harupy <hkawamura0130@gmail.com> * fix attr error Signed-off-by: harupy <hkawamura0130@gmail.com> Co-authored-by: harupy <hkawamura0130@gmail.com> * Run Model Registry SQLAlchemy tests in PostgreSQL, MySQL, and MSSQL (#6390) * Run Model Registry SQLAlchemy tests in postgres, mysql, and mssql Signed-off-by: harupy <hkawamura0130@gmail.com> * fix failed tests Signed-off-by: harupy <hkawamura0130@gmail.com> * remove uuid Signed-off-by: harupy <hkawamura0130@gmail.com> * clean up Signed-off-by: harupy <hkawamura0130@gmail.com> * do not drop tables Signed-off-by: harupy <hkawamura0130@gmail.com> * Fix filestore data loss on failed write & mitigate read / write race condition (#6388) * Fix Signed-off-by: dbczumar <corey.zumar@databricks.com> * Fix Signed-off-by: dbczumar <corey.zumar@databricks.com> * Format Signed-off-by: dbczumar <corey.zumar@databricks.com> * Added a custom filter from_json to render_and_merge_yaml (#6368) * Added a custom filter from_json to render_and_merge_yaml Signed-off-by: Jin Zhang <jin.zhang@databricks.com> * Fixed test Signed-off-by: Jin Zhang <jin.zhang@databricks.com> * Update tests/utils/test_file_utils.py Co-authored-by: Siddharth Murching <sid.murching@databricks.com> * Update tests/utils/test_file_utils.py Co-authored-by: Siddharth Murching <sid.murching@databricks.com> * Fixed test Signed-off-by: Jin Zhang <jin.zhang@databricks.com> * Fixed windows test attempt Signed-off-by: Jin Zhang <jin.zhang@databricks.com> * Fixed windows test Signed-off-by: Jin Zhang <jin.zhang@databricks.com> Co-authored-by: Siddharth Murching <sid.murching@databricks.com> * Fix `!=` operator for MySQL (#6397) * fix ne operator and improve tests Signed-off-by: harupy <hkawamura0130@gmail.com> * add () to other operators Signed-off-by: harupy <hkawamura0130@gmail.com> * Simplify the documentation section in the PR template (#6398) * Simplify the documentation section in the PR template Signed-off-by: harupy <hkawamura0130@gmail.com> * fix Signed-off-by: harupy <hkawamura0130@gmail.com> * Timebased gc (#6354) * Added delete_time field to runs table Signed-off-by: Jason Cheng <jasoncky96@gmail.com> * Updated latest_schema.sql to pass tests/store/tracking/test_sqlalchemy_store_schema.py Signed-off-by: Jason Cheng <jasoncky96@gmail.com> * Added delete_time column into mlflow/store/tracking/dbmodels/initial_models.py Signed-off-by: Jason Cheng <jasoncky96@gmail.com> * Moved the docstring to the end of the declaration instead of the start Signed-off-by: Jason Cheng <jasoncky96@gmail.com> * Regenerated protobuf for RunInfo new schema, and passed pytest Signed-off-by: Jason Cheng <jasoncky96@gmail.com> * Added --older-than flag in mlflow gc command, and added tests for that cli command Signed-off-by: Jason Cheng <jasoncky96@gmail.com> * Removed delete_time from RunInfo proto Signed-off-by: Jason Cheng <jasoncky96@gmail.com> * Set default older-than flag to None Signed-off-by: Jason Cheng <jasoncky96@gmail.com> * Remove delete_time from RunInfo, add optional parameter older_than to store.get_deleted_runs(), and write delete_time into yaml file for file storage instead of adding it to RunInfo Signed-off-by: Jason Cheng <jasoncky96@gmail.com> * Remove commented delete_time in RunInfo constructor in models.py Signed-off-by: Jason Cheng <jasoncky96@gmail.com> * added INVALID_PARAMETER_VALUE error code in mlflow gc Signed-off-by: Jason Cheng <jasoncky96@gmail.com> * Resolve case of delete time not in meta.yaml file, edited overwrite_run_info and gc Signed-off-by: Jason Cheng <jasoncky96@gmail.com> * Rename delete_time to deleted_time Signed-off-by: Jason Cheng <jasoncky96@gmail.com> * Rename delete_time to deleted_time, match error message in cli tests, filter SqlRuns by deleted_time, indicate float values accepted for gc --older-than flag Signed-off-by: Jason Cheng <jasoncky96@gmail.com> * Rename delete_time to deleted_time, fixed assert statement in test_mlflow_gc, and fixed deleted_time queries in test_file_store.py and test_sqlalchemy_store.py Signed-off-by: Jason Cheng <jasoncky96@gmail.com> * Only check if is in meta after restoring run in test_delte_restore_run Signed-off-by: Jason Cheng <jasoncky96@gmail.com> * Fixed lint errors Signed-off-by: Jason Cheng <jasoncky96@gmail.com> * Install mlflow from repository root Signed-off-by: harupy <hkawamura0130@gmail.com> Co-authored-by: harupy <hkawamura0130@gmail.com> * Separate protos job (#6399) * Separate protos job Signed-off-by: harupy <hkawamura0130@gmail.com> * test Signed-off-by: harupy <hkawamura0130@gmail.com> * Add paths Signed-off-by: harupy <hkawamura0130@gmail.com> * Revert "test" This reverts commit 5d04b02f55ec92e1e3ac18eb651ea989ca0d469a. Signed-off-by: harupy <hkawamura0130@gmail.com> * Fix flaky `TestFileStore.test_get_deleted_runs` (#6402) * use gt operator Signed-off-by: harupy <hkawamura0130@gmail.com> * fix sqlalchemy store Signed-off-by: harupy <hkawamura0130@gmail.com> * Migrate environment variables for SQLAlchemy store (#6396) Signed-off-by: harupy <hkawamura0130@gmail.com> * Deduplicate MlflowClient in MLflow docs, move to mlflow.client module (#6405) * Dedupe MlflowClient Signed-off-by: dbczumar <corey.zumar@databricks.com> * Improvements Signed-off-by: dbczumar <corey.zumar@databricks.com> * Add client Signed-off-by: dbczumar <corey.zumar@databricks.com> * Remove unused functions in `mlflow.data` module (#6408) * remove unused functions in data Signed-off-by: harupy <hkawamura0130@gmail.com> * Remove unused functions and variables Signed-off-by: harupy <hkawamura0130@gmail.com> * Tag based registered model search and model version search (#6320) * init Signed-off-by: Weichen Xu <weichen.xu@databricks.com> * update Signed-off-by: Weichen Xu <weichen.xu@databricks.com> * update Signed-off-by: Weichen Xu <weichen.xu@databricks.com> * update Signed-off-by: Weichen Xu <weichen.xu@databricks.com> * update Signed-off-by: Weichen Xu <weichen.xu@databricks.com> * update Signed-off-by: Weichen Xu <weichen.xu@databricks.com> * update Signed-off-by: Weichen Xu <weichen.xu@databricks.com> * update Signed-off-by: Weichen Xu <weichen.xu@databricks.com> * update Signed-off-by: Weichen Xu <weichen.xu@databricks.com> * update Signed-off-by: Weichen Xu <weichen.xu@databricks.com> * update Signed-off-by: Weichen Xu <weichen.xu@databricks.com> * update tests Signed-off-by: Weichen Xu <weichen.xu@databricks.com> * update Signed-off-by: Weichen Xu <weichen.xu@databricks.com> * update Signed-off-by: Weichen Xu <weichen.xu@databricks.com> * clean Signed-off-by: Weichen Xu <weichen.xu@databricks.com> * address comments Signed-off-by: Weichen Xu <weichen.xu@databricks.com> * add test Signed-off-by: Weichen Xu <weichen.xu@databricks.com> * address comments Signed-off-by: Weichen Xu <weichen.xu@databricks.com> * update Signed-off-by: Weichen Xu <weichen.xu@databricks.com> * de-dup tag based search result Signed-off-by: Weichen Xu <weichen.xu@databricks.com> * add () to fix case_sensitive_mysql_ne Signed-off-by: Weichen Xu <weichen.xu@databricks.com> * fix Signed-off-by: Weichen Xu <weichen.xu@databricks.com> * lint Signed-off-by: Weichen Xu <weichen.xu@databricks.com> * fix * fix test in TODO Signed-off-by: Weichen Xu <weichen.xu@databricks.com> * support dup tag key filter Signed-off-by: Weichen Xu <weichen.xu@databricks.com> * add mv order by Signed-off-by: Weichen Xu <weichen.xu@databricks.com> * cmp res list in test Signed-off-by: Weichen Xu <weichen.xu@databricks.com> * address comments Signed-off-by: Weichen Xu <weichen.xu@databricks.com> * address comments Signed-off-by: Weichen Xu <weichen.xu@databricks.com> * add doc Signed-off-by: Weichen Xu <weichen.xu@databricks.com> * fix Signed-off-by: Weichen Xu <weichen.xu@databricks.com> * fix lint Signed-off-by: Weichen Xu <weichen.xu@databricks.com> * address comments Signed-off-by: Weichen Xu <weichen.xu@databricks.com> * address comments Signed-off-by: Weichen Xu <weichen.xu@databricks.com> * update doc Signed-off-by: Weichen Xu <weichen.xu@databricks.com> * Increase maximum param value length to 500 (#6358) * increase validation of maximum param value length from 250 to 500 (squashed) Signed-off-by: Jan Sindlar <jan.sindlar@eureka.ai> * 1. added test for exceeding max length. 2. lint. 3. validation logic addresses null Signed-off-by: Jan Sindlar <jan.sindlar@eureka.ai> * file store should test already stringified params Signed-off-by: Jan Sindlar <jan.sindlar@eureka.ai> * update alembic chain Signed-off-by: Jan Sindlar <jan.sindlar@eureka.ai> * Autoformat: https://github.com/mlflow/mlflow/actions/runs/2800398158 Signed-off-by: mlflow-automation <mlflow-automation@users.noreply.github.com> * Update tests/store/tracking/test_file_store.py Co-authored-by: Harutaka Kawamura <hkawamura0130@gmail.com> Signed-off-by: Jan Sindlar <jan.sindlar@eureka.ai> Co-authored-by: Jan Sindlar <jan.sindlar@eureka.ai> Co-authored-by: mlflow-automation <mlflow-automation@users.noreply.github.com> Co-authored-by: Harutaka Kawamura <hkawamura0130@gmail.com> * Add workflow dispatch for requirements validation, remove duplicate file (#6418) * Workflow dispatch Signed-off-by: dbczumar <corey.zumar@databricks.com> * Remove unused Signed-off-by: dbczumar <corey.zumar@databricks.com> * Prevent `pytest.raises` context manager from containing multiple statements (#6409) * rename Signed-off-by: harupy <hkawamura0130@gmail.com> * rename Signed-off-by: harupy <hkawamura0130@gmail.com> * rename Signed-off-by: harupy <hkawamura0130@gmail.com> * Do not allow calling assert in pytest.raises Signed-off-by: harupy <hkawamura0130@gmail.com> * more strict rule Signed-off-by: harupy <hkawamura0130@gmail.com> * Improve error message Signed-off-by: harupy <hkawamura0130@gmail.com> * relax condition Signed-off-by: harupy <hkawamura0130@gmail.com> * fix lint errors Signed-off-by: harupy <hkawamura0130@gmail.com> * fix failed tests Signed-off-by: harupy <hkawamura0130@gmail.com> * clean up Signed-off-by: harupy <hkawamura0130@gmail.com> * rename _is_complex_pytest_raises Signed-off-by: harupy <hkawamura0130@gmail.com> * fix lint errors Signed-off-by: harupy <hkawamura0130@gmail.com> * fix failed tests Signed-off-by: harupy <hkawamura0130@gmail.com> * Fix regression in pandas type inference behavior for scoring server (#6417) * Fix impl Signed-off-by: dbczumar <corey.zumar@databricks.com> * Remove unused pandas import Signed-off-by: dbczumar <corey.zumar@databricks.com> * Adjust client to use SearchExperiments POST method (#6415) * POST Signed-off-by: dbczumar <corey.zumar@databricks.com> * Generate protos Signed-off-by: dbczumar <corey.zumar@databricks.com> * Fix test Signed-off-by: dbczumar <corey.zumar@databricks.com> * update (#6420) Signed-off-by: Weichen Xu <weichen.xu@databricks.com> * Check if artifacts are proxied in model_version_artifact_handler (#6355) * Check if artifacts are proxied in model_version_artifact_handler Signed-off-by: Mohammad Abbas <mohammad.abbas@digital.trade.gov.uk> * Use file prefix for artifact dest on windows Signed-off-by: dbczumar <corey.zumar@databricks.com> * Fix application of prefix Signed-off-by: dbczumar <corey.zumar@databricks.com> * Fixes Signed-off-by: dbczumar <corey.zumar@databricks.com> Co-authored-by: dbczumar <corey.zumar@databricks.com> * Allow sql filter IN operator right-hand side values including whitespaces (#6413) * support in rhs whitespace Signed-off-by: Weichen Xu <weichen.xu@databricks.com> * update Signed-off-by: Weichen Xu <weichen.xu@databricks.com> * Fix IN operator related issues in sql filter parser (#6411) * update Signed-off-by: Weichen Xu <weichen.xu@databricks.com> * update Signed-off-by: Weichen Xu <weichen.xu@databricks.com> * clean Signed-off-by: Weichen Xu <weichen.xu@databricks.com> * update Signed-off-by: Weichen Xu <weichen.xu@databricks.com> * update Signed-off-by: Weichen Xu <weichen.xu@databricks.com> * update Signed-off-by: Weichen Xu <weichen.xu@databricks.com> * address comment Signed-off-by: Weichen Xu <weichen.xu@databricks.com> * Remove unused method `SearchUtils._get_comparison_for_model_registry` (#6412) * init Signed-off-by: Weichen Xu <weichen.xu@databricks.com> * update Signed-off-by: Weichen Xu <weichen.xu@databricks.com> * Update MLflow UI (#6422) * Project import generated by Copybara. GitOrigin-RevId: 086b1200f4573559e37fcea6cb5472d52f166e42 Signed-off-by: dbczumar <corey.zumar@databricks.com> * Reconcile changes from universe with OSS Signed-off-by: dbczumar <corey.zumar@databricks.com> * Fix test fails Signed-off-by: dbczumar <corey.zumar@databricks.com> * Fix Signed-off-by: dbczumar <corey.zumar@databricks.com> * Apply model search fix Signed-off-by: dbczumar <corey.zumar@databricks.com> * Remove errant test line Signed-off-by: dbczumar <corey.zumar@databricks.com> Co-authored-by: Richard Zang <Zangr@users.noreply.github.com> * remove near from deprecation msg (#6430) Signed-off-by: ninabacc-db <Nina.Baccam@databricks.com> * Raise clear error message when pipeline path contains a space (#6426) * Add unsupported Signed-off-by: dbczumar <corey.zumar@databricks.com> * Fix Signed-off-by: dbczumar <corey.zumar@databricks.com> * Exc Signed-off-by: dbczumar <corey.zumar@databricks.com> * Test Signed-off-by: dbczumar <corey.zumar@databricks.com> * Format Signed-off-by: dbczumar <corey.zumar@databricks.com> * Test fix Signed-off-by: dbczumar <corey.zumar@databricks.com> * Fix Signed-off-by: dbczumar <corey.zumar@databricks.com> * [ALL TESTS] Changelog for MLflow 1.28.0 (#6433) * Generating changelog for MLflow 1.27.1.dev0 Signed-off-by: Jenkins <jenkins@databricks.com> * Changelog Signed-off-by: dbczumar <corey.zumar@databricks.com> * Add missing PR Signed-off-by: dbczumar <corey.zumar@databricks.com> * Some fixes Signed-off-by: dbczumar <corey.zumar@databricks.com> * More small fixes Signed-off-by: dbczumar <corey.zumar@databricks.com> * Fix Signed-off-by: dbczumar <corey.zumar@databricks.com> Signed-off-by: Jenkins <jenkins@databricks.com> Signed-off-by: dbczumar <corey.zumar@databricks.com> Co-authored-by: Jenkins <jenkins@databricks.com> Co-authored-by: dbczumar <corey.zumar@databricks.com> * Patch (#6440) Signed-off-by: dbczumar <corey.zumar@databricks.com> Signed-off-by: dbczumar <corey.zumar@databricks.com> * [ALL TESTS] Update (#6443) * python dev/update_ml_package_versions.py Signed-off-by: Jenkins <jenkins@databricks.com> * pyspark max ver 3.4 Signed-off-by: Weichen Xu <weichen.xu@databricks.com> Signed-off-by: Jenkins <jenkins@databricks.com> Signed-off-by: Weichen Xu <weichen.xu@databricks.com> Co-authored-by: Jenkins <jenkins@databricks.com> Co-authored-by: Weichen Xu <weichen.xu@databricks.com> * Update MLflow version to 1.28.1 (#6449) Signed-off-by: Jenkins <jenkins@databricks.com> Signed-off-by: Jenkins <jenkins@databricks.com> Co-authored-by: Jenkins <jenkins@databricks.com> * Add data capture config to sagemaker (#6423) Signed-off-by: jonwiggins <me@jonwiggins.org> Signed-off-by: jonwiggins <me@jonwiggins.org> * Fix small search_runs docs issues (#6452) * Fix Signed-off-by: dbczumar <corey.zumar@databricks.com> * Doc fix Signed-off-by: dbczumar <corey.zumar@databricks.com> * Fix Signed-off-by: dbczumar <corey.zumar@databricks.com> Signed-off-by: dbczumar <corey.zumar@databricks.com> * python dev/update_pypi_package_index.py (#6442) Signed-off-by: Jenkins <jenkins@databricks.com> Signed-off-by: Jenkins <jenkins@databricks.com> Co-authored-by: Jenkins <jenkins@databricks.com> * use xgboost on pypi (#6461) Signed-off-by: harupy <hkawamura0130@gmail.com> Signed-off-by: harupy <hkawamura0130@gmail.com> * Add environment variable for `echo` option in `create_engine` (#6460) * Add environment variable for echo Signed-off-by: harupy <hkawamura0130@gmail.com> * use _BooleanEnvironmentVariable Signed-off-by: harupy <hkawamura0130@gmail.com> * address comments Signed-off-by: harupy <hkawamura0130@gmail.com> * address comments and add tests Signed-off-by: harupy <hkawamura0130@gmail.com> * fix comment Signed-off-by: harupy <hkawamura0130@gmail.com> Signed-off-by: harupy <hkawamura0130@gmail.com> * Add more file extensions to show content on frontend side (#3292) Signed-off-by: Мартынов Максим Сергеевич <msmarty5@mts.ru> Signed-off-by: Мартынов Максим Сергеевич <msmarty5@mts.ru> Signed-off-by: harupy <hkawamura0130@gmail.com> Co-authored-by: harupy <hkawamura0130@gmail.com> * Avoid using assertions at top level in `mlflow/models/docker_utils.py` (#6463) * do not construct maven proxy options at top level Signed-off-by: harupy <hkawamura0130@gmail.com> * fix validation for port Signed-off-by: harupy <hkawamura0130@gmail.com> * unpack Signed-off-by: harupy <hkawamura0130@gmail.com> Signed-off-by: harupy <hkawamura0130@gmail.com> * Fix mlflowdbfs availability check (#6474) * Fix Signed-off-by: dbczumar <corey.zumar@databricks.com> * Fix Signed-off-by: dbczumar <corey.zumar@databricks.com> * Reverts Signed-off-by: dbczumar <corey.zumar@databricks.com> * Better fix Signed-off-by: dbczumar <corey.zumar@databricks.com> * Format Signed-off-by: dbczumar <corey.zumar@databricks.com> * Format Signed-off-by: dbczumar <corey.zumar@databricks.com> * Lint Signed-off-by: dbczumar <corey.zumar@databricks.com> * Clean up Signed-off-by: dbczumar <corey.zumar@databricks.com> Signed-off-by: dbczumar <corey.zumar@databricks.com> * Replace `assertRaisesRegex` with `pytest.raises` (#6464) * replace assertRaisesRegex with pytest.raises Signed-off-by: harupy <hkawamura0130@gmail.com> * fix message Signed-off-by: harupy <hkawamura0130@gmail.com> Signed-off-by: harupy <hkawamura0130@gmail.com> * CLI command suggestion after running the pipeline (#6376) * CLI command suggestion after running the pipeline Signed-off-by: Hubert Zub <hubert.zub@databricks.com> * fix: added git commands to MLP CLI suggestion * chore: lint fixes Signed-off-by: Hubert Zub <hubert.zub@databricks.com> * chore: test fixes Signed-off-by: Hubert Zub <hubert.zub@databricks.com> Signed-off-by: Hubert Zub <hubert.zub@databricks.com> * Remove redundant int conversion in create_sqlalchemy_engine (#6481) Signed-off-by: harupy <hkawamura0130@gmail.com> Signed-off-by: harupy <hkawamura0130@gmail.com> * Improve code-to-reproduce section in the bug report template (#6478) * Improve code-to-reproduce section Signed-off-by: harupy <hkawamura0130@gmail.com> * fix Signed-off-by: harupy <hkawamura0130@gmail.com> * problem -> issue Signed-off-by: harupy <hkawamura0130@gmail.com> * fix Signed-off-by: harupy <hkawamura0130@gmail.com> Signed-off-by: harupy <hkawamura0130@gmail.com> * Add Pipelines step option to skip step profiling (#6456) * changes Signed-off-by: apurva-koti <apurva.koti@databricks.com> * changes Signed-off-by: apurva-koti <apurva.koti@databricks.com> * partial-test Signed-off-by: apurva-koti <apurva.koti@databricks.com> * lint Signed-off-by: apurva-koti <apurva.koti@databricks.com> * missed Signed-off-by: apurva-koti <apurva.koti@databricks.com> * review Signed-off-by: apurva-koti <apurva.koti@databricks.com> Signed-off-by: apurva-koti <apurva.koti@databricks.com> * Implement _download_file in hdfs_artifact_repo (#6482) * Implement _download_file in hdfs_artifact_repo Signed-off-by: bowen_who <hubw8@chinaunicom.cn> * add ut on _download_file in test_hdfs_artifact_repo Signed-off-by: bowen_who <hubw8@chinaunicom.cn> * rewrite _download_artifact, change back _download_file in hdfs_artifact_repo Signed-off-by: bowen_who <hubw8@chinaunicom.cn> Signed-off-by: harupy <hkawamura0130@gmail.com> * rewrite test__download_file in test_hdfs_artifact_repo Signed-off-by: bowen_who <hubw8@chinaunicom.cn> Signed-off-by: bowen_who <hubw8@chinaunicom.cn> Signed-off-by: harupy <hkawamura0130@gmail.com> Co-authored-by: bowen_who <hubw8@chinaunicom.cn> * [ML-19871] R documentation fix (#6484) * R bug fix * Documentation fix * Revert breaking change * Updated doc file * Doc fix * Testing a broken test * fix * Revert test * feat: return param value for mlflow.log_param (#6483) * feat: return param value for mlflow.log_param Signed-off-by: Xinyue Ruan <serena.rxy@gmail.com> * update docstring and return value for MlflowClient Signed-off-by: Xinyue Ruan <serena.rxy@gmail.com> Signed-off-by: Xinyue Ruan <serena.rxy@gmail.com> * revert (#6490) Signed-off-by: harupy <hkawamura0130@gmail.com> Signed-off-by: harupy <hkawamura0130@gmail.com> * Ignore empty environment variables in _get_maven_proxy (#6492) Signed-off-by: harupy <hkawamura0130@gmail.com> Signed-off-by: harupy <hkawamura0130@gmail.com> * Migrate environment variables for S3 (#6438) * Migrated s3 Signed-off-by: changyonglik <theeahlag@gmail.com> * Migrated s3 for local.py Signed-off-by: changyonglik <theeahlag@gmail.com> * Changed default values Signed-off-by: changyonglik <theeahlag@gmail.com> * Added var imports Signed-off-by: changyonglik <theeahlag@gmail.com> * Added MLFLOW_S3_ENDPOINT_URL and MLFLOW_S3_IGNORE_TLS Signed-off-by: changyonglik <theeahlag@gmail.com> * Removed duplicated imports Signed-off-by: changyonglik <theeahlag@gmail.com> * Paraphrased specify to specifies Signed-off-by: changyonglik <theeahlag@gmail.com> * Changed _EnvironmentVariable to _BooleanEnvironmentVariable for MLFLOW_S3_IGNORE_TLS and defaults to False Signed-off-by: changyonglik <theeahlag@gmail.com> * Changed environment variables in hdfs_artifact_repo.py Signed-off-by: changyonglik <theeahlag@gmail.com> * Changed single to double quotes Signed-off-by: changyonglik <theeahlag@gmail.com> * Fixed comments Signed-off-by: changyonglik <theeahlag@gmail.com> * Small fixes Signed-off-by: dbczumar <corey.zumar@databricks.com> * Format Signed-off-by: dbczumar <corey.zumar@databricks.com> * Reset submodule Signed-off-by: dbczumar <corey.zumar@databricks.com> Signed-off-by: changyonglik <theeahlag@gmail.com> Signed-off-by: dbczumar <corey.zumar@databricks.com> Co-authored-by: dbczumar <corey.zumar@databricks.com> * Always run CI check on PR (#6505) Signed-off-by: harupy <hkawamura0130@gmail.com> Signed-off-by: harupy <hkawamura0130@gmail.com> * document how to install mlflow-pipelines from conda (#6509) Signed-off-by: harupy <hkawamura0130@gmail.com> Signed-off-by: harupy <hkawamura0130@gmail.com> * Move tests for pylint_plugins (#6506) Signed-off-by: harupy <hkawamura0130@gmail.com> Signed-off-by: harupy <hkawamura0130@gmail.com> * Retry pulling docker image on CircleCI (#6514) * Retry pulling docker image on CircleCI Signed-off-by: harupy <hkawamura0130@gmail.com> * fix indent Signed-off-by: harupy <hkawamura0130@gmail.com> Signed-off-by: harupy <hkawamura0130@gmail.com> * Remove unnecessary comprehension (#6516) Signed-off-by: harupy <hkawamura0130@gmail.com> Signed-off-by: harupy <hkawamura0130@gmail.com> * Don't use mlflowdbfs if dbutils is unavailable (#6508) * Dbutils Signed-off-by: dbczumar <corey.zumar@databricks.com> * Fix Signed-off-by: dbczumar <corey.zumar@databricks.com> * Fix Signed-off-by: dbczumar <corey.zumar@databricks.com> * Fix Signed-off-by: dbczumar <corey.zumar@databricks.com> Signed-off-by: dbczumar <corey.zumar@databricks.com> * bug-fix to allow non-string column names (#6503) (#6521) * Allow EvaluationDataset to accept dataframes that contain non-string column names (mlflow#6503) Signed-off-by: Daniel Gibbons <daniel.gibbons04@gmail.com> * Minor fix to remove redundant assignments from iris_pandas_df datasets (mlflow#6503) Signed-off-by: Daniel Gibbons <daniel.gibbons04@gmail.com> Signed-off-by: Daniel Gibbons <daniel.gibbons04@gmail.com> * Add `use-set-literal` rule for better performace (#6533) * Add use-set-literal rule Signed-off-by: harupy <hkawamura0130@gmail.com> * improve test Signed-off-by: harupy <hkawamura0130@gmail.com> Signed-off-by: harupy <hkawamura0130@gmail.com> * feat: Docker development scripts and readme (#6465) * feat: docker development scripts and readme Signed-off-by: Hubert Zub <hubert.zub@databricks.com> * chore: CR fixes Signed-off-by: Hubert Zub <hubert.zub@databricks.com> * chore: improved dependency install in Dockerfile.test Signed-off-by: Hubert Zub <hubert.zub@databricks.com> * Update CONTRIBUTING.rst Co-authored-by: Harutaka Kawamura <hkawamura0130@gmail.com> Signed-off-by: Hubert Zub <hubert.zub@databricks.com> * Update dev/Dockerfile.test Co-authored-by: Harutaka Kawamura <hkawamura0130@gmail.com> Signed-off-by: Hubert Zub <hubert.zub@databricks.com> * Update CONTRIBUTING.rst Signed-off-by: Hubert Zub <hubert.zub@databricks.com> * Update dev/run-test-container.sh Co-authored-by: Harutaka Kawamura <hkawamura0130@gmail.com> Signed-off-by: Hubert Zub <hubert.zub@databricks.com> * Update dev/Dockerfile.test.dockerignore Co-authored-by: Harutaka Kawamura <hkawamura0130@gmail.com> Signed-off-by: Hubert Zub <hubert.zub@databricks.com> * chore: CR fixes Signed-off-by: Hubert Zub <hubert.zub@databricks.com> * chore: shell script improvement Signed-off-by: Hubert Zub <hubert.zub@databricks.com> * chore: fixed dockerignore Signed-off-by: Hubert Zub <hubert.zub@databricks.com> * Update dev/Dockerfile.test.dockerignore Co-authored-by: Harutaka Kawamura <hkawamura0130@gmail.com> Signed-off-by: Hubert Zub <hubert.zub@databricks.com> * Update dev/Dockerfile.test Co-authored-by: Harutaka Kawamura <hkawamura0130@gmail.com> Signed-off-by: Hubert Zub <hubert.zub@databricks.com> * Update dev/Dockerfile.test.dockerignore Co-authored-by: Harutaka Kawamura <hkawamura0130@gmail.com> Signed-off-by: Hubert Zub <hubert.zub@databricks.com> * Update CONTRIBUTING.rst Co-authored-by: Harutaka Kawamura <hkawamura0130@gmail.com> Signed-off-by: Hubert Zub <hubert.zub@databricks.com> Signed-off-by: Hubert Zub <hubert.zub@databricks.com> Co-authored-by: Harutaka Kawamura <hkawamura0130@gmail.com> * Adding cd to run reproduce steps (#6495) * adding cd to repo Signed-off-by: Prithvi Kannan <prithvi.kannan@databricks.com> * drop git extension Signed-off-by: Prithvi Kannan <prithvi.kannan@databricks.com> * ran lint Signed-off-by: Prithvi Kannan <prithvi.kannan@databricks.com> Signed-off-by: Prithvi Kannan <prithvi.kannan@databricks.com> * fix use-set-literal errors batch 1 (#mlflow-6535) (#6542) * fix use-set-literal errors batch 1 (#mlflow-6535) Signed-off-by: Daniel Gibbons <daniel.gibbons04@gmail.com> * update set_checker batch 1 (#mlflow-6535) Signed-off-by: Daniel Gibbons <daniel.gibbons04@gmail.com> Signed-off-by: Daniel Gibbons <daniel.gibbons04@gmail.com> * fix use-set-literal errors batch 2 (#mlflow-6535) (#6543) * fix use-set-literal errors batch 2 (#mlflow-6535) Signed-off-by: Daniel Gibbons <daniel.gibbons04@gmail.com> * update set_checker batch 2 (#mlflow-6535) Signed-off-by: Daniel Gibbons <daniel.gibbons04@gmail.com> Signed-off-by: Daniel Gibbons <daniel.gibbons04@gmail.com> * Fix `use-set-literal` errors (#6534) * Fix use-set-literal errors Signed-off-by: harupy <hkawamura0130@gmail.com> * fix error message Signed-off-by: harupy <hkawamura0130@gmail.com> * fix error message Signed-off-by: harupy <hkawamura0130@gmail.com> * improve instructions Signed-off-by: harupy <hkawamura0130@gmail.com> * tuple Signed-off-by: harupy <hkawamura0130@gmail.com> Signed-off-by: harupy <hkawamura0130@gmail.com> * Remove run_uuid is None check (#6547) Signed-off-by: dbczumar <corey.zumar@databricks.com> Signed-off-by: dbczumar <corey.zumar@databricks.com> * Deprecate ListExperiments, ListRegisteredModels, ListRunInfos APIs (#6550) * Deprecate listing Signed-off-by: dbczumar <corey.zumar@databricks.com> * Deprecate Signed-off-by: dbczumar <corey.zumar@databricks.com> * Revert Signed-off-by: dbczumar <corey.zumar@databricks.com> Signed-off-by: dbczumar <corey.zumar@databricks.com> * fix missing_fields (#6549) Signed-off-by: harupy <hkawamura0130@gmail.com> Signed-off-by: harupy <hkawamura0130@gmail.com> * Improve deprecated decorator (#6552) Signed-off-by: harupy <hkawamura0130@gmail.com> Signed-off-by: harupy <hkawamura0130@gmail.com> * Fix violating consider-using-set-comprehension pylint rule (#6540) * Fix violating consider-using-set-comprehension pylint rule Signed-off-by: Evgenii Uvarov <evgenii.uvarov.post@gmail.com> * Fix linting issues. Fix PR review issues Signed-off-by: Evgenii Uvarov <evgenii.uvarov.post@gmail.com> Signed-off-by: Evgenii Uvarov <evgenii.uvarov.post@gmail.com> Co-authored-by: Evgenii Uvarov <e.uvarov@life-pay.ru> * Install conda in R development image (#6557) Signed-off-by: harupy <hkawamura0130@gmail.com> Signed-off-by: harupy <hkawamura0130@gmail.com> * MLP batch scoring DAG added to Regression Pipeline (#6455) * Added the skeletons for the 2 new steps (#6248) * Added the skeletons for the 2 new steps * CR comments * Linting errors * Added the skeletons for the 2 new steps (#6248) * Added the skeletons for the 2 new steps * CR comments * Linting errors * Added the skeletons for the 2 new steps (#6248) * Added the skeletons for the 2 new steps * CR comments * Linting errors * Refactored MLP infra to support additional template makefiles (#6259) * Added changes * fixed tests * Formatting and fixed tests * Added preprocessing step test (#6267) * Added preprocessing test * Fix linting errors * Added the skeletons for the 2 new steps (#6248) * Added the skeletons for the 2 new steps * CR comments * Linting errors * Refactored MLP infra to support additional template makefiles (#6259) * Added changes * fixed tests * Formatting and fixed tests * Added preprocessing step test (#6267) * Added preprocessing test * Fix linting errors * Fixed submodules * Build out predict step (#6296) Signed-off-by: Jerry Liang <jerry.liang@databricks.com> * Batch scoring framework (#6283) * Batch scoring pipeline added * Added gitmodule * updating the example? * Submodule fix and test fix * Submodule fix * black formatting * Added batch_scoring submodule correctly? * Lint errors * Fixed the integration test (removed predict for now) * Black formatting * Updated api doc * Documentation bloopers * Removed smart quotes * more documentation fixes * More fixes * Try to change the replacement name * Bad substitution * Add registered model test (#6327) Signed-off-by: Jerry Liang <jerry.liang@databricks.com> * Predict Step: output format and location, env restoration (#6336) * add output location writing logic Signed-off-by: Jerry Liang <jerry.liang@databricks.com> * change output param Signed-off-by: Jerry Liang <jerry.liang@databricks.com> * wip Signed-off-by: Jerry Liang <jerry.liang@databricks.com> * wip Signed-off-by: Jerry Liang <jerry.liang@databricks.com> * wip Signed-off-by: Jerry Liang <jerry.liang@databricks.com> * wip Signed-off-by: Jerry Liang <jerry.liang@databricks.com> * Batching scoring pipeline bug fixes, predict step card html (#6364) * wip Signed-off-by: Jerry Liang <jerry.liang@databricks.com> * wip Signed-off-by: Jerry Liang <jerry.liang@databricks.com> * wip Signed-off-by: Jerry Liang <jerry.liang@databricks.com> * Added the skeletons for the 2 new steps (#6248) * Added the skeletons for the 2 new steps * CR comments * Linting errors * Refactored MLP infra to support additional template makefiles (#6259) * Added changes * fixed tests * Formatting and fixed tests * Added preprocessing step test (#6267) * Added preprocessing test * Fix linting errors * Fixed submodules * Added the skeletons for the 2 new steps (#6248) * Added the skeletons for the 2 new steps * CR comments * Linting errors * Added the skeletons for the 2 new steps (#6248) * Added the skeletons for the 2 new steps * CR comments * Linting errors * Refactored MLP infra to support additional template makefiles (#6259) * Added changes * fixed tests * Formatting and fixed tests * Added preprocessing step test (#6267) * Added preprocessing test * Fix linting errors * Build out predict step (#6296) Signed-off-by: Jerry Liang <jerry.liang@databricks.com> * Batch scoring framework (#6283) * Batch scoring pipeline added * Added gitmodule * updating the example? * Submodule fix and test fix * Submodule fix * black formatting * Added batch_scoring submodule correctly? * Lint errors * Fixed the integration test (removed predict for now) * Black formatting * Updated api doc * Documentation bloopers * Removed smart quotes * more documentation fixes * More fixes * Try to change the replacement name * Bad substitution * Add registered model test (#6327) Signed-off-by: Jerry Liang <jerry.liang@databricks.com> * Predict Step: output format and location, env restoration (#6336) * add output location writing logic Signed-off-by: Jerry Liang <jerry.liang@databricks.com> * change output param Signed-off-by: Jerry Liang <jerry.liang@databricks.com> * wip Signed-off-by: Jerry Liang <jerry.liang@databricks.com> * wip Signed-off-by: Jerry Liang <jerry.liang@databricks.com> * wip Signed-off-by: Jerry Liang <jerry.liang@databricks.com> * wip Signed-off-by: Jerry Liang <jerry.liang@databricks.com> * Batching scoring pipeline bug fixes, predict step card html (#6364) * wip Signed-off-by: Jerry Liang <jerry.liang@databricks.com> * wip Signed-off-by: Jerry Liang <jerry.liang@databricks.com> * wip Signed-off-by: Jerry Liang <jerry.liang@databricks.com> * Convert to regression (#6371) * Convert feature branch to move batch scoring into regression pipeline * Black formatting * Add the new ingest_scoring step * Uupdated artifact name * Updated sklearn_regression example * Test fixes * Fixed broken tests * Linting errors * Augment predict step to use model name for register step (#6437) * wip Signed-off-by: Jerry Liang <jerry.liang@databricks.com> * wip Signed-off-by: Jerry Liang <jerry.liang@databricks.com> * wip Signed-off-by: Jerry Liang <jerry.liang@databricks.com> * wip Signed-off-by: Jerry Liang <jerry.liang@databricks.com> * wip Signed-off-by: Jerry Liang <jerry.liang@databricks.com> * wip Signed-off-by: Jerry Liang <jerry.liang@databricks.com> * wip Signed-off-by: Jerry Liang <jerry.liang@databricks.com> * wip Signed-off-by: Jerry Liang <jerry.liang@databricks.com> * wip Signed-off-by: Jerry Liang <jerry.liang@databricks.com> Signed-off-by: Jerry Liang <jerry.liang@databricks.com> * Remove preprocessing step * Fix ingest step rebase error * CR comments addressed * Fixed broken tests * Undo unknown split change * wip Signed-off-by: Jerry Liang <jerry.liang@databricks.com> * wip Signed-off-by: Jerry Liang <jerry.liang@databricks.com> * wip Signed-off-by: Jerry Liang <jerry.liang@databricks.com> * wip Signed-off-by: Jerry Liang <jerry.liang@databricks.com> Signed-off-by: Jerry Liang <jerry.liang@databricks.com> Co-authored-by: Jerry Liang <66143562+jerrylian-db@users.noreply.github.com> Co-authored-by: Jerry Liang <jerry.liang@databricks.com> * fix use-set-literal errors (#6544) * That was fun Signed-off-by: Uncle Hohotun <95711404+Monkero@users.noreply.github.com> * set_checker.py is updated Signed-off-by: Uncle Hohotun <95711404+Monkero@users.noreply.github.com> * Conflict in pylint_plagins/set_checker.py Signed-off-by: Uncle Hohotun <95711404+Monkero@users.noreply.github.com> * run black Signed-off-by: harupy <hkawamura0130@gmail.com> Signed-off-by: Uncle Hohotun <95711404+Monkero@users.noreply.github.com> Signed-off-by: harupy <hkawamura0130@gmail.com> Co-authored-by: harupy <hkawamura0130@gmail.com> * unwanted os calls removed (#6559) Signed-off-by: yogesh kumar <kumaryogesh.usict.ipu@gmail.com> Signed-off-by: yogesh kumar <kumaryogesh.usict.ipu@gmail.com> * Fix CircleCI artifact URL in `build_doc_r` job (#6558) * Fix CircleCI artifact URL Signed-off-by: harupy <hkawamura0130@gmail.com> * fix Signed-off-by: harupy <hkawamura0130@gmail.com> * revert Signed-off-by: harupy <hkawamura0130@gmail.com> Signed-off-by: harupy <hkawamura0130@gmail.com> * Update requirements.yaml files (#6564) Signed-off-by: mlflow-automation <mlflow-automation@users.noreply.github.com> Signed-off-by: mlflow-automation <mlflow-automation@users.noreply.github.com> Co-authored-by: mlflow-automation <mlflow-automation@users.noreply.github.com> * Add searchExperiments API to Java client, deprecate listExperiments (#6561) * SearchExperiments API for Java Signed-off-by: dbczumar <corey.zumar@databricks.com> * Search & tests Signed-off-by: dbczumar <corey.zumar@databricks.com> * Deprecate list Signed-off-by: dbczumar <corey.zumar@databricks.com> * Add page Signed-off-by: dbczumar <corey.zumar@databricks.com> Signed-off-by: dbczumar <corey.zumar@databricks.com> * Pin `pmdarima` in diviner test (#6565) * Pin pmdarima Signed-off-by: harupy <hkawamura0130@gmail.com> * Pin pmdarima Signed-off-by: harupy <hkawamura0130@gmail.com> * fix command arguments Signed-off-by: harupy <hkawamura0130@gmail.com> * quote CHANGED_FILES Signed-off-by: harupy <hkawamura0130@gmail.com> Signed-off-by: harupy <hkawamura0130@gmail.com> * fix use-set-literal errors batch 3 (#mlflow-6535) (#6560) * fix use-set-literal errors batch 3 (#mlflow-6535) Signed-off-by: Evgenii Uvarov <evgenii.uvarov.post@gmail.com> * Build trigger for CI Signed-off-by: evgenii.uvarov <evgenii.uvarov.post@gmail.com> * Trigger CI Signed-off-by: harupy <hkawamura0130@gmail.com> Signed-off-by: Evgenii Uvarov <evgenii.uvarov.post@gmail.com> Signed-off-by: evgenii.uvarov <evgenii.uvarov.post@gmail.com> Signed-off-by: harupy <hkawamura0130@gmail.com> Co-authored-by: harupy <hkawamura0130@gmail.com> * replaced all self.assertTrue with assert (#6570) * replaced all self.assertTrue with assert Signed-off-by: yogesh kumar <kumaryogesh.usict.ipu@gmail.com> * updated test-run-status file Signed-off-by: yogesh kumar <kumaryogesh.usict.ipu@gmail.com> * Autoformat: https://github.com/mlflow/mlflow/actions/runs/2917378175 Signed-off-by: mlflow-automation <mlflow-automation@users.noreply.github.com> Signed-off-by: yogesh kumar <kumaryogesh.usict.ipu@gmail.com> Signed-off-by: mlflow-automation <mlflow-automation@users.noreply.github.com> Co-authored-by: mlflow-automation <mlflow-automation@users.noreply.github.com> * [bug-fix] Removing the fixed width css for the experimentlist container (#6569) Signed-off-by: Sunish Sheth <sunishsheth2009@gmail.com> Signed-off-by: Sunish Sheth <sunishsheth2009@gmail.com> * Clarify ports in JS dev server instructions (#6563) * Clarify ports in JS dev server instructions Signed-off-by: Prithvi Kannan <prithvi.kannan@databricks.com> * update text Signed-off-by: Prithvi Kannan <prithvi.kannan@databricks.com> Signed-off-by: Prithvi Kannan <prithvi.kannan@databricks.com> * Allow specifying mlflow command line args as env variables. (#6457) Signed-off-by: Phil Xiaojun Hu <phil@cnphil.com> Signed-off-by: Phil Xiaojun Hu <phil@cnphil.com> * [ML-22889] Get artifact cli (#6517) * adding cli flag Signed-off-by: Prithvi Kannan <prithvi.kannan@databricks.com> * return datapath for dataframe artifacts Signed-off-by: Prithvi Kannan <prithvi.kannan@databricks.com> * registered_model_version path Signed-off-by: Prithvi Kannan <prithvi.kannan@databricks.com> * add support for model and transformer from cli Signed-off-by: Prithvi Kannan <prithvi.kannan@databricks.com> * adding support for run cmd Signed-off-by: Prithvi Kannan <prithvi.kannan@databricks.com> * change name to get_path Signed-off-by: Prithvi Kannan <prithvi.kannan@databricks.com> * adding a test Signed-off-by: Prithvi Kannan <prithvi.kannan@databricks.com> * idk why pyfunc was removed Signed-off-by: Prithvi Kannan <prithvi.kannan@databricks.com> * updating test and mandatory cli args Signed-off-by: Prithvi Kannan <prithvi.kannan@databricks.com> * fixing merge Signed-off-by: Prithvi Kannan <prithvi.kannan@databricks.com> * refactored into get_artifact_path Signed-off-by: Prithvi Kannan <prithvi.kannan@databricks.com> * fixing test case Signed-off-by: Prithvi Kannan <prithvi.kannan@databricks.com> * replaced Any with Optional Signed-off-by: Prithvi Kannan <prithvi.kannan@databricks.com> * idk how that happpened Signed-off-by: Prithvi Kannan <prithvi.kannan@databricks.com> * move output dirs to if-else Signed-off-by: Prithvi Kannan <prithvi.kannan@databricks.com> * addressing corey comments Signed-off-by: Prithvi Kannan <prithvi.kannan@databricks.com> * linter Signed-off-by: Prithvi Kannan <prithvi.kannan@databricks.com> Signed-off-by: Prithvi Kannan <prithvi.kannan@databricks.com> * fix final batch of use-set-literal errors #6535 (#6584) Signed-off-by: Daniel Gibbons <daniel.gibbons04@gmail.com> Signed-off-by: Daniel Gibbons <daniel.gibbons04@gmail.com> * Update doc to clarify tf.keras should not be logged with mlflow.tensorflow.log_model (#6573) Signed-off-by: Liang Zhang <liang.zhang@databricks.com> Signed-off-by: Liang Zhang <liang.zhang@databricks.com> * Add mlflow_search_experiments API to R client (#6576) * Make URLs clickable in the MLflow Tracking UI (#6526) * wrap urls in anchor tag in runs table Signed-off-by: Marijn Valk <marijncv@hotmail.com> * eslint fix & comment Signed-off-by: Marijn Valk <marijncv@hotmail.com> * added comment to util function Signed-off-by: Marijn Valk <marijncv@hotmail.com> Signed-off-by: Marijn Valk <marijncv@hotmail.com> * Removing actual code from DAG strings for MLP and just keeping user guide (#6595) Signed-off-by: Sunish Sheth <sunishsheth2009@gmail.com> Signed-off-by: Sunish Sheth <sunishsheth2009@gmail.com> * Add confs for metric prefix and metric dataset info to mlflow.evaluate(), rename accuracy to accuracy_score (#6593) * Actually include the prefix in eval results Signed-off-by: dbczumar <corey.zumar@databricks.com> * Naming Signed-off-by: dbczumar <corey.zumar@databricks.com> * Reset pipelines Signed-off-by: dbczumar <corey.zumar@databricks.com> * Rename Signed-off-by: dbczumar <corey.zumar@databricks.com> * Format Signed-off-by: dbczumar <corey.zumar@databricks.com> * Fix test Signed-off-by: dbczumar <corey.zumar@databricks.com> Signed-off-by: dbczumar <corey.zumar@databricks.com> * [Model Validation] Land model validation feature (#6582) * [Model Validation] Disable logging metrics and artifacts for baseline model evaluation (#6310) * add is_baseline_model boolean flag for disabling logging metrics and artifacts Signed-off-by: Zhe Wang <zhe.wang@databricks.com> * evaluation for baseline model should not generate artifact Signed-off-by: Zhe Wang <zhe.wang@databricks.com> * fix unit tests and dummy evaluator Signed-off-by: Zhe Wang <zhe.wang@databricks.com> * add unit test for validation interface Signed-off-by: Zhe Wang <zhe.wang@databricks.com> * change existing unit tests for testing validation Signed-off-by: Zhe Wang <zhe.wang@databricks.com> * add more unit tests for disabling logging of metrics and artifacts for baseline model evaluation Signed-off-by: Zhe Wang <zhe.wang@databricks.com> * add unit tests for vaildation Signed-off-by: Zhe Wang <zhe.wang@databricks.com> * Fix docstring for evaluate Signed-off-by: Zhe Wang <zhe.wang@databricks.com> * Refactor unit tests for evaluation and validation Signed-off-by: Zhe Wang <zhe.wang@databricks.com> * Refactor unit tests for evaluation and validation Signed-off-by: Zhe Wang <zhe.wang@databricks.com> * refactor unit tests Signed-off-by: Zhe Wang <zhe.wang@databricks.com> * remove typo Signed-off-by: Zhe Wang <zhe.wang@databricks.com> * refactor some code into functions Signed-off-by: Zhe Wang <zhe.wang@databricks.com> * add baseline_model to evaluator.evalaute and change corresponding unit tests Signed-off-by: Zhe Wang <zhe.wang@databricks.com> * refactor unit tests for better readability Signed-off-by: Zhe Wang <zhe.wang@databricks.com> * undo some unnecessary refactor Signed-off-by: Zhe Wang <zhe.wang@databricks.com> * remove unused methods Signed-off-by: Zhe Wang <zhe.wang@databricks.com> * refactor default evaluator evaluate Signed-off-by: Zhe Wang <zhe.wang@databricks.com> * add more info to doc string Signed-off-by: Zhe Wang <zhe.wang@databricks.com> * add more doc strings for helper methods Signed-off-by: Zhe Wang <zhe.wang@databricks.com> * Update mlflow/models/evaluation/base.py Co-authored-by: Jerry Liang <66143562+jerrylian-db@users.noreply.github.com> * Update mlflow/models/evaluation/base.py Co-authored-by: Jerry Liang <66143562+jerrylian-db@users.noreply.github.com> * address comments Signed-off-by: Zhe Wang <zhe.wang@databricks.com> * Update mlflow/models/evaluation/default_evaluator.py Co-authored-by: Jerry Liang <66143562+jerrylian-db@users.noreply.github.com> * Update mlflow/models/evaluation/default_evaluator.py Co-authored-by: Jerry Liang <66143562+jerrylian-db@users.noreply.github.com> * address comments Signed-off-by: Zhe Wang <zhe.wang@databricks.com> * add more comments Signed-off-by: Zhe Wang <zhe.wang@databricks.com> * Update tests/models/test_default_evaluator.py Co-authored-by: Jerry Liang <66143562+jerrylian-db@users.noreply.github.com> * rename test configs Signed-off-by: Zhe Wang <zhe.wang@databricks.com> * add doc string Signed-off-by: Zhe Wang <zhe.wang@databricks.com> * address comments Signed-off-by: Zhe Wang <zhe.wang@databricks.com> * Update mlflow/models/evaluation/default_evaluator.py Co-authored-by: Jerry Liang <66143562+jerrylian-db@users.noreply.github.com> Co-authored-by: Jerry Liang <66143562+jerrylian-db@users.noreply.github.com> * [Model Validation] Add MetricThreshold and Model Validation Functionality to MLflow.evaluate (#6328) Add model validation functionality to `mlflow.evaluate` - Add `MetricThreshold` class to let users define validation thresholds for model validation. - Add `_validate` for metric value threshold check and model comparison check - Add internal class `_ValidationResult` per metric for formatting failure message. If the model validation passed, no exception will be thrown and `EvaluationResult` will be returned. If the model validation failed, `mlflowModelValidationFailedException` will be thrown containing detailed failure message indicating why the model validation did not pass. Documentations: Two extra parameters for `mlflow.evalaute` <img width="895" alt="CleanShot 2022-08-05 at 13 17 13@2x" src="https://user-images.githubusercontent.com/25189433/183156569-06139716-3065-4493-9804-bcbc3538afa9.png"> One user facing class `MetricThreshold` for users to define thresholds for metrics. <img width="928" alt="CleanShot 2022-08-05 at 13 18 02@2x" src="https://user-images.githubusercontent.com/25189433/183156670-bdd1c147-9e45-4c77-87b0-d5dab308e7db.png"> * Update mlflow/models/evaluation/base.py Co-authored-by: Corey Zumar <39497902+dbczumar@users.noreply.github.com> * Update mlflow/models/evaluation/base.py Co-authored-by: Corey Zumar <39497902+dbczumar@users.noreply.github.com> * Update mlflow/models/evaluation/base.py Co-authored-by: Corey Zumar <39497902+dbczumar@users.noreply.github.com> * merge master pt2 Signed-off-by: Jerry Liang <jerry.liang@databricks.com> * [Model Validation] Finishing touches and documentation (#6577) * adapt feedback Signed-off-by: Jerry Liang <jerry.liang@databricks.com> * adapt feedback Signed-off-by: Jerry Liang <jerry.liang@databricks.com> * wip Signed-off-by: Jerry Liang <jerry.liang@databricks.com> * add MetricThreshold class checks Signed-off-by: Jerry Liang <jerry.liang@databricks.com> * fix missing baseline model test Signed-off-by: Jerry Liang <jerry.liang@databricks.com> * enhance baseline model test Signed-off-by: Jerry Liang <jerry.liang@databricks.com> * add baseline metric 0 tests Signed-off-by: Jerry Liang <jerry.liang@databricks.com> * add ; Signed-off-by: Jerry Liang <jerry.liang@databricks.com> * documentation Signed-off-by: Jerry Liang <jerry.liang@databricks.com> * documentation Signed-off-by: Jerry Liang <jerry.liang@databricks.com> * documentation Signed-off-by: Jerry Liang <jerry.liang@databricks.com> * wip Signed-off-by: Jerry Liang <jerry.liang@databricks.com> * wip Signed-off-by: Jerry Liang <jerry.liang@databricks.com> * Update mlflow/models/evaluation/validation.py Co-authored-by: Corey Zumar <39497902+dbczumar@users.noreply.github.com> Signed-off-by: Jerry Liang <jerry.liang@databricks.com> * Update docs/source/models.rst Co-authored-by: Corey Zumar <39497902+dbczumar@users.noreply.github.com> Signed-off-by: Jerry Liang <jerry.liang@databricks.com> * Update docs/source/models.rst Co-authored-by: Corey Zumar <39497902+dbczumar@users.noreply.github.com> Signed-off-by: Jerry Liang <jerry.liang@databricks.com> * Update docs/source/models.rst Co-authored-by: Corey Zumar <39497902+dbczumar@users.noreply.github.com> Signed-off-by: Jerry Liang <jerry.liang@databricks.com> * Update examples/evaluation/README.md Co-authored-by: Corey Zumar <39497902+dbczumar@users.noreply.github.com> Signed-off-by: Jerry Liang <jerry.liang@databricks.com> * wip Signed-off-by: Jerry Liang <jerry.liang@databricks.com> * wip Signed-off-by: Jerry Liang <jerry.liang@databricks.com> Signed-off-by: Jerry Liang <jerry.liang@databricks.com> Co-authored-by: Corey Zumar <39497902+dbczumar@users.noreply.github.com> * modify evaluate doc string Signed-off-by: Jerry Liang <jerry.liang@databricks.com> Signed-off-by: Jerry Liang <jerry.liang@databricks.com> Co-authored-by: Zhe Wang <zhe.wang@databricks.com> Co-authored-by: Corey Zumar <39497902+dbczumar@users.noreply.github.com> * [ML-20702] increase version font size (#6596) * Changed the name Start Time to Created (#6597) Signed-off-by: Jin Zhang <jin.zhang@databricks.com> Signed-off-by: Jin Zhang <jin.zhang@databricks.com> * feat(scoring_server): add /health endpoint (#6574) Signed-off-by: Gabriel Gazola Milan <gabriel.gazola@poli.ufrj.br> Signed-off-by: Gabriel Gazola Milan <gabriel.gazola@poli.ufrj.br> * Update start time col (#6614) * Changed the name Start Time to Created Signed-off-by: Jin Zhang <jin.zhang@databricks.com> * Added i18n change to Created Signed-off-by: Jin Zhang <jin.zhang@databricks.com> Signed-off-by: Jin Zhang <jin.zhang@databricks.com> * Add a script to update `CHANGELOG.md` (#6624) * Add update_changelog.py Signed-off-by: harupy <hkawamura0130@gmail.com> * fix Signed-off-by: harupy <hkawamura0130@gmail.com> * fix Signed-off-by: harupy <hkawamura0130@gmail.com> * comma Signed-off-by: harupy <hkawamura0130@gmail.com> Signed-off-by: harupy <hkawamura0130@gmail.com> * Improve styling of DAG and rearrage transform step node (#6612) Signed-off-by: Sunish Sheth <sunishsheth2009@gmail.com> Signed-off-by: Sunish Sheth <sunishsheth2009@gmail.com> * Update documentation for model version `run_link` field (#6632) * Update documentation for model version run_link field Signed-off-by: Arpit Jasapara <arpit.jasapara@databricks.com> * Updated protos Signed-off-by: Arpit Jasapara <arpit.jasapara@databricks.com> * Update mlflow/protos/model_registry.proto Co-authored-by: Siddharth Murching <smurching@gmail.com> Signed-off-by: Arpit Jasapara <arpit.jasapara@databricks.com> Signed-off-by: Arpit Jasapara <arpit.jasapara@databricks.com> Co-authored-by: Siddharth Murching <smurching@gmail.com> * Move data capture config to end (#6621) Signed-off-by: dbczumar <corey.zumar@databricks.com> Signed-off-by: dbczumar <corey.zumar@databricks.com> * Used tuple instead a comma-separated string in pytest.mark.parametrize. (#6623) * Used tuple instead a comma-separated string in pytest.mark.parametrize. Signed-off-by: Shraddha Falane <falane.shraddha18@gmail.com> * Used tuple instead a comma-separated string in pytest.mark.parametrize. Signed-off-by: Shraddha Falane <falane.shraddha18@gmail.com> Signed-off-by: Shraddha Falane <falane.shraddha18@gmail.com> * Stabilize SearchExperiments API (#6551) * Remove search experiments experimental Signed-off-by: dbczumar <corey.zumar@databricks.com> * Remove unused imports Signed-off-by: dbczumar <corey.zumar@databricks.com> Signed-off-by: dbczumar <corey.zumar@databricks.com> * Fix command to trigger auto-formatting (#6639) * Fix command to trigger auto-formatting Signed-off-by: harupy <hkawamura0130@gmail.com> * use test Signed-off-by: harupy <hkawamura0130@gmail.com> * fix script Signed-off-by: harupy <hkawamura0130@gmail.com> Signed-off-by: harupy <hkawamura0130@gmail.com> * Support argument for local_tmpdir when loading spark model. (#6642) * feat: Support argument for local_tmpdir when loading spark model. Signed-off-by: Li Jiang <bnujli@gmail.com> * update local_tmpdir with dst_path Signed-off-by: Li Jiang <bnujli@gmail.com> * Add soft delete in (#6637) * Add soft delete in Signed-off-by: ninabacc-db <Nina.Baccam@databricks.com> * Line item edit Signed-off-by: ninabacc-db <Nina.Baccam@databricks.com> Signed-off-by: ninabacc-db <Nina.Baccam@databricks.com> * adding load_text, load_image, load_dict to fluent.py (#6475) * adding load_text, load_json, load_image Signed-off-by: Subramaniam Shanmugam <subramaniam02@gmail.com> * adding pillow to requirements Signed-off-by: Subramaniam Shanmu…
nnethery
pushed a commit
to nnethery/mlflow
that referenced
this pull request
Feb 1, 2024
…e(), rename accuracy to accuracy_score (mlflow#6593) * Actually include the prefix in eval results Signed-off-by: dbczumar <corey.zumar@databricks.com> * Naming Signed-off-by: dbczumar <corey.zumar@databricks.com> * Reset pipelines Signed-off-by: dbczumar <corey.zumar@databricks.com> * Rename Signed-off-by: dbczumar <corey.zumar@databricks.com> * Format Signed-off-by: dbczumar <corey.zumar@databricks.com> * Fix test Signed-off-by: dbczumar <corey.zumar@databricks.com> Signed-off-by: dbczumar <corey.zumar@databricks.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What changes are proposed in this pull request?
Add confs for metric prefix and metric dataset info to mlflow.evaluate(), rename accuracy to accuracy_score
How is this patch tested?
Added integration tests
Does this PR change the documentation?
Detailslink on thePreview docscheck.Release Notes
Add confs for metric prefix and metric dataset info to mlflow.evaluate(), rename accuracy to accuracy_score
Is this a user-facing change?
What component(s), interfaces, languages, and integrations does this PR affect?
Components
area/artifacts: Artifact stores and artifact loggingarea/build: Build and test infrastructure for MLflowarea/docs: MLflow documentation pagesarea/examples: Example codearea/model-registry: Model Registry service, APIs, and the fluent client calls for Model Registryarea/models: MLmodel format, model serialization/deserialization, flavorsarea/pipelines: Pipelines, Pipeline APIs, Pipeline configs, Pipeline Templatesarea/projects: MLproject format, project running backendsarea/scoring: MLflow Model server, model deployment tools, Spark UDFsarea/server-infra: MLflow Tracking server backendarea/tracking: Tracking Service, tracking client APIs, autologgingInterface
area/uiux: Front-end, user experience, plotting, JavaScript, JavaScript dev serverarea/docker: Docker use across MLflow's components, such as MLflow Projects and MLflow Modelsarea/sqlalchemy: Use of SQLAlchemy in the Tracking Service or Model Registryarea/windows: Windows supportLanguage
language/r: R APIs and clientslanguage/java: Java APIs and clientslanguage/new: Proposals for new client languagesIntegrations
integrations/azure: Azure and Azure ML integrationsintegrations/sagemaker: SageMaker integrationsintegrations/databricks: Databricks integrationsHow should the PR be classified in the release notes? Choose one:
rn/breaking-change- The PR will be mentioned in the "Breaking Changes" sectionrn/none- No description will be included. The PR will be mentioned only by the PR number in the "Small Bugfixes and Documentation Updates" sectionrn/feature- A new user-facing feature worth mentioning in the release notesrn/bug-fix- A user-facing bug fix worth mentioning in the release notesrn/documentation- A user-facing documentation change worth mentioning in the release notes