Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Avoid using flask 2.2.0 #6380

Merged
merged 1 commit into from
Aug 2, 2022
Merged

Avoid using flask 2.2.0 #6380

merged 1 commit into from
Aug 2, 2022

Conversation

harupy
Copy link
Member

@harupy harupy commented Aug 2, 2022

Signed-off-by: harupy hkawamura0130@gmail.com

Related Issues/PRs

#xxx

What changes are proposed in this pull request?

Avoid using flask 2.2.0 until rycus86/prometheus_flask_exporter#138 is fixed.

How is this patch tested?

  • I have written tests (not required for typo or doc fix) and confirmed the proposed feature/bug-fix/change works.

Does this PR change the documentation?

  • No. You can skip the rest of this section.
  • Yes. Make sure the changed pages / sections render correctly by following the steps below.
  1. Check the status of the ci/circleci: build_doc check. If it's successful, proceed to the
    next step, otherwise fix it.
  2. Click Details on the right to open the job page of CircleCI.
  3. Click the Artifacts tab.
  4. Click docs/build/html/index.html.
  5. Find the changed pages / sections and make sure they render correctly.

Release Notes

Is this a user-facing change?

  • No. You can skip the rest of this section.
  • Yes. Give a description of this change to be included in the release notes for MLflow users.

(Details in 1-2 sentences. You can just refer to another PR with a description if this PR is part of a larger change.)

What component(s), interfaces, languages, and integrations does this PR affect?

Components

  • area/artifacts: Artifact stores and artifact logging
  • area/build: Build and test infrastructure for MLflow
  • area/docs: MLflow documentation pages
  • area/examples: Example code
  • area/model-registry: Model Registry service, APIs, and the fluent client calls for Model Registry
  • area/models: MLmodel format, model serialization/deserialization, flavors
  • area/pipelines: Pipelines, Pipeline APIs, Pipeline configs, Pipeline Templates
  • area/projects: MLproject format, project running backends
  • area/scoring: MLflow Model server, model deployment tools, Spark UDFs
  • area/server-infra: MLflow Tracking server backend
  • area/tracking: Tracking Service, tracking client APIs, autologging

Interface

  • area/uiux: Front-end, user experience, plotting, JavaScript, JavaScript dev server
  • area/docker: Docker use across MLflow's components, such as MLflow Projects and MLflow Models
  • area/sqlalchemy: Use of SQLAlchemy in the Tracking Service or Model Registry
  • area/windows: Windows support

Language

  • language/r: R APIs and clients
  • language/java: Java APIs and clients
  • language/new: Proposals for new client languages

Integrations

  • integrations/azure: Azure and Azure ML integrations
  • integrations/sagemaker: SageMaker integrations
  • integrations/databricks: Databricks integrations

How should the PR be classified in the release notes? Choose one:

  • rn/breaking-change - The PR will be mentioned in the "Breaking Changes" section
  • rn/none - No description will be included. The PR will be mentioned only by the PR number in the "Small Bugfixes and Documentation Updates" section
  • rn/feature - A new user-facing feature worth mentioning in the release notes
  • rn/bug-fix - A user-facing bug fix worth mentioning in the release notes
  • rn/documentation - A user-facing documentation change worth mentioning in the release notes

Signed-off-by: harupy <hkawamura0130@gmail.com>
@harupy harupy requested a review from dbczumar August 2, 2022 01:08
@github-actions github-actions bot added area/build Build and test infrastructure for MLflow rn/none List under Small Changes in Changelogs. labels Aug 2, 2022
Copy link
Collaborator

@dbczumar dbczumar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Thanks @harupy !

@harupy harupy enabled auto-merge (squash) August 2, 2022 01:17
@harupy harupy merged commit 2e421b0 into mlflow:master Aug 2, 2022
harupy added a commit that referenced this pull request Aug 2, 2022
@harupy harupy mentioned this pull request Aug 2, 2022
31 tasks
harupy added a commit that referenced this pull request Aug 2, 2022
* revert

Signed-off-by: harupy <hkawamura0130@gmail.com>

* workaround for test_prometheus_exporter

Signed-off-by: harupy <hkawamura0130@gmail.com>

* nit

Signed-off-by: harupy <hkawamura0130@gmail.com>
harupy added a commit that referenced this pull request Aug 16, 2022
* Fix java model server compatibility with models containing databricks_runtime conf (#6337)

* Javafix

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Resource fix

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Add run / model links to MLP train step card (#6294)

* One commit

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Submodule revert

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Fix client tests

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* fix ci

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Remove unused / invalid import

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Put run summary back to end, fix tests

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Use mocks instead

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Fix inefficient relative time computation for metric plot (#6341)

* compute minTimestamp only once

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix method name

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix lint

Signed-off-by: harupy <hkawamura0130@gmail.com>

* return empty array

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix test

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Add `MLFLOW_SQLALCHEMYSTORE_POOL_RECYCLE` flag (#6344)

* Add pool_recycle option to SQLAlchemy create_engine

Signed-off-by: Michal Karzynski <michal.karzynski@intel.com>

* Add new option to unit test

Signed-off-by: Michal Karzynski <michal.karzynski@intel.com>

* Add note in documentation

Signed-off-by: Michal Karzynski <michal.karzynski@intel.com>

* Add box plot in compare-runs page (#6308)

* Added BoxPlot

Signed-off-by: changyonglik <theeahlag@gmail.com>

* Fixed key error

Signed-off-by: changyonglik <theeahlag@gmail.com>

* Added default state

Signed-off-by: changyonglik <theeahlag@gmail.com>

* Fixed props passing

Signed-off-by: changyonglik <theeahlag@gmail.com>

* Fixed react hooks state

Signed-off-by: changyonglik <theeahlag@gmail.com>

* Added unit test

Signed-off-by: changyonglik <theeahlag@gmail.com>

* Added boxplot to CompareModelVersion

Signed-off-by: changyonglik <theeahlag@gmail.com>

* Changed keys numbering to reflect tab name

Signed-off-by: changyonglik <theeahlag@gmail.com>

* Made the ordering of tabs consistent accross CompareRunView and CompareModelVersion

Signed-off-by: changyonglik <theeahlag@gmail.com>

* Linted

Signed-off-by: changyonglik <theeahlag@gmail.com>

* Fixed FormattedMessage

Signed-off-by: changyonglik <theeahlag@gmail.com>

* refactoring

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix lint errors

Signed-off-by: harupy <hkawamura0130@gmail.com>

* i18n

Signed-off-by: harupy <hkawamura0130@gmail.com>

* minor comment fix

Signed-off-by: harupy <hkawamura0130@gmail.com>

Co-authored-by: harupy <hkawamura0130@gmail.com>

* Add script and workflow to update requirements.yaml specifications (#6287)

* add script to update requirements.yaml

Signed-off-by: harupy <hkawamura0130@gmail.com>

* update both skinny and core requirements

Signed-off-by: harupy <hkawamura0130@gmail.com>

* remove base

Signed-off-by: harupy <hkawamura0130@gmail.com>

* clean up

Signed-off-by: harupy <hkawamura0130@gmail.com>

* add MLFLOW_CONDA_HOME

Signed-off-by: harupy <hkawamura0130@gmail.com>

* create create-pr job

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix syntax

Signed-off-by: harupy <hkawamura0130@gmail.com>

* remove redundant if

Signed-off-by: harupy <hkawamura0130@gmail.com>

* clean up

Signed-off-by: harupy <hkawamura0130@gmail.com>

* doc

Signed-off-by: harupy <hkawamura0130@gmail.com>

* dynamically determine python version

Signed-off-by: harupy <hkawamura0130@gmail.com>

* remove python version mock

Signed-off-by: harupy <hkawamura0130@gmail.com>

* use setup-ssh.sh in master.yml

Signed-off-by: harupy <hkawamura0130@gmail.com>

* simplify

Signed-off-by: harupy <hkawamura0130@gmail.com>

* update both core and skinny requirements

Signed-off-by: harupy <hkawamura0130@gmail.com>

* use env

Signed-off-by: harupy <hkawamura0130@gmail.com>

* remove action version

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix pandas dtypes issue

Signed-off-by: harupy <hkawamura0130@gmail.com>

* check if PR already exists

Signed-off-by: harupy <hkawamura0130@gmail.com>

* add semicolons

Signed-off-by: harupy <hkawamura0130@gmail.com>

* test PR creation

Signed-off-by: harupy <hkawamura0130@gmail.com>

* set base

Signed-off-by: harupy <hkawamura0130@gmail.com>

* use run_number

Signed-off-by: harupy <hkawamura0130@gmail.com>

* set result-encoding

Signed-off-by: harupy <hkawamura0130@gmail.com>

* revert changes for test

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix more tests

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix run URL

Signed-off-by: harupy <hkawamura0130@gmail.com>

* add dot

Signed-off-by: harupy <hkawamura0130@gmail.com>

* remove minimum_version

Signed-off-by: harupy <hkawamura0130@gmail.com>

* remove create-pr job

Signed-off-by: harupy <hkawamura0130@gmail.com>

* rename workflow

Signed-off-by: harupy <hkawamura0130@gmail.com>

* mutate dict approach

Signed-off-by: harupy <hkawamura0130@gmail.com>

* use ruamel

Signed-off-by: harupy <hkawamura0130@gmail.com>

* remove PYTHON_VERSION patch

Signed-off-by: harupy <hkawamura0130@gmail.com>

* add ruamel.yaml to requirements

Signed-off-by: harupy <hkawamura0130@gmail.com>

* skip update-requirements on PR

Signed-off-by: harupy <hkawamura0130@gmail.com>

* rename

Signed-off-by: harupy <hkawamura0130@gmail.com>

* rename

Signed-off-by: harupy <hkawamura0130@gmail.com>

* print new version

Signed-off-by: harupy <hkawamura0130@gmail.com>

* preserve quotes

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix badge

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix README

Signed-off-by: harupy <hkawamura0130@gmail.com>

* run ci

Signed-off-by: harupy <hkawamura0130@gmail.com>

* remove validate-requirements.yml

Signed-off-by: harupy <hkawamura0130@gmail.com>

* rename

Signed-off-by: harupy <hkawamura0130@gmail.com>

* rename

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix test_dataframe_from_json

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Make `model_uri` optional for `mlflow models build-docker` (#6302)

* Make model_uri optional for mlflow models build-docker

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix doc

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Add generic

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Add pythou_function flavor

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix doc

Signed-off-by: harupy <hkawamura0130@gmail.com>

* quotes

Signed-off-by: harupy <hkawamura0130@gmail.com>

* use longer options

Signed-off-by: harupy <hkawamura0130@gmail.com>

* add index (#6347)

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix badge link (#6349)

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Enable consider-using-dict-items rule for pylint (#6351)

* Enable consider-using-dict-items

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix lint errors

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Enable consider-merging-isinstance (#6350)

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Enable consider-using-from-import rule for pylint (#6352)

* Enable consider-using-from-import

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix datasets import

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix failed tests

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Run model registry sqlalchemy tests in database job (#6353)

Signed-off-by: harupy <hkawamura0130@gmail.com>

* update to run profiles on the first 100 dataframe columns (#6297)

* update to run profiles on the first 100 dataframe columns

Signed-off-by: Sunish Sheth <sunishsheth2009@gmail.com>

* Adding replace=True so we can sample rows twice

Signed-off-by: Sunish Sheth <sunishsheth2009@gmail.com>

* Removing replace=true and fixing how the max_rows are calculated

Signed-off-by: Sunish Sheth <sunishsheth2009@gmail.com>

* Fix sklearn autologging input example & signature mutation for models containing transformers (#6230)

* Fix with test case

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Test fix

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Fix and simplification

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* more fixes & better test coveragE

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* test

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* LGBM test

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Format

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Address comments

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Fixes

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Use py37-compatible call args syntax

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* use list (#6363)

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Log runtime pipeline config to MLflow (#6359)

* Log runtime pipeline config to MLflow

Signed-off-by: Jin Zhang <jin.zhang@databricks.com>

* Fixed lint

Signed-off-by: Jin Zhang <jin.zhang@databricks.com>

* Add status badge for stale action (#6372)

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Add stack trace section to bug report template (#6374)

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Skipping the transform step by using Functional transformer (#6362)

Signed-off-by: Sunish Sheth <sunishsheth2009@gmail.com>

* Series schema (#6361)

* Added in name for ColSpec of pandas datatype

infer panda series name for ColSpec

Signed-off-by: Ryan Fogle <ryanfogle98@gmail.com>

* Added test for Series name inference

added one more pytest condition in test_schema:test_schema_inference_on_pandas_series

Signed-off-by: Ryan Fogle <ryanfogle98@gmail.com>

* Update test_schema.py

Added in one more assertion to test schema inference.

Signed-off-by: Ryan Fogle <ryanfogle98@gmail.com>

* ran black

Signed-off-by: Ryan Fogle <ryanfogle98@gmail.com>

* added in requested changes in issue #6361

Signed-off-by: Ryan Fogle <ryanfogle98@gmail.com>

* fix typo. changed hasattr to getattr

Signed-off-by: Ryan Fogle <ryanfogle98@gmail.com>

* integrate mlflowdbfs (#6282)

Signed-off-by: Brian Barnes <brian.barnes@databricks.com>

* Fix step card display breakage in Jupyter notebooks (#6378)

* remove

Signed-off-by: apurva-koti <apurva.koti@databricks.com>

* fix

Signed-off-by: apurva-koti <apurva.koti@databricks.com>

* Avoid using flask 2.2.0 (#6380)

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Migrate GCS environment variables to `mlflow.environment_variables` module (#6375)

* Migrate GCS environment variables to mlflow.environment_variables

Signed-off-by: harupy <hkawamura0130@gmail.com>

* add default

Signed-off-by: harupy <hkawamura0130@gmail.com>

* improve comments

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Avoid logging model signatures in pyspark ML autologging if model input/output dataframe contains unsupported data types (#6365)

* Avoid logging signatures

Signed-off-by: harupy <hkawamura0130@gmail.com>

* test

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix indent

Signed-off-by: harupy <hkawamura0130@gmail.com>

* check model output

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix warning message

Signed-off-by: harupy <hkawamura0130@gmail.com>

* todo

Signed-off-by: harupy <hkawamura0130@gmail.com>

* update test and doc

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix docstring

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix failed tests

Signed-off-by: harupy <hkawamura0130@gmail.com>

* improve _assert_autolog_infers_model_signature_correctly

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Refactor `set_matrix.py` (#6373)

* refacor set_matrix.py

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix tests

Signed-off-by: harupy <hkawamura0130@gmail.com>

* tset for --no-dev

Signed-off-by: harupy <hkawamura0130@gmail.com>

* install pytest and pytest-cov

Signed-off-by: harupy <hkawamura0130@gmail.com>

* test for --changed-files

Signed-off-by: harupy <hkawamura0130@gmail.com>

* add pyyaml to requirements

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix is_matrix_empty

Signed-off-by: harupy <hkawamura0130@gmail.com>

* use raw string and re.match

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Fix is_matrix_empty (#6385)

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Allow mlflow-skinny's autolog() to succeed when no scipy is installed (#5897)

* allow mlflow-skinny's autolog() to succeed when no scipy is installed

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* optional pyfunc scipy dependency

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* fix linter-discovered issues

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* add test case, autolog() should pass w/o scipy installed

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* minor cleanup

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* test creating _Example with / without scipy present

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* docs: do not warn about sparse matrix classes

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* linting

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* address review comment

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* run black

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* allow mlflow-skinny's autolog() to succeed when no scipy is installed

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* optional pyfunc scipy dependency

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* fix linter-discovered issues

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* add test case, autolog() should pass w/o scipy installed

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* minor cleanup

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* test creating _Example with / without scipy present

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* docs: do not warn about sparse matrix classes

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* linting

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* address review comment

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* run black

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* remove redundant None check after import

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* delete comment

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* Revert "remove redundant None check after import"

This reverts commit f2e198c.

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* review comments

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* remove redundant test

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* Apply suggestions from code review

Co-authored-by: Harutaka Kawamura <hkawamura0130@gmail.com>
Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* Autoformat: https://github.com/mlflow/mlflow/actions/runs/2779791161

Signed-off-by: mlflow-automation <mlflow-automation@users.noreply.github.com>

Co-authored-by: Harutaka Kawamura <hkawamura0130@gmail.com>
Co-authored-by: dbczumar <corey.zumar@databricks.com>
Co-authored-by: mlflow-automation <mlflow-automation@users.noreply.github.com>

* Fix extraneous warning about pyspark evaluation truncation (#6389)

* Revert #6380 (#6386)

* revert

Signed-off-by: harupy <hkawamura0130@gmail.com>

* workaround for test_prometheus_exporter

Signed-off-by: harupy <hkawamura0130@gmail.com>

* nit

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Validate metric value in `FileStore.log_metric` (#6381)

* add failing test

Signed-off-by: harupy <hkawamura0130@gmail.com>

* use _validate_metric

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix invalid metric timestamp

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix docstring

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix java test

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Make search experiment by tag key comparison case sensitive in MYSQL (#6333)

* init

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* add test

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* update

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* update

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* fix BINARY conditions in MySQL

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix attr error

Signed-off-by: harupy <hkawamura0130@gmail.com>

Co-authored-by: harupy <hkawamura0130@gmail.com>

* Run Model Registry SQLAlchemy tests in PostgreSQL, MySQL, and MSSQL (#6390)

* Run Model Registry SQLAlchemy tests in postgres, mysql, and mssql

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix failed tests

Signed-off-by: harupy <hkawamura0130@gmail.com>

* remove uuid

Signed-off-by: harupy <hkawamura0130@gmail.com>

* clean up

Signed-off-by: harupy <hkawamura0130@gmail.com>

* do not drop tables

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Fix filestore data loss on failed write & mitigate read / write race condition (#6388)

* Fix

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Fix

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Format

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Added a custom filter from_json to render_and_merge_yaml (#6368)

* Added a custom filter from_json to render_and_merge_yaml

Signed-off-by: Jin Zhang <jin.zhang@databricks.com>

* Fixed test

Signed-off-by: Jin Zhang <jin.zhang@databricks.com>

* Update tests/utils/test_file_utils.py

Co-authored-by: Siddharth Murching <sid.murching@databricks.com>

* Update tests/utils/test_file_utils.py

Co-authored-by: Siddharth Murching <sid.murching@databricks.com>

* Fixed test

Signed-off-by: Jin Zhang <jin.zhang@databricks.com>

* Fixed windows test attempt

Signed-off-by: Jin Zhang <jin.zhang@databricks.com>

* Fixed windows test

Signed-off-by: Jin Zhang <jin.zhang@databricks.com>

Co-authored-by: Siddharth Murching <sid.murching@databricks.com>

* Fix `!=` operator for MySQL (#6397)

* fix ne operator and improve tests

Signed-off-by: harupy <hkawamura0130@gmail.com>

* add () to other operators

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Simplify the documentation section in the PR template (#6398)

* Simplify the documentation section in the PR template

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Timebased gc (#6354)

* Added delete_time field to runs table

Signed-off-by: Jason Cheng <jasoncky96@gmail.com>

* Updated latest_schema.sql to pass tests/store/tracking/test_sqlalchemy_store_schema.py

Signed-off-by: Jason Cheng <jasoncky96@gmail.com>

* Added delete_time column into mlflow/store/tracking/dbmodels/initial_models.py

Signed-off-by: Jason Cheng <jasoncky96@gmail.com>

* Moved the docstring to the end of the declaration instead of the start

Signed-off-by: Jason Cheng <jasoncky96@gmail.com>

* Regenerated protobuf for RunInfo new schema, and passed pytest

Signed-off-by: Jason Cheng <jasoncky96@gmail.com>

* Added --older-than flag in mlflow gc command, and added tests for that cli command

Signed-off-by: Jason Cheng <jasoncky96@gmail.com>

* Removed delete_time from RunInfo proto

Signed-off-by: Jason Cheng <jasoncky96@gmail.com>

* Set default older-than flag to None

Signed-off-by: Jason Cheng <jasoncky96@gmail.com>

* Remove delete_time from RunInfo, add optional parameter older_than to store.get_deleted_runs(), and write delete_time into yaml file for file storage instead of adding it to RunInfo

Signed-off-by: Jason Cheng <jasoncky96@gmail.com>

* Remove commented delete_time in RunInfo constructor in models.py

Signed-off-by: Jason Cheng <jasoncky96@gmail.com>

* added INVALID_PARAMETER_VALUE error code in mlflow gc

Signed-off-by: Jason Cheng <jasoncky96@gmail.com>

* Resolve case of delete time not in meta.yaml file, edited overwrite_run_info and gc

Signed-off-by: Jason Cheng <jasoncky96@gmail.com>

* Rename delete_time to deleted_time

Signed-off-by: Jason Cheng <jasoncky96@gmail.com>

* Rename delete_time to deleted_time, match error message in cli tests, filter SqlRuns by deleted_time, indicate float values accepted for gc --older-than flag

Signed-off-by: Jason Cheng <jasoncky96@gmail.com>

* Rename delete_time to deleted_time, fixed assert statement in test_mlflow_gc, and fixed deleted_time queries in test_file_store.py and test_sqlalchemy_store.py

Signed-off-by: Jason Cheng <jasoncky96@gmail.com>

* Only check if  is in meta after restoring run in test_delte_restore_run

Signed-off-by: Jason Cheng <jasoncky96@gmail.com>

* Fixed lint errors

Signed-off-by: Jason Cheng <jasoncky96@gmail.com>

* Install mlflow from repository root

Signed-off-by: harupy <hkawamura0130@gmail.com>

Co-authored-by: harupy <hkawamura0130@gmail.com>

* Separate protos job (#6399)

* Separate protos job

Signed-off-by: harupy <hkawamura0130@gmail.com>

* test

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Add paths

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Revert "test"

This reverts commit 5d04b02.

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Fix flaky `TestFileStore.test_get_deleted_runs` (#6402)

* use gt operator

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix sqlalchemy store

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Migrate environment variables for SQLAlchemy store (#6396)

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Deduplicate MlflowClient in MLflow docs, move to mlflow.client module (#6405)

* Dedupe MlflowClient

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Improvements

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Add client

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Remove unused functions in `mlflow.data` module (#6408)

* remove unused functions in data

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Remove unused functions and variables

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Tag based registered model search and model version search (#6320)

* init

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* update

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* update

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* update

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* update

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* update

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* update

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* update

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* update

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* update

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* update

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* update tests

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* update

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* update

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* clean

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* address comments

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* add test

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* address comments

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* update

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* de-dup tag based search result

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* add () to fix case_sensitive_mysql_ne

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* fix

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* lint

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* fix

* fix test in TODO

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* support dup tag key filter

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* add mv order by

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* cmp res list in test

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* address comments

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* address comments

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* add doc

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* fix

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* fix lint

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* address comments

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* address comments

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* update doc

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* Increase maximum param value length to 500 (#6358)

* increase validation of maximum param value length from 250 to 500 (squashed)

Signed-off-by: Jan Sindlar <jan.sindlar@eureka.ai>

* 1. added test for exceeding max length. 2. lint. 3. validation logic addresses null

Signed-off-by: Jan Sindlar <jan.sindlar@eureka.ai>

* file store should test already stringified params

Signed-off-by: Jan Sindlar <jan.sindlar@eureka.ai>

* update alembic chain

Signed-off-by: Jan Sindlar <jan.sindlar@eureka.ai>

* Autoformat: https://github.com/mlflow/mlflow/actions/runs/2800398158

Signed-off-by: mlflow-automation <mlflow-automation@users.noreply.github.com>

* Update tests/store/tracking/test_file_store.py

Co-authored-by: Harutaka Kawamura <hkawamura0130@gmail.com>
Signed-off-by: Jan Sindlar <jan.sindlar@eureka.ai>

Co-authored-by: Jan Sindlar <jan.sindlar@eureka.ai>
Co-authored-by: mlflow-automation <mlflow-automation@users.noreply.github.com>
Co-authored-by: Harutaka Kawamura <hkawamura0130@gmail.com>

* Add workflow dispatch for requirements validation, remove duplicate file (#6418)

* Workflow dispatch

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Remove unused

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Prevent `pytest.raises` context manager from containing multiple statements (#6409)

* rename

Signed-off-by: harupy <hkawamura0130@gmail.com>

* rename

Signed-off-by: harupy <hkawamura0130@gmail.com>

* rename

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Do not allow calling assert in pytest.raises

Signed-off-by: harupy <hkawamura0130@gmail.com>

* more strict rule

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Improve error message

Signed-off-by: harupy <hkawamura0130@gmail.com>

* relax condition

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix lint errors

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix failed tests

Signed-off-by: harupy <hkawamura0130@gmail.com>

* clean up

Signed-off-by: harupy <hkawamura0130@gmail.com>

* rename _is_complex_pytest_raises

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix lint errors

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix failed tests

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Fix regression in pandas type inference behavior for scoring server (#6417)

* Fix impl

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Remove unused pandas import

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Adjust client to use SearchExperiments POST method (#6415)

* POST

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Generate protos

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Fix test

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* update (#6420)

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* Check if artifacts are proxied in model_version_artifact_handler (#6355)

* Check if artifacts are proxied in model_version_artifact_handler

Signed-off-by: Mohammad Abbas <mohammad.abbas@digital.trade.gov.uk>

* Use file prefix for artifact dest on windows

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Fix application of prefix

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Fixes

Signed-off-by: dbczumar <corey.zumar@databricks.com>

Co-authored-by: dbczumar <corey.zumar@databricks.com>

* Allow sql filter IN operator right-hand side values including whitespaces (#6413)

* support in rhs whitespace

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* update

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* Fix IN operator related issues in sql filter parser (#6411)

* update

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* update

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* clean

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* update

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* update

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* update

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* address comment

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* Remove unused method `SearchUtils._get_comparison_for_model_registry` (#6412)

* init

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* update

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* Update MLflow UI (#6422)

* Project import generated by Copybara.

GitOrigin-RevId: 086b1200f4573559e37fcea6cb5472d52f166e42
Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Reconcile changes from universe with OSS

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Fix test fails

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Fix

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Apply model search fix

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Remove errant test line

Signed-off-by: dbczumar <corey.zumar@databricks.com>

Co-authored-by: Richard Zang <Zangr@users.noreply.github.com>

* remove near from deprecation msg (#6430)

Signed-off-by: ninabacc-db <Nina.Baccam@databricks.com>

* Raise clear error message when pipeline path contains a space (#6426)

* Add unsupported

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Fix

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Exc

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Test

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Format

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Test fix

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Fix

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* [ALL TESTS] Changelog for MLflow 1.28.0 (#6433)

* Generating changelog for MLflow 1.27.1.dev0

Signed-off-by: Jenkins <jenkins@databricks.com>

* Changelog

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Add missing PR

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Some fixes

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* More small fixes

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Fix

Signed-off-by: dbczumar <corey.zumar@databricks.com>

Signed-off-by: Jenkins <jenkins@databricks.com>
Signed-off-by: dbczumar <corey.zumar@databricks.com>
Co-authored-by: Jenkins <jenkins@databricks.com>
Co-authored-by: dbczumar <corey.zumar@databricks.com>

* Patch (#6440)

Signed-off-by: dbczumar <corey.zumar@databricks.com>

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* [ALL TESTS] Update  (#6443)

* python dev/update_ml_package_versions.py

Signed-off-by: Jenkins <jenkins@databricks.com>

* pyspark max ver 3.4

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

Signed-off-by: Jenkins <jenkins@databricks.com>
Signed-off-by: Weichen Xu <weichen.xu@databricks.com>
Co-authored-by: Jenkins <jenkins@databricks.com>
Co-authored-by: Weichen Xu <weichen.xu@databricks.com>

* Update MLflow version to 1.28.1 (#6449)

Signed-off-by: Jenkins <jenkins@databricks.com>

Signed-off-by: Jenkins <jenkins@databricks.com>
Co-authored-by: Jenkins <jenkins@databricks.com>

* Add data capture config to sagemaker (#6423)

Signed-off-by: jonwiggins <me@jonwiggins.org>

Signed-off-by: jonwiggins <me@jonwiggins.org>

* Fix small search_runs docs issues (#6452)

* Fix

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Doc fix

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Fix

Signed-off-by: dbczumar <corey.zumar@databricks.com>

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* python dev/update_pypi_package_index.py (#6442)

Signed-off-by: Jenkins <jenkins@databricks.com>

Signed-off-by: Jenkins <jenkins@databricks.com>
Co-authored-by: Jenkins <jenkins@databricks.com>

* use xgboost on pypi (#6461)

Signed-off-by: harupy <hkawamura0130@gmail.com>

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Add environment variable for `echo` option in `create_engine` (#6460)

* Add environment variable for echo

Signed-off-by: harupy <hkawamura0130@gmail.com>

* use _BooleanEnvironmentVariable

Signed-off-by: harupy <hkawamura0130@gmail.com>

* address comments

Signed-off-by: harupy <hkawamura0130@gmail.com>

* address comments and add tests

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix comment

Signed-off-by: harupy <hkawamura0130@gmail.com>

Signed-off-by: harupy <hkawamura0130@gmail.com>

Signed-off-by: dbczumar <corey.zumar@databricks.com>
Signed-off-by: harupy <hkawamura0130@gmail.com>
Signed-off-by: Michal Karzynski <michal.karzynski@intel.com>
Signed-off-by: Sunish Sheth <sunishsheth2009@gmail.com>
Signed-off-by: Jin Zhang <jin.zhang@databricks.com>
Signed-off-by: Ryan Fogle <ryanfogle98@gmail.com>
Signed-off-by: Brian Barnes <brian.barnes@databricks.com>
Signed-off-by: apurva-koti <apurva.koti@databricks.com>
Signed-off-by: Weichen Xu <weichen.xu@databricks.com>
Signed-off-by: ninabacc-db <Nina.Baccam@databricks.com>
Signed-off-by: Jenkins <jenkins@databricks.com>
Signed-off-by: jonwiggins <me@jonwiggins.org>
Co-authored-by: Corey Zumar <39497902+dbczumar@users.noreply.github.com>
Co-authored-by: Michał Karzyński <michal.karzynski@intel.com>
Co-authored-by: Chang Yong Lik <51813538+ahlag@users.noreply.github.com>
Co-authored-by: Sunish Sheth <sunishsheth2009@gmail.com>
Co-authored-by: jinzhang21 <78067366+jinzhang21@users.noreply.github.com>
Co-authored-by: Ryan Fogle <40377123+RynoXLI@users.noreply.github.com>
Co-authored-by: bbarnes52 <brian.barnes@databricks.com>
Co-authored-by: apurva-koti <51172624+apurva-koti@users.noreply.github.com>
Co-authored-by: Hannes Schulz <hannes.schulz@microsoft.com>
Co-authored-by: dbczumar <corey.zumar@databricks.com>
Co-authored-by: mlflow-automation <mlflow-automation@users.noreply.github.com>
Co-authored-by: WeichenXu <weichen.xu@databricks.com>
Co-authored-by: Siddharth Murching <sid.murching@databricks.com>
Co-authored-by: Cheng Kuan Yong Jason <jasoncky96@gmail.com>
Co-authored-by: johnyNJ <77425417+johnyNJ@users.noreply.github.com>
Co-authored-by: Jan Sindlar <jan.sindlar@eureka.ai>
Co-authored-by: Mohammad Abbas <mohammad.abbas86@gmail.com>
Co-authored-by: Richard Zang <Zangr@users.noreply.github.com>
Co-authored-by: Nina Baccam <106279294+ninabacc-db@users.noreply.github.com>
Co-authored-by: mlflow-automation <61449322+mlflow-automation@users.noreply.github.com>
Co-authored-by: Jenkins <jenkins@databricks.com>
Co-authored-by: Jon Wiggins <35540058+jonwiggins@users.noreply.github.com>
harupy added a commit that referenced this pull request Aug 19, 2022
* Fix java model server compatibility with models containing databricks_runtime conf (#6337)

* Javafix

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Resource fix

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Add run / model links to MLP train step card (#6294)

* One commit

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Submodule revert

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Fix client tests

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* fix ci

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Remove unused / invalid import

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Put run summary back to end, fix tests

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Use mocks instead

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Fix inefficient relative time computation for metric plot (#6341)

* compute minTimestamp only once

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix method name

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix lint

Signed-off-by: harupy <hkawamura0130@gmail.com>

* return empty array

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix test

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Add `MLFLOW_SQLALCHEMYSTORE_POOL_RECYCLE` flag (#6344)

* Add pool_recycle option to SQLAlchemy create_engine

Signed-off-by: Michal Karzynski <michal.karzynski@intel.com>

* Add new option to unit test

Signed-off-by: Michal Karzynski <michal.karzynski@intel.com>

* Add note in documentation

Signed-off-by: Michal Karzynski <michal.karzynski@intel.com>

* Add box plot in compare-runs page (#6308)

* Added BoxPlot

Signed-off-by: changyonglik <theeahlag@gmail.com>

* Fixed key error

Signed-off-by: changyonglik <theeahlag@gmail.com>

* Added default state

Signed-off-by: changyonglik <theeahlag@gmail.com>

* Fixed props passing

Signed-off-by: changyonglik <theeahlag@gmail.com>

* Fixed react hooks state

Signed-off-by: changyonglik <theeahlag@gmail.com>

* Added unit test

Signed-off-by: changyonglik <theeahlag@gmail.com>

* Added boxplot to CompareModelVersion

Signed-off-by: changyonglik <theeahlag@gmail.com>

* Changed keys numbering to reflect tab name

Signed-off-by: changyonglik <theeahlag@gmail.com>

* Made the ordering of tabs consistent accross CompareRunView and CompareModelVersion

Signed-off-by: changyonglik <theeahlag@gmail.com>

* Linted

Signed-off-by: changyonglik <theeahlag@gmail.com>

* Fixed FormattedMessage

Signed-off-by: changyonglik <theeahlag@gmail.com>

* refactoring

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix lint errors

Signed-off-by: harupy <hkawamura0130@gmail.com>

* i18n

Signed-off-by: harupy <hkawamura0130@gmail.com>

* minor comment fix

Signed-off-by: harupy <hkawamura0130@gmail.com>

Co-authored-by: harupy <hkawamura0130@gmail.com>

* Add script and workflow to update requirements.yaml specifications (#6287)

* add script to update requirements.yaml

Signed-off-by: harupy <hkawamura0130@gmail.com>

* update both skinny and core requirements

Signed-off-by: harupy <hkawamura0130@gmail.com>

* remove base

Signed-off-by: harupy <hkawamura0130@gmail.com>

* clean up

Signed-off-by: harupy <hkawamura0130@gmail.com>

* add MLFLOW_CONDA_HOME

Signed-off-by: harupy <hkawamura0130@gmail.com>

* create create-pr job

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix syntax

Signed-off-by: harupy <hkawamura0130@gmail.com>

* remove redundant if

Signed-off-by: harupy <hkawamura0130@gmail.com>

* clean up

Signed-off-by: harupy <hkawamura0130@gmail.com>

* doc

Signed-off-by: harupy <hkawamura0130@gmail.com>

* dynamically determine python version

Signed-off-by: harupy <hkawamura0130@gmail.com>

* remove python version mock

Signed-off-by: harupy <hkawamura0130@gmail.com>

* use setup-ssh.sh in master.yml

Signed-off-by: harupy <hkawamura0130@gmail.com>

* simplify

Signed-off-by: harupy <hkawamura0130@gmail.com>

* update both core and skinny requirements

Signed-off-by: harupy <hkawamura0130@gmail.com>

* use env

Signed-off-by: harupy <hkawamura0130@gmail.com>

* remove action version

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix pandas dtypes issue

Signed-off-by: harupy <hkawamura0130@gmail.com>

* check if PR already exists

Signed-off-by: harupy <hkawamura0130@gmail.com>

* add semicolons

Signed-off-by: harupy <hkawamura0130@gmail.com>

* test PR creation

Signed-off-by: harupy <hkawamura0130@gmail.com>

* set base

Signed-off-by: harupy <hkawamura0130@gmail.com>

* use run_number

Signed-off-by: harupy <hkawamura0130@gmail.com>

* set result-encoding

Signed-off-by: harupy <hkawamura0130@gmail.com>

* revert changes for test

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix more tests

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix run URL

Signed-off-by: harupy <hkawamura0130@gmail.com>

* add dot

Signed-off-by: harupy <hkawamura0130@gmail.com>

* remove minimum_version

Signed-off-by: harupy <hkawamura0130@gmail.com>

* remove create-pr job

Signed-off-by: harupy <hkawamura0130@gmail.com>

* rename workflow

Signed-off-by: harupy <hkawamura0130@gmail.com>

* mutate dict approach

Signed-off-by: harupy <hkawamura0130@gmail.com>

* use ruamel

Signed-off-by: harupy <hkawamura0130@gmail.com>

* remove PYTHON_VERSION patch

Signed-off-by: harupy <hkawamura0130@gmail.com>

* add ruamel.yaml to requirements

Signed-off-by: harupy <hkawamura0130@gmail.com>

* skip update-requirements on PR

Signed-off-by: harupy <hkawamura0130@gmail.com>

* rename

Signed-off-by: harupy <hkawamura0130@gmail.com>

* rename

Signed-off-by: harupy <hkawamura0130@gmail.com>

* print new version

Signed-off-by: harupy <hkawamura0130@gmail.com>

* preserve quotes

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix badge

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix README

Signed-off-by: harupy <hkawamura0130@gmail.com>

* run ci

Signed-off-by: harupy <hkawamura0130@gmail.com>

* remove validate-requirements.yml

Signed-off-by: harupy <hkawamura0130@gmail.com>

* rename

Signed-off-by: harupy <hkawamura0130@gmail.com>

* rename

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix test_dataframe_from_json

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Make `model_uri` optional for `mlflow models build-docker` (#6302)

* Make model_uri optional for mlflow models build-docker

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix doc

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Add generic

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Add pythou_function flavor

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix doc

Signed-off-by: harupy <hkawamura0130@gmail.com>

* quotes

Signed-off-by: harupy <hkawamura0130@gmail.com>

* use longer options

Signed-off-by: harupy <hkawamura0130@gmail.com>

* add index (#6347)

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix badge link (#6349)

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Enable consider-using-dict-items rule for pylint (#6351)

* Enable consider-using-dict-items

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix lint errors

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Enable consider-merging-isinstance (#6350)

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Enable consider-using-from-import rule for pylint (#6352)

* Enable consider-using-from-import

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix datasets import

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix failed tests

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Run model registry sqlalchemy tests in database job (#6353)

Signed-off-by: harupy <hkawamura0130@gmail.com>

* update to run profiles on the first 100 dataframe columns (#6297)

* update to run profiles on the first 100 dataframe columns

Signed-off-by: Sunish Sheth <sunishsheth2009@gmail.com>

* Adding replace=True so we can sample rows twice

Signed-off-by: Sunish Sheth <sunishsheth2009@gmail.com>

* Removing replace=true and fixing how the max_rows are calculated

Signed-off-by: Sunish Sheth <sunishsheth2009@gmail.com>

* Fix sklearn autologging input example & signature mutation for models containing transformers (#6230)

* Fix with test case

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Test fix

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Fix and simplification

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* more fixes & better test coveragE

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* test

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* LGBM test

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Format

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Address comments

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Fixes

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Use py37-compatible call args syntax

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* use list (#6363)

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Log runtime pipeline config to MLflow (#6359)

* Log runtime pipeline config to MLflow

Signed-off-by: Jin Zhang <jin.zhang@databricks.com>

* Fixed lint

Signed-off-by: Jin Zhang <jin.zhang@databricks.com>

* Add status badge for stale action (#6372)

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Add stack trace section to bug report template (#6374)

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Skipping the transform step by using Functional transformer (#6362)

Signed-off-by: Sunish Sheth <sunishsheth2009@gmail.com>

* Series schema (#6361)

* Added in name for ColSpec of pandas datatype

infer panda series name for ColSpec

Signed-off-by: Ryan Fogle <ryanfogle98@gmail.com>

* Added test for Series name inference

added one more pytest condition in test_schema:test_schema_inference_on_pandas_series

Signed-off-by: Ryan Fogle <ryanfogle98@gmail.com>

* Update test_schema.py

Added in one more assertion to test schema inference.

Signed-off-by: Ryan Fogle <ryanfogle98@gmail.com>

* ran black

Signed-off-by: Ryan Fogle <ryanfogle98@gmail.com>

* added in requested changes in issue #6361

Signed-off-by: Ryan Fogle <ryanfogle98@gmail.com>

* fix typo. changed hasattr to getattr

Signed-off-by: Ryan Fogle <ryanfogle98@gmail.com>

* integrate mlflowdbfs (#6282)

Signed-off-by: Brian Barnes <brian.barnes@databricks.com>

* Fix step card display breakage in Jupyter notebooks (#6378)

* remove

Signed-off-by: apurva-koti <apurva.koti@databricks.com>

* fix

Signed-off-by: apurva-koti <apurva.koti@databricks.com>

* Avoid using flask 2.2.0 (#6380)

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Migrate GCS environment variables to `mlflow.environment_variables` module (#6375)

* Migrate GCS environment variables to mlflow.environment_variables

Signed-off-by: harupy <hkawamura0130@gmail.com>

* add default

Signed-off-by: harupy <hkawamura0130@gmail.com>

* improve comments

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Avoid logging model signatures in pyspark ML autologging if model input/output dataframe contains unsupported data types (#6365)

* Avoid logging signatures

Signed-off-by: harupy <hkawamura0130@gmail.com>

* test

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix indent

Signed-off-by: harupy <hkawamura0130@gmail.com>

* check model output

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix warning message

Signed-off-by: harupy <hkawamura0130@gmail.com>

* todo

Signed-off-by: harupy <hkawamura0130@gmail.com>

* update test and doc

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix docstring

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix failed tests

Signed-off-by: harupy <hkawamura0130@gmail.com>

* improve _assert_autolog_infers_model_signature_correctly

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Refactor `set_matrix.py` (#6373)

* refacor set_matrix.py

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix tests

Signed-off-by: harupy <hkawamura0130@gmail.com>

* tset for --no-dev

Signed-off-by: harupy <hkawamura0130@gmail.com>

* install pytest and pytest-cov

Signed-off-by: harupy <hkawamura0130@gmail.com>

* test for --changed-files

Signed-off-by: harupy <hkawamura0130@gmail.com>

* add pyyaml to requirements

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix is_matrix_empty

Signed-off-by: harupy <hkawamura0130@gmail.com>

* use raw string and re.match

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Fix is_matrix_empty (#6385)

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Allow mlflow-skinny's autolog() to succeed when no scipy is installed (#5897)

* allow mlflow-skinny's autolog() to succeed when no scipy is installed

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* optional pyfunc scipy dependency

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* fix linter-discovered issues

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* add test case, autolog() should pass w/o scipy installed

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* minor cleanup

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* test creating _Example with / without scipy present

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* docs: do not warn about sparse matrix classes

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* linting

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* address review comment

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* run black

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* allow mlflow-skinny's autolog() to succeed when no scipy is installed

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* optional pyfunc scipy dependency

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* fix linter-discovered issues

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* add test case, autolog() should pass w/o scipy installed

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* minor cleanup

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* test creating _Example with / without scipy present

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* docs: do not warn about sparse matrix classes

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* linting

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* address review comment

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* run black

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* remove redundant None check after import

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* delete comment

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* Revert "remove redundant None check after import"

This reverts commit f2e198c.

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* review comments

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* remove redundant test

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* Apply suggestions from code review

Co-authored-by: Harutaka Kawamura <hkawamura0130@gmail.com>
Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* Autoformat: https://github.com/mlflow/mlflow/actions/runs/2779791161

Signed-off-by: mlflow-automation <mlflow-automation@users.noreply.github.com>

Co-authored-by: Harutaka Kawamura <hkawamura0130@gmail.com>
Co-authored-by: dbczumar <corey.zumar@databricks.com>
Co-authored-by: mlflow-automation <mlflow-automation@users.noreply.github.com>

* Fix extraneous warning about pyspark evaluation truncation (#6389)

* Revert #6380 (#6386)

* revert

Signed-off-by: harupy <hkawamura0130@gmail.com>

* workaround for test_prometheus_exporter

Signed-off-by: harupy <hkawamura0130@gmail.com>

* nit

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Validate metric value in `FileStore.log_metric` (#6381)

* add failing test

Signed-off-by: harupy <hkawamura0130@gmail.com>

* use _validate_metric

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix invalid metric timestamp

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix docstring

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix java test

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Make search experiment by tag key comparison case sensitive in MYSQL (#6333)

* init

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* add test

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* update

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* update

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* fix BINARY conditions in MySQL

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix attr error

Signed-off-by: harupy <hkawamura0130@gmail.com>

Co-authored-by: harupy <hkawamura0130@gmail.com>

* Run Model Registry SQLAlchemy tests in PostgreSQL, MySQL, and MSSQL (#6390)

* Run Model Registry SQLAlchemy tests in postgres, mysql, and mssql

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix failed tests

Signed-off-by: harupy <hkawamura0130@gmail.com>

* remove uuid

Signed-off-by: harupy <hkawamura0130@gmail.com>

* clean up

Signed-off-by: harupy <hkawamura0130@gmail.com>

* do not drop tables

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Fix filestore data loss on failed write & mitigate read / write race condition (#6388)

* Fix

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Fix

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Format

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Added a custom filter from_json to render_and_merge_yaml (#6368)

* Added a custom filter from_json to render_and_merge_yaml

Signed-off-by: Jin Zhang <jin.zhang@databricks.com>

* Fixed test

Signed-off-by: Jin Zhang <jin.zhang@databricks.com>

* Update tests/utils/test_file_utils.py

Co-authored-by: Siddharth Murching <sid.murching@databricks.com>

* Update tests/utils/test_file_utils.py

Co-authored-by: Siddharth Murching <sid.murching@databricks.com>

* Fixed test

Signed-off-by: Jin Zhang <jin.zhang@databricks.com>

* Fixed windows test attempt

Signed-off-by: Jin Zhang <jin.zhang@databricks.com>

* Fixed windows test

Signed-off-by: Jin Zhang <jin.zhang@databricks.com>

Co-authored-by: Siddharth Murching <sid.murching@databricks.com>

* Fix `!=` operator for MySQL (#6397)

* fix ne operator and improve tests

Signed-off-by: harupy <hkawamura0130@gmail.com>

* add () to other operators

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Simplify the documentation section in the PR template (#6398)

* Simplify the documentation section in the PR template

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Timebased gc (#6354)

* Added delete_time field to runs table

Signed-off-by: Jason Cheng <jasoncky96@gmail.com>

* Updated latest_schema.sql to pass tests/store/tracking/test_sqlalchemy_store_schema.py

Signed-off-by: Jason Cheng <jasoncky96@gmail.com>

* Added delete_time column into mlflow/store/tracking/dbmodels/initial_models.py

Signed-off-by: Jason Cheng <jasoncky96@gmail.com>

* Moved the docstring to the end of the declaration instead of the start

Signed-off-by: Jason Cheng <jasoncky96@gmail.com>

* Regenerated protobuf for RunInfo new schema, and passed pytest

Signed-off-by: Jason Cheng <jasoncky96@gmail.com>

* Added --older-than flag in mlflow gc command, and added tests for that cli command

Signed-off-by: Jason Cheng <jasoncky96@gmail.com>

* Removed delete_time from RunInfo proto

Signed-off-by: Jason Cheng <jasoncky96@gmail.com>

* Set default older-than flag to None

Signed-off-by: Jason Cheng <jasoncky96@gmail.com>

* Remove delete_time from RunInfo, add optional parameter older_than to store.get_deleted_runs(), and write delete_time into yaml file for file storage instead of adding it to RunInfo

Signed-off-by: Jason Cheng <jasoncky96@gmail.com>

* Remove commented delete_time in RunInfo constructor in models.py

Signed-off-by: Jason Cheng <jasoncky96@gmail.com>

* added INVALID_PARAMETER_VALUE error code in mlflow gc

Signed-off-by: Jason Cheng <jasoncky96@gmail.com>

* Resolve case of delete time not in meta.yaml file, edited overwrite_run_info and gc

Signed-off-by: Jason Cheng <jasoncky96@gmail.com>

* Rename delete_time to deleted_time

Signed-off-by: Jason Cheng <jasoncky96@gmail.com>

* Rename delete_time to deleted_time, match error message in cli tests, filter SqlRuns by deleted_time, indicate float values accepted for gc --older-than flag

Signed-off-by: Jason Cheng <jasoncky96@gmail.com>

* Rename delete_time to deleted_time, fixed assert statement in test_mlflow_gc, and fixed deleted_time queries in test_file_store.py and test_sqlalchemy_store.py

Signed-off-by: Jason Cheng <jasoncky96@gmail.com>

* Only check if  is in meta after restoring run in test_delte_restore_run

Signed-off-by: Jason Cheng <jasoncky96@gmail.com>

* Fixed lint errors

Signed-off-by: Jason Cheng <jasoncky96@gmail.com>

* Install mlflow from repository root

Signed-off-by: harupy <hkawamura0130@gmail.com>

Co-authored-by: harupy <hkawamura0130@gmail.com>

* Separate protos job (#6399)

* Separate protos job

Signed-off-by: harupy <hkawamura0130@gmail.com>

* test

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Add paths

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Revert "test"

This reverts commit 5d04b02.

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Fix flaky `TestFileStore.test_get_deleted_runs` (#6402)

* use gt operator

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix sqlalchemy store

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Migrate environment variables for SQLAlchemy store (#6396)

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Deduplicate MlflowClient in MLflow docs, move to mlflow.client module (#6405)

* Dedupe MlflowClient

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Improvements

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Add client

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Remove unused functions in `mlflow.data` module (#6408)

* remove unused functions in data

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Remove unused functions and variables

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Tag based registered model search and model version search (#6320)

* init

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* update

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* update

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* update

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* update

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* update

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* update

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* update

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* update

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* update

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* update

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* update tests

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* update

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* update

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* clean

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* address comments

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* add test

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* address comments

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* update

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* de-dup tag based search result

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* add () to fix case_sensitive_mysql_ne

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* fix

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* lint

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* fix

* fix test in TODO

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* support dup tag key filter

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* add mv order by

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* cmp res list in test

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* address comments

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* address comments

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* add doc

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* fix

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* fix lint

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* address comments

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* address comments

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* update doc

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* Increase maximum param value length to 500 (#6358)

* increase validation of maximum param value length from 250 to 500 (squashed)

Signed-off-by: Jan Sindlar <jan.sindlar@eureka.ai>

* 1. added test for exceeding max length. 2. lint. 3. validation logic addresses null

Signed-off-by: Jan Sindlar <jan.sindlar@eureka.ai>

* file store should test already stringified params

Signed-off-by: Jan Sindlar <jan.sindlar@eureka.ai>

* update alembic chain

Signed-off-by: Jan Sindlar <jan.sindlar@eureka.ai>

* Autoformat: https://github.com/mlflow/mlflow/actions/runs/2800398158

Signed-off-by: mlflow-automation <mlflow-automation@users.noreply.github.com>

* Update tests/store/tracking/test_file_store.py

Co-authored-by: Harutaka Kawamura <hkawamura0130@gmail.com>
Signed-off-by: Jan Sindlar <jan.sindlar@eureka.ai>

Co-authored-by: Jan Sindlar <jan.sindlar@eureka.ai>
Co-authored-by: mlflow-automation <mlflow-automation@users.noreply.github.com>
Co-authored-by: Harutaka Kawamura <hkawamura0130@gmail.com>

* Add workflow dispatch for requirements validation, remove duplicate file (#6418)

* Workflow dispatch

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Remove unused

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Prevent `pytest.raises` context manager from containing multiple statements (#6409)

* rename

Signed-off-by: harupy <hkawamura0130@gmail.com>

* rename

Signed-off-by: harupy <hkawamura0130@gmail.com>

* rename

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Do not allow calling assert in pytest.raises

Signed-off-by: harupy <hkawamura0130@gmail.com>

* more strict rule

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Improve error message

Signed-off-by: harupy <hkawamura0130@gmail.com>

* relax condition

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix lint errors

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix failed tests

Signed-off-by: harupy <hkawamura0130@gmail.com>

* clean up

Signed-off-by: harupy <hkawamura0130@gmail.com>

* rename _is_complex_pytest_raises

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix lint errors

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix failed tests

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Fix regression in pandas type inference behavior for scoring server (#6417)

* Fix impl

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Remove unused pandas import

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Adjust client to use SearchExperiments POST method (#6415)

* POST

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Generate protos

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Fix test

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* update (#6420)

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* Check if artifacts are proxied in model_version_artifact_handler (#6355)

* Check if artifacts are proxied in model_version_artifact_handler

Signed-off-by: Mohammad Abbas <mohammad.abbas@digital.trade.gov.uk>

* Use file prefix for artifact dest on windows

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Fix application of prefix

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Fixes

Signed-off-by: dbczumar <corey.zumar@databricks.com>

Co-authored-by: dbczumar <corey.zumar@databricks.com>

* Allow sql filter IN operator right-hand side values including whitespaces (#6413)

* support in rhs whitespace

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* update

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* Fix IN operator related issues in sql filter parser (#6411)

* update

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* update

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* clean

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* update

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* update

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* update

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* address comment

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* Remove unused method `SearchUtils._get_comparison_for_model_registry` (#6412)

* init

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* update

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* Update MLflow UI (#6422)

* Project import generated by Copybara.

GitOrigin-RevId: 086b1200f4573559e37fcea6cb5472d52f166e42
Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Reconcile changes from universe with OSS

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Fix test fails

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Fix

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Apply model search fix

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Remove errant test line

Signed-off-by: dbczumar <corey.zumar@databricks.com>

Co-authored-by: Richard Zang <Zangr@users.noreply.github.com>

* remove near from deprecation msg (#6430)

Signed-off-by: ninabacc-db <Nina.Baccam@databricks.com>

* Raise clear error message when pipeline path contains a space (#6426)

* Add unsupported

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Fix

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Exc

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Test

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Format

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Test fix

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Fix

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* [ALL TESTS] Changelog for MLflow 1.28.0 (#6433)

* Generating changelog for MLflow 1.27.1.dev0

Signed-off-by: Jenkins <jenkins@databricks.com>

* Changelog

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Add missing PR

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Some fixes

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* More small fixes

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Fix

Signed-off-by: dbczumar <corey.zumar@databricks.com>

Signed-off-by: Jenkins <jenkins@databricks.com>
Signed-off-by: dbczumar <corey.zumar@databricks.com>
Co-authored-by: Jenkins <jenkins@databricks.com>
Co-authored-by: dbczumar <corey.zumar@databricks.com>

* Patch (#6440)

Signed-off-by: dbczumar <corey.zumar@databricks.com>

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* [ALL TESTS] Update  (#6443)

* python dev/update_ml_package_versions.py

Signed-off-by: Jenkins <jenkins@databricks.com>

* pyspark max ver 3.4

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

Signed-off-by: Jenkins <jenkins@databricks.com>
Signed-off-by: Weichen Xu <weichen.xu@databricks.com>
Co-authored-by: Jenkins <jenkins@databricks.com>
Co-authored-by: Weichen Xu <weichen.xu@databricks.com>

* Update MLflow version to 1.28.1 (#6449)

Signed-off-by: Jenkins <jenkins@databricks.com>

Signed-off-by: Jenkins <jenkins@databricks.com>
Co-authored-by: Jenkins <jenkins@databricks.com>

* Add data capture config to sagemaker (#6423)

Signed-off-by: jonwiggins <me@jonwiggins.org>

Signed-off-by: jonwiggins <me@jonwiggins.org>

* Fix small search_runs docs issues (#6452)

* Fix

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Doc fix

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Fix

Signed-off-by: dbczumar <corey.zumar@databricks.com>

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* python dev/update_pypi_package_index.py (#6442)

Signed-off-by: Jenkins <jenkins@databricks.com>

Signed-off-by: Jenkins <jenkins@databricks.com>
Co-authored-by: Jenkins <jenkins@databricks.com>

* use xgboost on pypi (#6461)

Signed-off-by: harupy <hkawamura0130@gmail.com>

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Add environment variable for `echo` option in `create_engine` (#6460)

* Add environment variable for echo

Signed-off-by: harupy <hkawamura0130@gmail.com>

* use _BooleanEnvironmentVariable

Signed-off-by: harupy <hkawamura0130@gmail.com>

* address comments

Signed-off-by: harupy <hkawamura0130@gmail.com>

* address comments and add tests

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix comment

Signed-off-by: harupy <hkawamura0130@gmail.com>

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Add more file extensions to show content on frontend side (#3292)

Signed-off-by: Мартынов Максим Сергеевич <msmarty5@mts.ru>

Signed-off-by: Мартынов Максим Сергеевич <msmarty5@mts.ru>
Signed-off-by: harupy <hkawamura0130@gmail.com>
Co-authored-by: harupy <hkawamura0130@gmail.com>

* Avoid using assertions at top level in `mlflow/models/docker_utils.py` (#6463)

* do not construct maven proxy options at top level

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix validation for port

Signed-off-by: harupy <hkawamura0130@gmail.com>

* unpack

Signed-off-by: harupy <hkawamura0130@gmail.com>

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Fix mlflowdbfs availability check (#6474)

* Fix

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Fix

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Reverts

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Better fix

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Format

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Format

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Lint

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Clean up

Signed-off-by: dbczumar <corey.zumar@databricks.com>

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Replace `assertRaisesRegex` with `pytest.raises` (#6464)

* replace assertRaisesRegex with pytest.raises

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix message

Signed-off-by: harupy <hkawamura0130@gmail.com>

Signed-off-by: harupy <hkawamura0130@gmail.com>

* CLI command suggestion after running the pipeline (#6376)

* CLI command suggestion after running the pipeline

Signed-off-by: Hubert Zub <hubert.zub@databricks.com>

* fix: added git commands to MLP CLI suggestion

* chore: lint fixes

Signed-off-by: Hubert Zub <hubert.zub@databricks.com>

* chore: test fixes

Signed-off-by: Hubert Zub <hubert.zub@databricks.com>

Signed-off-by: Hubert Zub <hubert.zub@databricks.com>

* Remove redundant int conversion in create_sqlalchemy_engine (#6481)

Signed-off-by: harupy <hkawamura0130@gmail.com>

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Improve code-to-reproduce section in the bug report template (#6478)

* Improve code-to-reproduce section

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix

Signed-off-by: harupy <hkawamura0130@gmail.com>

* problem -> issue

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix

Signed-off-by: harupy <hkawamura0130@gmail.com>

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Add Pipelines step option to skip step profiling (#6456)

* changes

Signed-off-by: apurva-koti <apurva.koti@databricks.com>

* changes

Signed-off-by: apurva-koti <apurva.koti@databricks.com>

* partial-test

Signed-off-by: apurva-koti <apurva.koti@databricks.com>

* lint

Signed-off-by: apurva-koti <apurva.koti@databricks.com>

* missed

Signed-off-by: apurva-koti <apurva.koti@databricks.com>

* review

Signed-off-by: apurva-koti <apurva.koti@databricks.com>

Signed-off-by: apurva-koti <apurva.koti@databricks.com>

* Implement _download_file in hdfs_artifact_repo (#6482)

* Implement _download_file in hdfs_artifact_repo

Signed-off-by: bowen_who <hubw8@chinaunicom.cn>

* add ut on _download_file in test_hdfs_artifact_repo

Signed-off-by: bowen_who <hubw8@chinaunicom.cn>

* rewrite _download_artifact, change back _download_file in hdfs_artifact_repo

Signed-off-by: bowen_who <hubw8@chinaunicom.cn>
Signed-off-by: harupy <hkawamura0130@gmail.com>

* rewrite test__download_file in test_hdfs_artifact_repo

Signed-off-by: bowen_who <hubw8@chinaunicom.cn>

Signed-off-by: bowen_who <hubw8@chinaunicom.cn>
Signed-off-by: harupy <hkawamura0130@gmail.com>
Co-authored-by: bowen_who <hubw8@chinaunicom.cn>

* [ML-19871] R documentation fix (#6484)

* R bug fix

* Documentation fix

* Revert breaking change

* Updated doc file

* Doc fix

* Testing a broken test

* fix

* Revert test

* feat: return param value for mlflow.log_param (#6483)

* feat: return param value for mlflow.log_param

Signed-off-by: Xinyue Ruan <serena.rxy@gmail.com>

* update docstring and return value for MlflowClient

Signed-off-by: Xinyue Ruan <serena.rxy@gmail.com>

Signed-off-by: Xinyue Ruan <serena.rxy@gmail.com>

* revert (#6490)

Signed-off-by: harupy <hkawamura0130@gmail.com>

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Ignore empty environment variables in _get_maven_proxy (#6492)

Signed-off-by: harupy <hkawamura0130@gmail.com>

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Migrate environment variables for S3 (#6438)

* Migrated s3

Signed-off-by: changyonglik <theeahlag@gmail.com>

* Migrated s3 for local.py

Signed-off-by: changyonglik <theeahlag@gmail.com>

* Changed default values

Signed-off-by: changyonglik <theeahlag@gmail.com>

* Added var imports

Signed-off-by: changyonglik <theeahlag@gmail.com>

* Added MLFLOW_S3_ENDPOINT_URL and MLFLOW_S3_IGNORE_TLS

Signed-off-by: changyonglik <theeahlag@gmail.com>

* Removed duplicated imports

Signed-off-by: changyonglik <theeahlag@gmail.com>

* Paraphrased specify to specifies

Signed-off-by: changyonglik <theeahlag@gmail.com>

* Changed _EnvironmentVariable to _BooleanEnvironmentVariable for MLFLOW_S3_IGNORE_TLS and defaults to False

Signed-off-by: changyonglik <theeahlag@gmail.com>

* Changed environment variables in hdfs_artifact_repo.py

Signed-off-by: changyonglik <theeahlag@gmail.com>

* Changed single to double quotes

Signed-off-by: changyonglik <theeahlag@gmail.com>

* Fixed comments

Signed-off-by: changyonglik <theeahlag@gmail.com>

* Small fixes

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Format

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Reset submodule

Signed-off-by: dbczumar <corey.zumar@databricks.com>

Signed-off-by: changyonglik <theeahlag@gmail.com>
Signed-off-by: dbczumar <corey.zumar@databricks.com>
Co-authored-by: dbczumar <corey.zumar@databricks.com>

* Always run CI check on PR (#6505)

Signed-off-by: harupy <hkawamura0130@gmail.com>

Signed-off-by: harupy <hkawamura0130@gmail.com>

Signed-off-by: dbczumar <corey.zumar@databricks.com>
Signed-off-by: harupy <hkawamura0130@gmail.com>
Signed-off-by: Michal Karzynski <michal.karzynski@intel.com>
Signed-off-by: Sunish Sheth <sunishsheth2009@gmail.com>
Signed-off-by: Jin Zhang <jin.zhang@databricks.com>
Signed-off-by: Ryan Fogle <ryanfogle98@gmail.com>
Signed-off-by: Brian Barnes <brian.barnes@databricks.com>
Signed-off-by: apurva-koti <apurva.koti@databricks.com>
Signed-off-by: Weichen Xu <weichen.xu@databricks.com>
Signed-off-by: ninabacc-db <Nina.Baccam@databricks.com>
Signed-off-by: Jenkins <jenkins@databricks.com>
Signed-off-by: jonwiggins <me@jonwiggins.org>
Signed-off-by: Мартынов Максим Сергеевич <msmarty5@mts.ru>
Signed-off-by: Hubert Zub <hubert.zub@databricks.com>
Signed-off-by: bowen_who <hubw8@chinaunicom.cn>
Signed-off-by: Xinyue Ruan <serena.rxy@gmail.com>
Signed-off-by: changyonglik <theeahlag@gmail.com>
Co-authored-by: Corey Zumar <39497902+dbczumar@users.noreply.github.com>
Co-authored-by: Michał Karzyński <michal.karzynski@intel.com>
Co-authored-by: Chang Yong Lik <51813538+ahlag@users.noreply.github.com>
Co-authored-by: Sunish Sheth <sunishsheth2009@gmail.com>
Co-authored-by: jinzhang21 <78067366+jinzhang21@users.noreply.github.com>
Co-authored-by: Ryan Fogle <40377123+RynoXLI@users.noreply.github.com>
Co-authored-by: bbarnes52 <brian.barnes@databricks.com>
Co-authored-by: apurva-koti <51172624+apurva-koti@users.noreply.github.com>
Co-authored-by: Hannes Schulz <hannes.schulz@microsoft.com>
Co-authored-by: dbczumar <corey.zumar@databricks.com>
Co-authored-by: mlflow-automation <mlflow-automation@users.noreply.github.com>
Co-authored-by: WeichenXu <weichen.xu@databricks.com>
Co-authored-by: Siddharth Murching <sid.murching@databricks.com>
Co-authored-by: Cheng Kuan Yong Jason <jasoncky96@gmail.com>
Co-authored-by: johnyNJ <77425417+johnyNJ@users.noreply.github.com>
Co-authored-by: Jan Sindlar <jan.sindlar@eureka.ai>
Co-authored-by: Mohammad Abbas <mohammad.abbas86@gmail.com>
Co-authored-by: Richard Zang <Zangr@users.noreply.github.com>
Co-authored-by: Nina Baccam <106279294+ninabacc-db@users.noreply.github.com>
Co-authored-by: mlflow-automation <61449322+mlflow-automation@users.noreply.github.com>
Co-authored-by: Jenkins <jenkins@databricks.com>
Co-authored-by: Jon Wiggins <35540058+jonwiggins@users.noreply.github.com>
Co-authored-by: Maxim Martynov <msmarty5@mts.ru>
Co-authored-by: Hubert Zub <104438646+hubertzub-db@users.noreply.github.com>
Co-authored-by: Bowen Hu <Bowen_who@163.com>
Co-authored-by: bowen_who <hubw8@chinaunicom.cn>
Co-authored-by: Kris Concepcion <84737625+kriscon-db@users.noreply.github.com>
Co-authored-by: Serena Ruan <82044803+serena-ruan@users.noreply.github.com>
dbczumar added a commit that referenced this pull request Sep 28, 2022
* Run model registry sqlalchemy tests in database job (#6353)

Signed-off-by: harupy <hkawamura0130@gmail.com>

* update to run profiles on the first 100 dataframe columns (#6297)

* update to run profiles on the first 100 dataframe columns

Signed-off-by: Sunish Sheth <sunishsheth2009@gmail.com>

* Adding replace=True so we can sample rows twice

Signed-off-by: Sunish Sheth <sunishsheth2009@gmail.com>

* Removing replace=true and fixing how the max_rows are calculated

Signed-off-by: Sunish Sheth <sunishsheth2009@gmail.com>

* Fix sklearn autologging input example & signature mutation for models containing transformers (#6230)

* Fix with test case

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Test fix

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Fix and simplification

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* more fixes & better test coveragE

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* test

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* LGBM test

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Format

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Address comments

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Fixes

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Use py37-compatible call args syntax

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* use list (#6363)

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Log runtime pipeline config to MLflow (#6359)

* Log runtime pipeline config to MLflow

Signed-off-by: Jin Zhang <jin.zhang@databricks.com>

* Fixed lint

Signed-off-by: Jin Zhang <jin.zhang@databricks.com>

* Add status badge for stale action (#6372)

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Add stack trace section to bug report template (#6374)

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Skipping the transform step by using Functional transformer (#6362)

Signed-off-by: Sunish Sheth <sunishsheth2009@gmail.com>

* Series schema (#6361)

* Added in name for ColSpec of pandas datatype

infer panda series name for ColSpec

Signed-off-by: Ryan Fogle <ryanfogle98@gmail.com>

* Added test for Series name inference

added one more pytest condition in test_schema:test_schema_inference_on_pandas_series

Signed-off-by: Ryan Fogle <ryanfogle98@gmail.com>

* Update test_schema.py

Added in one more assertion to test schema inference.

Signed-off-by: Ryan Fogle <ryanfogle98@gmail.com>

* ran black

Signed-off-by: Ryan Fogle <ryanfogle98@gmail.com>

* added in requested changes in issue #6361

Signed-off-by: Ryan Fogle <ryanfogle98@gmail.com>

* fix typo. changed hasattr to getattr

Signed-off-by: Ryan Fogle <ryanfogle98@gmail.com>

* integrate mlflowdbfs (#6282)

Signed-off-by: Brian Barnes <brian.barnes@databricks.com>

* Fix step card display breakage in Jupyter notebooks (#6378)

* remove

Signed-off-by: apurva-koti <apurva.koti@databricks.com>

* fix

Signed-off-by: apurva-koti <apurva.koti@databricks.com>

* Avoid using flask 2.2.0 (#6380)

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Migrate GCS environment variables to `mlflow.environment_variables` module (#6375)

* Migrate GCS environment variables to mlflow.environment_variables

Signed-off-by: harupy <hkawamura0130@gmail.com>

* add default

Signed-off-by: harupy <hkawamura0130@gmail.com>

* improve comments

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Avoid logging model signatures in pyspark ML autologging if model input/output dataframe contains unsupported data types (#6365)

* Avoid logging signatures

Signed-off-by: harupy <hkawamura0130@gmail.com>

* test

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix indent

Signed-off-by: harupy <hkawamura0130@gmail.com>

* check model output

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix warning message

Signed-off-by: harupy <hkawamura0130@gmail.com>

* todo

Signed-off-by: harupy <hkawamura0130@gmail.com>

* update test and doc

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix docstring

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix failed tests

Signed-off-by: harupy <hkawamura0130@gmail.com>

* improve _assert_autolog_infers_model_signature_correctly

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Refactor `set_matrix.py` (#6373)

* refacor set_matrix.py

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix tests

Signed-off-by: harupy <hkawamura0130@gmail.com>

* tset for --no-dev

Signed-off-by: harupy <hkawamura0130@gmail.com>

* install pytest and pytest-cov

Signed-off-by: harupy <hkawamura0130@gmail.com>

* test for --changed-files

Signed-off-by: harupy <hkawamura0130@gmail.com>

* add pyyaml to requirements

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix is_matrix_empty

Signed-off-by: harupy <hkawamura0130@gmail.com>

* use raw string and re.match

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Fix is_matrix_empty (#6385)

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Allow mlflow-skinny's autolog() to succeed when no scipy is installed (#5897)

* allow mlflow-skinny's autolog() to succeed when no scipy is installed

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* optional pyfunc scipy dependency

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* fix linter-discovered issues

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* add test case, autolog() should pass w/o scipy installed

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* minor cleanup

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* test creating _Example with / without scipy present

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* docs: do not warn about sparse matrix classes

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* linting

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* address review comment

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* run black

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* allow mlflow-skinny's autolog() to succeed when no scipy is installed

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* optional pyfunc scipy dependency

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* fix linter-discovered issues

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* add test case, autolog() should pass w/o scipy installed

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* minor cleanup

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* test creating _Example with / without scipy present

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* docs: do not warn about sparse matrix classes

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* linting

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* address review comment

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* run black

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* remove redundant None check after import

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* delete comment

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* Revert "remove redundant None check after import"

This reverts commit f2e198c944f41d9a96c20ae471d71bff39431b48.

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* review comments

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* remove redundant test

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* Apply suggestions from code review

Co-authored-by: Harutaka Kawamura <hkawamura0130@gmail.com>
Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* Autoformat: https://github.com/mlflow/mlflow/actions/runs/2779791161

Signed-off-by: mlflow-automation <mlflow-automation@users.noreply.github.com>

Co-authored-by: Harutaka Kawamura <hkawamura0130@gmail.com>
Co-authored-by: dbczumar <corey.zumar@databricks.com>
Co-authored-by: mlflow-automation <mlflow-automation@users.noreply.github.com>

* Fix extraneous warning about pyspark evaluation truncation (#6389)

* Revert #6380 (#6386)

* revert

Signed-off-by: harupy <hkawamura0130@gmail.com>

* workaround for test_prometheus_exporter

Signed-off-by: harupy <hkawamura0130@gmail.com>

* nit

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Validate metric value in `FileStore.log_metric` (#6381)

* add failing test

Signed-off-by: harupy <hkawamura0130@gmail.com>

* use _validate_metric

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix invalid metric timestamp

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix docstring

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix java test

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Make search experiment by tag key comparison case sensitive in MYSQL (#6333)

* init

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* add test

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* update

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* update

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* fix BINARY conditions in MySQL

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix attr error

Signed-off-by: harupy <hkawamura0130@gmail.com>

Co-authored-by: harupy <hkawamura0130@gmail.com>

* Run Model Registry SQLAlchemy tests in PostgreSQL, MySQL, and MSSQL (#6390)

* Run Model Registry SQLAlchemy tests in postgres, mysql, and mssql

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix failed tests

Signed-off-by: harupy <hkawamura0130@gmail.com>

* remove uuid

Signed-off-by: harupy <hkawamura0130@gmail.com>

* clean up

Signed-off-by: harupy <hkawamura0130@gmail.com>

* do not drop tables

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Fix filestore data loss on failed write & mitigate read / write race condition (#6388)

* Fix

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Fix

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Format

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Added a custom filter from_json to render_and_merge_yaml (#6368)

* Added a custom filter from_json to render_and_merge_yaml

Signed-off-by: Jin Zhang <jin.zhang@databricks.com>

* Fixed test

Signed-off-by: Jin Zhang <jin.zhang@databricks.com>

* Update tests/utils/test_file_utils.py

Co-authored-by: Siddharth Murching <sid.murching@databricks.com>

* Update tests/utils/test_file_utils.py

Co-authored-by: Siddharth Murching <sid.murching@databricks.com>

* Fixed test

Signed-off-by: Jin Zhang <jin.zhang@databricks.com>

* Fixed windows test attempt

Signed-off-by: Jin Zhang <jin.zhang@databricks.com>

* Fixed windows test

Signed-off-by: Jin Zhang <jin.zhang@databricks.com>

Co-authored-by: Siddharth Murching <sid.murching@databricks.com>

* Fix `!=` operator for MySQL (#6397)

* fix ne operator and improve tests

Signed-off-by: harupy <hkawamura0130@gmail.com>

* add () to other operators

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Simplify the documentation section in the PR template (#6398)

* Simplify the documentation section in the PR template

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Timebased gc (#6354)

* Added delete_time field to runs table

Signed-off-by: Jason Cheng <jasoncky96@gmail.com>

* Updated latest_schema.sql to pass tests/store/tracking/test_sqlalchemy_store_schema.py

Signed-off-by: Jason Cheng <jasoncky96@gmail.com>

* Added delete_time column into mlflow/store/tracking/dbmodels/initial_models.py

Signed-off-by: Jason Cheng <jasoncky96@gmail.com>

* Moved the docstring to the end of the declaration instead of the start

Signed-off-by: Jason Cheng <jasoncky96@gmail.com>

* Regenerated protobuf for RunInfo new schema, and passed pytest

Signed-off-by: Jason Cheng <jasoncky96@gmail.com>

* Added --older-than flag in mlflow gc command, and added tests for that cli command

Signed-off-by: Jason Cheng <jasoncky96@gmail.com>

* Removed delete_time from RunInfo proto

Signed-off-by: Jason Cheng <jasoncky96@gmail.com>

* Set default older-than flag to None

Signed-off-by: Jason Cheng <jasoncky96@gmail.com>

* Remove delete_time from RunInfo, add optional parameter older_than to store.get_deleted_runs(), and write delete_time into yaml file for file storage instead of adding it to RunInfo

Signed-off-by: Jason Cheng <jasoncky96@gmail.com>

* Remove commented delete_time in RunInfo constructor in models.py

Signed-off-by: Jason Cheng <jasoncky96@gmail.com>

* added INVALID_PARAMETER_VALUE error code in mlflow gc

Signed-off-by: Jason Cheng <jasoncky96@gmail.com>

* Resolve case of delete time not in meta.yaml file, edited overwrite_run_info and gc

Signed-off-by: Jason Cheng <jasoncky96@gmail.com>

* Rename delete_time to deleted_time

Signed-off-by: Jason Cheng <jasoncky96@gmail.com>

* Rename delete_time to deleted_time, match error message in cli tests, filter SqlRuns by deleted_time, indicate float values accepted for gc --older-than flag

Signed-off-by: Jason Cheng <jasoncky96@gmail.com>

* Rename delete_time to deleted_time, fixed assert statement in test_mlflow_gc, and fixed deleted_time queries in test_file_store.py and test_sqlalchemy_store.py

Signed-off-by: Jason Cheng <jasoncky96@gmail.com>

* Only check if  is in meta after restoring run in test_delte_restore_run

Signed-off-by: Jason Cheng <jasoncky96@gmail.com>

* Fixed lint errors

Signed-off-by: Jason Cheng <jasoncky96@gmail.com>

* Install mlflow from repository root

Signed-off-by: harupy <hkawamura0130@gmail.com>

Co-authored-by: harupy <hkawamura0130@gmail.com>

* Separate protos job (#6399)

* Separate protos job

Signed-off-by: harupy <hkawamura0130@gmail.com>

* test

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Add paths

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Revert "test"

This reverts commit 5d04b02f55ec92e1e3ac18eb651ea989ca0d469a.

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Fix flaky `TestFileStore.test_get_deleted_runs` (#6402)

* use gt operator

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix sqlalchemy store

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Migrate environment variables for SQLAlchemy store (#6396)

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Deduplicate MlflowClient in MLflow docs, move to mlflow.client module (#6405)

* Dedupe MlflowClient

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Improvements

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Add client

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Remove unused functions in `mlflow.data` module (#6408)

* remove unused functions in data

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Remove unused functions and variables

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Tag based registered model search and model version search (#6320)

* init

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* update

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* update

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* update

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* update

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* update

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* update

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* update

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* update

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* update

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* update

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* update tests

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* update

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* update

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* clean

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* address comments

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* add test

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* address comments

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* update

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* de-dup tag based search result

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* add () to fix case_sensitive_mysql_ne

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* fix

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* lint

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* fix

* fix test in TODO

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* support dup tag key filter

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* add mv order by

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* cmp res list in test

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* address comments

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* address comments

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* add doc

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* fix

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* fix lint

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* address comments

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* address comments

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* update doc

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* Increase maximum param value length to 500 (#6358)

* increase validation of maximum param value length from 250 to 500 (squashed)

Signed-off-by: Jan Sindlar <jan.sindlar@eureka.ai>

* 1. added test for exceeding max length. 2. lint. 3. validation logic addresses null

Signed-off-by: Jan Sindlar <jan.sindlar@eureka.ai>

* file store should test already stringified params

Signed-off-by: Jan Sindlar <jan.sindlar@eureka.ai>

* update alembic chain

Signed-off-by: Jan Sindlar <jan.sindlar@eureka.ai>

* Autoformat: https://github.com/mlflow/mlflow/actions/runs/2800398158

Signed-off-by: mlflow-automation <mlflow-automation@users.noreply.github.com>

* Update tests/store/tracking/test_file_store.py

Co-authored-by: Harutaka Kawamura <hkawamura0130@gmail.com>
Signed-off-by: Jan Sindlar <jan.sindlar@eureka.ai>

Co-authored-by: Jan Sindlar <jan.sindlar@eureka.ai>
Co-authored-by: mlflow-automation <mlflow-automation@users.noreply.github.com>
Co-authored-by: Harutaka Kawamura <hkawamura0130@gmail.com>

* Add workflow dispatch for requirements validation, remove duplicate file (#6418)

* Workflow dispatch

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Remove unused

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Prevent `pytest.raises` context manager from containing multiple statements (#6409)

* rename

Signed-off-by: harupy <hkawamura0130@gmail.com>

* rename

Signed-off-by: harupy <hkawamura0130@gmail.com>

* rename

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Do not allow calling assert in pytest.raises

Signed-off-by: harupy <hkawamura0130@gmail.com>

* more strict rule

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Improve error message

Signed-off-by: harupy <hkawamura0130@gmail.com>

* relax condition

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix lint errors

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix failed tests

Signed-off-by: harupy <hkawamura0130@gmail.com>

* clean up

Signed-off-by: harupy <hkawamura0130@gmail.com>

* rename _is_complex_pytest_raises

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix lint errors

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix failed tests

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Fix regression in pandas type inference behavior for scoring server (#6417)

* Fix impl

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Remove unused pandas import

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Adjust client to use SearchExperiments POST method (#6415)

* POST

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Generate protos

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Fix test

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* update (#6420)

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* Check if artifacts are proxied in model_version_artifact_handler (#6355)

* Check if artifacts are proxied in model_version_artifact_handler

Signed-off-by: Mohammad Abbas <mohammad.abbas@digital.trade.gov.uk>

* Use file prefix for artifact dest on windows

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Fix application of prefix

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Fixes

Signed-off-by: dbczumar <corey.zumar@databricks.com>

Co-authored-by: dbczumar <corey.zumar@databricks.com>

* Allow sql filter IN operator right-hand side values including whitespaces (#6413)

* support in rhs whitespace

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* update

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* Fix IN operator related issues in sql filter parser (#6411)

* update

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* update

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* clean

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* update

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* update

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* update

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* address comment

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* Remove unused method `SearchUtils._get_comparison_for_model_registry` (#6412)

* init

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* update

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* Update MLflow UI (#6422)

* Project import generated by Copybara.

GitOrigin-RevId: 086b1200f4573559e37fcea6cb5472d52f166e42
Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Reconcile changes from universe with OSS

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Fix test fails

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Fix

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Apply model search fix

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Remove errant test line

Signed-off-by: dbczumar <corey.zumar@databricks.com>

Co-authored-by: Richard Zang <Zangr@users.noreply.github.com>

* remove near from deprecation msg (#6430)

Signed-off-by: ninabacc-db <Nina.Baccam@databricks.com>

* Raise clear error message when pipeline path contains a space (#6426)

* Add unsupported

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Fix

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Exc

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Test

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Format

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Test fix

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Fix

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* [ALL TESTS] Changelog for MLflow 1.28.0 (#6433)

* Generating changelog for MLflow 1.27.1.dev0

Signed-off-by: Jenkins <jenkins@databricks.com>

* Changelog

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Add missing PR

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Some fixes

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* More small fixes

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Fix

Signed-off-by: dbczumar <corey.zumar@databricks.com>

Signed-off-by: Jenkins <jenkins@databricks.com>
Signed-off-by: dbczumar <corey.zumar@databricks.com>
Co-authored-by: Jenkins <jenkins@databricks.com>
Co-authored-by: dbczumar <corey.zumar@databricks.com>

* Patch (#6440)

Signed-off-by: dbczumar <corey.zumar@databricks.com>

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* [ALL TESTS] Update  (#6443)

* python dev/update_ml_package_versions.py

Signed-off-by: Jenkins <jenkins@databricks.com>

* pyspark max ver 3.4

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

Signed-off-by: Jenkins <jenkins@databricks.com>
Signed-off-by: Weichen Xu <weichen.xu@databricks.com>
Co-authored-by: Jenkins <jenkins@databricks.com>
Co-authored-by: Weichen Xu <weichen.xu@databricks.com>

* Update MLflow version to 1.28.1 (#6449)

Signed-off-by: Jenkins <jenkins@databricks.com>

Signed-off-by: Jenkins <jenkins@databricks.com>
Co-authored-by: Jenkins <jenkins@databricks.com>

* Add data capture config to sagemaker (#6423)

Signed-off-by: jonwiggins <me@jonwiggins.org>

Signed-off-by: jonwiggins <me@jonwiggins.org>

* Fix small search_runs docs issues (#6452)

* Fix

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Doc fix

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Fix

Signed-off-by: dbczumar <corey.zumar@databricks.com>

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* python dev/update_pypi_package_index.py (#6442)

Signed-off-by: Jenkins <jenkins@databricks.com>

Signed-off-by: Jenkins <jenkins@databricks.com>
Co-authored-by: Jenkins <jenkins@databricks.com>

* use xgboost on pypi (#6461)

Signed-off-by: harupy <hkawamura0130@gmail.com>

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Add environment variable for `echo` option in `create_engine` (#6460)

* Add environment variable for echo

Signed-off-by: harupy <hkawamura0130@gmail.com>

* use _BooleanEnvironmentVariable

Signed-off-by: harupy <hkawamura0130@gmail.com>

* address comments

Signed-off-by: harupy <hkawamura0130@gmail.com>

* address comments and add tests

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix comment

Signed-off-by: harupy <hkawamura0130@gmail.com>

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Add more file extensions to show content on frontend side (#3292)

Signed-off-by: Мартынов Максим Сергеевич <msmarty5@mts.ru>

Signed-off-by: Мартынов Максим Сергеевич <msmarty5@mts.ru>
Signed-off-by: harupy <hkawamura0130@gmail.com>
Co-authored-by: harupy <hkawamura0130@gmail.com>

* Avoid using assertions at top level in `mlflow/models/docker_utils.py` (#6463)

* do not construct maven proxy options at top level

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix validation for port

Signed-off-by: harupy <hkawamura0130@gmail.com>

* unpack

Signed-off-by: harupy <hkawamura0130@gmail.com>

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Fix mlflowdbfs availability check (#6474)

* Fix

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Fix

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Reverts

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Better fix

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Format

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Format

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Lint

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Clean up

Signed-off-by: dbczumar <corey.zumar@databricks.com>

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Replace `assertRaisesRegex` with `pytest.raises` (#6464)

* replace assertRaisesRegex with pytest.raises

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix message

Signed-off-by: harupy <hkawamura0130@gmail.com>

Signed-off-by: harupy <hkawamura0130@gmail.com>

* CLI command suggestion after running the pipeline (#6376)

* CLI command suggestion after running the pipeline

Signed-off-by: Hubert Zub <hubert.zub@databricks.com>

* fix: added git commands to MLP CLI suggestion

* chore: lint fixes

Signed-off-by: Hubert Zub <hubert.zub@databricks.com>

* chore: test fixes

Signed-off-by: Hubert Zub <hubert.zub@databricks.com>

Signed-off-by: Hubert Zub <hubert.zub@databricks.com>

* Remove redundant int conversion in create_sqlalchemy_engine (#6481)

Signed-off-by: harupy <hkawamura0130@gmail.com>

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Improve code-to-reproduce section in the bug report template (#6478)

* Improve code-to-reproduce section

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix

Signed-off-by: harupy <hkawamura0130@gmail.com>

* problem -> issue

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix

Signed-off-by: harupy <hkawamura0130@gmail.com>

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Add Pipelines step option to skip step profiling (#6456)

* changes

Signed-off-by: apurva-koti <apurva.koti@databricks.com>

* changes

Signed-off-by: apurva-koti <apurva.koti@databricks.com>

* partial-test

Signed-off-by: apurva-koti <apurva.koti@databricks.com>

* lint

Signed-off-by: apurva-koti <apurva.koti@databricks.com>

* missed

Signed-off-by: apurva-koti <apurva.koti@databricks.com>

* review

Signed-off-by: apurva-koti <apurva.koti@databricks.com>

Signed-off-by: apurva-koti <apurva.koti@databricks.com>

* Implement _download_file in hdfs_artifact_repo (#6482)

* Implement _download_file in hdfs_artifact_repo

Signed-off-by: bowen_who <hubw8@chinaunicom.cn>

* add ut on _download_file in test_hdfs_artifact_repo

Signed-off-by: bowen_who <hubw8@chinaunicom.cn>

* rewrite _download_artifact, change back _download_file in hdfs_artifact_repo

Signed-off-by: bowen_who <hubw8@chinaunicom.cn>
Signed-off-by: harupy <hkawamura0130@gmail.com>

* rewrite test__download_file in test_hdfs_artifact_repo

Signed-off-by: bowen_who <hubw8@chinaunicom.cn>

Signed-off-by: bowen_who <hubw8@chinaunicom.cn>
Signed-off-by: harupy <hkawamura0130@gmail.com>
Co-authored-by: bowen_who <hubw8@chinaunicom.cn>

* [ML-19871] R documentation fix (#6484)

* R bug fix

* Documentation fix

* Revert breaking change

* Updated doc file

* Doc fix

* Testing a broken test

* fix

* Revert test

* feat: return param value for mlflow.log_param (#6483)

* feat: return param value for mlflow.log_param

Signed-off-by: Xinyue Ruan <serena.rxy@gmail.com>

* update docstring and return value for MlflowClient

Signed-off-by: Xinyue Ruan <serena.rxy@gmail.com>

Signed-off-by: Xinyue Ruan <serena.rxy@gmail.com>

* revert (#6490)

Signed-off-by: harupy <hkawamura0130@gmail.com>

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Ignore empty environment variables in _get_maven_proxy (#6492)

Signed-off-by: harupy <hkawamura0130@gmail.com>

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Migrate environment variables for S3 (#6438)

* Migrated s3

Signed-off-by: changyonglik <theeahlag@gmail.com>

* Migrated s3 for local.py

Signed-off-by: changyonglik <theeahlag@gmail.com>

* Changed default values

Signed-off-by: changyonglik <theeahlag@gmail.com>

* Added var imports

Signed-off-by: changyonglik <theeahlag@gmail.com>

* Added MLFLOW_S3_ENDPOINT_URL and MLFLOW_S3_IGNORE_TLS

Signed-off-by: changyonglik <theeahlag@gmail.com>

* Removed duplicated imports

Signed-off-by: changyonglik <theeahlag@gmail.com>

* Paraphrased specify to specifies

Signed-off-by: changyonglik <theeahlag@gmail.com>

* Changed _EnvironmentVariable to _BooleanEnvironmentVariable for MLFLOW_S3_IGNORE_TLS and defaults to False

Signed-off-by: changyonglik <theeahlag@gmail.com>

* Changed environment variables in hdfs_artifact_repo.py

Signed-off-by: changyonglik <theeahlag@gmail.com>

* Changed single to double quotes

Signed-off-by: changyonglik <theeahlag@gmail.com>

* Fixed comments

Signed-off-by: changyonglik <theeahlag@gmail.com>

* Small fixes

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Format

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Reset submodule

Signed-off-by: dbczumar <corey.zumar@databricks.com>

Signed-off-by: changyonglik <theeahlag@gmail.com>
Signed-off-by: dbczumar <corey.zumar@databricks.com>
Co-authored-by: dbczumar <corey.zumar@databricks.com>

* Always run CI check on PR (#6505)

Signed-off-by: harupy <hkawamura0130@gmail.com>

Signed-off-by: harupy <hkawamura0130@gmail.com>

* document how to install mlflow-pipelines from conda (#6509)

Signed-off-by: harupy <hkawamura0130@gmail.com>

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Move tests for pylint_plugins (#6506)

Signed-off-by: harupy <hkawamura0130@gmail.com>

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Retry pulling docker image on CircleCI (#6514)

* Retry pulling docker image on CircleCI

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix indent

Signed-off-by: harupy <hkawamura0130@gmail.com>

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Remove unnecessary comprehension (#6516)

Signed-off-by: harupy <hkawamura0130@gmail.com>

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Don't use mlflowdbfs if dbutils is unavailable (#6508)

* Dbutils

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Fix

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Fix

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Fix

Signed-off-by: dbczumar <corey.zumar@databricks.com>

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* bug-fix to allow non-string column names (#6503) (#6521)

* Allow EvaluationDataset to accept dataframes that contain non-string column names (mlflow#6503)

Signed-off-by: Daniel Gibbons <daniel.gibbons04@gmail.com>

* Minor fix to remove redundant assignments from iris_pandas_df datasets (mlflow#6503)

Signed-off-by: Daniel Gibbons <daniel.gibbons04@gmail.com>

Signed-off-by: Daniel Gibbons <daniel.gibbons04@gmail.com>

* Add `use-set-literal` rule for better performace (#6533)

* Add use-set-literal rule

Signed-off-by: harupy <hkawamura0130@gmail.com>

* improve test

Signed-off-by: harupy <hkawamura0130@gmail.com>

Signed-off-by: harupy <hkawamura0130@gmail.com>

* feat: Docker development scripts and readme (#6465)

* feat: docker development scripts and readme

Signed-off-by: Hubert Zub <hubert.zub@databricks.com>

* chore: CR fixes

Signed-off-by: Hubert Zub <hubert.zub@databricks.com>

* chore: improved dependency install in Dockerfile.test

Signed-off-by: Hubert Zub <hubert.zub@databricks.com>

* Update CONTRIBUTING.rst

Co-authored-by: Harutaka Kawamura <hkawamura0130@gmail.com>
Signed-off-by: Hubert Zub <hubert.zub@databricks.com>

* Update dev/Dockerfile.test

Co-authored-by: Harutaka Kawamura <hkawamura0130@gmail.com>
Signed-off-by: Hubert Zub <hubert.zub@databricks.com>

* Update CONTRIBUTING.rst

Signed-off-by: Hubert Zub <hubert.zub@databricks.com>

* Update dev/run-test-container.sh

Co-authored-by: Harutaka Kawamura <hkawamura0130@gmail.com>
Signed-off-by: Hubert Zub <hubert.zub@databricks.com>

* Update dev/Dockerfile.test.dockerignore

Co-authored-by: Harutaka Kawamura <hkawamura0130@gmail.com>
Signed-off-by: Hubert Zub <hubert.zub@databricks.com>

* chore: CR fixes

Signed-off-by: Hubert Zub <hubert.zub@databricks.com>

* chore: shell script improvement

Signed-off-by: Hubert Zub <hubert.zub@databricks.com>

* chore: fixed dockerignore

Signed-off-by: Hubert Zub <hubert.zub@databricks.com>

* Update dev/Dockerfile.test.dockerignore

Co-authored-by: Harutaka Kawamura <hkawamura0130@gmail.com>
Signed-off-by: Hubert Zub <hubert.zub@databricks.com>

* Update dev/Dockerfile.test

Co-authored-by: Harutaka Kawamura <hkawamura0130@gmail.com>
Signed-off-by: Hubert Zub <hubert.zub@databricks.com>

* Update dev/Dockerfile.test.dockerignore

Co-authored-by: Harutaka Kawamura <hkawamura0130@gmail.com>
Signed-off-by: Hubert Zub <hubert.zub@databricks.com>

* Update CONTRIBUTING.rst

Co-authored-by: Harutaka Kawamura <hkawamura0130@gmail.com>
Signed-off-by: Hubert Zub <hubert.zub@databricks.com>

Signed-off-by: Hubert Zub <hubert.zub@databricks.com>
Co-authored-by: Harutaka Kawamura <hkawamura0130@gmail.com>

* Adding cd to run reproduce steps (#6495)

* adding cd to repo

Signed-off-by: Prithvi Kannan <prithvi.kannan@databricks.com>

* drop git extension

Signed-off-by: Prithvi Kannan <prithvi.kannan@databricks.com>

* ran lint

Signed-off-by: Prithvi Kannan <prithvi.kannan@databricks.com>

Signed-off-by: Prithvi Kannan <prithvi.kannan@databricks.com>

* fix use-set-literal errors batch 1 (#mlflow-6535) (#6542)

* fix use-set-literal errors batch 1 (#mlflow-6535)

Signed-off-by: Daniel Gibbons <daniel.gibbons04@gmail.com>

* update set_checker batch 1 (#mlflow-6535)

Signed-off-by: Daniel Gibbons <daniel.gibbons04@gmail.com>

Signed-off-by: Daniel Gibbons <daniel.gibbons04@gmail.com>

* fix use-set-literal errors batch 2 (#mlflow-6535) (#6543)

* fix use-set-literal errors batch 2 (#mlflow-6535)

Signed-off-by: Daniel Gibbons <daniel.gibbons04@gmail.com>

* update set_checker batch 2 (#mlflow-6535)

Signed-off-by: Daniel Gibbons <daniel.gibbons04@gmail.com>

Signed-off-by: Daniel Gibbons <daniel.gibbons04@gmail.com>

* Fix `use-set-literal` errors (#6534)

* Fix use-set-literal errors

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix error message

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix error message

Signed-off-by: harupy <hkawamura0130@gmail.com>

* improve instructions

Signed-off-by: harupy <hkawamura0130@gmail.com>

* tuple

Signed-off-by: harupy <hkawamura0130@gmail.com>

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Remove run_uuid is None check (#6547)

Signed-off-by: dbczumar <corey.zumar@databricks.com>

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Deprecate ListExperiments, ListRegisteredModels, ListRunInfos APIs (#6550)

* Deprecate listing

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Deprecate

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Revert

Signed-off-by: dbczumar <corey.zumar@databricks.com>

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* fix missing_fields (#6549)

Signed-off-by: harupy <hkawamura0130@gmail.com>

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Improve deprecated decorator (#6552)

Signed-off-by: harupy <hkawamura0130@gmail.com>

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Fix violating consider-using-set-comprehension pylint rule (#6540)

* Fix violating consider-using-set-comprehension pylint rule

Signed-off-by: Evgenii Uvarov <evgenii.uvarov.post@gmail.com>

* Fix linting issues. Fix PR review issues

Signed-off-by: Evgenii Uvarov <evgenii.uvarov.post@gmail.com>

Signed-off-by: Evgenii Uvarov <evgenii.uvarov.post@gmail.com>
Co-authored-by: Evgenii Uvarov <e.uvarov@life-pay.ru>

* Install conda in R development image (#6557)

Signed-off-by: harupy <hkawamura0130@gmail.com>

Signed-off-by: harupy <hkawamura0130@gmail.com>

* MLP batch scoring DAG added to Regression Pipeline (#6455)

* Added the skeletons for the 2 new steps (#6248)

* Added the skeletons for the 2 new steps

* CR comments

* Linting errors

* Added the skeletons for the 2 new steps (#6248)

* Added the skeletons for the 2 new steps

* CR comments

* Linting errors

* Added the skeletons for the 2 new steps (#6248)

* Added the skeletons for the 2 new steps

* CR comments

* Linting errors

* Refactored MLP infra to support additional template makefiles (#6259)

* Added changes

* fixed tests

* Formatting and fixed tests

* Added preprocessing step test (#6267)

* Added preprocessing test

* Fix linting errors

* Added the skeletons for the 2 new steps (#6248)

* Added the skeletons for the 2 new steps

* CR comments

* Linting errors

* Refactored MLP infra to support additional template makefiles (#6259)

* Added changes

* fixed tests

* Formatting and fixed tests

* Added preprocessing step test (#6267)

* Added preprocessing test

* Fix linting errors

* Fixed submodules

* Build out predict step (#6296)

Signed-off-by: Jerry Liang <jerry.liang@databricks.com>

* Batch scoring framework (#6283)

* Batch scoring pipeline added

* Added gitmodule

* updating the example?

* Submodule fix and test fix

* Submodule fix

* black formatting

* Added batch_scoring submodule correctly?

* Lint errors

* Fixed the integration test (removed predict for now)

* Black formatting

* Updated api doc

* Documentation bloopers

* Removed smart quotes

* more documentation fixes

* More fixes

* Try to change the replacement name

* Bad substitution

* Add registered model test (#6327)

Signed-off-by: Jerry Liang <jerry.liang@databricks.com>

* Predict Step: output format and location, env restoration (#6336)

* add output location writing logic

Signed-off-by: Jerry Liang <jerry.liang@databricks.com>

* change output param

Signed-off-by: Jerry Liang <jerry.liang@databricks.com>

* wip

Signed-off-by: Jerry Liang <jerry.liang@databricks.com>

* wip

Signed-off-by: Jerry Liang <jerry.liang@databricks.com>

* wip

Signed-off-by: Jerry Liang <jerry.liang@databricks.com>

* wip

Signed-off-by: Jerry Liang <jerry.liang@databricks.com>

* Batching scoring pipeline bug fixes, predict step card html (#6364)

* wip

Signed-off-by: Jerry Liang <jerry.liang@databricks.com>

* wip

Signed-off-by: Jerry Liang <jerry.liang@databricks.com>

* wip

Signed-off-by: Jerry Liang <jerry.liang@databricks.com>

* Added the skeletons for the 2 new steps (#6248)

* Added the skeletons for the 2 new steps

* CR comments

* Linting errors

* Refactored MLP infra to support additional template makefiles (#6259)

* Added changes

* fixed tests

* Formatting and fixed tests

* Added preprocessing step test (#6267)

* Added preprocessing test

* Fix linting errors

* Fixed submodules

* Added the skeletons for the 2 new steps (#6248)

* Added the skeletons for the 2 new steps

* CR comments

* Linting errors

* Added the skeletons for the 2 new steps (#6248)

* Added the skeletons for the 2 new steps

* CR comments

* Linting errors

* Refactored MLP infra to support additional template makefiles (#6259)

* Added changes

* fixed tests

* Formatting and fixed tests

* Added preprocessing step test (#6267)

* Added preprocessing test

* Fix linting errors

* Build out predict step (#6296)

Signed-off-by: Jerry Liang <jerry.liang@databricks.com>

* Batch scoring framework (#6283)

* Batch scoring pipeline added

* Added gitmodule

* updating the example?

* Submodule fix and test fix

* Submodule fix

* black formatting

* Added batch_scoring submodule correctly?

* Lint errors

* Fixed the integration test (removed predict for now)

* Black formatting

* Updated api doc

* Documentation bloopers

* Removed smart quotes

* more documentation fixes

* More fixes

* Try to change the replacement name

* Bad substitution

* Add registered model test (#6327)

Signed-off-by: Jerry Liang <jerry.liang@databricks.com>

* Predict Step: output format and location, env restoration (#6336)

* add output location writing logic

Signed-off-by: Jerry Liang <jerry.liang@databricks.com>

* change output param

Signed-off-by: Jerry Liang <jerry.liang@databricks.com>

* wip

Signed-off-by: Jerry Liang <jerry.liang@databricks.com>

* wip

Signed-off-by: Jerry Liang <jerry.liang@databricks.com>

* wip

Signed-off-by: Jerry Liang <jerry.liang@databricks.com>

* wip

Signed-off-by: Jerry Liang <jerry.liang@databricks.com>

* Batching scoring pipeline bug fixes, predict step card html (#6364)

* wip

Signed-off-by: Jerry Liang <jerry.liang@databricks.com>

* wip

Signed-off-by: Jerry Liang <jerry.liang@databricks.com>

* wip

Signed-off-by: Jerry Liang <jerry.liang@databricks.com>

* Convert to regression (#6371)

* Convert feature branch to move batch scoring into regression pipeline

* Black formatting

* Add the new ingest_scoring step

* Uupdated artifact name

* Updated sklearn_regression example

* Test fixes

* Fixed broken tests

* Linting errors

* Augment predict step to use model name for register step (#6437)

* wip

Signed-off-by: Jerry Liang <jerry.liang@databricks.com>

* wip

Signed-off-by: Jerry Liang <jerry.liang@databricks.com>

* wip

Signed-off-by: Jerry Liang <jerry.liang@databricks.com>

* wip

Signed-off-by: Jerry Liang <jerry.liang@databricks.com>

* wip

Signed-off-by: Jerry Liang <jerry.liang@databricks.com>

* wip

Signed-off-by: Jerry Liang <jerry.liang@databricks.com>

* wip

Signed-off-by: Jerry Liang <jerry.liang@databricks.com>

* wip

Signed-off-by: Jerry Liang <jerry.liang@databricks.com>

* wip

Signed-off-by: Jerry Liang <jerry.liang@databricks.com>

Signed-off-by: Jerry Liang <jerry.liang@databricks.com>

* Remove preprocessing step

* Fix ingest step rebase error

* CR comments addressed

* Fixed broken tests

* Undo unknown split change

* wip

Signed-off-by: Jerry Liang <jerry.liang@databricks.com>

* wip

Signed-off-by: Jerry Liang <jerry.liang@databricks.com>

* wip

Signed-off-by: Jerry Liang <jerry.liang@databricks.com>

* wip

Signed-off-by: Jerry Liang <jerry.liang@databricks.com>

Signed-off-by: Jerry Liang <jerry.liang@databricks.com>
Co-authored-by: Jerry Liang <66143562+jerrylian-db@users.noreply.github.com>
Co-authored-by: Jerry Liang <jerry.liang@databricks.com>

* fix use-set-literal errors (#6544)

* That was fun

Signed-off-by: Uncle Hohotun <95711404+Monkero@users.noreply.github.com>

* set_checker.py is updated

Signed-off-by: Uncle Hohotun <95711404+Monkero@users.noreply.github.com>

* Conflict in pylint_plagins/set_checker.py

Signed-off-by: Uncle Hohotun <95711404+Monkero@users.noreply.github.com>

* run black

Signed-off-by: harupy <hkawamura0130@gmail.com>

Signed-off-by: Uncle Hohotun <95711404+Monkero@users.noreply.github.com>
Signed-off-by: harupy <hkawamura0130@gmail.com>
Co-authored-by: harupy <hkawamura0130@gmail.com>

* unwanted os calls removed (#6559)

Signed-off-by: yogesh kumar <kumaryogesh.usict.ipu@gmail.com>

Signed-off-by: yogesh kumar <kumaryogesh.usict.ipu@gmail.com>

* Fix CircleCI artifact URL in `build_doc_r` job (#6558)

* Fix CircleCI artifact URL

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix

Signed-off-by: harupy <hkawamura0130@gmail.com>

* revert

Signed-off-by: harupy <hkawamura0130@gmail.com>

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Update requirements.yaml files (#6564)

Signed-off-by: mlflow-automation <mlflow-automation@users.noreply.github.com>

Signed-off-by: mlflow-automation <mlflow-automation@users.noreply.github.com>
Co-authored-by: mlflow-automation <mlflow-automation@users.noreply.github.com>

* Add searchExperiments API to Java client, deprecate listExperiments (#6561)

* SearchExperiments API for Java

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Search & tests

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Deprecate list

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Add page

Signed-off-by: dbczumar <corey.zumar@databricks.com>

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Pin `pmdarima` in diviner test (#6565)

* Pin pmdarima

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Pin pmdarima

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix command arguments

Signed-off-by: harupy <hkawamura0130@gmail.com>

* quote CHANGED_FILES

Signed-off-by: harupy <hkawamura0130@gmail.com>

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix use-set-literal errors batch 3 (#mlflow-6535) (#6560)

* fix use-set-literal errors batch 3 (#mlflow-6535)

Signed-off-by: Evgenii Uvarov <evgenii.uvarov.post@gmail.com>

* Build trigger for CI

Signed-off-by: evgenii.uvarov <evgenii.uvarov.post@gmail.com>

* Trigger CI

Signed-off-by: harupy <hkawamura0130@gmail.com>

Signed-off-by: Evgenii Uvarov <evgenii.uvarov.post@gmail.com>
Signed-off-by: evgenii.uvarov <evgenii.uvarov.post@gmail.com>
Signed-off-by: harupy <hkawamura0130@gmail.com>
Co-authored-by: harupy <hkawamura0130@gmail.com>

* replaced all self.assertTrue with assert (#6570)

* replaced all self.assertTrue with assert

Signed-off-by: yogesh kumar <kumaryogesh.usict.ipu@gmail.com>

* updated test-run-status file

Signed-off-by: yogesh kumar <kumaryogesh.usict.ipu@gmail.com>

* Autoformat: https://github.com/mlflow/mlflow/actions/runs/2917378175

Signed-off-by: mlflow-automation <mlflow-automation@users.noreply.github.com>

Signed-off-by: yogesh kumar <kumaryogesh.usict.ipu@gmail.com>
Signed-off-by: mlflow-automation <mlflow-automation@users.noreply.github.com>
Co-authored-by: mlflow-automation <mlflow-automation@users.noreply.github.com>

* [bug-fix] Removing the fixed width css for the experimentlist container (#6569)

Signed-off-by: Sunish Sheth <sunishsheth2009@gmail.com>

Signed-off-by: Sunish Sheth <sunishsheth2009@gmail.com>

* Clarify ports in JS dev server instructions (#6563)

* Clarify ports in JS dev server instructions

Signed-off-by: Prithvi Kannan <prithvi.kannan@databricks.com>

* update text

Signed-off-by: Prithvi Kannan <prithvi.kannan@databricks.com>

Signed-off-by: Prithvi Kannan <prithvi.kannan@databricks.com>

* Allow specifying mlflow command line args as env variables. (#6457)

Signed-off-by: Phil Xiaojun Hu <phil@cnphil.com>

Signed-off-by: Phil Xiaojun Hu <phil@cnphil.com>

* [ML-22889] Get artifact cli (#6517)

* adding cli flag

Signed-off-by: Prithvi Kannan <prithvi.kannan@databricks.com>

* return datapath for dataframe artifacts

Signed-off-by: Prithvi Kannan <prithvi.kannan@databricks.com>

* registered_model_version path

Signed-off-by: Prithvi Kannan <prithvi.kannan@databricks.com>

* add support for model and transformer from cli

Signed-off-by: Prithvi Kannan <prithvi.kannan@databricks.com>

* adding support for run cmd

Signed-off-by: Prithvi Kannan <prithvi.kannan@databricks.com>

* change name to get_path

Signed-off-by: Prithvi Kannan <prithvi.kannan@databricks.com>

* adding a test

Signed-off-by: Prithvi Kannan <prithvi.kannan@databricks.com>

* idk why pyfunc was removed

Signed-off-by: Prithvi Kannan <prithvi.kannan@databricks.com>

* updating test and mandatory cli args

Signed-off-by: Prithvi Kannan <prithvi.kannan@databricks.com>

* fixing merge

Signed-off-by: Prithvi Kannan <prithvi.kannan@databricks.com>

* refactored into get_artifact_path

Signed-off-by: Prithvi Kannan <prithvi.kannan@databricks.com>

* fixing test case

Signed-off-by: Prithvi Kannan <prithvi.kannan@databricks.com>

* replaced Any with Optional

Signed-off-by: Prithvi Kannan <prithvi.kannan@databricks.com>

* idk how that happpened

Signed-off-by: Prithvi Kannan <prithvi.kannan@databricks.com>

* move output dirs to if-else

Signed-off-by: Prithvi Kannan <prithvi.kannan@databricks.com>

* addressing corey comments

Signed-off-by: Prithvi Kannan <prithvi.kannan@databricks.com>

* linter

Signed-off-by: Prithvi Kannan <prithvi.kannan@databricks.com>

Signed-off-by: Prithvi Kannan <prithvi.kannan@databricks.com>

* fix final batch of use-set-literal errors #6535 (#6584)

Signed-off-by: Daniel Gibbons <daniel.gibbons04@gmail.com>

Signed-off-by: Daniel Gibbons <daniel.gibbons04@gmail.com>

* Update doc to clarify tf.keras should not be logged with mlflow.tensorflow.log_model (#6573)

Signed-off-by: Liang Zhang <liang.zhang@databricks.com>

Signed-off-by: Liang Zhang <liang.zhang@databricks.com>

* Add mlflow_search_experiments API to R client (#6576)

* Make URLs clickable in the MLflow Tracking UI (#6526)

* wrap urls in anchor tag in runs table

Signed-off-by: Marijn Valk <marijncv@hotmail.com>

* eslint fix & comment

Signed-off-by: Marijn Valk <marijncv@hotmail.com>

* added comment to util function

Signed-off-by: Marijn Valk <marijncv@hotmail.com>

Signed-off-by: Marijn Valk <marijncv@hotmail.com>

* Removing actual code from DAG strings for MLP and just keeping user guide (#6595)

Signed-off-by: Sunish Sheth <sunishsheth2009@gmail.com>

Signed-off-by: Sunish Sheth <sunishsheth2009@gmail.com>

* Add confs for metric prefix and metric dataset info to mlflow.evaluate(), rename accuracy to accuracy_score (#6593)

* Actually include the prefix in eval results

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Naming

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Reset pipelines

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Rename

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Format

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Fix test

Signed-off-by: dbczumar <corey.zumar@databricks.com>

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* [Model Validation] Land model validation feature (#6582)

* [Model Validation] Disable logging metrics and artifacts for baseline model evaluation (#6310)

* add is_baseline_model boolean flag for disabling logging metrics and artifacts

Signed-off-by: Zhe Wang <zhe.wang@databricks.com>

* evaluation for baseline model should not generate artifact

Signed-off-by: Zhe Wang <zhe.wang@databricks.com>

* fix unit tests and dummy evaluator

Signed-off-by: Zhe Wang <zhe.wang@databricks.com>

* add unit test for validation interface

Signed-off-by: Zhe Wang <zhe.wang@databricks.com>

* change existing unit tests for testing validation

Signed-off-by: Zhe Wang <zhe.wang@databricks.com>

* add more unit tests for disabling logging of metrics and artifacts for baseline model evaluation

Signed-off-by: Zhe Wang <zhe.wang@databricks.com>

* add unit tests for vaildation

Signed-off-by: Zhe Wang <zhe.wang@databricks.com>

* Fix docstring for evaluate

Signed-off-by: Zhe Wang <zhe.wang@databricks.com>

* Refactor unit tests for evaluation and validation

Signed-off-by: Zhe Wang <zhe.wang@databricks.com>

* Refactor unit tests for evaluation and validation

Signed-off-by: Zhe Wang <zhe.wang@databricks.com>

* refactor unit tests

Signed-off-by: Zhe Wang <zhe.wang@databricks.com>

* remove typo

Signed-off-by: Zhe Wang <zhe.wang@databricks.com>

* refactor some code into functions

Signed-off-by: Zhe Wang <zhe.wang@databricks.com>

* add baseline_model to evaluator.evalaute and change corresponding unit tests

Signed-off-by: Zhe Wang <zhe.wang@databricks.com>

* refactor unit tests for better readability

Signed-off-by: Zhe Wang <zhe.wang@databricks.com>

* undo some unnecessary refactor

Signed-off-by: Zhe Wang <zhe.wang@databricks.com>

* remove unused methods

Signed-off-by: Zhe Wang <zhe.wang@databricks.com>

* refactor default evaluator evaluate

Signed-off-by: Zhe Wang <zhe.wang@databricks.com>

* add more info to doc string

Signed-off-by: Zhe Wang <zhe.wang@databricks.com>

* add more doc strings for helper methods

Signed-off-by: Zhe Wang <zhe.wang@databricks.com>

* Update mlflow/models/evaluation/base.py

Co-authored-by: Jerry Liang <66143562+jerrylian-db@users.noreply.github.com>

* Update mlflow/models/evaluation/base.py

Co-authored-by: Jerry Liang <66143562+jerrylian-db@users.noreply.github.com>

* address comments

Signed-off-by: Zhe Wang <zhe.wang@databricks.com>

* Update mlflow/models/evaluation/default_evaluator.py

Co-authored-by: Jerry Liang <66143562+jerrylian-db@users.noreply.github.com>

* Update mlflow/models/evaluation/default_evaluator.py

Co-authored-by: Jerry Liang <66143562+jerrylian-db@users.noreply.github.com>

* address comments

Signed-off-by: Zhe Wang <zhe.wang@databricks.com>

* add more comments

Signed-off-by: Zhe Wang <zhe.wang@databricks.com>

* Update tests/models/test_default_evaluator.py

Co-authored-by: Jerry Liang <66143562+jerrylian-db@users.noreply.github.com>

* rename test configs

Signed-off-by: Zhe Wang <zhe.wang@databricks.com>

* add doc string

Signed-off-by: Zhe Wang <zhe.wang@databricks.com>

* address comments

Signed-off-by: Zhe Wang <zhe.wang@databricks.com>

* Update mlflow/models/evaluation/default_evaluator.py

Co-authored-by: Jerry Liang <66143562+jerrylian-db@users.noreply.github.com>

Co-authored-by: Jerry Liang <66143562+jerrylian-db@users.noreply.github.com>

* [Model Validation] Add MetricThreshold and Model Validation Functionality to MLflow.evaluate (#6328)

Add model validation functionality to `mlflow.evaluate`
- Add `MetricThreshold` class to let users define validation thresholds for model validation.
- Add `_validate` for metric value threshold check and model comparison check
- Add internal class `_ValidationResult` per metric for formatting failure message.

If the model validation passed, no exception will be thrown and `EvaluationResult` will be returned. If the model validation failed, `mlflowModelValidationFailedException` will be thrown containing detailed failure message indicating why the model validation did not pass.

Documentations:
Two extra parameters for `mlflow.evalaute`
<img width="895" alt="CleanShot 2022-08-05 at 13 17 13@2x" src="https://user-images.githubusercontent.com/25189433/183156569-06139716-3065-4493-9804-bcbc3538afa9.png">
One user facing class `MetricThreshold` for users to define thresholds for metrics.
<img width="928" alt="CleanShot 2022-08-05 at 13 18 02@2x" src="https://user-images.githubusercontent.com/25189433/183156670-bdd1c147-9e45-4c77-87b0-d5dab308e7db.png">

* Update mlflow/models/evaluation/base.py

Co-authored-by: Corey Zumar <39497902+dbczumar@users.noreply.github.com>

* Update mlflow/models/evaluation/base.py

Co-authored-by: Corey Zumar <39497902+dbczumar@users.noreply.github.com>

* Update mlflow/models/evaluation/base.py

Co-authored-by: Corey Zumar <39497902+dbczumar@users.noreply.github.com>

* merge master pt2

Signed-off-by: Jerry Liang <jerry.liang@databricks.com>

* [Model Validation] Finishing touches and documentation (#6577)

* adapt feedback

Signed-off-by: Jerry Liang <jerry.liang@databricks.com>

* adapt feedback

Signed-off-by: Jerry Liang <jerry.liang@databricks.com>

* wip

Signed-off-by: Jerry Liang <jerry.liang@databricks.com>

* add MetricThreshold class checks

Signed-off-by: Jerry Liang <jerry.liang@databricks.com>

* fix missing baseline model test

Signed-off-by: Jerry Liang <jerry.liang@databricks.com>

* enhance baseline model test

Signed-off-by: Jerry Liang <jerry.liang@databricks.com>

* add baseline metric 0 tests

Signed-off-by: Jerry Liang <jerry.liang@databricks.com>

* add ;

Signed-off-by: Jerry Liang <jerry.liang@databricks.com>

* documentation

Signed-off-by: Jerry Liang <jerry.liang@databricks.com>

* documentation

Signed-off-by: Jerry Liang <jerry.liang@databricks.com>

* documentation

Signed-off-by: Jerry Liang <jerry.liang@databricks.com>

* wip

Signed-off-by: Jerry Liang <jerry.liang@databricks.com>

* wip

Signed-off-by: Jerry Liang <jerry.liang@databricks.com>

* Update mlflow/models/evaluation/validation.py

Co-authored-by: Corey Zumar <39497902+dbczumar@users.noreply.github.com>
Signed-off-by: Jerry Liang <jerry.liang@databricks.com>

* Update docs/source/models.rst

Co-authored-by: Corey Zumar <39497902+dbczumar@users.noreply.github.com>
Signed-off-by: Jerry Liang <jerry.liang@databricks.com>

* Update docs/source/models.rst

Co-authored-by: Corey Zumar <39497902+dbczumar@users.noreply.github.com>
Signed-off-by: Jerry Liang <jerry.liang@databricks.com>

* Update docs/source/models.rst

Co-authored-by: Corey Zumar <39497902+dbczumar@users.noreply.github.com>
Signed-off-by: Jerry Liang <jerry.liang@databricks.com>

* Update examples/evaluation/README.md

Co-authored-by: Corey Zumar <39497902+dbczumar@users.noreply.github.com>
Signed-off-by: Jerry Liang <jerry.liang@databricks.com>

* wip

Signed-off-by: Jerry Liang <jerry.liang@databricks.com>

* wip

Signed-off-by: Jerry Liang <jerry.liang@databricks.com>

Signed-off-by: Jerry Liang <jerry.liang@databricks.com>
Co-authored-by: Corey Zumar <39497902+dbczumar@users.noreply.github.com>

* modify evaluate doc string

Signed-off-by: Jerry Liang <jerry.liang@databricks.com>

Signed-off-by: Jerry Liang <jerry.liang@databricks.com>
Co-authored-by: Zhe Wang <zhe.wang@databricks.com>
Co-authored-by: Corey Zumar <39497902+dbczumar@users.noreply.github.com>

* [ML-20702] increase version font size (#6596)

* Changed the name Start Time to Created (#6597)

Signed-off-by: Jin Zhang <jin.zhang@databricks.com>

Signed-off-by: Jin Zhang <jin.zhang@databricks.com>

* feat(scoring_server): add /health endpoint (#6574)

Signed-off-by: Gabriel Gazola Milan <gabriel.gazola@poli.ufrj.br>

Signed-off-by: Gabriel Gazola Milan <gabriel.gazola@poli.ufrj.br>

* Update start time col (#6614)

* Changed the name Start Time to Created

Signed-off-by: Jin Zhang <jin.zhang@databricks.com>

* Added i18n change to Created

Signed-off-by: Jin Zhang <jin.zhang@databricks.com>

Signed-off-by: Jin Zhang <jin.zhang@databricks.com>

* Add a script to update `CHANGELOG.md` (#6624)

* Add update_changelog.py

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix

Signed-off-by: harupy <hkawamura0130@gmail.com>

* comma

Signed-off-by: harupy <hkawamura0130@gmail.com>

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Improve styling of DAG and rearrage transform step node (#6612)

Signed-off-by: Sunish Sheth <sunishsheth2009@gmail.com>

Signed-off-by: Sunish Sheth <sunishsheth2009@gmail.com>

* Update documentation for model version `run_link` field (#6632)

* Update documentation for model version run_link field

Signed-off-by: Arpit Jasapara <arpit.jasapara@databricks.com>

* Updated protos

Signed-off-by: Arpit Jasapara <arpit.jasapara@databricks.com>

* Update mlflow/protos/model_registry.proto

Co-authored-by: Siddharth Murching <smurching@gmail.com>
Signed-off-by: Arpit Jasapara <arpit.jasapara@databricks.com>

Signed-off-by: Arpit Jasapara <arpit.jasapara@databricks.com>
Co-authored-by: Siddharth Murching <smurching@gmail.com>

* Move data capture config to end (#6621)

Signed-off-by: dbczumar <corey.zumar@databricks.com>

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Used tuple instead a comma-separated string in pytest.mark.parametrize. (#6623)

* Used tuple instead a comma-separated string in pytest.mark.parametrize.

Signed-off-by: Shraddha Falane <falane.shraddha18@gmail.com>

* Used tuple instead a comma-separated string in pytest.mark.parametrize.

Signed-off-by: Shraddha Falane <falane.shraddha18@gmail.com>

Signed-off-by: Shraddha Falane <falane.shraddha18@gmail.com>

* Stabilize SearchExperiments API (#6551)

* Remove search experiments experimental

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Remove unused imports

Signed-off-by: dbczumar <corey.zumar@databricks.com>

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Fix command to trigger auto-formatting (#6639)

* Fix command to trigger auto-formatting

Signed-off-by: harupy <hkawamura0130@gmail.com>

* use test

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix script

Signed-off-by: harupy <hkawamura0130@gmail.com>

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Support argument for local_tmpdir when loading spark model. (#6642)

* feat: Support argument for local_tmpdir when loading spark model.
Signed-off-by: Li Jiang <bnujli@gmail.com>

* update local_tmpdir with dst_path
Signed-off-by: Li Jiang <bnujli@gmail.com>

* Add soft delete in (#6637)

* Add soft delete in

Signed-off-by: ninabacc-db <Nina.Baccam@databricks.com>

* Line item edit

Signed-off-by: ninabacc-db <Nina.Baccam@databricks.com>

Signed-off-by: ninabacc-db <Nina.Baccam@databricks.com>

* adding load_text, load_image, load_dict to fluent.py (#6475)

* adding load_text, load_json, load_image

Signed-off-by: Subramaniam Shanmugam <subramaniam02@gmail.com>

* adding pillow to requirements

Signed-off-by: Subramaniam Shanmu…
BenWilson2 added a commit that referenced this pull request Oct 27, 2022
* Rebase `branch-2.0` on master (#6467)

* Fix java model server compatibility with models containing databricks_runtime conf (#6337)

* Javafix

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Resource fix

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Add run / model links to MLP train step card (#6294)

* One commit

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Submodule revert

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Fix client tests

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* fix ci

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Remove unused / invalid import

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Put run summary back to end, fix tests

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Use mocks instead

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Fix inefficient relative time computation for metric plot (#6341)

* compute minTimestamp only once

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix method name

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix lint

Signed-off-by: harupy <hkawamura0130@gmail.com>

* return empty array

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix test

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Add `MLFLOW_SQLALCHEMYSTORE_POOL_RECYCLE` flag (#6344)

* Add pool_recycle option to SQLAlchemy create_engine

Signed-off-by: Michal Karzynski <michal.karzynski@intel.com>

* Add new option to unit test

Signed-off-by: Michal Karzynski <michal.karzynski@intel.com>

* Add note in documentation

Signed-off-by: Michal Karzynski <michal.karzynski@intel.com>

* Add box plot in compare-runs page (#6308)

* Added BoxPlot

Signed-off-by: changyonglik <theeahlag@gmail.com>

* Fixed key error

Signed-off-by: changyonglik <theeahlag@gmail.com>

* Added default state

Signed-off-by: changyonglik <theeahlag@gmail.com>

* Fixed props passing

Signed-off-by: changyonglik <theeahlag@gmail.com>

* Fixed react hooks state

Signed-off-by: changyonglik <theeahlag@gmail.com>

* Added unit test

Signed-off-by: changyonglik <theeahlag@gmail.com>

* Added boxplot to CompareModelVersion

Signed-off-by: changyonglik <theeahlag@gmail.com>

* Changed keys numbering to reflect tab name

Signed-off-by: changyonglik <theeahlag@gmail.com>

* Made the ordering of tabs consistent accross CompareRunView and CompareModelVersion

Signed-off-by: changyonglik <theeahlag@gmail.com>

* Linted

Signed-off-by: changyonglik <theeahlag@gmail.com>

* Fixed FormattedMessage

Signed-off-by: changyonglik <theeahlag@gmail.com>

* refactoring

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix lint errors

Signed-off-by: harupy <hkawamura0130@gmail.com>

* i18n

Signed-off-by: harupy <hkawamura0130@gmail.com>

* minor comment fix

Signed-off-by: harupy <hkawamura0130@gmail.com>

Co-authored-by: harupy <hkawamura0130@gmail.com>

* Add script and workflow to update requirements.yaml specifications (#6287)

* add script to update requirements.yaml

Signed-off-by: harupy <hkawamura0130@gmail.com>

* update both skinny and core requirements

Signed-off-by: harupy <hkawamura0130@gmail.com>

* remove base

Signed-off-by: harupy <hkawamura0130@gmail.com>

* clean up

Signed-off-by: harupy <hkawamura0130@gmail.com>

* add MLFLOW_CONDA_HOME

Signed-off-by: harupy <hkawamura0130@gmail.com>

* create create-pr job

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix syntax

Signed-off-by: harupy <hkawamura0130@gmail.com>

* remove redundant if

Signed-off-by: harupy <hkawamura0130@gmail.com>

* clean up

Signed-off-by: harupy <hkawamura0130@gmail.com>

* doc

Signed-off-by: harupy <hkawamura0130@gmail.com>

* dynamically determine python version

Signed-off-by: harupy <hkawamura0130@gmail.com>

* remove python version mock

Signed-off-by: harupy <hkawamura0130@gmail.com>

* use setup-ssh.sh in master.yml

Signed-off-by: harupy <hkawamura0130@gmail.com>

* simplify

Signed-off-by: harupy <hkawamura0130@gmail.com>

* update both core and skinny requirements

Signed-off-by: harupy <hkawamura0130@gmail.com>

* use env

Signed-off-by: harupy <hkawamura0130@gmail.com>

* remove action version

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix pandas dtypes issue

Signed-off-by: harupy <hkawamura0130@gmail.com>

* check if PR already exists

Signed-off-by: harupy <hkawamura0130@gmail.com>

* add semicolons

Signed-off-by: harupy <hkawamura0130@gmail.com>

* test PR creation

Signed-off-by: harupy <hkawamura0130@gmail.com>

* set base

Signed-off-by: harupy <hkawamura0130@gmail.com>

* use run_number

Signed-off-by: harupy <hkawamura0130@gmail.com>

* set result-encoding

Signed-off-by: harupy <hkawamura0130@gmail.com>

* revert changes for test

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix more tests

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix run URL

Signed-off-by: harupy <hkawamura0130@gmail.com>

* add dot

Signed-off-by: harupy <hkawamura0130@gmail.com>

* remove minimum_version

Signed-off-by: harupy <hkawamura0130@gmail.com>

* remove create-pr job

Signed-off-by: harupy <hkawamura0130@gmail.com>

* rename workflow

Signed-off-by: harupy <hkawamura0130@gmail.com>

* mutate dict approach

Signed-off-by: harupy <hkawamura0130@gmail.com>

* use ruamel

Signed-off-by: harupy <hkawamura0130@gmail.com>

* remove PYTHON_VERSION patch

Signed-off-by: harupy <hkawamura0130@gmail.com>

* add ruamel.yaml to requirements

Signed-off-by: harupy <hkawamura0130@gmail.com>

* skip update-requirements on PR

Signed-off-by: harupy <hkawamura0130@gmail.com>

* rename

Signed-off-by: harupy <hkawamura0130@gmail.com>

* rename

Signed-off-by: harupy <hkawamura0130@gmail.com>

* print new version

Signed-off-by: harupy <hkawamura0130@gmail.com>

* preserve quotes

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix badge

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix README

Signed-off-by: harupy <hkawamura0130@gmail.com>

* run ci

Signed-off-by: harupy <hkawamura0130@gmail.com>

* remove validate-requirements.yml

Signed-off-by: harupy <hkawamura0130@gmail.com>

* rename

Signed-off-by: harupy <hkawamura0130@gmail.com>

* rename

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix test_dataframe_from_json

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Make `model_uri` optional for `mlflow models build-docker` (#6302)

* Make model_uri optional for mlflow models build-docker

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix doc

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Add generic

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Add pythou_function flavor

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix doc

Signed-off-by: harupy <hkawamura0130@gmail.com>

* quotes

Signed-off-by: harupy <hkawamura0130@gmail.com>

* use longer options

Signed-off-by: harupy <hkawamura0130@gmail.com>

* add index (#6347)

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix badge link (#6349)

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Enable consider-using-dict-items rule for pylint (#6351)

* Enable consider-using-dict-items

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix lint errors

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Enable consider-merging-isinstance (#6350)

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Enable consider-using-from-import rule for pylint (#6352)

* Enable consider-using-from-import

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix datasets import

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix failed tests

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Run model registry sqlalchemy tests in database job (#6353)

Signed-off-by: harupy <hkawamura0130@gmail.com>

* update to run profiles on the first 100 dataframe columns (#6297)

* update to run profiles on the first 100 dataframe columns

Signed-off-by: Sunish Sheth <sunishsheth2009@gmail.com>

* Adding replace=True so we can sample rows twice

Signed-off-by: Sunish Sheth <sunishsheth2009@gmail.com>

* Removing replace=true and fixing how the max_rows are calculated

Signed-off-by: Sunish Sheth <sunishsheth2009@gmail.com>

* Fix sklearn autologging input example & signature mutation for models containing transformers (#6230)

* Fix with test case

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Test fix

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Fix and simplification

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* more fixes & better test coveragE

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* test

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* LGBM test

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Format

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Address comments

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Fixes

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Use py37-compatible call args syntax

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* use list (#6363)

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Log runtime pipeline config to MLflow (#6359)

* Log runtime pipeline config to MLflow

Signed-off-by: Jin Zhang <jin.zhang@databricks.com>

* Fixed lint

Signed-off-by: Jin Zhang <jin.zhang@databricks.com>

* Add status badge for stale action (#6372)

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Add stack trace section to bug report template (#6374)

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Skipping the transform step by using Functional transformer (#6362)

Signed-off-by: Sunish Sheth <sunishsheth2009@gmail.com>

* Series schema (#6361)

* Added in name for ColSpec of pandas datatype

infer panda series name for ColSpec

Signed-off-by: Ryan Fogle <ryanfogle98@gmail.com>

* Added test for Series name inference

added one more pytest condition in test_schema:test_schema_inference_on_pandas_series

Signed-off-by: Ryan Fogle <ryanfogle98@gmail.com>

* Update test_schema.py

Added in one more assertion to test schema inference.

Signed-off-by: Ryan Fogle <ryanfogle98@gmail.com>

* ran black

Signed-off-by: Ryan Fogle <ryanfogle98@gmail.com>

* added in requested changes in issue #6361

Signed-off-by: Ryan Fogle <ryanfogle98@gmail.com>

* fix typo. changed hasattr to getattr

Signed-off-by: Ryan Fogle <ryanfogle98@gmail.com>

* integrate mlflowdbfs (#6282)

Signed-off-by: Brian Barnes <brian.barnes@databricks.com>

* Fix step card display breakage in Jupyter notebooks (#6378)

* remove

Signed-off-by: apurva-koti <apurva.koti@databricks.com>

* fix

Signed-off-by: apurva-koti <apurva.koti@databricks.com>

* Avoid using flask 2.2.0 (#6380)

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Migrate GCS environment variables to `mlflow.environment_variables` module (#6375)

* Migrate GCS environment variables to mlflow.environment_variables

Signed-off-by: harupy <hkawamura0130@gmail.com>

* add default

Signed-off-by: harupy <hkawamura0130@gmail.com>

* improve comments

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Avoid logging model signatures in pyspark ML autologging if model input/output dataframe contains unsupported data types (#6365)

* Avoid logging signatures

Signed-off-by: harupy <hkawamura0130@gmail.com>

* test

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix indent

Signed-off-by: harupy <hkawamura0130@gmail.com>

* check model output

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix warning message

Signed-off-by: harupy <hkawamura0130@gmail.com>

* todo

Signed-off-by: harupy <hkawamura0130@gmail.com>

* update test and doc

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix docstring

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix failed tests

Signed-off-by: harupy <hkawamura0130@gmail.com>

* improve _assert_autolog_infers_model_signature_correctly

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Refactor `set_matrix.py` (#6373)

* refacor set_matrix.py

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix tests

Signed-off-by: harupy <hkawamura0130@gmail.com>

* tset for --no-dev

Signed-off-by: harupy <hkawamura0130@gmail.com>

* install pytest and pytest-cov

Signed-off-by: harupy <hkawamura0130@gmail.com>

* test for --changed-files

Signed-off-by: harupy <hkawamura0130@gmail.com>

* add pyyaml to requirements

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix is_matrix_empty

Signed-off-by: harupy <hkawamura0130@gmail.com>

* use raw string and re.match

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Fix is_matrix_empty (#6385)

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Allow mlflow-skinny's autolog() to succeed when no scipy is installed (#5897)

* allow mlflow-skinny's autolog() to succeed when no scipy is installed

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* optional pyfunc scipy dependency

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* fix linter-discovered issues

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* add test case, autolog() should pass w/o scipy installed

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* minor cleanup

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* test creating _Example with / without scipy present

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* docs: do not warn about sparse matrix classes

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* linting

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* address review comment

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* run black

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* allow mlflow-skinny's autolog() to succeed when no scipy is installed

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* optional pyfunc scipy dependency

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* fix linter-discovered issues

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* add test case, autolog() should pass w/o scipy installed

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* minor cleanup

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* test creating _Example with / without scipy present

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* docs: do not warn about sparse matrix classes

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* linting

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* address review comment

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* run black

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* remove redundant None check after import

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* delete comment

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* Revert "remove redundant None check after import"

This reverts commit f2e198c944f41d9a96c20ae471d71bff39431b48.

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* review comments

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* remove redundant test

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* Apply suggestions from code review

Co-authored-by: Harutaka Kawamura <hkawamura0130@gmail.com>
Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* Autoformat: https://github.com/mlflow/mlflow/actions/runs/2779791161

Signed-off-by: mlflow-automation <mlflow-automation@users.noreply.github.com>

Co-authored-by: Harutaka Kawamura <hkawamura0130@gmail.com>
Co-authored-by: dbczumar <corey.zumar@databricks.com>
Co-authored-by: mlflow-automation <mlflow-automation@users.noreply.github.com>

* Fix extraneous warning about pyspark evaluation truncation (#6389)

* Revert #6380 (#6386)

* revert

Signed-off-by: harupy <hkawamura0130@gmail.com>

* workaround for test_prometheus_exporter

Signed-off-by: harupy <hkawamura0130@gmail.com>

* nit

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Validate metric value in `FileStore.log_metric` (#6381)

* add failing test

Signed-off-by: harupy <hkawamura0130@gmail.com>

* use _validate_metric

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix invalid metric timestamp

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix docstring

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix java test

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Make search experiment by tag key comparison case sensitive in MYSQL (#6333)

* init

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* add test

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* update

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* update

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* fix BINARY conditions in MySQL

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix attr error

Signed-off-by: harupy <hkawamura0130@gmail.com>

Co-authored-by: harupy <hkawamura0130@gmail.com>

* Run Model Registry SQLAlchemy tests in PostgreSQL, MySQL, and MSSQL (#6390)

* Run Model Registry SQLAlchemy tests in postgres, mysql, and mssql

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix failed tests

Signed-off-by: harupy <hkawamura0130@gmail.com>

* remove uuid

Signed-off-by: harupy <hkawamura0130@gmail.com>

* clean up

Signed-off-by: harupy <hkawamura0130@gmail.com>

* do not drop tables

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Fix filestore data loss on failed write & mitigate read / write race condition (#6388)

* Fix

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Fix

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Format

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Added a custom filter from_json to render_and_merge_yaml (#6368)

* Added a custom filter from_json to render_and_merge_yaml

Signed-off-by: Jin Zhang <jin.zhang@databricks.com>

* Fixed test

Signed-off-by: Jin Zhang <jin.zhang@databricks.com>

* Update tests/utils/test_file_utils.py

Co-authored-by: Siddharth Murching <sid.murching@databricks.com>

* Update tests/utils/test_file_utils.py

Co-authored-by: Siddharth Murching <sid.murching@databricks.com>

* Fixed test

Signed-off-by: Jin Zhang <jin.zhang@databricks.com>

* Fixed windows test attempt

Signed-off-by: Jin Zhang <jin.zhang@databricks.com>

* Fixed windows test

Signed-off-by: Jin Zhang <jin.zhang@databricks.com>

Co-authored-by: Siddharth Murching <sid.murching@databricks.com>

* Fix `!=` operator for MySQL (#6397)

* fix ne operator and improve tests

Signed-off-by: harupy <hkawamura0130@gmail.com>

* add () to other operators

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Simplify the documentation section in the PR template (#6398)

* Simplify the documentation section in the PR template

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Timebased gc (#6354)

* Added delete_time field to runs table

Signed-off-by: Jason Cheng <jasoncky96@gmail.com>

* Updated latest_schema.sql to pass tests/store/tracking/test_sqlalchemy_store_schema.py

Signed-off-by: Jason Cheng <jasoncky96@gmail.com>

* Added delete_time column into mlflow/store/tracking/dbmodels/initial_models.py

Signed-off-by: Jason Cheng <jasoncky96@gmail.com>

* Moved the docstring to the end of the declaration instead of the start

Signed-off-by: Jason Cheng <jasoncky96@gmail.com>

* Regenerated protobuf for RunInfo new schema, and passed pytest

Signed-off-by: Jason Cheng <jasoncky96@gmail.com>

* Added --older-than flag in mlflow gc command, and added tests for that cli command

Signed-off-by: Jason Cheng <jasoncky96@gmail.com>

* Removed delete_time from RunInfo proto

Signed-off-by: Jason Cheng <jasoncky96@gmail.com>

* Set default older-than flag to None

Signed-off-by: Jason Cheng <jasoncky96@gmail.com>

* Remove delete_time from RunInfo, add optional parameter older_than to store.get_deleted_runs(), and write delete_time into yaml file for file storage instead of adding it to RunInfo

Signed-off-by: Jason Cheng <jasoncky96@gmail.com>

* Remove commented delete_time in RunInfo constructor in models.py

Signed-off-by: Jason Cheng <jasoncky96@gmail.com>

* added INVALID_PARAMETER_VALUE error code in mlflow gc

Signed-off-by: Jason Cheng <jasoncky96@gmail.com>

* Resolve case of delete time not in meta.yaml file, edited overwrite_run_info and gc

Signed-off-by: Jason Cheng <jasoncky96@gmail.com>

* Rename delete_time to deleted_time

Signed-off-by: Jason Cheng <jasoncky96@gmail.com>

* Rename delete_time to deleted_time, match error message in cli tests, filter SqlRuns by deleted_time, indicate float values accepted for gc --older-than flag

Signed-off-by: Jason Cheng <jasoncky96@gmail.com>

* Rename delete_time to deleted_time, fixed assert statement in test_mlflow_gc, and fixed deleted_time queries in test_file_store.py and test_sqlalchemy_store.py

Signed-off-by: Jason Cheng <jasoncky96@gmail.com>

* Only check if  is in meta after restoring run in test_delte_restore_run

Signed-off-by: Jason Cheng <jasoncky96@gmail.com>

* Fixed lint errors

Signed-off-by: Jason Cheng <jasoncky96@gmail.com>

* Install mlflow from repository root

Signed-off-by: harupy <hkawamura0130@gmail.com>

Co-authored-by: harupy <hkawamura0130@gmail.com>

* Separate protos job (#6399)

* Separate protos job

Signed-off-by: harupy <hkawamura0130@gmail.com>

* test

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Add paths

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Revert "test"

This reverts commit 5d04b02f55ec92e1e3ac18eb651ea989ca0d469a.

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Fix flaky `TestFileStore.test_get_deleted_runs` (#6402)

* use gt operator

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix sqlalchemy store

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Migrate environment variables for SQLAlchemy store (#6396)

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Deduplicate MlflowClient in MLflow docs, move to mlflow.client module (#6405)

* Dedupe MlflowClient

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Improvements

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Add client

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Remove unused functions in `mlflow.data` module (#6408)

* remove unused functions in data

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Remove unused functions and variables

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Tag based registered model search and model version search (#6320)

* init

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* update

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* update

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* update

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* update

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* update

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* update

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* update

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* update

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* update

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* update

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* update tests

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* update

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* update

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* clean

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* address comments

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* add test

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* address comments

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* update

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* de-dup tag based search result

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* add () to fix case_sensitive_mysql_ne

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* fix

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* lint

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* fix

* fix test in TODO

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* support dup tag key filter

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* add mv order by

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* cmp res list in test

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* address comments

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* address comments

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* add doc

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* fix

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* fix lint

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* address comments

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* address comments

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* update doc

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* Increase maximum param value length to 500 (#6358)

* increase validation of maximum param value length from 250 to 500 (squashed)

Signed-off-by: Jan Sindlar <jan.sindlar@eureka.ai>

* 1. added test for exceeding max length. 2. lint. 3. validation logic addresses null

Signed-off-by: Jan Sindlar <jan.sindlar@eureka.ai>

* file store should test already stringified params

Signed-off-by: Jan Sindlar <jan.sindlar@eureka.ai>

* update alembic chain

Signed-off-by: Jan Sindlar <jan.sindlar@eureka.ai>

* Autoformat: https://github.com/mlflow/mlflow/actions/runs/2800398158

Signed-off-by: mlflow-automation <mlflow-automation@users.noreply.github.com>

* Update tests/store/tracking/test_file_store.py

Co-authored-by: Harutaka Kawamura <hkawamura0130@gmail.com>
Signed-off-by: Jan Sindlar <jan.sindlar@eureka.ai>

Co-authored-by: Jan Sindlar <jan.sindlar@eureka.ai>
Co-authored-by: mlflow-automation <mlflow-automation@users.noreply.github.com>
Co-authored-by: Harutaka Kawamura <hkawamura0130@gmail.com>

* Add workflow dispatch for requirements validation, remove duplicate file (#6418)

* Workflow dispatch

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Remove unused

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Prevent `pytest.raises` context manager from containing multiple statements (#6409)

* rename

Signed-off-by: harupy <hkawamura0130@gmail.com>

* rename

Signed-off-by: harupy <hkawamura0130@gmail.com>

* rename

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Do not allow calling assert in pytest.raises

Signed-off-by: harupy <hkawamura0130@gmail.com>

* more strict rule

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Improve error message

Signed-off-by: harupy <hkawamura0130@gmail.com>

* relax condition

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix lint errors

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix failed tests

Signed-off-by: harupy <hkawamura0130@gmail.com>

* clean up

Signed-off-by: harupy <hkawamura0130@gmail.com>

* rename _is_complex_pytest_raises

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix lint errors

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix failed tests

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Fix regression in pandas type inference behavior for scoring server (#6417)

* Fix impl

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Remove unused pandas import

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Adjust client to use SearchExperiments POST method (#6415)

* POST

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Generate protos

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Fix test

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* update (#6420)

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* Check if artifacts are proxied in model_version_artifact_handler (#6355)

* Check if artifacts are proxied in model_version_artifact_handler

Signed-off-by: Mohammad Abbas <mohammad.abbas@digital.trade.gov.uk>

* Use file prefix for artifact dest on windows

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Fix application of prefix

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Fixes

Signed-off-by: dbczumar <corey.zumar@databricks.com>

Co-authored-by: dbczumar <corey.zumar@databricks.com>

* Allow sql filter IN operator right-hand side values including whitespaces (#6413)

* support in rhs whitespace

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* update

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* Fix IN operator related issues in sql filter parser (#6411)

* update

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* update

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* clean

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* update

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* update

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* update

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* address comment

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* Remove unused method `SearchUtils._get_comparison_for_model_registry` (#6412)

* init

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* update

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* Update MLflow UI (#6422)

* Project import generated by Copybara.

GitOrigin-RevId: 086b1200f4573559e37fcea6cb5472d52f166e42
Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Reconcile changes from universe with OSS

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Fix test fails

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Fix

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Apply model search fix

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Remove errant test line

Signed-off-by: dbczumar <corey.zumar@databricks.com>

Co-authored-by: Richard Zang <Zangr@users.noreply.github.com>

* remove near from deprecation msg (#6430)

Signed-off-by: ninabacc-db <Nina.Baccam@databricks.com>

* Raise clear error message when pipeline path contains a space (#6426)

* Add unsupported

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Fix

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Exc

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Test

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Format

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Test fix

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Fix

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* [ALL TESTS] Changelog for MLflow 1.28.0 (#6433)

* Generating changelog for MLflow 1.27.1.dev0

Signed-off-by: Jenkins <jenkins@databricks.com>

* Changelog

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Add missing PR

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Some fixes

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* More small fixes

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Fix

Signed-off-by: dbczumar <corey.zumar@databricks.com>

Signed-off-by: Jenkins <jenkins@databricks.com>
Signed-off-by: dbczumar <corey.zumar@databricks.com>
Co-authored-by: Jenkins <jenkins@databricks.com>
Co-authored-by: dbczumar <corey.zumar@databricks.com>

* Patch (#6440)

Signed-off-by: dbczumar <corey.zumar@databricks.com>

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* [ALL TESTS] Update  (#6443)

* python dev/update_ml_package_versions.py

Signed-off-by: Jenkins <jenkins@databricks.com>

* pyspark max ver 3.4

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

Signed-off-by: Jenkins <jenkins@databricks.com>
Signed-off-by: Weichen Xu <weichen.xu@databricks.com>
Co-authored-by: Jenkins <jenkins@databricks.com>
Co-authored-by: Weichen Xu <weichen.xu@databricks.com>

* Update MLflow version to 1.28.1 (#6449)

Signed-off-by: Jenkins <jenkins@databricks.com>

Signed-off-by: Jenkins <jenkins@databricks.com>
Co-authored-by: Jenkins <jenkins@databricks.com>

* Add data capture config to sagemaker (#6423)

Signed-off-by: jonwiggins <me@jonwiggins.org>

Signed-off-by: jonwiggins <me@jonwiggins.org>

* Fix small search_runs docs issues (#6452)

* Fix

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Doc fix

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Fix

Signed-off-by: dbczumar <corey.zumar@databricks.com>

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* python dev/update_pypi_package_index.py (#6442)

Signed-off-by: Jenkins <jenkins@databricks.com>

Signed-off-by: Jenkins <jenkins@databricks.com>
Co-authored-by: Jenkins <jenkins@databricks.com>

* use xgboost on pypi (#6461)

Signed-off-by: harupy <hkawamura0130@gmail.com>

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Add environment variable for `echo` option in `create_engine` (#6460)

* Add environment variable for echo

Signed-off-by: harupy <hkawamura0130@gmail.com>

* use _BooleanEnvironmentVariable

Signed-off-by: harupy <hkawamura0130@gmail.com>

* address comments

Signed-off-by: harupy <hkawamura0130@gmail.com>

* address comments and add tests

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix comment

Signed-off-by: harupy <hkawamura0130@gmail.com>

Signed-off-by: harupy <hkawamura0130@gmail.com>

Signed-off-by: dbczumar <corey.zumar@databricks.com>
Signed-off-by: harupy <hkawamura0130@gmail.com>
Signed-off-by: Michal Karzynski <michal.karzynski@intel.com>
Signed-off-by: Sunish Sheth <sunishsheth2009@gmail.com>
Signed-off-by: Jin Zhang <jin.zhang@databricks.com>
Signed-off-by: Ryan Fogle <ryanfogle98@gmail.com>
Signed-off-by: Brian Barnes <brian.barnes@databricks.com>
Signed-off-by: apurva-koti <apurva.koti@databricks.com>
Signed-off-by: Weichen Xu <weichen.xu@databricks.com>
Signed-off-by: ninabacc-db <Nina.Baccam@databricks.com>
Signed-off-by: Jenkins <jenkins@databricks.com>
Signed-off-by: jonwiggins <me@jonwiggins.org>
Co-authored-by: Corey Zumar <39497902+dbczumar@users.noreply.github.com>
Co-authored-by: Michał Karzyński <michal.karzynski@intel.com>
Co-authored-by: Chang Yong Lik <51813538+ahlag@users.noreply.github.com>
Co-authored-by: Sunish Sheth <sunishsheth2009@gmail.com>
Co-authored-by: jinzhang21 <78067366+jinzhang21@users.noreply.github.com>
Co-authored-by: Ryan Fogle <40377123+RynoXLI@users.noreply.github.com>
Co-authored-by: bbarnes52 <brian.barnes@databricks.com>
Co-authored-by: apurva-koti <51172624+apurva-koti@users.noreply.github.com>
Co-authored-by: Hannes Schulz <hannes.schulz@microsoft.com>
Co-authored-by: dbczumar <corey.zumar@databricks.com>
Co-authored-by: mlflow-automation <mlflow-automation@users.noreply.github.com>
Co-authored-by: WeichenXu <weichen.xu@databricks.com>
Co-authored-by: Siddharth Murching <sid.murching@databricks.com>
Co-authored-by: Cheng Kuan Yong Jason <jasoncky96@gmail.com>
Co-authored-by: johnyNJ <77425417+johnyNJ@users.noreply.github.com>
Co-authored-by: Jan Sindlar <jan.sindlar@eureka.ai>
Co-authored-by: Mohammad Abbas <mohammad.abbas86@gmail.com>
Co-authored-by: Richard Zang <Zangr@users.noreply.github.com>
Co-authored-by: Nina Baccam <106279294+ninabacc-db@users.noreply.github.com>
Co-authored-by: mlflow-automation <61449322+mlflow-automation@users.noreply.github.com>
Co-authored-by: Jenkins <jenkins@databricks.com>
Co-authored-by: Jon Wiggins <35540058+jonwiggins@users.noreply.github.com>

* Use `virtualenv` by default when running an ML project (#6489)

* Use virtualenv by default when running an ML project

Signed-off-by: harupy <hkawamura0130@gmail.com>

* use _PYTHON_ENV_FILE_NAME

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix doc

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix env type

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix env_type

Signed-off-by: harupy <hkawamura0130@gmail.com>

* swap conda and virtualenv

Signed-off-by: harupy <hkawamura0130@gmail.com>

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Use `virtualenv` by default when restoring a model environment (#6459)

* Use virtualenv by default when restoring a model environment

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix failed test

Signed-off-by: harupy <hkawamura0130@gmail.com>

* set up pyenv and virtualenv

Signed-off-by: harupy <hkawamura0130@gmail.com>

* get conda_env file path from MLmodel file

Signed-off-by: harupy <hkawamura0130@gmail.com>

* pyenv

Signed-off-by: harupy <hkawamura0130@gmail.com>

* use current python version

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix failed test

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix failed test

Signed-off-by: harupy <hkawamura0130@gmail.com>

* update default value of spark_udf

Signed-off-by: harupy <hkawamura0130@gmail.com>

* rename test

Signed-off-by: harupy <hkawamura0130@gmail.com>

* revert spark_udf change

Signed-off-by: harupy <hkawamura0130@gmail.com>

* revert

Signed-off-by: harupy <hkawamura0130@gmail.com>

* change default value to virtualenv

Signed-off-by: harupy <hkawamura0130@gmail.com>

* use local

Signed-off-by: harupy <hkawamura0130@gmail.com>

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Remove `--no-conda` (#6501)

* remove --no-conda

Signed-off-by: harupy <hkawamura0130@gmail.com>

* update R docs

Signed-off-by: harupy <hkawamura0130@gmail.com>

* remove tests for --no-conda

Signed-off-by: harupy <hkawamura0130@gmail.com>

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Sync `branch-2.0` with master (#6510)

* Fix java model server compatibility with models containing databricks_runtime conf (#6337)

* Javafix

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Resource fix

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Add run / model links to MLP train step card (#6294)

* One commit

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Submodule revert

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Fix client tests

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* fix ci

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Remove unused / invalid import

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Put run summary back to end, fix tests

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Use mocks instead

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Fix inefficient relative time computation for metric plot (#6341)

* compute minTimestamp only once

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix method name

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix lint

Signed-off-by: harupy <hkawamura0130@gmail.com>

* return empty array

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix test

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Add `MLFLOW_SQLALCHEMYSTORE_POOL_RECYCLE` flag (#6344)

* Add pool_recycle option to SQLAlchemy create_engine

Signed-off-by: Michal Karzynski <michal.karzynski@intel.com>

* Add new option to unit test

Signed-off-by: Michal Karzynski <michal.karzynski@intel.com>

* Add note in documentation

Signed-off-by: Michal Karzynski <michal.karzynski@intel.com>

* Add box plot in compare-runs page (#6308)

* Added BoxPlot

Signed-off-by: changyonglik <theeahlag@gmail.com>

* Fixed key error

Signed-off-by: changyonglik <theeahlag@gmail.com>

* Added default state

Signed-off-by: changyonglik <theeahlag@gmail.com>

* Fixed props passing

Signed-off-by: changyonglik <theeahlag@gmail.com>

* Fixed react hooks state

Signed-off-by: changyonglik <theeahlag@gmail.com>

* Added unit test

Signed-off-by: changyonglik <theeahlag@gmail.com>

* Added boxplot to CompareModelVersion

Signed-off-by: changyonglik <theeahlag@gmail.com>

* Changed keys numbering to reflect tab name

Signed-off-by: changyonglik <theeahlag@gmail.com>

* Made the ordering of tabs consistent accross CompareRunView and CompareModelVersion

Signed-off-by: changyonglik <theeahlag@gmail.com>

* Linted

Signed-off-by: changyonglik <theeahlag@gmail.com>

* Fixed FormattedMessage

Signed-off-by: changyonglik <theeahlag@gmail.com>

* refactoring

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix lint errors

Signed-off-by: harupy <hkawamura0130@gmail.com>

* i18n

Signed-off-by: harupy <hkawamura0130@gmail.com>

* minor comment fix

Signed-off-by: harupy <hkawamura0130@gmail.com>

Co-authored-by: harupy <hkawamura0130@gmail.com>

* Add script and workflow to update requirements.yaml specifications (#6287)

* add script to update requirements.yaml

Signed-off-by: harupy <hkawamura0130@gmail.com>

* update both skinny and core requirements

Signed-off-by: harupy <hkawamura0130@gmail.com>

* remove base

Signed-off-by: harupy <hkawamura0130@gmail.com>

* clean up

Signed-off-by: harupy <hkawamura0130@gmail.com>

* add MLFLOW_CONDA_HOME

Signed-off-by: harupy <hkawamura0130@gmail.com>

* create create-pr job

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix syntax

Signed-off-by: harupy <hkawamura0130@gmail.com>

* remove redundant if

Signed-off-by: harupy <hkawamura0130@gmail.com>

* clean up

Signed-off-by: harupy <hkawamura0130@gmail.com>

* doc

Signed-off-by: harupy <hkawamura0130@gmail.com>

* dynamically determine python version

Signed-off-by: harupy <hkawamura0130@gmail.com>

* remove python version mock

Signed-off-by: harupy <hkawamura0130@gmail.com>

* use setup-ssh.sh in master.yml

Signed-off-by: harupy <hkawamura0130@gmail.com>

* simplify

Signed-off-by: harupy <hkawamura0130@gmail.com>

* update both core and skinny requirements

Signed-off-by: harupy <hkawamura0130@gmail.com>

* use env

Signed-off-by: harupy <hkawamura0130@gmail.com>

* remove action version

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix pandas dtypes issue

Signed-off-by: harupy <hkawamura0130@gmail.com>

* check if PR already exists

Signed-off-by: harupy <hkawamura0130@gmail.com>

* add semicolons

Signed-off-by: harupy <hkawamura0130@gmail.com>

* test PR creation

Signed-off-by: harupy <hkawamura0130@gmail.com>

* set base

Signed-off-by: harupy <hkawamura0130@gmail.com>

* use run_number

Signed-off-by: harupy <hkawamura0130@gmail.com>

* set result-encoding

Signed-off-by: harupy <hkawamura0130@gmail.com>

* revert changes for test

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix more tests

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix run URL

Signed-off-by: harupy <hkawamura0130@gmail.com>

* add dot

Signed-off-by: harupy <hkawamura0130@gmail.com>

* remove minimum_version

Signed-off-by: harupy <hkawamura0130@gmail.com>

* remove create-pr job

Signed-off-by: harupy <hkawamura0130@gmail.com>

* rename workflow

Signed-off-by: harupy <hkawamura0130@gmail.com>

* mutate dict approach

Signed-off-by: harupy <hkawamura0130@gmail.com>

* use ruamel

Signed-off-by: harupy <hkawamura0130@gmail.com>

* remove PYTHON_VERSION patch

Signed-off-by: harupy <hkawamura0130@gmail.com>

* add ruamel.yaml to requirements

Signed-off-by: harupy <hkawamura0130@gmail.com>

* skip update-requirements on PR

Signed-off-by: harupy <hkawamura0130@gmail.com>

* rename

Signed-off-by: harupy <hkawamura0130@gmail.com>

* rename

Signed-off-by: harupy <hkawamura0130@gmail.com>

* print new version

Signed-off-by: harupy <hkawamura0130@gmail.com>

* preserve quotes

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix badge

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix README

Signed-off-by: harupy <hkawamura0130@gmail.com>

* run ci

Signed-off-by: harupy <hkawamura0130@gmail.com>

* remove validate-requirements.yml

Signed-off-by: harupy <hkawamura0130@gmail.com>

* rename

Signed-off-by: harupy <hkawamura0130@gmail.com>

* rename

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix test_dataframe_from_json

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Make `model_uri` optional for `mlflow models build-docker` (#6302)

* Make model_uri optional for mlflow models build-docker

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix doc

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Add generic

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Add pythou_function flavor

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix doc

Signed-off-by: harupy <hkawamura0130@gmail.com>

* quotes

Signed-off-by: harupy <hkawamura0130@gmail.com>

* use longer options

Signed-off-by: harupy <hkawamura0130@gmail.com>

* add index (#6347)

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix badge link (#6349)

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Enable consider-using-dict-items rule for pylint (#6351)

* Enable consider-using-dict-items

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix lint errors

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Enable consider-merging-isinstance (#6350)

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Enable consider-using-from-import rule for pylint (#6352)

* Enable consider-using-from-import

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix datasets import

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix failed tests

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Run model registry sqlalchemy tests in database job (#6353)

Signed-off-by: harupy <hkawamura0130@gmail.com>

* update to run profiles on the first 100 dataframe columns (#6297)

* update to run profiles on the first 100 dataframe columns

Signed-off-by: Sunish Sheth <sunishsheth2009@gmail.com>

* Adding replace=True so we can sample rows twice

Signed-off-by: Sunish Sheth <sunishsheth2009@gmail.com>

* Removing replace=true and fixing how the max_rows are calculated

Signed-off-by: Sunish Sheth <sunishsheth2009@gmail.com>

* Fix sklearn autologging input example & signature mutation for models containing transformers (#6230)

* Fix with test case

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Test fix

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Fix and simplification

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* more fixes & better test coveragE

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* test

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* LGBM test

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Format

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Address comments

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Fixes

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Use py37-compatible call args syntax

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* use list (#6363)

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Log runtime pipeline config to MLflow (#6359)

* Log runtime pipeline config to MLflow

Signed-off-by: Jin Zhang <jin.zhang@databricks.com>

* Fixed lint

Signed-off-by: Jin Zhang <jin.zhang@databricks.com>

* Add status badge for stale action (#6372)

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Add stack trace section to bug report template (#6374)

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Skipping the transform step by using Functional transformer (#6362)

Signed-off-by: Sunish Sheth <sunishsheth2009@gmail.com>

* Series schema (#6361)

* Added in name for ColSpec of pandas datatype

infer panda series name for ColSpec

Signed-off-by: Ryan Fogle <ryanfogle98@gmail.com>

* Added test for Series name inference

added one more pytest condition in test_schema:test_schema_inference_on_pandas_series

Signed-off-by: Ryan Fogle <ryanfogle98@gmail.com>

* Update test_schema.py

Added in one more assertion to test schema inference.

Signed-off-by: Ryan Fogle <ryanfogle98@gmail.com>

* ran black

Signed-off-by: Ryan Fogle <ryanfogle98@gmail.com>

* added in requested changes in issue #6361

Signed-off-by: Ryan Fogle <ryanfogle98@gmail.com>

* fix typo. changed hasattr to getattr

Signed-off-by: Ryan Fogle <ryanfogle98@gmail.com>

* integrate mlflowdbfs (#6282)

Signed-off-by: Brian Barnes <brian.barnes@databricks.com>

* Fix step card display breakage in Jupyter notebooks (#6378)

* remove

Signed-off-by: apurva-koti <apurva.koti@databricks.com>

* fix

Signed-off-by: apurva-koti <apurva.koti@databricks.com>

* Avoid using flask 2.2.0 (#6380)

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Migrate GCS environment variables to `mlflow.environment_variables` module (#6375)

* Migrate GCS environment variables to mlflow.environment_variables

Signed-off-by: harupy <hkawamura0130@gmail.com>

* add default

Signed-off-by: harupy <hkawamura0130@gmail.com>

* improve comments

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Avoid logging model signatures in pyspark ML autologging if model input/output dataframe contains unsupported data types (#6365)

* Avoid logging signatures

Signed-off-by: harupy <hkawamura0130@gmail.com>

* test

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix indent

Signed-off-by: harupy <hkawamura0130@gmail.com>

* check model output

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix warning message

Signed-off-by: harupy <hkawamura0130@gmail.com>

* todo

Signed-off-by: harupy <hkawamura0130@gmail.com>

* update test and doc

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix docstring

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix failed tests

Signed-off-by: harupy <hkawamura0130@gmail.com>

* improve _assert_autolog_infers_model_signature_correctly

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Refactor `set_matrix.py` (#6373)

* refacor set_matrix.py

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix tests

Signed-off-by: harupy <hkawamura0130@gmail.com>

* tset for --no-dev

Signed-off-by: harupy <hkawamura0130@gmail.com>

* install pytest and pytest-cov

Signed-off-by: harupy <hkawamura0130@gmail.com>

* test for --changed-files

Signed-off-by: harupy <hkawamura0130@gmail.com>

* add pyyaml to requirements

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix is_matrix_empty

Signed-off-by: harupy <hkawamura0130@gmail.com>

* use raw string and re.match

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Fix is_matrix_empty (#6385)

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Allow mlflow-skinny's autolog() to succeed when no scipy is installed (#5897)

* allow mlflow-skinny's autolog() to succeed when no scipy is installed

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* optional pyfunc scipy dependency

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* fix linter-discovered issues

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* add test case, autolog() should pass w/o scipy installed

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* minor cleanup

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* test creating _Example with / without scipy present

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* docs: do not warn about sparse matrix classes

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* linting

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* address review comment

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* run black

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* allow mlflow-skinny's autolog() to succeed when no scipy is installed

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* optional pyfunc scipy dependency

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* fix linter-discovered issues

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* add test case, autolog() should pass w/o scipy installed

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* minor cleanup

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* test creating _Example with / without scipy present

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* docs: do not warn about sparse matrix classes

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* linting

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* address review comment

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* run black

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* remove redundant None check after import

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* delete comment

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* Revert "remove redundant None check after import"

This reverts commit f2e198c944f41d9a96c20ae471d71bff39431b48.

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* review comments

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* remove redundant test

Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* Apply suggestions from code review

Co-authored-by: Harutaka Kawamura <hkawamura0130@gmail.com>
Signed-off-by: Hannes Schulz <hannes.schulz@microsoft.com>

* Autoformat: https://github.com/mlflow/mlflow/actions/runs/2779791161

Signed-off-by: mlflow-automation <mlflow-automation@users.noreply.github.com>

Co-authored-by: Harutaka Kawamura <hkawamura0130@gmail.com>
Co-authored-by: dbczumar <corey.zumar@databricks.com>
Co-authored-by: mlflow-automation <mlflow-automation@users.noreply.github.com>

* Fix extraneous warning about pyspark evaluation truncation (#6389)

* Revert #6380 (#6386)

* revert

Signed-off-by: harupy <hkawamura0130@gmail.com>

* workaround for test_prometheus_exporter

Signed-off-by: harupy <hkawamura0130@gmail.com>

* nit

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Validate metric value in `FileStore.log_metric` (#6381)

* add failing test

Signed-off-by: harupy <hkawamura0130@gmail.com>

* use _validate_metric

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix invalid metric timestamp

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix docstring

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix java test

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Make search experiment by tag key comparison case sensitive in MYSQL (#6333)

* init

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* add test

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* update

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* update

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* fix BINARY conditions in MySQL

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix attr error

Signed-off-by: harupy <hkawamura0130@gmail.com>

Co-authored-by: harupy <hkawamura0130@gmail.com>

* Run Model Registry SQLAlchemy tests in PostgreSQL, MySQL, and MSSQL (#6390)

* Run Model Registry SQLAlchemy tests in postgres, mysql, and mssql

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix failed tests

Signed-off-by: harupy <hkawamura0130@gmail.com>

* remove uuid

Signed-off-by: harupy <hkawamura0130@gmail.com>

* clean up

Signed-off-by: harupy <hkawamura0130@gmail.com>

* do not drop tables

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Fix filestore data loss on failed write & mitigate read / write race condition (#6388)

* Fix

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Fix

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Format

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Added a custom filter from_json to render_and_merge_yaml (#6368)

* Added a custom filter from_json to render_and_merge_yaml

Signed-off-by: Jin Zhang <jin.zhang@databricks.com>

* Fixed test

Signed-off-by: Jin Zhang <jin.zhang@databricks.com>

* Update tests/utils/test_file_utils.py

Co-authored-by: Siddharth Murching <sid.murching@databricks.com>

* Update tests/utils/test_file_utils.py

Co-authored-by: Siddharth Murching <sid.murching@databricks.com>

* Fixed test

Signed-off-by: Jin Zhang <jin.zhang@databricks.com>

* Fixed windows test attempt

Signed-off-by: Jin Zhang <jin.zhang@databricks.com>

* Fixed windows test

Signed-off-by: Jin Zhang <jin.zhang@databricks.com>

Co-authored-by: Siddharth Murching <sid.murching@databricks.com>

* Fix `!=` operator for MySQL (#6397)

* fix ne operator and improve tests

Signed-off-by: harupy <hkawamura0130@gmail.com>

* add () to other operators

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Simplify the documentation section in the PR template (#6398)

* Simplify the documentation section in the PR template

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Timebased gc (#6354)

* Added delete_time field to runs table

Signed-off-by: Jason Cheng <jasoncky96@gmail.com>

* Updated latest_schema.sql to pass tests/store/tracking/test_sqlalchemy_store_schema.py

Signed-off-by: Jason Cheng <jasoncky96@gmail.com>

* Added delete_time column into mlflow/store/tracking/dbmodels/initial_models.py

Signed-off-by: Jason Cheng <jasoncky96@gmail.com>

* Moved the docstring to the end of the declaration instead of the start

Signed-off-by: Jason Cheng <jasoncky96@gmail.com>

* Regenerated protobuf for RunInfo new schema, and passed pytest

Signed-off-by: Jason Cheng <jasoncky96@gmail.com>

* Added --older-than flag in mlflow gc command, and added tests for that cli command

Signed-off-by: Jason Cheng <jasoncky96@gmail.com>

* Removed delete_time from RunInfo proto

Signed-off-by: Jason Cheng <jasoncky96@gmail.com>

* Set default older-than flag to None

Signed-off-by: Jason Cheng <jasoncky96@gmail.com>

* Remove delete_time from RunInfo, add optional parameter older_than to store.get_deleted_runs(), and write delete_time into yaml file for file storage instead of adding it to RunInfo

Signed-off-by: Jason Cheng <jasoncky96@gmail.com>

* Remove commented delete_time in RunInfo constructor in models.py

Signed-off-by: Jason Cheng <jasoncky96@gmail.com>

* added INVALID_PARAMETER_VALUE error code in mlflow gc

Signed-off-by: Jason Cheng <jasoncky96@gmail.com>

* Resolve case of delete time not in meta.yaml file, edited overwrite_run_info and gc

Signed-off-by: Jason Cheng <jasoncky96@gmail.com>

* Rename delete_time to deleted_time

Signed-off-by: Jason Cheng <jasoncky96@gmail.com>

* Rename delete_time to deleted_time, match error message in cli tests, filter SqlRuns by deleted_time, indicate float values accepted for gc --older-than flag

Signed-off-by: Jason Cheng <jasoncky96@gmail.com>

* Rename delete_time to deleted_time, fixed assert statement in test_mlflow_gc, and fixed deleted_time queries in test_file_store.py and test_sqlalchemy_store.py

Signed-off-by: Jason Cheng <jasoncky96@gmail.com>

* Only check if  is in meta after restoring run in test_delte_restore_run

Signed-off-by: Jason Cheng <jasoncky96@gmail.com>

* Fixed lint errors

Signed-off-by: Jason Cheng <jasoncky96@gmail.com>

* Install mlflow from repository root

Signed-off-by: harupy <hkawamura0130@gmail.com>

Co-authored-by: harupy <hkawamura0130@gmail.com>

* Separate protos job (#6399)

* Separate protos job

Signed-off-by: harupy <hkawamura0130@gmail.com>

* test

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Add paths

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Revert "test"

This reverts commit 5d04b02f55ec92e1e3ac18eb651ea989ca0d469a.

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Fix flaky `TestFileStore.test_get_deleted_runs` (#6402)

* use gt operator

Signed-off-by: harupy <hkawamura0130@gmail.com>

* fix sqlalchemy store

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Migrate environment variables for SQLAlchemy store (#6396)

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Deduplicate MlflowClient in MLflow docs, move to mlflow.client module (#6405)

* Dedupe MlflowClient

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Improvements

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Add client

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* Remove unused functions in `mlflow.data` module (#6408)

* remove unused functions in data

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Remove unused functions and variables

Signed-off-by: harupy <hkawamura0130@gmail.com>

* Tag based registered model search and model version search (#6320)

* init

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* update

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* update

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* update

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* update

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* update

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* update

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* update

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* update

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* update

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* update

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* update tests

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* update

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* update

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* clean

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* address comments

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* add test

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* address comments

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* update

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* de-dup tag based search result

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* add () to fix case_sensitive_mysql_ne

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* fix

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* lint

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* fix

* fix test in TODO

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* support dup tag key filter

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* add mv order by

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* cmp res list in test

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* address comments

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* address comments

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* add doc

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* fix

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* fix lint

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* address comments

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* address comments

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* update doc

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* Increase maximum param value length to 500 (#6358)

* increase validation of maximum param value length from 250 to 500 (squashed)

Signed-off-by: Jan Sindlar <jan.sindlar@eureka.ai>

* 1. added test for exceedin…
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/build Build and test infrastructure for MLflow rn/none List under Small Changes in Changelogs.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants