Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update client to pass request headers created by all RequestHeaderProviders #4042

Merged
merged 14 commits into from
Feb 5, 2021

Conversation

jimmyxu-db
Copy link
Contributor

@jimmyxu-db jimmyxu-db commented Feb 1, 2021

What changes are proposed in this pull request?

In this pull request, we will pass all request headers created by RequestHeaderProviders in outgoing requests. This follows up #3946

How is this patch tested?

New unit tests

Release Notes

MLflow will now add all request headers from registered RequestHeaderProviders to outgoing requests.

Is this a user-facing change?

  • No. You can skip the rest of this section.
  • Yes. Give a description of this change to be included in the release notes for MLflow users.

All outgoing requests will now have request headers provided by all RequestHeaderProviders attached.

What component(s), interfaces, languages, and integrations does this PR affect?

Components

  • area/artifacts: Artifact stores and artifact logging
  • area/build: Build and test infrastructure for MLflow
  • area/docs: MLflow documentation pages
  • area/examples: Example code
  • area/model-registry: Model Registry service, APIs, and the fluent client calls for Model Registry
  • area/models: MLmodel format, model serialization/deserialization, flavors
  • area/projects: MLproject format, project running backends
  • area/scoring: Local serving, model deployment tools, spark UDFs
  • area/server-infra: MLflow server, JavaScript dev server
  • area/tracking: Tracking Service, tracking client APIs, autologging

Interface

  • area/uiux: Front-end, user experience, JavaScript, plotting
  • area/docker: Docker use across MLflow's components, such as MLflow Projects and MLflow Models
  • area/sqlalchemy: Use of SQLAlchemy in the Tracking Service or Model Registry
  • area/windows: Windows support

Language

  • language/r: R APIs and clients
  • language/java: Java APIs and clients
  • language/new: Proposals for new client languages

Integrations

  • integrations/azure: Azure and Azure ML integrations
  • integrations/sagemaker: SageMaker integrations
  • integrations/databricks: Databricks integrations

How should the PR be classified in the release notes? Choose one:

  • rn/breaking-change - The PR will be mentioned in the "Breaking Changes" section
  • rn/none - No description will be included. The PR will be mentioned only by the PR number in the "Small Bugfixes and Documentation Updates" section
  • rn/feature - A new user-facing feature worth mentioning in the release notes
  • rn/bug-fix - A user-facing bug fix worth mentioning in the release notes
  • rn/documentation - A user-facing documentation change worth mentioning in the release notes

@jimmyxu-db jimmyxu-db force-pushed the ML-13222 branch 2 times, most recently from e9bf15b to 369698a Compare February 2, 2021 22:42
@github-actions github-actions bot added area/tracking Tracking service, tracking client APIs, autologging rn/feature Mention under Features in Changelogs. labels Feb 2, 2021
@jimmyxu-db jimmyxu-db changed the title Ml 13222 Update client to pass request headers created by all RequestHeaderProviders Feb 2, 2021
headers = dict(_DEFAULT_HEADERS)
from mlflow.tracking.request_header.registry import resolve_request_headers

headers = dict({**_DEFAULT_HEADERS, **resolve_request_headers()})
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a way we can easily unit test the addition of these provider headers to a request?

Copy link
Collaborator

@dbczumar dbczumar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jimmyxu-db This is awesome! Do you have a manual test plan that you can share (i.e. one that tries this out on Databricks)?

Signed-off-by: Jimmy Xu <jimmy.xu@databricks.com>
Signed-off-by: Jimmy Xu <jimmy.xu@databricks.com>
Signed-off-by: Jimmy Xu <jimmy.xu@databricks.com
Signed-off-by: Jimmy Xu <jimmy.xu@databricks.com>
Signed-off-by: Jimmy Xu <jimmy.xu@databricks.com>
Signed-off-by: Jimmy Xu <jimmy.xu@databricks.com>
Signed-off-by: Jimmy Xu <jimmy.xu@databricks.com>
Signed-off-by: Jimmy Xu <jimmy.xu@databricks.com>
Signed-off-by: Jimmy Xu <jimmy.xu@databricks.com>
Signed-off-by: Jimmy Xu <jimmy.xu@databricks.com>
Signed-off-by: Jimmy Xu <jimmy.xu@databricks.com>
Signed-off-by: Jimmy Xu <jimmy.xu@databricks.com>
Signed-off-by: Jimmy Xu <jimmy.xu@databricks.com>
Copy link
Collaborator

@dbczumar dbczumar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM once lint failures are addressed. Thanks @jimmyxu-db !

Signed-off-by: Jimmy Xu <jimmy.xu@databricks.com>
@jimmyxu-db jimmyxu-db merged commit 83c55be into mlflow:master Feb 5, 2021
eedeleon pushed a commit to eedeleon/mlflow that referenced this pull request Feb 5, 2021
…viders (mlflow#4042)

* added request header provider registry

Signed-off-by: Jimmy Xu <jimmy.xu@databricks.com>

* added registry tests

Signed-off-by: Jimmy Xu <jimmy.xu@databricks.com>

* updated rest_utils

Signed-off-by: Jimmy Xu <jimmy.xu@databricks.com

* lint

Signed-off-by: Jimmy Xu <jimmy.xu@databricks.com>

* fixed circular import

Signed-off-by: Jimmy Xu <jimmy.xu@databricks.com>

* fixed dict syntax

Signed-off-by: Jimmy Xu <jimmy.xu@databricks.com>

* f-string

Signed-off-by: Jimmy Xu <jimmy.xu@databricks.com>

* f-string

Signed-off-by: Jimmy Xu <jimmy.xu@databricks.com>

* lint

Signed-off-by: Jimmy Xu <jimmy.xu@databricks.com>

* lint

Signed-off-by: Jimmy Xu <jimmy.xu@databricks.com>

* lint

Signed-off-by: Jimmy Xu <jimmy.xu@databricks.com>

* removed unused param

Signed-off-by: Jimmy Xu <jimmy.xu@databricks.com>

* added test to rest_utils

Signed-off-by: Jimmy Xu <jimmy.xu@databricks.com>

* lint

Signed-off-by: Jimmy Xu <jimmy.xu@databricks.com>
aloosley pushed a commit to aloosley/mlflow that referenced this pull request Feb 9, 2021
…viders (mlflow#4042)

* added request header provider registry

Signed-off-by: Jimmy Xu <jimmy.xu@databricks.com>

* added registry tests

Signed-off-by: Jimmy Xu <jimmy.xu@databricks.com>

* updated rest_utils

Signed-off-by: Jimmy Xu <jimmy.xu@databricks.com

* lint

Signed-off-by: Jimmy Xu <jimmy.xu@databricks.com>

* fixed circular import

Signed-off-by: Jimmy Xu <jimmy.xu@databricks.com>

* fixed dict syntax

Signed-off-by: Jimmy Xu <jimmy.xu@databricks.com>

* f-string

Signed-off-by: Jimmy Xu <jimmy.xu@databricks.com>

* f-string

Signed-off-by: Jimmy Xu <jimmy.xu@databricks.com>

* lint

Signed-off-by: Jimmy Xu <jimmy.xu@databricks.com>

* lint

Signed-off-by: Jimmy Xu <jimmy.xu@databricks.com>

* lint

Signed-off-by: Jimmy Xu <jimmy.xu@databricks.com>

* removed unused param

Signed-off-by: Jimmy Xu <jimmy.xu@databricks.com>

* added test to rest_utils

Signed-off-by: Jimmy Xu <jimmy.xu@databricks.com>

* lint

Signed-off-by: Jimmy Xu <jimmy.xu@databricks.com>
harupy pushed a commit to chauhang/mlflow that referenced this pull request Apr 8, 2021
…viders (mlflow#4042)

* added request header provider registry

Signed-off-by: Jimmy Xu <jimmy.xu@databricks.com>

* added registry tests

Signed-off-by: Jimmy Xu <jimmy.xu@databricks.com>

* updated rest_utils

Signed-off-by: Jimmy Xu <jimmy.xu@databricks.com

* lint

Signed-off-by: Jimmy Xu <jimmy.xu@databricks.com>

* fixed circular import

Signed-off-by: Jimmy Xu <jimmy.xu@databricks.com>

* fixed dict syntax

Signed-off-by: Jimmy Xu <jimmy.xu@databricks.com>

* f-string

Signed-off-by: Jimmy Xu <jimmy.xu@databricks.com>

* f-string

Signed-off-by: Jimmy Xu <jimmy.xu@databricks.com>

* lint

Signed-off-by: Jimmy Xu <jimmy.xu@databricks.com>

* lint

Signed-off-by: Jimmy Xu <jimmy.xu@databricks.com>

* lint

Signed-off-by: Jimmy Xu <jimmy.xu@databricks.com>

* removed unused param

Signed-off-by: Jimmy Xu <jimmy.xu@databricks.com>

* added test to rest_utils

Signed-off-by: Jimmy Xu <jimmy.xu@databricks.com>

* lint

Signed-off-by: Jimmy Xu <jimmy.xu@databricks.com>
Signed-off-by: harupy <17039389+harupy@users.noreply.github.com>
harupy pushed a commit to wamartin-aml/mlflow that referenced this pull request Jun 7, 2021
…viders (mlflow#4042)

* added request header provider registry

Signed-off-by: Jimmy Xu <jimmy.xu@databricks.com>

* added registry tests

Signed-off-by: Jimmy Xu <jimmy.xu@databricks.com>

* updated rest_utils

Signed-off-by: Jimmy Xu <jimmy.xu@databricks.com

* lint

Signed-off-by: Jimmy Xu <jimmy.xu@databricks.com>

* fixed circular import

Signed-off-by: Jimmy Xu <jimmy.xu@databricks.com>

* fixed dict syntax

Signed-off-by: Jimmy Xu <jimmy.xu@databricks.com>

* f-string

Signed-off-by: Jimmy Xu <jimmy.xu@databricks.com>

* f-string

Signed-off-by: Jimmy Xu <jimmy.xu@databricks.com>

* lint

Signed-off-by: Jimmy Xu <jimmy.xu@databricks.com>

* lint

Signed-off-by: Jimmy Xu <jimmy.xu@databricks.com>

* lint

Signed-off-by: Jimmy Xu <jimmy.xu@databricks.com>

* removed unused param

Signed-off-by: Jimmy Xu <jimmy.xu@databricks.com>

* added test to rest_utils

Signed-off-by: Jimmy Xu <jimmy.xu@databricks.com>

* lint

Signed-off-by: Jimmy Xu <jimmy.xu@databricks.com>
Signed-off-by: harupy <17039389+harupy@users.noreply.github.com>
@afparsons
Copy link

afparsons commented Jun 29, 2021

Hello; this might be related: #4507
Never mind!

@ddl-ebrown
Copy link

Has any thought been given to implementing an equivalent for the R client? I see no way to easily modify outgoing requests in https://github.com/mlflow/mlflow/blob/master/mlflow/R/mlflow/R/tracking-rest.R#L34

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/tracking Tracking service, tracking client APIs, autologging rn/feature Mention under Features in Changelogs.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants