Add backoff jitter

[ajinkyavbhandare]

🛠 DevTools 🛠

Install mlflow from this PR

pip install git+https://github.com/mlflow/mlflow.git@refs/pull/10486/merge

Checkout with GitHub CLI

gh pr checkout 10486

Related Issues/PRs

Resolve #10482

What changes are proposed in this pull request?

This PR adds a backoff jitter to HTTP requests in MLflow. This is achieved by introducing a new environment variable MLFLOW_HTTP_REQUEST_BACKOFF_JITTER and using this variable when making HTTP requests.

How is this PR tested?

• Existing unit/integration tests
• New unit/integration tests
• Manual tests

Does this PR require documentation update?

• No. You can skip the rest of this section.
• Yes. I've updated:
  • Examples
  • API references
  • Instructions

Release Notes

Is this a user-facing change?

• No. You can skip the rest of this section.
• Yes. Give a description of this change to be included in the release notes for MLflow users.

Yes. This change introduces a backoff jitter to HTTP requests, which can help improve the efficiency and reliability of the requests.

What component(s), interfaces, languages, and integrations does this PR affect?

Components

• area/artifacts: Artifact stores and artifact logging
• area/build: Build and test infrastructure for MLflow
• area/docs: MLflow documentation pages
• area/examples: Example code
• area/gateway: AI Gateway service, Gateway client APIs, third-party Gateway integrations
• area/model-registry: Model Registry service, APIs, and the fluent client calls for Model Registry
• area/models: MLmodel format, model serialization/deserialization, flavors
• area/recipes: Recipes, Recipe APIs, Recipe configs, Recipe Templates
• area/projects: MLproject format, project running backends
• area/scoring: MLflow Model server, model deployment tools, Spark UDFs
• area/server-infra: MLflow Tracking server backend
• area/tracking: Tracking Service, tracking client APIs, autologging

Interface

• area/uiux: Front-end, user experience, plotting, JavaScript, JavaScript dev server
• area/docker: Docker use across MLflow's components, such as MLflow Projects and MLflow Models
• area/sqlalchemy: Use of SQLAlchemy in the Tracking Service or Model Registry
• area/windows: Windows support

Language

• language/r: R APIs and clients
• language/java: Java APIs and clients
• language/new: Proposals for new client languages

Integrations

• integrations/azure: Azure and Azure ML integrations
• integrations/sagemaker: SageMaker integrations
• integrations/databricks: Databricks integrations

How should the PR be classified in the release notes? Choose one:

• rn/none - No description will be included. The PR will be mentioned only by the PR number in the "Small Bugfixes and Documentation Updates" section
• rn/breaking-change - The PR will be mentioned in the "Breaking Changes" section
• rn/feature - A new user-facing feature worth mentioning in the release notes
• rn/bug-fix - A user-facing bug fix worth mentioning in the release notes
• rn/documentation - A user-facing documentation change worth mentioning in the release notes

[github-actions]

Documentation preview for 33fcf70 will be available here when this CircleCI job completes successfully.

More info

• Ignore this comment if this PR does not change the documentation.
• It takes a few minutes for the preview to be available.
• The preview is updated when a new commit is pushed to this PR.
• This comment was created by https://github.com/mlflow/mlflow/actions/runs/6963900977.

[harupy]

@ajinkyavbhandare why you closed it?

[ajinkyavbhandare]

hello @harupy ,I made some changes to the code for the and created a new pull request, #10486. Could you please review the new pull request and reassign me the task? thank you.

[harupy]

where is the new PR?

[harupy]

@ajinkyavbhandare what's the difference?

[harupy]

we can keep working on this one.

[harupy]

let me push a commit.

[harupy]

In the original code, max_retries = 0 defaults to MLFLOW_HTTP_REQUEST_MAX_RETRIES.get() but it should not.

[harupy]

LGTM

[harupy]

Manually tested using this toy fastapi app:

from fastapi import FastAPI, HTTPException, status


app = FastAPI()


counter = 0


@app.get("/test")
def test():
    global counter
    counter += 1

    if counter < 3:
        raise HTTPException(
            status_code=status.HTTP_502_BAD_GATEWAY,
            detail="Service unavailable",
        )

    else:
        # Reset counter
        counter = 0
        return {"counter": counter}


# Command to run this app using uvicorn:
# uvicorn app:app --reload --port 8000

[harupy]

@ajinkyavbhandare Thanks for the contribution!

[ajinkyavbhandare]

@harupy Thank you , I'm grateful for the opportunity to be a part of this project.