Skip to content

flmu/mlflow-tracking-server

Repository files navigation

MLFLow Tracking Server Based on Docker and AWS S3

Build Status Code Score

This repo provides a docker image of MLFLow Tracking Server which is based on sqlite, an internal file system for metadata (e.g. parameters, metrics) and an AWS S3 Bucket for files and artifacts.

Architecture

Prerequisites

Before you start the MLFlow Tracking Server, you must create an AWS S3 Bucket and the corresponding credentials.

  1. AWS Account
  2. AWS CLI
  3. Create a new AWS S3 Bucket
  4. Add a new AWS User to to the system
  5. Assign the following policy to the user:
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": [
                "s3:ListAllMyBuckets",
                "s3:HeadBucket"
            ],
            "Resource": "*"
        },
        {
            "Sid": "VisualEditor1",
            "Effect": "Allow",
            "Action": "s3:*",
            "Resource": [
                "arn:aws:s3:::<_AWS_BUCKET_NAME_>",
                "arn:aws:s3:::<_AWS_BUCKET_NAME_>/*"
            ]
        }
    ]
}

Run the MLFlow Tracking Server

$ docker run \
    --rm \
    --name mlflow-tracking-server \
    -p 5000:5000 \
    -e PORT=5000 \
    -e FILE_DIR=/mlflow \
    -e AWS_BUCKET=<YOUR_AWS_BUCKET> \
    -e AWS_ACCESS_KEY_ID=<YOUR_AWS_ACCESS_KEY_ID> \
    -e AWS_SECRET_ACCESS_KEY=<YOUR_AWS_SECRET_ACCESS_KEY> \
    foxrider/mlflow-tracking-server:0.2.0

Access to http://127.0.0.1:5000

Environment Variables for the MLFlow Tracking Server

Required

Key Description
FILE_DIR Directory for artifacts and metadata (e.g. parameters, metrics)
AWS_BUCKET Name of AWS S3 Bucket that will contain the artifacts
AWS_ACCESS_KEY_ID AWS-Access-Key that you have created in the Prerequisite section
AWS_SECRET_ACCESS_KEY AWS-Secret-Access-Key that you have created in the Prerequisite section

Optional

Key Description Default
PORT Value for listen directive 5000

Test the MLFlow Tracking Server

Prerequisite

  1. Install the AWS CLI
  2. Add the AWS credentials to the client. You can either set the environment variables AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY on your client or use the AWS CLI command aws configure.

Simple Python Script

See this link

Contribution

Pull requests are welcome :)

Project Initiator

Florian Muchow

Buy Me A Coffee

Contributors

Thanks to all the contributors