AWS MLOps Handson

This repository is designed to provide a comprehensive ML infrastructure for CTR (Click-Through Rate) prediction. With a focus on AWS services, this repository offer practical learning experience for MLOps.

This repository is based on nsakki55/aws-mlops-handson(2023). I have made some version upgrades and bug fixes and have published this repository with the consent of nsakki55.

Slide[japanese]: https://speakerdeck.com/tyaba/mlops-handson

Key Features

Python Development Environment

We guide you through setting up a Python development environment that ensures code quality and maintainability. This environment is carefully configured to enable efficient development practices and facilitate collaboration.

Train Pipeline

This repository includes the implementation of a training pipeline. This pipeline covers the stages, including data preprocessing, model training, and evaluation.

Prediction Server

This repository provides an implementation of a prediction server that serves predictions based on your trained CTR prediction model.

AWS Deployment

To showcase industry-standard practices, this repository guide you in deploying the training pipeline and inference server on AWS.

AWS Infra Architecture

AWS Infra Architecture made by this repository.

ML Pipeline

Predict Server

Requirements

Software	Install (Mac)
pyenv	brew install pyenv
Poetry	curl -sSL https://install.python-poetry.org | python3 -
direnv	brew install direnv
Terraform	brew install terraform
Docker	install via dmg
awscli	curl "https://awscli.amazonaws.com/AWSCLIV2.pkg" -o "AWSCLIV2.pkg"

Setup

Install Python Dependencies

Use pyenv to install Python 3.11.7 environment

$ pyenv install 3.11.7
$ pyenv local 3.11.7

Use poetry to install library dependencies

$ poetry install

Configure environment variable

Use direnv to configure environment variable

$ cp .env.sample .env
$ direnv allow .

Usage

Build ML Pipeline

$ make build-ml

Run ML Pipeline

$ make run-ml

Build Predict API

$ make build-predictor

Run Predict API locally

$ make up

Shutdown Predict API locally

$ make down

Run formatter

$ make format

Run linter

$ make lint

Run pytest

$ make test

Show options

$ make help