Skip to content


Switch branches/tags

Name already in use

A tag already exists with the provided branch name. Many Git commands accept both tag and branch names, so creating this branch may cause unexpected behavior. Are you sure you want to create this branch?

Latest commit


Git stats


Failed to load latest commit information.
Latest commit message
Commit time

Jupyter Notebook Pipelines on Kubernetes using Bodywork


This repository contains a Bodywork project that demonstrates how to run a ML pipeline on Kubernetes, using Jupyter notebooks with Bodywork. The example pipeline has two stages, defined in two notebooks:

  1. train_model.ipynb - download data from an AWS S3 bucket, train a classifier and then uploaded it back to the same S3 bucket.
  2. score_data.ipynb - download the trained model from AWS S3, together with data needs to be scored, and then score the data and upload the results back to S3.

To run this project, follow the steps below.

Get Access to a Kubernetes Cluster

Use our Quickstart Guide to Kubernetes for MLOps to spin-up a local Minikube cluster in minutes.

Install the Bodywork Python Package

$ pip install bodywork

Run the ML Pipeline

$ bodywork create deployment

The orchestrator logs will be streamed to your terminal until the job has been successfully completed.

Running the ML Pipeline on a Schedule

If you're happy with the test results, you can schedule the workflow-controller to operate remotely on the cluster on a pre-defined schedule. For example, to setup the the workflow to run every hour, use the following command,

$ bodywork create cronjob \
    --name=jupyter-pipeline \
    --schedule="0 * * * *"

Each scheduled workflow will attempt to re-run the batch-job, as defined by the state of this repository's master branch at the time of execution.

To get the execution history for all jupyter-pipeline jobs use,

$ bodywork get cronjob jupyter-pipeline --history

Make this Project Your Own

This repository is a GitHub template repository that can be automatically copied into your own GitHub account by clicking the Use this template button above.

After you've cloned the template project, use official Bodywork documentation to help modify the project to meet your own requirements.