Example project using Tasks as Containers architecture
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
tac
Dockerfile
README.md
requirements.txt
setup.py

README.md

Task as Containers example project

Details

Please read the detailed article at Data Revenue's Blog.

How to run

  • Clone the repo and cd into its root.
  • Install requirements and the project itself.
  • Export your AWS credentials into AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY environment variables.
  • Export name of your S3 bucket name into S3_BUCKET env variable.
  • Spin up a Minikube cluster.
  • Build a docker image inside Minikube VM:
eval $(minikube docker-env)
docker build -t tac-example:v1 . --build-arg AWS_ACCESS_KEY_ID=$AWS_ACCESS_KEY_ID --build-arg AWS_SECRET_ACCESS_KEY=$AWS_SECRET_ACCESS_KEY --build-arg S3_BUCKET=$S3_BUCKET
  • Run the pipeline:
luigi --module tac.task MakePredictions --date 2018-01-01