Skip to content

Dockerized Apache Airflow 2 with Anaconda. Perfect marriage for data-science ETL projects.

Notifications You must be signed in to change notification settings

andgineer/airflow-conda

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

24 Commits
 
 
 
 
 
 
 
 

Repository files navigation

Docker Automated build

Dockerized Apache Airflow 2 with Anaconda

This repository offers a Docker container to run Airflow 2 on your local machine, using an Anaconda (miniconda3) environment.

It comes with handy libraries and tools like Pandas, PyArrow, and Celery, as well as adapters for Redis and Postgres.

Features

  • Airflow 2: Easy management and scaling with a modular setup and a message queue.
  • Anaconda Environment: Simplify setup and execution with Anaconda managing dependencies.
  • Faster with Mamba: Speed up dependency resolution by swapping conda with mamba in your commands.

Usage

Start the Container

docker run -p 8080:8080 andgineer/airflow-conda

Access Apache Airflow WebUI

Open the Apache Airflow web UI in your browser: localhost:8080/. See Apache Airflow UI docs for more.

Default Login

  • Username: admin
  • Password: admin

Airflow DAGs

The container initially sets up a local DB at /root/airflow/airflow.db and loads some tutorial examples from Apache Airflow.

For a more realistic environment, with a local DB and DAGs mounted into the container, see this example.

About

Dockerized Apache Airflow 2 with Anaconda. Perfect marriage for data-science ETL projects.

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published