A simple ETL project that fetches 10 records and loads them into a postgres database representing our data warehouse.
The Purpose of this simple project is to help on setting up a local development environment for future data projects using Docker
To run the code, you will need
- Docker and Docker Compose(v1.27.0 or later)
- git
Clone the git repo and run the ETL as shown below.
git clone https://github.com/josephmachado/local_dev.git
cd local_dev
make up
make ci # run tests and format code
make run-etl # run the ETL process
make down # spins down the containers
For more details, please read the post: Setting up local dev environment with Docker.