DataDuck ETL - the extract-transform-load framework for data warehousing
Ruby HTML Shell
Latest commit aff3c77 Sep 5, 2016 @pickhardt pickhardt Updated README
Permalink
Failed to load latest commit information.
bin Add the quickstart command. Sep 28, 2015
docs
examples/example Include tables in example Oct 8, 2015
lib Add Airbrake integration Sep 5, 2016
spec Add Airbrake integration Sep 5, 2016
static Updated README Sep 5, 2016
.gitignore Update spec version to 1.0.0 Jun 7, 2016
.rspec Committing files Sep 21, 2015
DEV_README.md Switch to mysql2 gem because it is supposed to be faster and it shoul… Oct 28, 2015
Gemfile Committing files Sep 21, 2015
README.md Updated README Sep 5, 2016
Rakefile
dataduck.gemspec Add Airbrake integration Sep 5, 2016

README.md

DataDuck ETL

Set up in under 5 minutes

DataDuck ETL is probably the quickest extract-transform-load framework system to set up. If you want to set up a data warehouse, give DataDuck ETL a try.

Extract-transform-load to Amazon Redshift

DataDuck ETL is currently focused on loading to Amazon Redshift (through Amazon S3).

DataDuck ETL

Installation

Instructions for using DataDuck ETL

Create a new, empty directory. Inside this directory, create a file named Gemfile, and add the following to it:

source 'https://rubygems.org'

gem 'dataduck'

Then execute:

$ bundle install

Finally, run the quickstart command:

$ dataduck quickstart

The quickstart wizard will ask you for credentials to your database, then create the basic setup for your project. After the quickstart, you'll still need to edit the .env and config/base.yml files, as well as customize your tables. Finally, your project's ETL can be run by running the dataduck etl all command.

If you'd like to run this regularly, such as every night, it's recommended to use the whenever gem to manage a cron job to regularly run the ETL.

Documentation

Visit the docs page to read the documentation. The docs page is autogenerated from the files in this project's docs directory.

Third Party Integrations

There are many third party integrations, including Optimizely, SEMrush, Sentry, and Airbrake. Visit the docs page to learn more about integrating.

Contributing

To contribute, get in touch at http://DataDuckETL.com/ so that we can share the Contributor License Agreement (CLA) with you, then create a pull request.

License

You can buy a license directly from Supported Source at https://supportedsource.org/projects/dataduck-etl

You're also welcome to get in touch at http://dataducketl.com/ for invoicing or anything else.