This project seeks to provide a simple set of tools libraries for expressing and orchestrating Extract, Transform and Load solutions in Node.js.
honeybadger package is itself split into 3 packages:
honeybadger-etl, independent libraries for implementing ETL in Node.js Streams
honeybadger-service, core Node.js services, workers and scheduling for automated ETL solutions
honeybadger-admin, a browser-based Solution Designer, System Admin and Reporting for Honey Badger services.
- configuration for services and admin
- metadata caching
- running production stack
- VirtualBox (OSX)
- required to run docker-machine
- Docker Machine (OSX)
- running docker on OSX hosts
Whole Stack from Docker
For convenience you can run the HoneyBadger stack locally or in production via Docker. Work is being done to run/scale in Kubernetes for large scale production deployments.
docker run honeybadger/honeybadger:latest
Note: See Getting Started - this will not run CouchDB
In Your Own Services Stack
You can also run a headless version of the HoneyBadger Service engine within your existing stack. The Service engine provides the scheduled running of ETL solutions you have configured via either the admin or directly through JSON configs stored in CouchDB.
core Node.js services, workers and scheduling for automated ETL solutions.
In Your Own Application
HoneyBadger ETL libraries are nicely packaged for doing some low-level ETL within your own applications. You can directly consume the npm packages to provide simple ETL workflows to your application with robust support for various sources, transforms.
independent libraries for implementing ETL in Node.js Streams.
If you are wanting to run the complete solution to start your ETL workloads, you'll want to use the docker container approach. To make this extremely simple, this meta-package provides a method for automatically running CouchDB in a docker container, and firing up the HoneyBadger stack in another container. From this package, simply run:
*Note you must have docker installed properly.
Contributions are welcomed from the greater community via GitHub pull request. For questions or help contributing please create an issue.
You must have CouchDB installed and running locally. This is pretty easy to do. Any local development or locally running stack will require a local CouchDB.
Running Stack for Local Development
This package is a meta-package and is not where actual development is done. However, you can fire up the stack locally from this package to see it running before you start cloning the other repos.
You should be able to browse to [http://localhost:8090/admin] and see the HoneyBadger Admin Console.
Honey Badger is maintained under the Semantic Versioning guidelines.