A set of tools for performing ETL tasks on the Minerva database platform
Python Makefile Shell PLpgSQL SaltStack Ruby
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
dispatcher
harvesting
importer-csv
materialize
node-dummy
node-harvest
node-materialize
node-transform
node
transform
.gitignore
README.md
set-version

README.md

Minerva ETL

A set of tools for performing ETL tasks on the Minerva database platform.

The main tools are:

  • Dispatcher - For picking up jobs (files) and placing them on the Minerva job queue.
  • Node - A generic worker process for picking up jobs from the Minerva job queue and processing them.
  • Node plugins - Different types of processing logic for different types of jobs.