A simple transformation/data processing pipeline for CrisisNET
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
docs
etc
src
tests
.gitignore
README.md
fabfile.py
requirements.txt

README.md

Grimlock

"Me, Grimlock."

A simple transformation/data processing pipeline for CrisisNET. Pulls jobs from a FIFO queue and runs each through a series of predefined tasks. These tasks add metadata to content retrieved by Sucka.

New tasks can be added to the tasks package, and included in the pipeline in app.set_pipeline_steps.

Very much a work in progress. Watch this space.