snorkel is a real time data analysis tool, meant to answer questions about the now. At its heart, snorkel is built on the idea that iterative analysis is key to understanding. In other words - when graphs move, it should be easy to drill down on what is happening and why, not a frustrating exercise in adjusting failed queries.
Using sybil as a backend, snorkel can support querying datasets with tens of millions of samples in realtime.
snorkel-lite is a re-implementation of snorkel in python, to read more about why we moved to python from nodejs, see the motivation docs
June 2019
Initial E2E testing with cypress has been setup, now CircleCI will run e2e tests on snorkel.
Map View has been added, but it is only python2 compatible for now.
Feb 2019
snorkel-lite is now in Alpha status. All but one view (map view) has been ported over from snorkel, data ingestion works and grafana endpoints are compatible. I have switched over from using the original snorkel to using snorkel-lite since November of 2018.
Sept. 2018
Development has started on porting snorkel to python.
The simplest way to deploy snorkel will be to use the snorkel-lite pypi package and the instructions on the wiki.
To develop on snorkel, clone snorkel and get going:
git clone git@github.com:logv/snorkel.git
cd snorkel
for python2:
make virtualenv2
source dev2/bin/activate
for python3:
make virtualenv3
source dev3/bin/activate
to finish installation:
make setup
make run
motivation and features
grafana snorkel datasource is a plugin for grafana (3.0+) to query snorkel. While this plugin is in early stages, grafana is quite amazing.