Data transformations for the ML era
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
.circleci
.github Improved the issue templates (#231) Apr 24, 2018
blurr
docs
scripts Renaming DTC to BTS across the project (#308) May 31, 2018
tests Add support to sanitize field after evaluation. Make sure that dateti… Jun 4, 2018
.coveralls.yml
.gitignore
CNAME Docs v1 for review Mar 28, 2018
LICENSE Docs v1 for review (#79) Mar 30, 2018
Pipfile Removed extra space Jun 1, 2018
Pipfile.lock Updated pipenv lock file. May 31, 2018
README.md
build.sh dynamodb is downloaded May 9, 2018
mkdocs.yml
setup.cfg
setup.py

README.md

Blurr

CircleCI Documentation Status Coverage Status PyPI version

Table of contents

What is Blurr?

Blurr transforms structured, streaming raw data into features for model training and prediction using a high-level expressive YAML-based language called the Blurr Transform Spec (BTS). The BTS merges the schema and computation model for data processing.

The BTS is a data transform definition for structured data. The BTS encapsulates the business logic of data transforms and Blurr orchestrates the execution of data transforms. Blurr is runner-agnostic, so BTSs can be run by event processors such as Spark, Spark Streaming or Flink.

Is Blurr for you?

Yes, if: you are well on your way on the ML 'curve of enlightenment', and are thinking about how to do online scoring

Curve

Playground

Launch playground

Tutorial and Docs

Coming up with features is difficult, time-consuming, requires expert knowledge. 'Applied machine learning' is basically feature engineering --- Andrew Ng

Read the docs

Streaming BTS Tutorial | Window BTS Tutorial

Preparing data for specific use cases using Blurr:

Contribute to Blurr

Welcome to the Blurr community! We are so glad that you share our passion for building MLOps!

Please create a new issue to begin a discussion. Alternatively, feel free to pick up an existing issue!

Please sign the Contributor License Agreement before raising a pull request.

Data Science 'Joel Test'

Inspired by the (old school) Joel Test to rate software teams, here's our version for data science teams. What's your score?

  1. Data pipelines are versioned and reproducible
  2. Pipelines (re)build in one step
  3. Deploying to production needs minimal engineering help
  4. Successful ML is a long game. You play it like it is
  5. Kaizen. Experimentation and iterations are a way of life

Roadmap

Blurr is currently in Developer Preview. Stay in touch!: Star this project or email hello@blurr.ai

  • Local transformations only
  • Support for custom functions and other python libraries in the BTS
  • Spark runner
  • S3 support for data sink
  • DynamoDB as an Intermediate Store
  • Features server