Skip to content
(Hopefully) The fastest way to integrate Machine Learning into new or existing solutions
Branch: master
Clone or download
pdxjohnny CHANGELOG: Add Standardization of API to changes
Signed-off-by: John Andersen <>
Latest commit 8e224b4 May 21, 2019
Type Name Latest commit message Commit time
Failed to load latest commit information.
.github/ISSUE_TEMPLATE community: Create issue templates May 9, 2019
dffml feature: git: Remove non-data-flow features May 21, 2019
docs feature: auth: Example of thread pool usage May 21, 2019
model/tensorflow model: tensorflow: Update to new model API May 21, 2019
scripts scripts: skel: Update model May 21, 2019
.coveragerc release: Initial Open Source Release Mar 8, 2019
.dockerignore release: Initial Open Source Release Mar 8, 2019
.gitattributes release: Initial Open Source Release Mar 8, 2019
.gitignore df: Added Data Flow Facilitator Mar 29, 2019
.pylintrc release: Initial Open Source Release Mar 8, 2019
.travis.yml ci: Download tokei every time May 21, 2019 CHANGELOG: Add Standardization of API to changes May 21, 2019 README: Add link to May 10, 2019
Dockerfile release: Initial Open Source Release Mar 8, 2019
LICENSE release: Initial Open Source Release Mar 8, 2019 README: Add gitter badge May 15, 2019
pyproject.toml source: Use standardized pattern May 21, 2019

Data Flow Facilitator for Machine Learning (dffml)

Build Status codecov CII Gitter chat

DFFML provides APIs for dataset generation and storage, and model definition using any machine learning framework, from high level down to low level use is supported.

The goal of DFFML is to build a community driven library of plugins for dataset generation and model definition. So that we as developers and researchers can quickly and easily plug and play various pieces of data with various model implementations.

Here's a quick demo showing how DFFML can be used to train on the iris dataset. The more we build up the library of plugins (which anyone can maintain, they don't have to be contributed upstream unless you want to) the more variations on model implementations and feature data generators we all have to work with.


Right now we've released a wrapper around the Tensorflow DNN estimator, and a set of feature generators which gather data from git repositories.


DFFML currently should work with Python 3.6. However, only Python 3.7 is officially supported. This is because there are a lot of nice helper methods Python 3.7 implemented that we intend to use instead of re-implementing.

python3.7 -m pip install -U dffml

You can also install the Features for Git Version Control, and Models for Tensorflow Library all at once.

If you want a quick how to on the iris dataset head to the DFFML Models for Tensorflow Library repo.

python3.7 -m pip install -U dffml[git,tensorflow]

If you don't have Python 3.7 we have a docker image for you, or you can install pyenv which will quickly and easily give you Python 3.7. See docs/ for more details.


To start using dffml for data set generation with a single CLI command see DFFML Features for Git Version Control.

To start using dffml for machine learning with a few CLI commands see DFFML Models for Tensorflow Library.


Start with Architecture.


DFFML is meant to be a community driven application. There are various segments you can help with:

  • Found a bug, error in the docs or have a new idea, create an issue here.
  • Help us fix an issue, refer to HACKING in the docs.
  • Make sure you go through before contributing.


Got an idea for a new feature/model, tutorials will help you write code that takes full advantage of the DFFML API. Making your next machine learning project a breeze to write!

  • Features: The new feature tutorial will walk you through how to write a new DFFML feature to generate new data for a dataset.
  • Models: The new model tutorial will walk you through how to wrap your favorite framework or a custom implementation in the DFFML library's model API.


dffml is distributed under the MIT License.


This software is subject to the U.S. Export Administration Regulations and other U.S. law, and may not be exported or re-exported to certain countries (Cuba, Iran, Crimea Region of Ukraine, North Korea, Sudan, and Syria) or to persons or entities prohibited from receiving U.S. exports (including Denied Parties, Specially Designated Nationals, and entities on the Bureau of Export Administration Entity List or involved with missile technology or nuclear, chemical or biological weapons).

You can’t perform that action at this time.