Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP
A tool for managing data processing in Python
Python Shell
branch: master

Fetching latest commit…

Cannot retrieve the latest commit at this time

Failed to load latest commit information.
doc
pypeline
test
.gitignore
.travis.yml
LICENSE
README.md
TODO.md
requirements-test.txt
run_tests.sh
setup.cfg
setup.py
upload.py

README.md

This project has been closed, and my development effort will be transitioned to dat. Dat has a similar mission, is also built on levelDB, and supports many features that pypeline doesn't have.

Pypeline DB

Pypeline DB is designed to simplify the creation and management of datasets. It has a friendly and easy-to-master API backed by the power of LevelDB. This allows it to manage datasets too large to fit in RAM without sacrificing data access performance.

Pypeline is great for:

  • Exploring data without eating all your RAM
  • Transforming data with maps, filters and reductions
  • Stopping you from losing or overwriting your data (unless you explicitly ask it to)

It's also easy to export a dataset from Pypeline to Pandas for further analysis.

Links

Something went wrong with that request. Please try again.