Skip to content
Python client for Pachyderm
Branch: master
Clone or download
Latest commit f1d34c9 Mar 20, 2019
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
docs/python_pachyderm
proto
src/python_pachyderm merge with original master Mar 20, 2019
tests
.bumpversion.cfg
.editorconfig Add editorconfig Jan 13, 2018
.gitignore Add test automation tools Dec 29, 2017
.gitmodules Move submodule for proto building Feb 15, 2018
.travis.yml
BUILD_NUMBER
CHANGELOG.rst
LICENSE Update readme, makefile, and proto helpers Feb 15, 2018
MANIFEST.in
Makefile Fix bug Mar 20, 2019
README.md
VERSION
setup.cfg
setup.py
tox.ini

README.md

Python Pachyderm

PyPI Package latest releasee Supported versions Slack Status

Official Python Pachyderm client. Created by kalugny (formerly kalugny/pypachy.)

See the API docs. Most of values are auto-generated from protobufs. It's generally easier to rely on the higher-level classes if they provide the functionality you need:

Installation

pip install python-pachyderm

Usage and options

All of the PFS functions used in pachctl are supported (almost) as-is.

There are some helper functions that help make things more pythonic:

  • commit which is a context manager wrapper for start_commit and finish_commit
  • get_files which supports getting the data from multiple files

Naming commits

All functions that accept a commit argument will accept a tuple of (repo, branch) or (repo, commit_id), a string like repo/branch or repo/commit_id and a Commit object.

e.g:

client.list_file(('my_repo', 'branch'), '/')  # tuple
client.list_file('my_repo/commit_id', '/')    # string
c = client.list_commit('my_repo')[0]          # get some commit
client.list_file(c, '/')                      # and use it directly

Basic usage example

import python_pachyderm
client = python_pachyderm.PfsClient()
client.create_repo('test')
with client.commit('test', 'master') as c:
    client.put_file_bytes(c, '/dir_a/data', b'DATA')
    client.put_file_url(c, '/dir_b/icon.png', 'http://www.pearl-guide.com/forum/images/smilies/biggrin.png')

client.get_files('test/master', '/', recursive=True)

As of version 0.1.4, there is also limited support for PPS:

pps_client = python_pachyderm.PpsClient()
pps_client.list_pipeline()
...

Contributing

This driver is co-maintained by Pachyderm and the community. If you're looking to contribute to the project, this is a fantastic place to get involved.

Getting started

To run tests, clone the repo, then run:

make init
tox
You can’t perform that action at this time.