Skip to content
IO library for Chainer to access various filesystem with unified API
Python Shell Dockerfile
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
.pfnci
chainerio Add `path` support to recursive list in zip Oct 8, 2019
docs Change theme of docs Oct 2, 2019
examples Init commit for release Apr 23, 2019
tests Add tests with path to recursive list Oct 8, 2019
.gitignore Init commit for release Apr 23, 2019
LICENSE Init commit for release Apr 23, 2019
README.md Add Classifiers to setup.py May 30, 2019
setup.cfg
setup.py Change theme of docs Oct 2, 2019
tox.ini Add tox May 10, 2019

README.md

ChainerIO

ChainerIO is an IO abstraction library for Chainer, optimized for deep learning training with batteries included. It supports

  • Filesystem API abstraction with unified error semantics,
  • Explicit user-land caching system,
  • IO performance tracing and metrics stats, and
  • Fileset container utilities to save metadata.

Dependency

  • HDFS client and libhdfs for HDFS access
  • Python 3

Installation and Document build

Installation

$ git clone https://github.com/chainer/chainerio.git
$ cd chainerio
$ pip install .

Documentation

$ cd chainerio/docs
$ make html
$ open build/html/index.html

Test

$ cd chainerio
$ pip install .[test]
$ pytest

How to use

Please refer to the Documentation for more information about the usage. Also you can find some examples in examples directory for usage in Chainer training script examples.

You can’t perform that action at this time.