Skip to content
Fast Avro for Python
Branch: master
Clone or download
Latest commit b94ab90 Apr 3, 2019
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
benchmark
docs add more documentation for reader objects Sep 20, 2018
fastavro 0.21.20 Apr 3, 2019
tests Add Writer.write_block() (#324) Apr 3, 2019
.flake8.cython
.gitignore
.travis.yml remove support for python 3.4 Apr 3, 2019
.travis_build_wheel.sh
ChangeLog
Dockerfile
LICENSE
MANIFEST.in
Makefile have `fresh` and `clean` do actions with the html files Jun 28, 2018
NOTICE.txt Even better compliense with Apache license Jan 30, 2012
README.md
appveyor.yml
developer_requirements.txt
publish.sh
pytest.ini
run-tests.sh Code coverage (#216) Jun 11, 2018
setup.cfg also ignore benchmark folder Dec 13, 2017
setup.py remove support for python 3.4 Apr 3, 2019
tag.sh add block_reader interface (#208) Jun 13, 2018
tox.ini remove support for python 3.4 Apr 3, 2019

README.md

fastavro

Build Status Documentation Status codecov

Because the Apache Python avro package is written in pure Python, it is relatively slow. In one test case, it takes about 14 seconds to iterate through a file of 10,000 records. By comparison, the JAVA avro SDK reads the same file in 1.9 seconds.

The fastavro library was written to offer performance comparable to the Java library. With regular CPython, fastavro uses C extensions which allow it to iterate the same 10,000 record file in 1.7 seconds. With PyPy, this drops to 1.5 seconds (to be fair, the JAVA benchmark is doing some extra JSON encoding/decoding).

fastavro supports the following Python versions:

  • Python 2.7
  • Python 3.5
  • Python 3.6
  • Python 3.7
  • PyPy
  • PyPy3

Supported Features

  • File Writer
  • File Reader (iterating via records or blocks)
  • Schemaless Writer
  • Schemaless Reader
  • Snappy and Deflate codecs
  • Schema resolution
  • Aliases
  • Logical Types

Missing Features

  • Anything involving Avro's RPC features
  • Parsing schemas into the canonical form
  • Schema fingerprinting

Documentation

Documentation is available at http://fastavro.readthedocs.io/en/latest/

Installing

fastavro is available both on PyPi

pip install fastavro

and on conda-forge conda channel.

conda install -c conda-forge fastavro

Contributing

  • Bugs and new feature requests typically start as github issues where they can be discussed. I try to resolve these as time affords, but PRs are welcome from all.
  • Get approval from discussing on the github issue before opening the pull request
  • Tests must be passing for pull request to be considered

Developer requirements can be installed with pip install -r developer_requirements.txt. If those are installed, you can run the tests with ./run-tests.sh. If you have trouble installing those dependencies, you can run docker build . to run the tests inside a docker container. This won't test on all versions of python or on pypy, so it's possible to still get CI failures after making a pull request, but we can work through those errors if/when they happen.

Releasing

We release both to pypi and to conda-forge.

We assume you have twine installed and that you've created your own fork of fastavro-feedstock.

Changes

See the ChangeLog

Contact

Project Home

You can’t perform that action at this time.