Skip to content
An Avro file reader/writer for Python
C Python Shell C++
Branch: master
Clone or download
walshb and facboy More Travis CI builds (#33)
* Travis: MacOS build.

* Travis: Python 3.6.
Latest commit fed8531 Jan 16, 2019
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
.travis
examples
pyavroc Bumped version to 0.7.2. Sep 30, 2015
src More error checks. Dec 8, 2018
tests Test empty record. Dec 8, 2018
.gitignore Now building against mainline apache/avro. Dec 8, 2018
.travis.yml More Travis CI builds (#33) Jan 15, 2019
LICENSE Initial public commit. Jan 14, 2015
MANIFEST.in MANIFEST.in: include README.md. Sep 20, 2015
README.md Now building against mainline apache/avro. Dec 8, 2018
clone_avro_and_build.sh More Travis CI builds (#33) Jan 15, 2019
run_benchmark.sh More Travis CI builds (#33) Jan 15, 2019
setup.py More Travis CI builds (#33) Jan 15, 2019

README.md

pyavroc

Build Status

An Avro file reader/writer for Python.

Usage

>>> import pyavroc
>>> with open('myfile.avro') as fp:
>>>     reader = pyavroc.AvroFileReader(fp, types=True)
>>>     for record in reader:
>>>         print record

Comparison with original Avro Python API

pyavroc is a Python API on top of upstream Avro-C. This means it reads about 40 times faster than Avro's Python implementation. (The exact timings depend on the version of Python used).

Name Description Relative speed (bigger is better)
python-avro Avro's implementation (pure Python) 1
fastavro python-avro improved, using Cython 10
pyavroc Python/C API on upstream Avro-C 40

Building the module

You will need to build Avro-C. This is available at https://github.com/apache/avro.git.

Then you can build pyavroc, linking against the Avro-C shared library.

The pyavroc repository contains the script clone_avro_and_build.sh which automates this process:

./clone_avro_and_build.sh --static

Installing the module

  • Make sure cmake is available (eg. apt-get install cmake).
  • Run ./clone_avro_and_build.sh
  • This generates a build folder
  • python setup.py bdist_wheel -d build makes an executable wheel of the project and saves in the build/ folder
  • cd build
  • pip install pyavroc-0.7.2-cp36-cp36m-linux_x86_64.whl

Writing records

pyavroc supports writing, both for records created as dictionaries, and for records created as Python objects.

More examples

More examples are available in the tests directory.

License

Copyright 2015 Byhiras (Europe) Limited

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at:

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

You can’t perform that action at this time.