Skip to content
This repository has been archived by the owner on Jan 15, 2020. It is now read-only.

AsyncIO backend #46

Closed
wants to merge 8 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
3 changes: 2 additions & 1 deletion .travis.yml
Expand Up @@ -2,9 +2,10 @@ language: python
python:
- "2.7"
- "3.4"
- "3.5"
before_install:
- sudo apt-get update -qq
- sudo apt-get install -y libyajl1
install:
- pip install .
script: python tests.py
script: python -m tests
42 changes: 42 additions & 0 deletions README.rst
Expand Up @@ -58,6 +58,35 @@ immediately producing some result::
stream.write('</geo>')


For ``asyncio`` backend usage will be a bit different, depending on your Python
version. For Python 3.5 you can stay with the familiar API, thanks to
`async for` feature::

import ijson.backends.asyncio as ijson

async def go():
resp = await aiohttp.get('http://localhost:5984')
async for item in ijson.items(resp.content, ''):
...

Python 3.3 and 3.4 lacks that feature, so you'll have to go with the `while`
loop instead::

import asyncio
import ijson.backends.asyncio as ijson

@asyncio.coroutine
def go():
resp = yield from aiohttp.get('http://localhost:5984')
items = ijson.items(resp.content, '')
while True:
try:
item = yield from items.next()
except ijson.StopAsyncIteration: # signal that steam is over
break
else:
...

Backends
========

Expand All @@ -70,6 +99,7 @@ backends located in ijson/backends:
for some reason.
- ``yajl``: deprecated YAJL 1.x + ctypes wrapper, for even older systems.
- ``python``: pure Python parser, good to use with PyPy
- ``asyncio``: pure Python parser made for asyncio

You can import a specific backend and use it in the same way as the top level
library::
Expand All @@ -79,6 +109,18 @@ library::
for item in ijson.items(...):
# ...

With `asyncio` backend it's possible to reuse `yajl*` ones for parsing JSON
data. This will improve parsing performance with preserving async IO::

import ijson.backends.asyncio as ijson
from ijson.backends import yajl2

async def go():
resp = await aiohttp.get('http://localhost:5984')
items = ijson.items(resp.content, '')
async for item for ijson.items(resp.content, '', yajl_backend=yajl2)
...

Importing the top level library as ``import ijson`` uses the pure Python
backend.

Expand Down