1059-Fixing a bug with multiple json objects

[tescalada]

This splits the text by CRLF and then json.loads each part
independently instead of attempting the parse the whole string.

Signed-off-by: Tristan Escalada tristan@escalada.us

[tescalada]

I was also running into issue #1059. This fixes it for me but does not seem like a good solution.

I was also unable to update the unit tests to show the bug. For some reason it works fine in testing.

Here is the test I was using:

from io import BytesIO
from docker import Client

dockerfile = '''
FROM alpine:latest
RUN apk add --update python-dev
CMD ["/bin/sh"]
'''

f = BytesIO(dockerfile.encode('utf-8'))
cli = Client()

response = cli.build(
    fileobj=f,
    rm=True,
    tag='yourname/volume',
    stream=True,
    decode=True,
    nocache=True,
)

for line in response:
    print line

[danpalmer]

Any update on this being merged? The build looks green (although Janky hasn't updated correctly).

[aanand]

It's difficult to integration test because the API streaming endpoint won't necessarily always concatenate multiple JSON objects in one chunk. It's even difficult to unit test in its current state, because _stream_helper is coupled to an HTTP response object.

If you can decouple the inner logic so it just operates on an IO object, then you should be able to write a unit test that passes in a mock object which emits a chunk of multiple JSON objects when read from.

[pahaz]

I have the same problem and I think we must split chunks before decode. Otherwise, we will have the problem with the glued raw data.

Now, I use similar patch in my code (Python 3.x only):

def _events(client, filters=None):
    events = client.events(filters=filters)
    for event in events:
        raw_event = event = event.decode('utf-8').strip('\n')
        try:
            if '\n' in event:
                # multiple events!
                for x in event.split('\n'):
                    yield json.loads(x)
            else:
                yield json.loads(event)
        except json.JSONDecodeError:
            warn('bad event json: %r' % raw_event)

[rmb938]

Any updates on this? I recently ran into this issue.

[robbiet480]

This PR fixes the issue for me. Can it be merged soon?

[shin-]

Thank you!
Looks like a good solution. Jenkins' failure is just a fluke. I'll go ahead and merge this.