question: using StreamingResponsefor json data

[matthdsm]

Hi,

I'm trying to use the Starlette StreamingResponse with a generator that yields a dict.
However, when trying to get a return I get the following error

Traceback (most recent call last):
  File "/Users/matdsmet/Projects/variantExpress/variantexpress_server/venv/lib/python3.7/site-packages/uvicorn/protocols/http/httptools_impl.py", line 379, in run_asgi
    result = await asgi(self.receive, self.send)
  File "/Users/matdsmet/Projects/variantExpress/variantexpress_server/venv/lib/python3.7/site-packages/uvicorn/middleware/debug.py", line 83, in __call__
    raise exc from None
  File "/Users/matdsmet/Projects/variantExpress/variantexpress_server/venv/lib/python3.7/site-packages/uvicorn/middleware/debug.py", line 80, in __call__
    await asgi(receive, self.send)
  File "/Users/matdsmet/Projects/variantExpress/variantexpress_server/venv/lib/python3.7/site-packages/starlette/middleware/errors.py", line 125, in asgi
    raise exc from None
  File "/Users/matdsmet/Projects/variantExpress/variantexpress_server/venv/lib/python3.7/site-packages/starlette/middleware/errors.py", line 103, in asgi
    await asgi(receive, _send)
  File "/Users/matdsmet/Projects/variantExpress/variantexpress_server/venv/lib/python3.7/site-packages/starlette/middleware/cors.py", line 138, in simple_response
    await inner(receive, send)
  File "/Users/matdsmet/Projects/variantExpress/variantexpress_server/venv/lib/python3.7/site-packages/starlette/middleware/gzip.py", line 33, in __call__
    await self.inner(receive, self.send_with_gzip)
  File "/Users/matdsmet/Projects/variantExpress/variantexpress_server/venv/lib/python3.7/site-packages/starlette/exceptions.py", line 74, in app
    raise exc from None
  File "/Users/matdsmet/Projects/variantExpress/variantexpress_server/venv/lib/python3.7/site-packages/starlette/exceptions.py", line 63, in app
    await instance(receive, sender)
  File "/Users/matdsmet/Projects/variantExpress/variantexpress_server/venv/lib/python3.7/site-packages/starlette/routing.py", line 44, in awaitable
    await response(receive, send)
  File "/Users/matdsmet/Projects/variantExpress/variantexpress_server/venv/lib/python3.7/site-packages/starlette/responses.py", line 194, in __call__
    chunk = chunk.encode(self.charset)
AttributeError: 'dict' object has no attribute 'encode'

Some googling showed that the error is because the response expect a string input from the generator. Is there a way to fix this?

code snippet

async def some_func():
    for x in list():
        yield dict()

@router.post("/postroute")
async def postroute():
    generator = some_func()
    return StreamingResponse(generator, media_type="application/json")

Am I doing this wrong or am I expecting something that's not done?

Thanks a lot
M

[tomchristie]

Streaming responses expect a stream of bytes or text.

If the JSON you're returning is a list of objects then I guess you could do something like this...

async def some_func():
    yield "["
    for idx, item in enumerate(...):
        if idx > 0:
            yield ","
        yield json.dumps(item)
    yield "]"

[sigg3]

Hi! I can't see that this was resolved yet. What is the current workaround?

Minimal working example:

from starlette.responses import StreamingResponse
import asyncio
list_of_dicts = [
                 {"key": "value"},
                 {"nice": 0},
                 {"true": True}
                 ]

async def slow_dicts(maximum):
    yield "["
    for number in range(maximum):
        yield list_of_dicts[number]
        await asyncio.sleep(0.5)
    yield "]"

async def app(scope, receive, send):
    global list_of_dicts
    assert scope['type'] == 'http'
    #generator = slow_numbers(1, 100)
    generator = slow_dicts(3)
    response = StreamingResponse(generator, media_type='application/json')
    await response(scope, receive, send)

AttributeError: 'dict' object has no attribute 'encode'

I get the same error if the objects are Meh objects, e.g. class Meh: pass.

[jamesbraza]

You can use multiple inheritance with StreamingResponse and JSONResponse to create a JSONStreamingResponse.

Sample code can be found at #2363 (comment)