'async for' requires an object with __aiter__ method, got AIOTracedCursor

[ewjoachim]

Problem

Using ddtrace and aiopg, if I do:

await cur.execute(query)
async for value in cur:
    yield value

If my connection is not patched, I get:

TypeError: 'async for' requires an object with __aiter__ method, got AIOTracedCursor
(...)
  File "path/to/my/file.py", line 241, in get_many
    async for value in cur:

(if my connection is not patched, it works)

Analysis

The cursor class is replaced with AIOTracedCursor which inherits wrapt.ObjectProxy.

Problem is, while thanks to ObjectProxy, AIOTracedCursor().__aiter__() would most probably work and return whatever the real proxy would return, this is not enough for Python to accept that the cursor is an iterator.

A small example with simple objects:

class A():
    def iter(self):
        return iter([])

    async def aiter(self):
        return iter([])

    def __getattr__(self, attr):
        if attr.endswith("iter__"):
            return getattr(self, attr.strip("_"))
a = A()

We implement a.__iter__() and a.__aiter__() but Python doesn't see it:

In [6]: a.__iter__()                                                                                                                                                                                               
Out[6]: <list_iterator at 0x7fdff00de860>

In [7]: a.__aiter__()                                                                                                                                                                                              
Out[7]: <coroutine object A.aiter at 0x7fdff00ddba0>

In [8]: async for e in a: print(e)                                                                                                                                                                                 
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
cell_name in async-def-wrapper()

TypeError: 'async for' requires an object with __aiter__ method, got A                                                                                                                                                                                                   

In [9]: iter(a)                                                                                                                                                                                                    
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-9-2b64cb055077> in <module>
----> 1 iter(a)

TypeError: 'A' object is not iterable

[ewjoachim]

I'm going to try a fix: explicitely define __aiter__ and make it call underlying_cursor.__aiter__

[brettlangdon]

hey @ewjoachim thanks for the report.

Nice investigation, yeah, if you wanted to submit a PR what we probably need to do is add __aiter__ method to the object proxy we are returning.

It should be something like:

def __aiter__(self):
    return self.__wrapped__.__aiter__()

Not 100% sure if we need it to be async def __aiter__ or not, but just wanted to showcase the self.__wrapped__.__aiter__() part as the way to access the thing wrapped by the object proxy.

[ewjoachim]

Yep, I had this part alright. I'm struggling to write tests that will not SyntaxError under python versions that don't have "async for" at the moment :D

[brettlangdon]

This might help: https://github.com/DataDog/dd-trace-py/blob/master/conftest.py#L23

TL;DR; you can use folders named py{27,34,35,36,37,38} to specify minimum required Python version for running tests.

e.g. tests/contrib/asyncio/py37/test.py