tornado.queues consumers are difficult to stop cleanly

[zkdfbb]

While using tornado 5.0.2, the following code will report
ERROR:asyncio:Task was destroyed but it is pending! task: <Task pending coro=<consumer() running at a.py:18> wait_for=<Future pending cb=[<TaskWakeupMethWrapper object at 0x10eb7a6a8>()]> cb=[IOLoop.add_future.<locals>.<lambda>() at /usr/local/lib/python3.6/site-packages/tornado/ioloop.py:720]>

this code is copied from http://www.tornadoweb.org/en/stable/queues.html

from tornado import gen
from tornado.ioloop import IOLoop
from tornado.queues import Queue

q = Queue(maxsize=2)

async def consumer():
    async for item in q:
        try:
            print('Doing work on %s' % item)
        finally:
            q.task_done()


async def producer():
    for item in range(1):
        await q.put(item)
        print('Put %s' % item)


async def main():
    # Start consumer without waiting (since it never finishes).
    IOLoop.current().spawn_callback(consumer)
    await producer()     # Wait for producer to put all tasks.
    await q.join()       # Wait for consumer to finish all tasks.
    print('Done')

IOLoop.current().run_sync(main)

[bdarnell]

The queue is working, it is just logging a message at shutdown because the consumer task never exited. I recommend not worrying about this message - getting rid of all messages like this is generally more trouble than it's worth. We could add a close() method to the queue that would make the async for loop terminate, but synchronizing everything properly would make this example much more complicated.