Asyncio support

[stkubr]

Hi devs,
a question - it looks like current Pyota package doesn't support async requests to IRI. Is it possible to have it in future?

Or may be one can share some ideas about an easy way how to amend the library, because the only way I thought of is to add async/await to every function up from _send_http_request().

Thanks.

[todofixthis]

Hey @stkubr I love this idea, and would be very interested to see an async version of the lib 😻

My feeling is that we would not be able to make this work with PyOTA as-is; one of the goals of this project is to support applications written in Python 2. There might be a way to write the code in such a way that we can maintain that backwards-compatibility, but it is definitely beyond my ken 🥋

[stkubr]

Python 2 is going to be deprecated in 8 months 🤔 Why python is so low priority for IOTA? 😕

Well not the whole library should be rewritten, just the iota.adapter I think. I could have a look, but I would need some advising from you or other IOTA devs.

[Aconitin]

I really struggle to understand why we would uphold the Python 2 support. Just let it die already.
If it comes down to either async or Python 2, I'd drop Python 2 like a hot potato.
@stkubr How is this (https://github.com/stkubr/iota_async.lib.py) coming along? I wish Python support was taken more seriously by the IF either ....

[lzpap]

Hey @stkubr and @todofixthis I investigated a bit the asyncio support for PyOTA.

IRI just ditched Python 2 (used for testing), and I think it's time for us too. I heard no objections from IF side 😃 .

With this in mind, I think we should add asyncio support, which might also be great for the future implementation of account module. Supporting asynchronous api commands would speed up applications using them for non-blocking calls. Some extended API commands, such as get_bundles would also be faster.

My approach would be to keep the support for synchronous commands as well, possibly implement the changes in a way that they are backwards compatible.
For this to work, we can do the following:

• Implement all API commands (***Command classes in PyOTA) as asynchronous. ( their __call__ and _execute methods with async def ... )
• Furthermore, requests in adapters should be replaced with requests_async or aiohttp to support async in the http client.
• A new AsyncIota API class that has all API methods defined as async, calling the async commands from above.

import asyncio

class AsyncIota(object):
    """
    Asynchronous API.
    """
    def init(self,...):
        ...
    
    async def add_neighbors(self, uris):
        # type: (Iterable[Text]) -> dict
        """
        Add one or more neighbors to the node.  Lasts until the node is
        restarted. Asynchronous API command.
        """
        return await core.AddNeighborsCommand(self.adapter)(uris=uris)

• Restructure Iota and StrictIota API classes to work in synchronous mode, but use the underlying async commands. One way to do this could be to inherit Iota class from AsyncIota, and override the parent's API methods. An example:

import asyncio

class Iota(AsyncIota):
    """
    Synchronous API.
    """
    def init(self,...):
        ...
    
    def add_neighbors(self, uris):
        # type: (Iterable[Text]) -> dict
        """
        Add one or more neighbors to the node.  Lasts until the node is
        restarted. Synchronous API command.
        """
        return asyncio.get_event_loop().run_until_complete(
            super(Iota, self).add_neighbors(uris)
        )

An option would be to add a metaclass to Iota that looks up the api commands of the parent class and modifies them at runtime to be synchronous, like above. However, this would mean loosing IDE static analysis support, which is not the best for developer experience.

Another possibility is to just simplify the code example above with a decorator that does the same. I like this one as it is compact, but still readable.

Conclusion:

• Adding asyncio support is possible while keeping the synchronous nature of the commands.
• Even some synchronous extended api commands could be improved in terms of performance.
• Python 2 has to go. 😿
• Most async http clients work only 3.6+, so Python 3.5 should go as well.

Please let me know what you think :)

[Aconitin]

Agreed 100%

[lzpap]

Released with https://github.com/iotaledger/iota.py/releases/tag/3.0.0b1