Add asyncio support for all components

[joncinque]

In order to minimize the copy-paste between the async and sync
implementations, all of the non-base components are sym-linked to their
sync counterparts. Since these import AlphaVantage locally, they
automatically pick up the async version, which is really nifty!

There was some refactoring in SectorPerformance to make the re-use
possible.

The async verison of AlphaVantage tries to re-use as much as possible
from the sync version. The main differences are:

• a new session for the HTTP calls
• a new close() function which should be called when you're done
• proxy is a string instead of a dict

Using it:

import asyncio
from alpha_vantage.async_support.timeseries import TimeSeries

async def get_data():
    ts = TimeSeries(key='YOUR_API_KEY')
    data, meta_data = await ts.get_intraday('GOOGL')
    await ts.close()

loop = asyncio.get_event_loop()
loop.run_until_complete(get_data())
loop.close()

[PatrickAlphaC]

This is fantastic!
For some context: http://masnun.rocks/2016/10/06/async-python-the-different-forms-of-concurrency/

I'm trying to figure out if their might be a simpler way than this? Like maybe:

import asyncio
from alpha_vantage.timeseries import TimeSeries

async def get_data():
    ts = TimeSeries(key='YOUR_API_KEY', async = True)
    data, meta_data = await ts.get_intraday('GOOGL')
    await ts.close()
loop = asyncio.get_event_loop()
loop.run_until_complete(get_data())
loop.close()

So this may show my lack of knowledge on the package, but do you know how would someone look to run async programming currently for this package? I only know how to multithread at the moment.

What do you think?

[PatrickAlphaC]

Or maybe even something like:

from alpha_vantage.async_support.timeseries import TimeSeries
ts = TimeSeries(key='YOUR_API_KEY')
data, meta_data = ts.get_intraday(['GOOGL', 'AAPL', 'TSLA'], async = True)

[joncinque]

The library has changed a lot over the past few iterations of Python 3, so the code provided will only work for Python 3.5+, which is when the async and await keywords were introduced. A side note on that, I didn't include any Python version checks, so it would be best to provide some documentation in the README saying that the async version is only compatible for Python 3.5 and above.
As for the example I provided, starting with python 3.7, there's a helper function which can make the example cleaner:

import asyncio
from alpha_vantage.async_support.timeseries import TimeSeries

async def get_data():
    ts = TimeSeries(key='YOUR_API_KEY', async = True)
    data, meta_data = await ts.get_intraday('GOOGL')
    await ts.close()
asyncio.run(get_data())

As for the placement of the files, I went with a model of another library, which makes it extremely clear when you're using the async version over the sync by explicitly importing from the async_support part. Reference for that library: https://github.com/ccxt/ccxt#python

Otherwise, we would need to create async versions of all the functions in the base class and name them differently, ie get_intraday_async, since async functions must be called from an async environment, which is some form of an asyncio event loop.

There's some interesting thoughts on this SO thread for duplication of code in libraries that want to support both sync and async: https://stackoverflow.com/questions/55152952/duplication-of-code-for-synchronous-and-asynchronous-implementations

To summarize, your first example would only work if the whole library were to be async, without the sync versions, and you'll want to keep supporting sync calls. Your second example wouldn't work unfortunately, because an async function must be awaited to even run.

The idea of using asyncio is to achieve really good performance all from one thread. The event loop does the job of intelligently scheduling the work between simultaneous tasks, switching whenever there's some sort of "heavy" operation, usually I/O.

For some background, I recently converted a large codebase into asyncio. The program essentially did many requests calls using 30-50 threads. After the conversion, there was a slight increase in CPU usage, but the performance went up roughly 6x! This way, if people want to fetch a lot of AlphaVantage data, instead of creating many threads, they could just schedule a bunch of get_data() calls and gather them in one place, ie:

loop.run_until_complete(asyncio.gather(get_data(), get_data(), get_data()))

and asyncio would do the job of handling concurrency.
https://docs.python.org/3/library/asyncio-task.html#running-tasks-concurrently for reference as well

Let me know how if there's any confusion or unclear parts to my answer!

[PatrickAlphaC]

This is great @joncinque, I'm going to add this to the develop branch as-is and will do some testing from there.

Are you currently looking to use this for a project at the moment? Do you mind installing from develop and looking for some fine tuning of this install? I'll do some testing to see how it will affect different versions of python, and I've been meaning to upgrade the tests with github actions here anyways....

I love how you modeled it after the ccxt library, that's a project that I LOVE (and I've been hoping they add decentralized exchange support soon...)

Thank you!

After doing some more research I found out this won't work in IPython/Jupyter Notebooks, so this is my note to update the README.md with that information as well.

One more thing, I think an easier example is as follows:

import asyncio
from alpha_vantage.async_support.timeseries import TimeSeries


async def get_data(symbol):
    ts = TimeSeries(key='YOUR_API_KEY')
    data, meta_data = await ts.get_intraday(symbol)
    await ts.close()
    return data

loop = asyncio.get_event_loop()
cat = loop.run_until_complete(get_data('AAPL'))
print(cat)
loop.close()

This way users can see the data they returned, and for multiple calls:

import asyncio
from alpha_vantage.async_support.timeseries import TimeSeries

symbols = ['AAPL', 'GOOG', 'TSLA', 'MSFT']


async def get_data(symbol):
    ts = TimeSeries()
    data, _ = await ts.get_quote_endpoint(symbol)
    await ts.close()
    return data

loop = asyncio.get_event_loop()
tasks = [get_data(symbol) for symbol in symbols]
group1 = asyncio.gather(*tasks)
results = loop.run_until_complete(group1)
loop.close()
print(results)

Let me know if you think otherwise.

[joncinque]

Those changes for the README are great, and feel free to make any additional changes to work with your dependencies and environments. For Python 3.4, the uses of await will need to be to yield from and all async def functions will need the @asyncio.coroutine decorator. There's some more info about that difference at https://stackoverflow.com/questions/44251045/what-does-the-yield-from-syntax-do-in-asyncio-and-how-is-it-different-from-aw

Otherwise, I do need this for a project, but I can use a local version no problem. Let me know if you need any other help or clarifications!

[PatrickAlphaC]

Perfect! Let's keep it in develop for a little bit and we and the community can both play with it. As you continue your project, if you see improvements, feel free to add them here.

Otherwise, once we can figure out how to make sure the code won't break for different versions of python, we can merge it to the production/master branch.

Thanks again for this great PR.