ENH: Add support for AlphaVantage API #490

addisonlynch · 2018-02-19T07:46:16Z

AlphaVantage Readers

Adds support for nearly all of the AlphaVantage API endpoints.

Additions

AVTimeSeriesReader - class to handle requests for data from the Stock Time Series endpoints
- top-level get_data_av, for TIME_SERIES_DAILY
- av-daily for TIME_SERIES_DAILY
- av-daily-adjusted for TIME_SERIES_DAILY_ADJUSTED
- av-weekly for TIME_SERIES_WEEKLY
- av-weekly-adjusted for TIME_SERIES_WEEKLY_ADJUSTED
- av-monthly for TIME_SERIES_MONTHLY
- av-monthly-adjusted TIME_SERIES_MONTHLY_ADJUSTED
AVQuotesReader - class to handle requests for Batch Stock Quotes using top-level get_quote_av
AVForexReader - class to handle FX Exchange Rates endpoint (av-forex or top-level get_exchange_rate_av)
AVSectorPerformanceReader - class to handle Sector Performances SECTOR endpoint av-sector or top-level get_sector_performance_av
AlphaVantage- base class for handling AlphaVantage requests

Questions

What format should the sector performances return? As of now, they're strings with a percent sign at the end. Using int64 and float64 for the other new readers.
Is it okay for TimeSeriesReader to take the argument function which allows the selection of the daily/monthly/weekly endpoint? I have an implementation (see this gist) which uses separate readers for each (AVDailyReader, AVDailyAdjustedReader, etc.). Can switch over to that if it is more in-line with the current style.
Used AV instead of AlphaVantage in names...readers such as AlphaVantageTimeSeriesReader seemed a bit long.

Needs Help

Will need to obtain an API key from AlphaVantage and contact them (support@alphavantage.co) to request an exemption from throttling to allow the tests to run. Seems they don't like repeated calls to the same endpoints without additional permission. As such, the tests now will pass with an API key that does not have these restrictions, but may fail otherwise.
- This key will be stored as the environment variable ALPHAVANTAGE_API_KEY

Near Future

The above takes care of most of the AlphaVantage endpoints. There are two groups remaining:

Digital/Crypto Currencies - is this something we would want to implement? I don't know if this is a direction you guys would want to head in for PDR. I don't have a lot of knowledge in this sphere, but it seems as if these endpoints could be easily implemented if desired.
Technical Indicators - what is the preferred way to implement this?
AlphaVantage provides 52 technical indicators, each of which seems to take the same default (required) parameters: function, symbol, interval, apikey. However, certain indicators (such as EMA) require additional parameters time_period, series_type, etc.

In addition, will need to add a reader to handle TIME_SERIES_INTRADAY

closes Add Alpha Vantage as data source? #389
tests added / passed
passes git diff upstream/master -u -- "*.py" | flake8 --diff
added entry to docs/source/whatsnew/vLATEST.txt

gliptak · 2018-02-19T13:18:16Z

@addisonlynch Please review the Travis failures

https://travis-ci.org/pydata/pandas-datareader/builds/343250680

While most of them API key related, this ones fails on import pandas.testing

https://travis-ci.org/pydata/pandas-datareader/jobs/343250681

@davidastephens @jreback Who can add the AlphaVantage key to Travis and ReadTheDocs?

Thanks

bashtage · 2018-02-19T21:53:21Z

Need to use same test pattern to skip module if API key is not available.

bashtage · 2018-02-19T21:55:02Z

@gliptak I can add a key to Travis. It won't actually be used for testing until the PR is merged since PRs can't access encryption variables to prevent leaking them by someone who could submit a useless PR that had print(os.environ.get['SEKRIT'])

addisonlynch · 2018-02-20T02:03:20Z

I added the pytest.skipifs and fixed pandas 0.19.2 compat. Do you want me to add @skip_on_exception(RemoteDataError) to the tests as well? Certain tests will still fail unexpectedly because of:

{'Information': 'Thank you for using the CURRENCY_EXCHANGE_RATE_API! Please kindly contact support@alphavantage.co if you would like to have a higher API call volume'}

unless PDR does obtain a higher call volume key. Let me know what works best.

bashtage

Looks really good. Some small changes and doc string fixes. Also, could you add to readers/index.rst

bashtage · 2018-02-20T08:57:55Z

docs/source/remote_data.rst

+                       access_key=os.getenv('ALPHAVANTAGE_API_KEY'))
+    f.loc["2017-02-09"]
+
+The top-level function ``get_data_av`` is also provided. This function will


I don't really like get_data_av -- how could a new user know what av is? Maybe just the long get_data_alphavantage? Autocomplete is pretty good.

bashtage · 2018-02-20T08:58:33Z

pandas_datareader/av/__init__.py

+
+class AlphaVantage(_BaseReader):
+    """
+    Base class for all AlphaVantage queries


I think there should be docstring here. Maybe we should use a doc string inheriter to simplify this?

Nevermind - this is a base class.

bashtage · 2018-02-20T08:59:12Z

pandas_datareader/av/forex.py

+
+    Parameters
+    ----------
+    pairs : string, array-like object (list, tuple, Series)


Why not just keep the standard symbols?

bashtage · 2018-02-20T09:01:11Z

pandas_datareader/av/sector.py

+
+    Parameters
+    ----------
+    retry_count : int, default 3


I don't think this docstring has all the arguments from the base class.

bashtage · 2018-02-20T09:01:35Z

pandas_datareader/data.py

@@ -47,6 +51,10 @@
           'DataReader']


+def get_data_av(*args, **kwargs):


get_data_alphavantage?

addisonlynch · 2018-02-21T06:38:30Z

Done

addisonlynch · 2018-02-21T20:27:04Z

@bashtage was going to add the remaining endpoints today or tomorrow. Do you want me to push them to this pull or a separate one? Also, any direction you're able to provide on the preferred way to implement the technical indicators (see above) would be great. Thanks

gliptak · 2018-03-03T15:35:18Z

@bashtage Were your review items addressed?

@addisonlynch Could you resolve the conflict?

Thanks

addisonlynch · 2018-03-04T17:34:45Z

@gliptak conflicts resolved. Did indeed repair the requested changes

addisonlynch · 2018-03-07T18:17:56Z

Travis fails on unrelated errors (FRED and tiingo?). Are we ready to merge this otherwise?

gliptak · 2018-04-04T17:50:13Z

@bashtage Did you complete your review? Thanks

addisonlynch added 3 commits February 19, 2018 01:04

Added AlphaVantage readeres

03fb53f

DOCS: Cleanup AlphaVantage docs

3286fff

Added AlphaVantage Quotes Reader

72ee5ad

Updated AV tests

bed2ffb

bashtage requested changes Feb 20, 2018

View reviewed changes

addisonlynch added 2 commits February 20, 2018 17:54

Updated AV docstrings, repaired names

1188ddb

DOCS: Added Alphavantage to readers index

7ef54a4

bashtage added this to the 0.7.0 milestone Feb 21, 2018

rsvp mentioned this pull request Mar 4, 2018

Disruption of equities data :: pandas_datareader dependency on Yahoo and Google Finance API rsvp/fecon235#7

Open

Merge branch 'master' into alphavantage

680ca16

addisonlynch mentioned this pull request Mar 10, 2018

Add Investopedia as a Data Source? #497

Closed

bashtage approved these changes Apr 13, 2018

View reviewed changes

bashtage merged commit 1763dbd into pydata:master Apr 13, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ENH: Add support for AlphaVantage API #490

ENH: Add support for AlphaVantage API #490

addisonlynch commented Feb 19, 2018 •

edited

Loading

gliptak commented Feb 19, 2018 •

edited

Loading

bashtage commented Feb 19, 2018

bashtage commented Feb 19, 2018

addisonlynch commented Feb 20, 2018 •

edited

Loading

bashtage left a comment

bashtage Feb 20, 2018

bashtage Feb 20, 2018

bashtage Feb 20, 2018

bashtage Feb 20, 2018

bashtage Feb 20, 2018

bashtage Feb 20, 2018

addisonlynch commented Feb 21, 2018

addisonlynch commented Feb 21, 2018 •

edited

Loading

gliptak commented Mar 3, 2018

addisonlynch commented Mar 4, 2018

addisonlynch commented Mar 7, 2018

gliptak commented Apr 4, 2018

		@@ -47,6 +51,10 @@
		'DataReader']


		def get_data_av(args, *kwargs):

ENH: Add support for AlphaVantage API #490

ENH: Add support for AlphaVantage API #490

Conversation

addisonlynch commented Feb 19, 2018 • edited Loading

AlphaVantage Readers

Additions

Questions

Needs Help

Near Future

gliptak commented Feb 19, 2018 • edited Loading

bashtage commented Feb 19, 2018

bashtage commented Feb 19, 2018

addisonlynch commented Feb 20, 2018 • edited Loading

bashtage left a comment

Choose a reason for hiding this comment

bashtage Feb 20, 2018

Choose a reason for hiding this comment

bashtage Feb 20, 2018

Choose a reason for hiding this comment

bashtage Feb 20, 2018

Choose a reason for hiding this comment

bashtage Feb 20, 2018

Choose a reason for hiding this comment

bashtage Feb 20, 2018

Choose a reason for hiding this comment

bashtage Feb 20, 2018

Choose a reason for hiding this comment

addisonlynch commented Feb 21, 2018

addisonlynch commented Feb 21, 2018 • edited Loading

gliptak commented Mar 3, 2018

addisonlynch commented Mar 4, 2018

addisonlynch commented Mar 7, 2018

gliptak commented Apr 4, 2018

addisonlynch commented Feb 19, 2018 •

edited

Loading

gliptak commented Feb 19, 2018 •

edited

Loading

addisonlynch commented Feb 20, 2018 •

edited

Loading

addisonlynch commented Feb 21, 2018 •

edited

Loading