Incorporate Alpha Vantage as a source of historical equity prices #176
Comments
|
Great idea! Please use the current Great suggestion to use quantmod already uses jsonlite for a few things (e.g. |
|
This is great! Thanks.
…On Tue, Jul 4, 2017 at 13:43 Joshua Ulrich ***@***.***> wrote:
Great idea!
Please use the current getSymbols "methods" as a template for
getSymbols.av(). There is a getSymbols.skeleton(), but I'm not sure if
it's up-to-date. Should we consider creating one (or more?) "alias"
functions? Maybe getSymbols.alphavantage()?
Great suggestion to use setDefaults() to store the user's API key.
quantmod already uses jsonlite for a few things (e.g. getOptionChain()),
so please take a look at that code to see how it uses jsonlite to
manipulate JSON data.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#176 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AA1swNwYXghpCdGeintmIRPJhos5TGBBks5sKofWgaJpZM4ONmTL>
.
|
|
Thanks, Josh and Jeff, for the encouragement and suggestions. Yes, I checked Finally, I did create an alias, You can see the updated code in the feature branch of my fork. |
|
It would be great if we can have adjusted prices outside of Yahoo which is not reliable. It would be great if we can have a function with the same features of getSymbols using a different data source. Thank you |
|
getSymbol.av and its expected behavior would be ideal. I do not have teh skill set to do it. Just a user! My apologies |
|
@anozari , getSymbols is a wrapper which calls methods for different data sources. by default, it has historically called Yahoo. |
|
Understood. Thank you. I was not complaining. I naively used getSymbols.alphavantage instead of getSymbols and gave me an error. I probably should have not done that. Thank you for your suggestion. |
|
I used alphavantage today into an R code. I have found no issues to report. It does look very promising. Thank you. |
|
Brian had a very valid point about alphavantage. I have been using it for a week. I have found serious quality issues with their data. I am sure that over time they will get better and would be to get the kinks out. But, for now their data is not robust enough. |
|
@anozari Thanks for the testing and feedback. Can you summarize what problems you found with the data? I would like to replicate the problems if possible and study them. |
|
They are working very hard to fix their problems. But their data quality does not appear to be robust. For a few days they had issues for SPY. They had prices for some holidays -- including Christmas. And, they just fixed it. They seem to have problems with some mutual funds. For example, if you look at PTTRX, you will see data on holidays -- Like April 30, 2017. These are simple things that I noticed while comparing long series against each other. I noticed that the dates are out of sync. There might be other issues. I was surprised that they did not know about these things. So, their quality assurance may not be as robust. I am just guessing I think their processes are just new and over time things will only get better. They will get the kinks out. Good folks! |
|
Thanks for the detailed description of the problems. They are pretty serious. We can keep an eye on AlphaVantage and watch for improvements. |
|
Thank you @pteetor for proposing the quantmod-alphavantage integration! We are members of the product development team at Alpha Vantage Inc., and would like to share the following updates with the community:
Many thanks again - we appreciate the continued feedback and support from the community. |
|
Excellent post, @pteetor! Apparently there are a number of libraries built on top of Alpha Vantage on GitHub. Below is a review article on Alpha Vantage (written by a Python developer) that may provide some additional context on the Alpha Vantage API. Source: RomelTorres/alpha_vantage#13 Alpha Vantage API Review from a Python DeveloperThe ExperienceI was one of the "yahoo refugees" who stumbled upon Alpha Vantage via Google search. My first interactions with their website could be summarized by the good-old term WYSIWYG (what-you-see-is-what-you-get). Their documentation promised three things: (1) time series data with various resolutions, (2) over 50 technical signals, (3) sector data, and they kept their promise by providing demo URLs for all the said API functions.
They promised free API keys, and they also delivered. They promised "no promotional materials to your inbox," and indeed the only email I got from them so far was the announcement email for their new CSV feature. This being said, there are a couple areas they could optimize upon.
The DataOf the thousands of US-based equities I have analyzed so far, their historical data and technical indicators seem to match other reputable data sources. Their intraday data is realtime up to the current minute, which is fine for my research purposes but may not satisfy users who want to beat the market with millisecond precision. Perhaps a premium feature for this down the road? Their JSON output is easily readable and python-parsable. For the daily time series, however, I understand that their most recent data point is the cumulative information of the current trading day (updated realtime), but why is the first timestamp in YYYY-MM-DD HH:MM:SS format while all the others are in the normal YYYY-MM-DD format typical of the EOD data?
I would love to see a consistent YYYY-MM-DD format across all the timestamps. The "last refreshed" timestamp can be specified in Meta Data instead:
In addition to the data presentation aspects, below are couple other data-related proposals:
In SummaryIt is always a pleasure to have an API service that is well documented, platform/language-agnostic, and easily integratable. The fact that we have several third-party libraries built on top of Alpha Vantage on GitHub is in a sense testament to its developer-friendly nature. While there is still room for them to become a better version of themselves, I hope they thrive and stay true to the description on their home page - "driven by rigorous research, cutting edge technology, and a disciplined focus on democratizing access to data." |
|
was just about to cruft up a version of this when I stumbled upon this fork. Any plans to merge this into main anytime soon? Anything i can do to help? |
|
@ebs238 I do plan to merge it. @pteetor has done an excellent job. I'm the roadblock. |
|
Thanks for the great work gents. I plan on doing data comparisons with my existing database over the next week or 2. will post my findings here. Also, I've asked them to add an API to replace yahoos getQuote. That's my last current dependency on yahoo |
|
This all sounds good and heading in the right direction. I chuckle at Josh
tagging himself as a "roadblock". The guy is prolific. As far as I'm
concerned, the rest of the world is the roadblock, trying to catch up with
*him*.
I download AV's data nightly for a dozen ETFs. I am seeing a small data
problem: duplicated last row. Haven't reported this to the vendor yet
because I haven't created a minimal reproducible example yet! Could be my
code. Could be b/c I download shortly after midnight (Eastern time), when
the calendar is switching over. We'll see.
…On Sun, Sep 17, 2017 at 5:51 PM, ebs238 ***@***.***> wrote:
Thanks for the great work gents. I plan on doing data comparisons with my
existing database over the next week or 2. will post my findings here.
Also, I've asked them to add an API to replace yahoos getQuote. That's my
last current dependency on yahoo
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#176 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/ABH0cfBo-wBonEuvDZc46v-bmiT359RVks5sjaJcgaJpZM4ONmTL>
.
--
Paul Teetor
Elgin, IL, USA
http://quantdevel.com/public/
|
Since Paul's initial version, Alpha Vantage has added the option to download the data via CSV. Add the data.type argument so users can specify which source they prefer. Note that the JSON API provides timezone information that is not included in the CSV data. That's a reason to prefer the JSON API. See #176.
|
@pteetor @joshuaulrich Just a quick update that Alpha Vantage has added Forex quotes and digital/cryto-currency time series to their API portfolio. FYI. Forex: https://www.alphavantage.co/documentation/#fx |
|
Has anyone had any interaction with Alpha Vantage other than the one comment here? They offer support by email on their web site. I wrote them a couple of days ago, but haven't had a reply. My question was about support for the TSX. By trial and error, requests like TSX:POT will get intraday quotes for regular stocks. I haven't found how to get current quotes for ETFs or REITs. |
|
Hi i'm a refugee from Yahoo Finance Api, like many other. I'm interested in Alpha Vantage, it seems promising. I have a question : Is the EU stocks (ETFplus) market covered? I refer to Italy Stock Market, Milan Stock Exchange. I searched but found nothing. Thank you in advance for infos. |
|
|
@braverock |
|
Why I can't apply for API key？ |
|
@lishixin7 , Alpha Vantage is not quantmod. You need to get the API key from them. |
|
Has anyone seen a way to query a date range, it appears to be 100 rows or everything. I was able to do this on the Yahoo query. |
|
Correct. The underlying API provided by AlphaVantage provides only two choices ( If you'd enjoy the benefits of having the downloader window the data for you, please feel free to contribute that feature. Thank you! |
|
This is the second link on google when you type in alpha vantage and data quality was discussed here so I thought I would post. For OTC stocks their data can be a bit wonkish. Ex: FMBM has a volume of -23 on 2017-08-24. There are a couple other OTC stocks I've seen with negative or humongous adjusted closing prices. FTR, I don't remember seeing any non-OTC stocks from alpha vantage where the data was rough like this. |
|
I know this isn't the right place to post about this, but again it's one of the first places people will when they google alpha vantage. Some more odd numbers
I tried to email them about discrepancies but they have not gotten back to me |
|
Most of their data is fine, but when it's not, you're on your own to both detect and fix. Let's just say that their data quality is questionable and they are unresponsive and leave it at that. some examples for the record (these have all been raised as issues to them with no response):
as with many things, you get what you pay for on this one. |
|
Can we change the Time zone to another one ? |
|
Which time zone? The time zone of the data? We use the time zone given by the vendor, Alpha Vantage. It wouldn't make sense for us to override the reported time zone. You can change the time zone yourself by using R's facilities for working with dates and times. |
|
I was loving what I found from Alpha Vantage (I am doing some historical analysis) and I found that with CTXS they have all the values wrong from 1999 to 2017-01-31, when it FINALLY suddenly corrects itself: Yahoo has: That's enough proof to discourage me from using this service :'( as it is years and years of wrong data!!! |
|
^check out the tiingo api, I find their data pretty consistent |
|
Thanks Steve - Similar deviation for Tiingo... (A bit harder to read!) $18 dividend? I'd remember that one!!! |
|
looks like a split issue and CTXS has had many splits. are you looking for adjusted or unadjusted data. You can specify which you want with the relevant parameter: Also: |
|
Not sure if @tiingo is checked too often but yeah if you email tiingo they are pretty quick to figure out a fix for thing sort of thing. Had some v weird pennystock data they managed to fixup a while back |
|
@ethanbsmith |
|
Vendor data quality issues are kind of off topic for this ticket. To try to wrap up the digression, my experience is that all vendors occasionally have problems. Older data is of correspondingly lower quality (fewer eyeballs). Cheaper data is of correspondingly lower quality too (less/zero leverage on your vendor to fix it). Basically, you get what you pay for. If issues are found in paid data sources, you need to report them and work with the vendor to fix them. Data quality is always an issue. "Clean" data sources are extremely expensive, with services like CRSP/Compustat or Reuters or Bloomberg or Factset costing hundreds or thousands of dollars per month. Those costs come with corresponding increases in base data quality, and with dedicated tech support who can usually resolve issues when they inevitably arise. As for adjusted data, it is useful for getting adjusted returns. Adjusted prices have all sorts of issues when considering analysis, and those issues compound the more splits and adjustments there are going back through time. |
|
Hi @ozchamo Normally we try not to speak too much in github issues, but you can always E-mail our support team at support@tiingo.com In the case of Citrix (CTXS), it appears as if we receive conflicting information. The Feb 2017 split/distribution has to do with the spin-off of GoTo. Generally we reflect spinoffs as special stock dividends when possible. This is because splits should generally mean that the entity has not changed. That share prices are just now $X/split factor. In the case of a spin-off, splits should not be the standard because the shares now mean something fundamentally different than pre-spinoff. A stock dividend is a more accurate way to describe the spin-off - especially if we have to condense it to a table. In the case of Citrix, it appears as if we were passed a dissemination message of a split. In recent corp events we check for this (split disseminations instead of stock dividend), to help prevent this issue - hence why it happens in rare, specific situations like this one. While all vendors have an error term, I want to make sure all users who report them are heard. When we hear of an error we do extensive checks to ensure it does not appear elsewhere and add a check if possible. Given the weird events that often happen, sometimes fully automated checks are not possible. Nonetheless, if you ever reach out to the support team, we take all error reports seriously. I modeled Tiingo to be the data firm that I myself always wanted. Anyway, you should see CTXS now corrected - thanks for the report. And @braverock - generally I agree with you (that you get what you pay for), but we are working hard to change that. We want users to get far more than what they pay for - and many people in this thread have been instrumental in that. |
|
Thank you all. Much appreciated. And I apologise to both Tilingo and Alpha Vantage: They do reflect the adjustment through the "splitFactor":1.2558. And they both agree on that amount. Good work! |
|
Hi, how can I get 5 years of a time series ?? I use, for instance, to get ExxonMobil data (XOM): key = MyAPIKey xom_data, xom_meta_data = ts.get_daily(symbol='XOM') How can i get 5 years of data instead of the default 100 last days |
|
Hi, please ask your question on Stackoverflow or a similar site.
This issue tracker is only for the quantmod R package. It's not a general
purpose support forum for the services quantmod provides access to.
… |
Description
The Alpha Vantage web service provides real-time and historical equity data. The service is free, requiring only a free, one-time registration. The service provides daily, weekly, and monthly history for both domestic and international markets, with up to 20 years of history. For daily data, adjusted close prices are available to account for dividends and splits. The service can also provide real-time price bars at a resolution of 1 minute or higher, for up to 10 recent days. They have a clean, documented, public API that returns JSON-encoded data.
I propose incorporating Alpha Vantage into
getSymbolsas a source of historical data, much like Yahoo and Google.Expected behavior
The caller would invoke the Alpha Vantage downloader through the usual mechanism; i.e.,
The associated function,
getSymbols.av, would have this signature.You must register with Alpha Vantage in order to download their data, but the on-time registration is fast and free. A.V. will assign an API key to you, a short string of alphanumeric characters (e.g., "FOO4").
You must provide your API key every time you call
getSymbols.av. Alternatively, you can set it globally usingThe Alpha Vantage site provides daily, weekly, monthly, and intraday data. Use
periodicityto select one.Set
adjusted=TRUEto include a column of closing prices adjusted for dividends and stock splits (available only for daily data).The intraday data is provided as a sequence of OHLC bars. Use the
intervalargument to determine the "width" of the bars: 1 minute, 5 minutes, 15 minutes, etc.By default Alpha Vantage returns the 100 most-recent data points. Set
outputsize="full"to obtain the entire available history (which requires more download time, of course). Alpha Vantage says they provide up to 20 years of daily data: or the most recent 10 or 15 days of intraday data.Issues and Limitations
The web service is free, but registration is required. Alpha Vantage provides an API key at registration, and that key is needed to download data.
The A.V. API returns its data via JSON. In my experience, pulling large, historical datasets can be slower than Google or Yahoo (but not crippling slow). At this point, I don't know if the bottleneck is the A.V. server itself or the encoding/decoding process for (inefficient) JSON.
Beyond historical data, the A.V. API can also provide sector performance data and technical indicators (e.g., moving averages). I am not planning to provide access to those data elements. The sector performance stuff is kind of orthogonal to
quantmodscope; and the technical indicators are already available via theTTRpackage.I believe the intraday data is available only for domestic stocks, not international; but I'm not sure.
The text was updated successfully, but these errors were encountered: