Getting benckmark data via IEX API does not work anymore #2480
Comments
|
Same issue here. Looking for a way to stop this whole mechanics either by providing the data from another source (a file, etc.) or just remove it completely. |
|
IEX has deprecated the old API in favor of the new one: iexcloud.io offers the chart endpoint used in zipline, but user registration |
|
My temporal fix (on zipline-live, which is branched off from 1.1.0): |
|
Inspired by this comment in #1951, here is the workaround for people who does NOT need benchmarks at all: By default, zipline downloads benchmark data by making an http request in First, replace Then in In this example This is only a hack. In the long run I would suggest to change the benchmark downloading method to request other API in case you would like to use benchmarks in the future. |
|
@MarikoKujo @tibkiss |
|
here's a fix that goes back to yahoo as a benchmark source. |
|
@MarikoKujo I did the test with MarikoKujo's and shlomikushchi 's temporary fix. /home/seungyong/zipline/lib/python3.5/site-packages/empyrical/stats.py:704: RuntimeWarning: invalid value encountered in true_divide During handling of the above exception, another exception occurred: Traceback (most recent call last): |
|
This seems to have a few issues going now so Ill weigh in on the most commented one. I tried all of the workarounds with no luck. Replacing the get_bencmark_returns() method just defers to other null references down the line. Fixing the IEX API defers to the treasury data failing for me. Since I am using BTC data, OHLCV. Is there any supported way to backtest crypto currency data? Are the benchmark and treasury data pulls really that crucial? |
|
@zipper-123 Perhaps you have a faulty cached file. Remember that ensure_benchmark_data in loader.py first attempts to read from disk. That happened to my while I was tinkering with a solution. I ended up implementing a minor variation of the solution suggested by @marketneutral to sidestep the issue until it's properly fixed. I kept the signature of get_benchmark_returns, and just used a wider date range than I'll ever need. In benchmark_py And bypassing the cache in loader.py |
|
Another fix is getting a free IEX api key and altering the API request in benchmarks.py |
|
@tibkiss |
|
jesus christ .. just spent two hours on this |
|
Has the solution been found for this? |
|
yes... mine is not a temporary fix. |
|
getting the same error, tried all the solutions on this thread as well as the one here is there a solution forthcoming? |
|
Getting the same error using local data (csv -> DataFrame -> panel) and it's not clear to me how your fix works @shlomikushchi as I'm not using yahoo data. Maybe you have an idea how to tweak your code in order to fix the problem when using local data. |
|
This is still not working. @weu39843 I'm having the same issue with local data as well, passing a CSV as a DataFrame. When trying @AndreasClenow solution and all the others I get the following error: |
|
@jecker7 when you say passing a local csv, do you mean that you provide a DataPanel as the data parameter for zipline.run_algorith? That way of providing data is no longer supported and Q are strongly recommending against using it. I didn't try that method in quite some time, after they told me a couple of years ago not to use it. I'd suggest using the regular bundle/ingest process to read your csv. |
|
@shlomikushchi I tried your solution, it works well for me!! Thanks BTW, what is the point of doing the following operations: |
|
@sinnergarden glad to hear that |
|
Zipline will have to move their datasource for SPY data somewhere else besides IEX, or change the setup to require an API key. Another way to fix is:
token = 'pk_numbersnumbersnumbers'
r = requests.get(
'https://cloud.iexapis.com/stable/stock/{}/chart/5y?token={}'.format(symbol, token)
)I guess it's the nature of the beast with financial data that it's hard to find for free...but annoying. |
|
It would make far more sense for the benchmark routines to use a symbol from a bundle rather than any hardcoded external source. |
|
@stocktrader8888 no free bundles on Quandl have SPY (default benchmark I think). Could get it with the yfinance package, but that requires the latest pandas and zipline can't use latest pandas. |
|
Plus IEX only has 5y of history for SPY which is pretty weak. yfinance could get everything |
|
My take on solution:
People who are serious enough about backtesting to bother with setting up a local Zipline are not very likely to rely on Yahoo, Quandle, Google or other free sources, and they are very likely to use proper benchmarks instead of price series of an ETF. |
|
Just curious, what would be a good source for S&P 500 benchmark data, or other benchmarks that you use? |
|
I like Andreas' suggestion, namely that not having benchmark data should be a warning rather than an error. There should also, as he said, be an option for users to provide their own csv/dataframe as benchmark data. Looking at the code, the TradingEnvironment object is where the benchmark returns are ultimately loaded. I'm not sure where TradingEnvironment gets used by zipline, but I could Imagine a good way to do this would be to add a class method setter to TE where you just provide a benchmark dataframe manually. This seems like it would require the least overhead. However, referencing the first suggestion, I'm not sure what the expected behavior of zipline should be if there is no benchmark data found. It could be non-trivial to run a test without a benchmark. A workaround to this has been discussed above, where you can simply pass a dataframe of all zeros, but this certainly isn't elegant either. It would be nice to be able to circumvent this entirely, but I think for now the best solution is find some sort of consistent (albeit low quality) data source for zipline to use (like google, yahoo, etc.) I suggest creating a PR for adding @shlomikushchi 's change in the meantime, and further discussing detail on the TradingEnvironment change I added above. I can make the branch if you'd like. EDIT: It actually looks like a commit was made that addresses this issue (intentionally or not). It's not on the current pip or conda repos for zipline. But it is on the master branch of zipline. @ssanderson added an argument to run_algorithm called benchmark_returns. If you just run your algorithm with run_algorithm(), you can dowload your own benchmark in csv, run pd.read_csv() to load, and pass it to run_algorithm as benchmark_data. The only issue I'm running into now is I'm not sure the exact format that benchmark_data expects. Looking at _load_cache_data in loader.py, it looksl like this is the command you should run to read the csv, but I'm still having some issues. If anyone knows the proper formatting for benchmark data we can basically close this issue as people can just provide their own benchmark on a per-test basis. EDIT 2: I got it to work. In the process I discovered that there has been an attempt to allow custom benchmark data via creating a bundle with csvdir and adding it to set_benchmarks in initialize. https://stackoverflow.com/questions/44199678/how-to-manually-provide-a-benchmark-in-zipline https://www.zipline.io/bundles.html However, I don't like this because it requires you to have a specific ohlcvs data and seems like a lot of overhead. If you want a custom benchmark, just use the current master branch of zipline and pass benchmark_returns as an argument to run_algorithm. The format of benchmark returns is a pandas series with the adj. close (or whatever data you want) as the data and the Timestamp as the index. |
|
#2488 is related. |
The 'Publishable' Token can be used from https://iexcloud.io/ |
how can i bypass the cache? Where exactly should i paste the code? Sorry just learning and thank you. |
|
@shlomikushchi looks like your permanent solution is working for me. Thanks |
This resolved it for me (had to +import json though). Any chance this will be in the official release? |
Hi I changed the benchmark.py and loader.py accordingly.... Inside the code I set: def initialize(context): which gives an error: 'Symbol 'SPY' was not found.' using 'AAPL' as anSymbol gives no error. Than returns, positions, transactions, benchmark_rets = pf.utils.extract_rets_pos_txn_from_zipline(perf) gives again an error: ValueError: not enough values to unpack (expected 4, got 3) so it did not find: benchmark_rets I think I'm using this wrong with zipline.run_algorithm Could someone please share the correct usage. |
|
@carstenf you don't need to call |
|
@shlomikushchi ok, understood, thanks. I'm not getting any more errors. But I don't get a benchmark if I want to pull it from benchmark.py. I did the changes in benchmark.py(copied you fix), and I changed one line in the and were is the benchmark symbol defined? thanks |
|
benchmark symbol is defined in loader.py: |
|
Hey all. I'm taking a look at this this morning. |
|
We've got a bunch of issues related to the IEX-sourced benchmark data failing. In the interest of having a single canonical location for the issue, I'm closing this in favor of the newly added #2627. Feel free to comment there if there's information I've missed from this thread. |
Thank you @shlomikushchi |
|
I’m not able to find loader.py file — hows that possible ?
Anyone have a path to it by chance ?
Zach
Get Outlook for iOS<https://aka.ms/o0ukef>
…________________________________
From: awesomefrank <notifications@github.com>
Sent: Wednesday, March 4, 2020 12:13:56 PM
To: quantopian/zipline <zipline@noreply.github.com>
Cc: Zach Mazz <zach_oakes@outlook.com>; Mention <mention@noreply.github.com>
Subject: Re: [quantopian/zipline] Getting benckmark data via IEX API does not work anymore (#2480)
here's a fix that goes back to yahoo as a benchmark source.
replace this method in benchmarks.py and don't forget to change the call to it in loader.py
import numpy as np
import pandas as pd
import pandas_datareader.data as pd_reader
def get_benchmark_returns(symbol, first_date, last_date):
"""
Get a Series of benchmark returns from Yahoo associated with `symbol`.
Default is `SPY`.
Parameters
----------
symbol : str
Benchmark symbol for which we're getting the returns.
The data is provided by Yahoo Finance
"""
data = pd_reader.DataReader(
symbol,
'yahoo',
first_date,
last_date
)
data = data['Close']
data[pd.Timestamp('2008-12-15')] = np.nan
data[pd.Timestamp('2009-08-11')] = np.nan
data[pd.Timestamp('2012-02-02')] = np.nan
data = data.fillna(method='ffill')
return data.sort_index().tz_localize('UTC').pct_change(1).iloc[1:]
Thank you @shlomikushchi<https://eur04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fshlomikushchi&data=02%7C01%7C%7C681668adb70445608d1108d7c067ce5c%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637189424377455029&sdata=xUqug511h3s2xt3m%2FTsoKCPRs%2F1afHK2zq4vC4g8Y78%3D&reserved=0>
The solution works very well for me.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub<https://eur04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fquantopian%2Fzipline%2Fissues%2F2480%3Femail_source%3Dnotifications%26email_token%3DALPUCIETFFDQUCB2KWAOZPTRF2K6JA5CNFSM4HYSHEN2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOENZLJ6Y%23issuecomment-594719995&data=02%7C01%7C%7C681668adb70445608d1108d7c067ce5c%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637189424377455029&sdata=Zm1GtpfLOEmgNCuHF9%2BqorCtkJz648H51VAY4BNqJoo%3D&reserved=0>, or unsubscribe<https://eur04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FALPUCID4BLTZ7OLNF7QNJPLRF2K6JANCNFSM4HYSHENQ&data=02%7C01%7C%7C681668adb70445608d1108d7c067ce5c%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637189424377465037&sdata=l1DnsxWz5qkcYyy%2FhEtJhkqeCHQzJOoQNRZu1d%2Fjhbk%3D&reserved=0>.
|
..\site-packages\zipline\data |
|
slight variation on one of the posted answers to use the new IEX Cloud API endpoint. this one will read your token from env var --- benchmarks.py 2020-03-16 19:45:33.000000000 -0700
+++ benchmarks_new.py 2020-03-16 19:48:08.000000000 -0700
@@ -12,10 +12,15 @@
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
+from os import getenv
+
import pandas as pd
import requests
+IEX_CLOUD_API_TOKEN = getenv("IEX_CLOUD_API_TOKEN", "<IEX_CLOUD_API_TOKEN>")
+
+
def get_benchmark_returns(symbol):
"""
Get a Series of benchmark returns from IEX associated with `symbol`.
@@ -30,7 +35,7 @@
get up to 5 years worth of data.
"""
r = requests.get(
- 'https://api.iextrading.com/1.0/stock/{}/chart/5y'.format(symbol)
+ "https://cloud.iexapis.com/stable/stock/{}/chart/5y?chartCloseOnly=True&token={}".format(symbol, IEX_CLOUD_API_TOKEN)
)
data = r.json()Can apply by saving the above to a file |
Hi @k-s30011, did you find a solution? I have exactly the same issue. |
|
I am also not able to get this working following the suggestions above. It seems like it is timing out trying to divide, possibly because the benchmark is just a bunch of zeros? I am just trying to run a basic file. I am getting this same error: RuntimeWarning: invalid value encountered in true_divide I am wondering if it might be due to the version of numpy and other packages I am using. Using numpy version 1.14.6, python 3.5.6 |
I have been able to narrow down this issue to zipline trying to calculate the sharpe ratio and sorting ratio. I commented out the divide in the stats.py file and set the sharp and sorting ratio to a fixed value and it caused this error to go away. I believe the problem is that when we implemented the fix above for the API issue, the benchmark data that is loaded is zeros and this is causing the divide function issues. I don't think this is a very good solution, but I cannot find where in the zippline algorithm it is calling the stats functions so this will have to do for now. Does anyone know where in the zipline package the functions for sharpe and sorting are called? it seems like it would be better to comment those out in zipline rather than the stats module. |
|
@k-s30011 @marlowequart Looks like I'm late to the game, and a newbie but I've gotten as far as repairing the benchmarks and loader files, got my first backtest to work, albeit with the divide by zero error. Have you been successful with this onward in Andreas book? I am getting the idea that passing zero's into the code is causing it to throw those errors. Thanks for the feedback. |
|
@PatrickTunni You are better off using your own data, for example if you have your own SPY data you can use it for the benchmark returns. Andreas's is book is pretty nice in terms of programming for beginners and is probably top 5 for algo trading for beginners. Also quantopian lectures slides are really usefull and a good place to start. If you dont have SPY data, I have data from 1990 to 2020 of data, check my github |
* fix seaborn to 0.10.1 to resolve conflict * remove obsolete patch: #2480 has been fixed [#2480](quantopian/zipline#2480)
and others
Zipline uses IEX API to get benchmark data in
benchmarks.py:However, according to the IEX FAQ page, the chart api was already removed on June 15, 2019. Currently, using this api to try to download any stock data such as SPY will return nothing but an HTTP 403 error. The functions of deprecated APIs are now transferred to their new API, IEX Cloud, which requires a unique token per user in any request.
Any idea how to fix this issue in the long run?
The text was updated successfully, but these errors were encountered: