Quandl price handler #53
Conversation
|
After thought: |
|
My opinion is that a more general price handler with Pandas DataFrame (or Pandas Panel when several tickers are given) as input will be better. An example with Pandas DataReader In [1]: import pandas_datareader as pdr
In [12]: pdr.DataReader(["GOOG", "IBM"], "yahoo")
Out[2]:
<class 'pandas.core.panel.Panel'>
Dimensions: 6 (items) x 1630 (major_axis) x 2 (minor_axis)
Items axis: Open to Adj Close
Major_axis axis: 2010-01-04 00:00:00 to 2016-06-23 00:00:00
Minor_axis axis: GOOG to IBMI'm just wondering if Quandl Python client support 3D datastructure... |
|
Quandl does return data already in Pandas DataFrame format. The only problem with pandas_datareader is that they don't support Quandl as a data source. This change was specifically to leverage the databases provided by Quandl; in particular futures data and other hard to find economic data. As to your suggestion about a more general handler. I guess what we have is price handlers that are starting with data in some format (i.e. CSV), and the price handler is written to recognize the format. i.e. YahooPriceHandler isn't getting the data from Yahoo (someone/something already downloaded that data into CSV format), rather its built to just recognize the format of data in a CSV file. I guess I did something a bit different here. I could have downloaded the CSV file from Quandl, and then had my price handler read the CSV file and load it into a DataFrame data structure. But I found the Python Quandl API already returns a Pandas DataFrame. So I just kept that. The downside is that it goes across the wire for each backtest, which I was originally concerned about, but I found the execution to be surprisingly fast (100Mbps network) - even on a daily bar /ES eminis going back to 1982 or whatever it was. But of course we'll need something that's more friendly when running "1000 backtests". One ideaI that I was tossing. If it retrieved the data, then store it as CSV (or I prefer sqlite db) as sort of a cache. Then basically use this as a data cache when deciding if it needs to go to Quandl or get it locally. Stepping back a bit. I feel like this is over engineering the price handler. Some other considerations. How does this affect a live trader. Do brokerages (i.e. IB) already give you historical data in their APIs? or does a live trader need also use this local db for strategy analysis? I just downloaded IBs API to learn more. Sorry for the long post. Just trying to flush this out some more. |
|
I understand perfectly that pandas_datareader doesn't have Quandl source... it's just an example to show what Panels are and how they can be an interesting datastructure to create a price handler (for several tickers) About caching, you might have a look at quandl/quandl-python#30 See http://pandas-datareader.readthedocs.io/en/latest/cache.html |
|
Sorry, thanks for clarifying. Let me digest it some more. Very interesting links on the caching mechanism, thanks for sharing! |
|
@jacohn16 You should have a look at quandl/quandl-python#74 |
|
Closed due to deprecation of Quandl data |
Okay I was able to get a quandl price handler working with Futures data. Although its currently using Equity pricing. The next step is to fix up QSTrader to recognize if the Ticker should be priced using Futures pricing or Equities pricing.
Note: this PR also included my previous PR to ignore Textmate files. Please suggest if there's a better workflow to handle all of this. I am pushing these from non-master branches. But I wanted to work off the same code base as the previous PR (because i wanted git to ignore Textmate files). Also, I noticed each one of my commits is showing up. Should I being squashing these? or if you accept this PR will you be able to squash? (sorry for the newbie questions. still getting used to the workflows in GitHub)