Example run error "sqlalchemy.exc.InvalidRequestError: .."

[guoweihw]

Dear Zipline Maintainers,

Before I tell you about my issue, let me describe my environment:

Environment

• Operating System:# (Windows 7)
• Python Version: $ python --version 2.7.11
• Python Bitness: $ python -c 'import math, sys;print(int(math.log(sys.maxsize + 1, 2) + 1))' 64
• How did you install Zipline: (pip, conda, or other (please explain)) conda
• Python packages: $ pip freeze or $ conda list

  packages in environment at C:\Anaconda2:

  _nb_ext_conf 0.2.0 py27_0
  alabaster 0.7.8 py27_0
  anaconda 4.1.0 np111py27_0
  anaconda-client 1.4.0 py27_0
  anaconda-navigator 1.2.1 py27_0
  argcomplete 1.0.0 py27_1
  astropy 1.2.1 np111py27_0
  babel 2.3.3 py27_0
  backports 1.0 py27_0
  backports_abc 0.4 py27_0
  beautifulsoup4 4.4.1 py27_0
  bitarray 0.8.1 py27_1
  bokeh 0.11.1 py27_0
  boto 2.40.0 py27_0
  bottleneck 1.0.0 np110py27_1
  bzip2 1.0.6 vc9_3 [vc9]
  cdecimal 2.3 py27_2
  cffi 1.6.0 py27_0
  chest 0.2.3 py27_0
  click 6.6 py27_0
  cloudpickle 0.2.1 py27_0
  clyent 1.2.2 py27_0
  colorama 0.3.7 py27_0
  comtypes 1.1.2 py27_0
  conda 4.1.6 py27_0
  conda-build 1.21.2 py27_0
  conda-env 2.5.1 py27_0
  configobj 5.0.6 py27_0
  configparser 3.5.0b2 py27_1
  console_shortcut 0.1.1 py27_1
  contextlib2 0.5.3 py27_0
  cryptography 1.4 py27_0
  curl 7.49.0 vc9_0 [vc9]
  cycler 0.10.0 py27_0
  cython 0.24 py27_0
  cytoolz 0.8.0 py27_0
  dask 0.10.0 py27_0
  datashape 0.5.2 py27_0
  decorator 4.0.10 py27_0
  dill 0.2.5 py27_0
  docutils 0.12 py27_2
  entrypoints 0.2.2 py27_0
  enum34 1.1.6 py27_0
  et_xmlfile 1.0.1 py27_0
  fastcache 1.0.2 py27_1
  flask 0.11.1 py27_0
  flask-cors 2.1.2 py27_0
  freetype 2.5.5 vc9_1 [vc9]
  funcsigs 1.0.2 py27_0
  functools32 3.2.3.2 py27_0
  futures 3.0.5 py27_0
  get_terminal_size 1.0.0 py27_0
  gevent 1.1.1 py27_0
  greenlet 0.4.10 py27_0
  grin 1.2.1 py27_3
  h5py 2.6.0 np111py27_0
  hdf5 1.8.15.1 vc9_4 [vc9]
  heapdict 1.0.0 py27_1
  idna 2.1 py27_0
  imagesize 0.7.1 py27_0
  ipaddress 1.0.16 py27_0
  ipykernel 4.3.1 py27_0
  ipython 4.2.0 py27_0
  ipython_genutils 0.1.0 py27_0
  ipywidgets 4.1.1 py27_0
  itsdangerous 0.24 py27_0
  jdcal 1.2 py27_1
  jedi 0.9.0 py27_1
  jinja2 2.8 py27_1
  jpeg 8d vc9_0 [vc9]
  jsonschema 2.5.1 py27_0
  jupyter 1.0.0 py27_3
  jupyter_client 4.3.0 py27_0
  jupyter_console 4.1.1 py27_0
  jupyter_core 4.1.0 py27_0
  libpng 1.6.22 vc9_0 [vc9]
  libtiff 4.0.6 vc9_2 [vc9]
  llvmlite 0.11.0 py27_0
  locket 0.2.0 py27_1
  Logbook 1.0.0
  lxml 3.6.0 py27_0
  markupsafe 0.23 py27_2
  matplotlib 1.5.1 np110py27_0
  menuinst 1.4.1 py27_0
  mistune 0.7.2 py27_0
  mkl 11.3.3 1
  mkl-service 1.1.2 py27_2
  mpmath 0.19 py27_1
  multipledispatch 0.4.8 py27_0
  nb_anacondacloud 1.1.0 py27_0
  nb_conda 1.1.0 py27_0
  nb_conda_kernels 1.0.3 py27_0
  nbconvert 4.2.0 py27_0
  nbformat 4.0.1 py27_0
  nbpresent 3.0.2 py27_0
  networkx 1.11 py27_0
  nltk 3.2.1 py27_0
  nose 1.3.7 py27_1
  notebook 4.2.1 py27_0
  numba 0.26.0 np111py27_0
  numexpr 2.6.0 np110py27_0
  numpy 1.10.4 py27_2
  odo 0.5.0 py27_0
  openpyxl 2.3.2 py27_0
  openssl 1.0.2h vc9_0 [vc9]
  pandas 0.17.1
  pandas 0.17.1 np110py27_0
  pandas-datareader 0.2.1
  pandas-datareader 0.2.1 py27_0
  partd 0.3.4 py27_0
  path.py 8.2.1 py27_0
  pathlib2 2.1.0 py27_0
  patsy 0.4.1 py27_0
  pep8 1.7.0 py27_0
  pickleshare 0.7.2 py27_0
  pillow 3.2.0 py27_1
  pip 8.1.2 py27_0
  ply 3.8 py27_0
  progressbar 2.3
  psutil 4.3.0 py27_0
  py 1.4.31 py27_0
  pyasn1 0.1.9 py27_0
  pycosat 0.6.1 py27_1
  pycparser 2.14 py27_1
  pycrypto 2.6.1 py27_4
  pycurl 7.43.0 py27_0
  pyflakes 1.2.3 py27_0
  pygments 2.1.3 py27_0
  pyopenssl 0.16.0 py27_0
  pyparsing 2.1.4 py27_0
  pyqt 4.11.4 py27_6
  pyreadline 2.1 py27_0
  pytables 3.2.2 np111py27_4
  pytest 2.9.2 py27_0
  python 2.7.11 5
  python-dateutil 2.5.3 py27_0
  pytz 2016.4 py27_0
  pywin32 220 py27_1
  pyyaml 3.11 py27_4
  pyzmq 15.2.0 py27_0
  qt 4.8.7 vc9_8 [vc9]
  qtconsole 4.2.1 py27_0
  qtpy 1.0.2 py27_0
  bcolz 0.12.1 np110py27_0 Quantopian
  cachetools 1.1.6 py27_0 Quantopian
  cyordereddict 0.2.2 py27_0 Quantopian
  intervaltree 2.1.0 py27_0 Quantopian
  logbook 0.12.5 py27_0 Quantopian
  sortedcontainers 1.4.4 py27_0 Quantopian
  zipline 1.0.1 np110py27_2_g85327d8 Quantopian
  requests 2.10.0 py27_0
  requests-file 1.4
  requests-file 1.4 py27_0
  rope 0.9.4 py27_1
  ruamel_yaml 0.11.7 py27_0
  scikit-image 0.12.3 np111py27_1
  scikit-learn 0.17.1 np110py27_1
  scipy 0.17.1 np110py27_1
  seaborn 0.7.0 py27_0
  setuptools 23.0.0 py27_0
  simplegeneric 0.8.1 py27_1
  singledispatch 3.4.0.3 py27_0
  sip 4.16.9 py27_2
  six 1.10.0 py27_0
  snowballstemmer 1.2.1 py27_0
  sockjs-tornado 1.0.3 py27_0
  sphinx 1.4.1 py27_0
  sphinx_rtd_theme 0.1.9 py27_0
  spyder 2.3.9 py27_0
  sqlalchemy 1.0.13 py27_0
  sqlite 3.13.0 vc9_1 [vc9]
  ssl_match_hostname 3.4.0.2 py27_1
  statsmodels 0.6.1 np110py27_1
  sympy 1.0 py27_0
  tk 8.5.18 vc9_0 [vc9]
  toolz 0.8.0 py27_0
  tornado 4.3 py27_1
  traitlets 4.2.1 py27_0
  unicodecsv 0.14.1 py27_0
  vs2008_runtime 9.00.30729.1 2
  werkzeug 0.11.10 py27_0
  wheel 0.29.0 py27_0
  xlrd 1.0.0 py27_0
  xlsxwriter 0.9.2 py27_0
  xlwings 0.7.2 py27_0
  xlwt 1.1.2 py27_0
  zlib 1.2.8 vc9_3 [vc9]

Now that you know a little about me, let me tell you about the issue I am
having:

Description of Issue

• What did you expect to happen?
  zipline is installed, I expect I can run the example.
  zipline run -f ../zipline/examples/buyapple.py --start 2011-1-1 --end 2012-1-1 -o dma.pickle
• What happened instead?
  .......
  (bind.engine.url, s, ', '.join(missing)))
  sqlalchemy.exc.InvalidRequestError: Could not reflect: requested table(s) not av
  ailable in sqlite:///C:\Users\Guowei/.zipline\data\quantopian-quandl\2016-07-05T
  09;28;22.970000\assets-3.sqlite: (futures_root_symbols, version_info, asset_rout
  er, equities, futures_contracts, futures_exchanges)

Here is how you can reproduce this issue on your machine:

Reproduction Steps

...

What steps have you taken to resolve this already?

...

Anything else?

...

Sincerely,
$ whoami
Wei

[guoweihw]

if i use the following command:
python run_algo.py -f ../examples/buyapple.py --start 2011-1-1 --end 2012-1-1 -o out.pickle
The error information is as follows:

Traceback (most recent call last):
File "run_algo.py", line 23, in
from zipline.utils.calendars import default_nyse_schedule
ImportError: No module named calendars

Thanks for any solutions or advices.

[enriquedecote]

having the exact same InvalidRequestError

[rjmoggach]

same issue here - probably related to assets-5.sqlite being 0 bytes?

[rjmoggach]

try using a release instead of master branch - master is not always stable

https://github.com/quantopian/zipline/releases

[smujahed]

Dear Devs, is there anything happening with this? Maybe @richafrank could help? I'm having the same issue...

EDIT: I have built a new data bundle, filled it with APPL info from Yahoo finance and used that to test buyapple.py . It works using my dataset, so I can only assume that the issue is with the quantopian-quandl bundled data.

To create data using Yahoo! look here http://www.zipline.io/bundles.html#yahoo-bundle-factories

[richafrank]

I've reproduced the InvalidRequestError. Agreed that the 0-byte assets db is the issue - looks to be the same as #1228, which is fixed now on master.

When I update to the latest build from master:

> conda install -c https://conda.anaconda.org/Quantopian/label/ci -c quantopian zipline

I'm able to run

> zipline run -f Miniconda2\envs\zp\Lib\site-packages\zipline\examples\buyapple.py  --start 2011-1-1 --end 2012-1-1
[2016-08-23 02:12:21.764000] INFO: Performance: Simulated 252 trading days out of 252.
[2016-08-23 02:12:21.764000] INFO: Performance: first open: 2011-01-03 14:31:00+00:00
[2016-08-23 02:12:21.764000] INFO: Performance: last close: 2011-12-30 21:00:00+00:00
...

successfully.

[richafrank]

Regarding the other error mentioned here (ImportError: No module named calendars), it's definitely confusing. run_algo.py is the pre-1.0 cli, but the default_nyse_schedule name was a short-lived piece of code that existed in some commits of master only after the latest release, i.e. post-1.0.1.

I'm not sure what commit was installed at the time, but I'd try the latest master @guoweihw , using the above command. Maybe zipline was installed some way other than conda or pip and didn't compile the c extensions?

If that doesn't solve the issue, let's make a new one in github, since it seems unrelated to the InvalidRequestError.

[smujahed]

Hey @richafrank I need advice on whether to open a new issue, since this is still happening on my system, although I am using a different operating system (OS X) and possibly different packages. What would you advise?

[scarrick68]

I'm getting the same error on mac. I tried uninstalling zipline via pip and reinstalling, but that didn't solve it. I also tried clearing all the data via "zipline clean..." but that didn't do anything as seen by "zipline bundles". I'm using python 3.4.4 and pip3.

[d3ld0t]

I am having the same problem. I tried installing the master branch as suggested by @richafrank ,with pip:
sudo -H pip3 install -e git+https://github.com/quantopian/zipline@master#egg=zipline_master
and am having the same InvalidRequestError.

[richafrank]

I'm getting the same error on mac.

If this is more general than windows, then I'll just re-label this.

Also, #1443 was spotted today, but using latest master should actually be a solution to that issue, so it's likely different from this one.

[d3ld0t]

If this is more general than windows, then I'll just re-label this.

Indeed, I was getting this error on Kubuntu 16.04.1

[parrondo]

Same error on Centos 7, solved with richafrank suggestion:
conda install -c https://conda.anaconda.org/Quantopian/label/ci -c quantopian zipline

[BlackForgeOne]

Having the same issue. I tried richafrank's suggestion to
conda install -c https://conda.anaconda.org/Quantopian/label/ci -c quantopian zipline, which downloaded and updated fine, but when I tried running my algo, I still get the same:

InvalidRequestError: Could not reflect: requested table(s) not available in sqlite:///C:\Users\JKLIMCAK/.zipline\data\quantopian-quandl\2016-08-31T11;05;48.373000\assets-5.sqlite: (asset_router, futures_root_symbols, version_info, equity_symbol_mappings, equities, futures_contracts, futures_exchanges)

Any ideas or anything else I can provide to help out? Do you think uploading my own data bundles would solve this issue?

[richafrank]

@CaballeroJoe Can you tell if the file at that path is empty?

[BlackForgeOne]

@richafrank Indeed looks like it. The conda install -c https://conda.anaconda.org/Quantopian/label/ci -c quantopian zipline went through fine however, what's the best way of getting that asset db file?

[richafrank]

Have you run zipline ingest after updating to that CI version zipline?

[nicoCalvo]

I had the same issue and was solved by re-running zipline ingest

[ColeMurray]

Also experiencing InvalidRequestError when running the buyApple.py. Running on OSX python2.7

Zipline1.0.2 was installed via pip. Have tried installing / reinstalling. Cleaning & re-ingesting.
Asset-5 is not created until running the example. After running, it is an empty file.

Any update on this?

Update:
Running sudo -H pip install -e git+https://github.com/quantopian/zipline@master#egg=zipline_master

worked for me. One thing I observed was that zipline master is using pandas 18.1, while pip's is using 17.1.

[turboslon]

Ubuntu 16.04 x64, python 2.7
zipline 1.0.2 (via pip). Confirm the same error (InvalidRequestError) and assets-5.sqlite is zero-length.

[pbharrin]

I had this issue as well, a bundle I ingested on 2016/11/29 worked but a new bundle ingested on 12/19/2016 did not work. The only thing that worked for me was clearing all installed packages and reinstalling Zipline via:
sudo -H pip install -e git+https://github.com/quantopian/zipline@master#egg=zipline_master
also I had to remove the original ~/.zipline/ folder for this work.

[TimMurnaghan]

Another me too. Fresh install on windows 7 from conda.
Not even close to working "out-of-the-box"

First can't get the quandl data
requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://s3.amazonaws.com/quantopian-public-zipline-data/quandl

Tried a yahoo bundle and now get:
sqlalchemy.exc.InvalidRequestError: Could not reflect: requested table(s) not available in sqlite:///C:\Users\Tim/.zipline\data\quantopian-quandl\2017-04-01T17;
52;27.676000\assets-5.sqlite: (asset_router, futures_root_symbols, version_info,
equity_symbol_mappings, equities, futures_contracts, futures_exchanges)

Just too hard to get going if the examples are this tough - bounced off and trying some alternatives.

[freddiev4]

@TimMurnaghan that was an issue we had for about a day but was fixed (see #1736)

Otherwise it looks like this works fine on master when I ran zipline ingest -b quantopian-quandl and then ran my example algorithm

[acanacar]

on my ubuntu 16.04 i got same error :
SQLite file '/home/cem/.zipline/data/csvdir/2017-12-23T22;58;03.941248/assets-6.sqlite' doesn't exist.
i tried with conda and zipline master version but still got same issue.

[freddiev4]

@acanacar that looks like it's coming from the csvdir bundle and not quantopian-quandl. Did you run zipline ingest -b csvdir with your own set of equities?