New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CLI #325

Merged
merged 10 commits into from May 7, 2014

Conversation

Projects
None yet
2 participants
@twiecki
Contributor

twiecki commented Apr 23, 2014

Adds a CLI and IPython magic for zipline.

CLI example:

python run_algo.py -f dual_moving_avg.py --symbols AAPL --start 2011-1-1 --end 2012-1-1 -o dma.pickle

Grabs the data from yahoo finance, runs the file dual_moving_avg.py (and looks for dual_moving_avg_analyze.py which, if found, will be executed after the algorithm has been run), and outputs the perf dataframe to dma.pickle.

IPython magic command (at the top of an IPython notebook cell):

%%zipline --symbols AAPL --start 2011-1-1 --end 2012-1-1 -o perf

Does the same as above except instead of executing the file looks for the algorithm in the cell and instead of outputting the perf df to a file, creates a variable in the namespace called perf.

# Do argv default this way, as doing it in the functional
# declaration sets it at compile time.
if argv is None:
argv = sys.argv

This comment has been minimized.

@ehebert

ehebert Apr 24, 2014

Member

Is the argv setting still needed here?
Or does the use of argparse supplant it?

sharpe = [risk['sharpe'] for risk in dma.risk_report['one_month']]
print("Monthly Sharpe ratios: {0}".format(sharpe))
plt.gcf().set_size_inches(18, 8)
"""Dual Moving Average Crossover algorithm.

This comment has been minimized.

@ehebert

ehebert Apr 24, 2014

Member

I love how this cleans up the examples.

@@ -0,0 +1,64 @@
#

This comment has been minimized.

@ehebert

ehebert Apr 24, 2014

Member

What do you thank about adding an alias in the environment bin that is zipline-runner or some better name.

import os
from copy import copy
import cProfile

This comment has been minimized.

@ehebert

ehebert Apr 24, 2014

Member

Should we add a try/except around these, like Pygments, as wel?

@twiecki

This comment has been minimized.

Contributor

twiecki commented May 1, 2014

OK, I think this is ready for another round of reviews. Has now also the IPython magic!

@ehebert

This comment has been minimized.

Member

ehebert commented May 2, 2014

Awesome. Looking forward to digging into it!
On May 1, 2014 7:50 PM, "Thomas Wiecki" notifications@github.com wrote:

OK, I think this is ready for another round of reviews. Has now also the
IPython magic!


Reply to this email directly or view it on GitHubhttps://github.com//pull/325#issuecomment-41970020
.

.travis.yml Outdated
@@ -14,6 +14,6 @@ install:
- conda install --yes -c https://conda.binstar.org/twiecki numpy==1.8.0 scipy nose matplotlib pandas Cython patsy statsmodels tornado pyparsing xlrd mock pytz requests six dateutil ta-lib logbook
- pip install pyflakes==0.7.3 pep8==1.4.6 mccabe==0.2.1 flake8==2.1.0 nose-parameterized==0.3.3 nose-ignore-docstring==0.2
before_script:
- "flake8 zipline tests"
- "flake8 --ignore=F821 zipline tests"

This comment has been minimized.

@ehebert

ehebert May 5, 2014

Member

What's the ignore here?

Been shying away from having any customization on flake8, but if we do I think we should use tox.ini or something else that will be picked up both in dev and the CI server.

This comment has been minimized.

@twiecki

twiecki May 5, 2014

Contributor

That's the call below to get_ipython() which is present only if IPython NB is active. So this ignores unimported function calls. Haven't used tox before.

This comment has been minimized.

@ehebert

ehebert May 5, 2014

Member

I'd recommend adding # flake8: noqa to that specific line, since the unimported function call is a useful check. (It has prevented my accidentally omitting an import or having a typo in the function call at least a few times.)

This comment has been minimized.

@twiecki

twiecki May 5, 2014

Contributor

Ah, didn't know that was possible. That's a better option.

@@ -0,0 +1,10 @@
import matplotlib.pyplot as plt

This comment has been minimized.

@ehebert

ehebert May 5, 2014

Member

Copyright.

This comment has been minimized.

@twiecki

twiecki May 5, 2014

Contributor

I had that initially but removed it because it garbles up the output when ran from command line.

@@ -0,0 +1,22 @@
import matplotlib.pyplot as plt

This comment has been minimized.

@ehebert

ehebert May 5, 2014

Member

Copyright.

This comment has been minimized.

@ehebert

ehebert May 5, 2014

Member

Also might be good to have a module level docstring explaining how this file is paired with dual_moving_average.py, and how it provides the analyze functionality.

This comment has been minimized.

@twiecki

twiecki May 5, 2014

Contributor

See above.

@ehebert

This comment has been minimized.

Member

ehebert commented May 5, 2014

@twiecki can you provide me with an example invocation of the script?

From the zipline root directory I tried both python scripts/run_algo.py and python scripts.run_algo.
The former errors with ImportError: No module named zipline and the latter with No module named scripts

Does the invocation require add2virtualenv having been done for zipline from its own directory?

For my setup, which is using the root directory of the repo, adding an __init__.py to the scripts directory allows python -m scripts.run_algo to find the zipline module directory which is a sibling to scripts.

Though I could be missing a setup step in my dev environment to get the paths aligned correctly.

@twiecki

This comment has been minimized.

Contributor

twiecki commented May 5, 2014

python scripts/run_algo.py should work. It's just trying to import zipline I think. Can you import zipline in general?

@ehebert

This comment has been minimized.

Member

ehebert commented May 5, 2014

Yes, from my root directory:

python -c "import zipline"; echo $?
0
from zipline.utils import parse_args, run_pipeline
if __name__ == "__main__":
parsed = parse_args(sys.argv[1:])

This comment has been minimized.

@ehebert

ehebert May 6, 2014

Member

Curious why do we use argv instead of letting argparse handle it?

If I run python -m scripts.run_algo this error occurs:

$ python -m scripts.run_algo
AAPL
Traceback (most recent call last):
  File "/usr/lib64/python2.7/runpy.py", line 162, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/lib64/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/home/ehebert/src/quantopian/qexec/zipline_repo/scripts/run_algo.py", line 23, in <module>
    run_pipeline(print_algo=True, **parsed)
  File "zipline/utils/cli.py", line 171, in run_pipeline
    with open(algo_fname, 'r') as fd:
TypeError: coercing to Unicode: need string or buffer, NoneType found

But if we let argparse handle all of the command line handling, it would output a message about the algo filename being required and print out the available options.

This comment has been minimized.

@twiecki

twiecki May 6, 2014

Contributor

The reason is that the ipython magic calls the same code for parsing the magic arguments. It's very odd that you're getting these errors which didn't occur for me. Is this python3 by any chance?

This comment has been minimized.

@ehebert

ehebert May 6, 2014

Member

Using 2.7. I think the difference might be your use of add2virtualenv vs.
my use of the repo's directory structure.

Should we standardize on one or the other. Or consider something like the
dev install that pandas uses. (Which I think is analogous to add2virtualenv)
On May 6, 2014 10:21 AM, "Thomas Wiecki" notifications@github.com wrote:

In scripts/run_algo.py:

+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import sys
+
+import zipline
+from zipline.utils import parse_args, run_pipeline
+
+if name == "main":

  • parsed = parse_args(sys.argv[1:])

The reason is that the ipython magic calls the same code for parsing the
magic arguments. It's very odd that you're getting these errors which
didn't occur for me. Is this python3 by any chance?


Reply to this email directly or view it on GitHubhttps://github.com//pull/325/files#r12326281
.

This comment has been minimized.

@twiecki

twiecki May 6, 2014

Contributor

Oh I see. Yeah, you need zipline installed for this (either via setup.py
develop or add2virtualenv). I think that's the common use-case unless you
think otherwise?

On Tue, May 6, 2014 at 10:25 AM, Eddie Hebert notifications@github.comwrote:

In scripts/run_algo.py:

+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import sys
+
+import zipline
+from zipline.utils import parse_args, run_pipeline
+
+if name == "main":

  • parsed = parse_args(sys.argv[1:])

Using 2.7. I think the difference might be your use of add2virtualenv vs.
my use of the repo's directory structure. Should we standardize on one or
the other. Or consider something like the dev install that pandas uses.
(Which I think is analogous to add2virtualenv)
… <#145d1ecbe9ff1dc1_>
On May 6, 2014 10:21 AM, "Thomas Wiecki" notifications@github.com
wrote: In scripts/run_algo.py: > +# > +#
http://www.apache.org/licenses/LICENSE-2.0 > +# > +# Unless required by
applicable law or agreed to in writing, software > +# distributed under the
License is distributed on an "AS IS" BASIS, > +# WITHOUT WARRANTIES OR
CONDITIONS OF ANY KIND, either express or implied. > +# See the License for
the specific language governing permissions and > +# limitations under the
License. > + > +import sys > + > +import zipline > +from zipline.utils
import parse_args, run_pipeline > + > +if name == "main": > +
parsed = parse_args(sys.argv[1:]) The reason is that the ipython magic
calls the same code for parsing the magic arguments. It's very odd that
you're getting these errors which didn't occur for me. Is this python3 by
any chance? — Reply to this email directly or view it on GitHub<
https://github.com/quantopian/zipline/pull/325/files#r12326281> .


Reply to this email directly or view it on GitHubhttps://github.com//pull/325/files#r12326556
.

Thomas Wiecki
PhD candidate, Brown University
Quantitative Researcher, Quantopian Inc, Boston

This comment has been minimized.

@ehebert

ehebert May 6, 2014

Member

I can change my dev setup to match the more common setup.

Not sure if we have that step in the development docs, if not let's add a
note about running setup.py develop

ehebert and others added some commits Apr 10, 2014

ENH: Implement CLI.
Add a CLI that reads in an algorithm, loads data,
run the algorithm, and output performance metrics.

The examples are adapted to the new zipline API and
analyses are split into separate files.

Also add config files that run the example
algorithms with preset settings.
ENH: Add IPython cell magic.
When zipline is imported it checks whether
it runs in the IPython notebook. If it does,
it registers a %%zipline magic that takes the
same arguments as the CLI with the addition of
a -o for specifying the output variable to store
the performance frame in.

The algo code in the cell is, as of yet, executed
in its own environment rather than that of the
IPython NB which is probably what we want.

Also adds cli option to save the perf dataframe
to a pickle file.

Also adds an IPython notebook buyapple example.

@twiecki twiecki merged commit c444587 into master May 7, 2014

1 check passed

continuous-integration/travis-ci The Travis CI build passed
Details

@ehebert ehebert deleted the cmdline_run branch Feb 2, 2015

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment