A script to help new devs

[ericmjl]

At the recent PyCon 2016, there was a bunch of us who wanted to work on documentation, but had issues getting setup.

In response to these frustrations, I created a script (pasted below) that I think may help others. Tested only on Mac OS X, likely will work on Ubuntu Linux. Windows - sorry folks 😦, I don't have access to a Windows Box.

Name this script dev-start.sh, place it in the bokeh/scripts directory, and execute the script.

# A script that gets new contributors up and running asap.
# 
# Author: Eric J. Ma, MIT, github: @ericmjl
# Help from:
# - Andrea Grant, Univ. Minnesota, github: @andreagrant
# - Michael Gilbert, github: @zyell
# 
# Written during the PyCon 2016 Sprints.
# 
# Purpose: The script creates a new conda environment called 
# "bokehdev", activates it, installs all dependencies as determined by
# going through the documentation page and manually checking,
# and executes the commands in order. We wrote this script because
# we ourselves were going through some trouble getting setup, and
# want to save others their time.
# 
#! /usr/bin/sh
cd ..
conda create -n "bokehdev" python=3.5
source activate bokehdev
cd bokehjs
# Install bokeh and node.js & dependencies.
conda install -y bokeh nodejs jupyter flexx nbformat
# Then remove bokeh so that we can re-install in development mode.
conda remove bokeh -y
npm install
conda install -y -c javascript gulp
gulp dev-build
cd ..
python setup.py develop --build_js
conda install -y flask matplotlib docutils pygments pyyaml seaborn sphinx yaml patsy statsmodels pytest
pip install --upgrade pip
pip install brewer2mpl graphviz
pip install ggplot sphinxcontrib-httpdomain
pip install beautifulsoup4 colorama pytest pytest-cov pytest-selenium mock websocket-client
cd sphinx
export BOKEH_DOCS_CDN="local"
make all
make serve

[ericmjl]

Works on OS X.

[damianavila]

We have a script to install all the dependencies you need here: https://github.com/bokeh/bokeh/blob/master/scripts/dev_environment

We should advertise that better... I guess...

[ericmjl]

@damianavila totally!

[birdsarah]

@damian - I can't get that script to run - any suggestions.

First, it requires conda-build, which isn't obvious.

If I run ./scripts/dev_environment -e bokeh-dev:

Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
  File "/Users/caged/miniconda3/lib/python3.4/site-packages/conda_build/metadata.py", line 333, in __init__
    assert isdir(path)
AssertionError
Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
  File "/Users/caged/miniconda3/lib/python3.4/site-packages/conda_build/metadata.py", line 333, in __init__
    assert isdir(path)
AssertionError
Error: too few arguments, must supply command line package specs or --file
BUILD dependecies installed.
Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
  File "/Users/caged/miniconda3/lib/python3.4/site-packages/conda_build/metadata.py", line 333, in __init__
    assert isdir(path)
AssertionError
Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
  File "/Users/caged/miniconda3/lib/python3.4/site-packages/conda_build/metadata.py", line 333, in __init__
    assert isdir(path)
AssertionError
Error: too few arguments, must supply command line package specs or --file
RUN dependecies installed.
Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
  File "/Users/caged/miniconda3/lib/python3.4/site-packages/conda_build/metadata.py", line 333, in __init__
    assert isdir(path)
AssertionError
Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
  File "/Users/caged/miniconda3/lib/python3.4/site-packages/conda_build/metadata.py", line 333, in __init__
    assert isdir(path)
AssertionError
Error: too few arguments, must supply command line package specs or --file
TEST (and examples) dependecies installed.

If I run ./dev_environment -e bokeh-dev:

usage: conda [-h] [-V] [--debug] command ...
conda: error: unrecognized arguments: python distribute setuptools nodejs>=4.1 six python-dateutil numpy yaml pyyaml jinja2 tornado>=4.3
BUILD dependecies installed.
usage: conda [-h] [-V] [--debug] command ...
conda: error: unrecognized arguments: python six requests pyyaml python-dateutil jinja2 numpy tornado>=4.3 futures-compat
RUN dependecies installed.
usage: conda [-h] [-V] [--debug] command ...
conda: error: unrecognized arguments: graphviz pygments sphinx>=1.3.1 colorama coverage flake8 mock pandas pytest>=2.9.0 pytest-cov==1.8.1 pytest-selenium>=1.2.1 pytest-xdist pytest-rerunfailures beautiful-soup scipy pillow boto pdiff jupyter matplotlib sympy ggplot seaborn icalendar flask flexx
TEST (and examples) dependecies installed.

[birdsarah]

Also, does this let you pick a version of python? I'm not sure it would. My conda root is py3.4., but I wanted to set-up a py 3.5 environment.

[willingc]

@damianavila :( I couldn't get it to work either. Clearly I'm missing something too. I've been using py 3.5.

[willingc]

@birdsarah Eric's script didn't work flawlessly for me. Though even with debugging it was less than 5 minutes. 😵

[damianavila]

First, it requires conda-build, which isn't obvious.

Yep, it uses conda build to be able to always be in sync with the packages outlined in the conda recipe... so, theoretically, the script will be always updated from the packages point of view... BUT, I have to test it with the latest conda-build 😉, because it could be outdated in that specific point.
I will try to do it test it and report back...

Also, does this let you pick a version of python? I'm not sure it would. My conda root is py3.4., but I wanted to set-up a py 3.5 environment.

If you create a py35 env, and use the script pointing to that env, it should install the py35 packages as expected...

[willingc]

Thanks @damianavila

[birdsarah]

@damianavila - also worth noting this bug I reported here: conda/conda#2602
which means that >= has problems fro me.

I also wrote-up a guide here: https://bokeh-cookbook.github.io/blog/2016/06/05/setting-up-your-dev-environment/

[StevenCHowell]

There is also the doc page, generated from the source code then built using sphinx. There are some inconsistencies in directions contained in that file, e.g., multiple ways to do the same thing (including multiple shortcuts) and mis-ordered bits.

[bryevdv]

I've made a substantial pass at the dev docs "getting started" as well as simplified some install steps greatly with a deps.py that reads the single source of truth from the conda recipe to install. There is also an open PR to add some docker files for users that want to go that route. I think this can be closed at this point.