New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Getting started guide #1537

Merged
merged 99 commits into from Jun 27, 2017

Conversation

Projects
None yet
3 participants
@jlstevens
Member

jlstevens commented Jun 14, 2017

This PR will add the new getting started guide notebooks to examples, the assets and the related .rst files.

@jlstevens

This comment has been minimized.

Member

jlstevens commented Jun 14, 2017

The tentative notebook names in examples/start that I decided with Philipp are:

I-Introduction.ipynb 
II-Customization.ipynb 
III-Datasets.ipynb 
IV-Live_Data.ipynb 
V-Pipelines.ipynb 
VI-Principles.ipynb

These are filenames and the sections will probably have longer names associated with them as well.

@jbednar

This comment has been minimized.

Member

jbednar commented Jun 14, 2017

Roman numerals look a bit too formal for a software getting started guide! Those look good, apart from me not being able to guess what Pipelines are.

@jlstevens

This comment has been minimized.

Member

jlstevens commented Jun 14, 2017

The guides are now in a top level guides/ directory. I would strongly recommend renaming doc/ to sphinx/ at a later date.

@jlstevens

This comment has been minimized.

Member

jlstevens commented Jun 15, 2017

The new listing is:

1-Introduction.ipynb
2-Customization.ipynb
3-Tabular_Datasets.ipynb
4-Gridded_Datasets.ipynb
5-Live_Data.ipynb
6-Pipelines.ipynb
7-Principles.ipynb

@jbednar @philippjfr Can we try to think of a better word than 'gridded' before inventing an entirely new word?

@jbednar

This comment has been minimized.

Member

jbednar commented Jun 15, 2017

Can we try to think of a better word than 'gridded' before inventing an entirely new word?

Not sure what you mean; "gridded" is in wide use in at least some contexts (see e.g. http://scitools.org.uk/iris/docs/latest/userguide/iris_cubes.html).

@jlstevens

This comment has been minimized.

Member

jlstevens commented Jun 15, 2017

@jbednar You've convinced me about gridded being used elsewhere!

@jlstevens

This comment has been minimized.

Member

jlstevens commented Jun 15, 2017

In 08e5354 I remove the 'hidden' cell as I expect #1518 will be merged to fix the issue with bgcolor.

@@ -9,7 +9,7 @@
"\n",
"The 'Getting Started' Guide has up until this point demonstrate how HoloViews objects can wrap your data and be given a rich, useful representation. This has assumed that the data was available in memory so that it could be used to construct the appropriate object.\n",
"\n",
"The assumption that the data is immediately available in memory for analysis and visualization does not hold in many different scenarios. The data of interest may exist on some remote server making it unavailable locally until it is fetched. In other situations, the data may exist on the local disk but be too large to fit into memory. Perhaps the data doesn't even exit yet: it may be the result of some computation yet to be performed or the outcome of some live live process with the corresponding measurement not yet made.\n",
"The assumption that the data is immediately available in memory for analysis and visualization does not hold in many different scenarios. The data of interest may exist on some remote server making it unavailable locally until it is fetched. In other situations, the data may exist on the local disk but be too large to fit into memory. Perhaps the data doesn't even exist yet: it may be the result of some computation yet to be performed or the outcome of some live live process with the corresponding measurement not yet made.\n",

This comment has been minimized.

@jbednar

jbednar Jun 16, 2017

Member

live live

This comment has been minimized.

@jlstevens

jlstevens Jun 16, 2017

Member

Fixed locally, thanks! I'll push if you tell me you don't have any other fixes for me to add right now...

This comment has been minimized.

@jbednar

jbednar Jun 16, 2017

Member

Never mind; I found others so I already edited the file and pushed it. Please abandon that bit.

@jlstevens

This comment has been minimized.

Member

jlstevens commented Jun 19, 2017

@philippjfr I've initialized some rst for the two guides. Are you happy with this structure before I continue?

@jlstevens

This comment has been minimized.

Member

jlstevens commented Jun 20, 2017

In commit 7f6772e I improve the Exporting user guide, making it more robust and thus addressing #1472.

@jlstevens

This comment has been minimized.

Member

jlstevens commented Jun 21, 2017

Updated plan for user guides:

  1. Annotating your Data [redim, explicit dimension objects, label, group]
  2. Compositions of elements [+, *]
  3. Dimensioned Containers
  • Mention and link to split notebooks: HoloMap, NdLayout, GridSpace, NdOverlay, DynamicMap
  • Methods: casting, faceting, .overlay, .layout
  1. Nesting data (replacing Composing data, hierarchy of structure, traversal, map)
  2. DynamicMap: Live Data. Setting ranges etc.
  3. Gridded data [gets you to 1,2,3,5]
  4. Tabular data [gets you to 1,2,3,5]
  5. Options: Customizing Plots
  6. Transforming elements
  7. Data transformation pipelines.
  8. Streams: Responding to Events
  9. Linked Streams: Custom Interactivity
  10. Large Data
  11. Indexing and selecting Data

Supplementary" guides (no strong ordering):

  • Plotting with Matplotlib
  • Plotting with Bokeh
  • Plotting with Plotly
  • Exporting
  • Bokeh apps
  • Deployment
  • Dashboards
  • Continuous Coordinates
  • Renderer and plots (user-centric)
  • Magics
@jlstevens

This comment has been minimized.

Member

jlstevens commented Jun 22, 2017

I've added the current homepage demo (mandelbrot) to examples/demos. Note I had to heavily downsample (100x100 array instead of 400x400) and use half the frames with the bokeh version to keep the file size down. And it is still 3x larger than the matplotlib version (6MB instead of 2MB).

@jlstevens

This comment has been minimized.

Member

jlstevens commented Jun 25, 2017

@philippjfr I just fixed a syntax error in the game of life app, but it still doesn't work for me (nothing happens after loading).

I've just tested all the apps and gapminder also doesn't work..at least on Python 3:

Traceback (most recent call last):
  File "/Users/jstevens/miniconda3/envs/analytics/lib/python3.6/site-packages/bokeh/application/handlers/code_runner.py", line 81, in run
    exec(self._code, module.__dict__)
  File "/Users/jstevens/Desktop/development/holoviews/examples/apps/bokeh/gapminder.py", line 80, in <module>
    hvplot = BokehRenderer.get_plot(hvgapminder, doc)
  File "/Users/jstevens/Desktop/development/holoviews/holoviews/plotting/bokeh/renderer.py", line 113, in get_plot
    plot = super(BokehRenderer, self_or_cls).get_plot(obj, renderer)
  File "/Users/jstevens/Desktop/development/holoviews/holoviews/plotting/renderer.py", line 177, in get_plot
    **plot_opts)
  File "/Users/jstevens/Desktop/development/holoviews/holoviews/plotting/plot.py", line 797, in __init__
    self.hmap = self._apply_compositor(self.hmap, ranges, self.keys)
  File "/Users/jstevens/Desktop/development/holoviews/holoviews/plotting/plot.py", line 823, in _apply_compositor
    mapwise_ranges = self.compute_ranges(holomap, None, None)
  File "/Users/jstevens/Desktop/development/holoviews/holoviews/plotting/plot.py", line 352, in compute_ranges
    self._compute_group_range(group, elements, ranges)
  File "/Users/jstevens/Desktop/development/holoviews/holoviews/plotting/plot.py", line 412, in _compute_group_range
    dim_range = el.range(dim)
  File "/Users/jstevens/Desktop/development/holoviews/holoviews/core/data/__init__.py", line 263, in range
    drange = self.interface.range(self, dim)
  File "/Users/jstevens/Desktop/development/holoviews/holoviews/core/data/pandas.py", line 102, in range
    column = column.sort_values()
  File "/Users/jstevens/miniconda3/envs/analytics/lib/python3.6/site-packages/pandas/core/series.py", line 1710, in sort_values
    argsorted = _try_kind_sort(arr[good])
  File "/Users/jstevens/miniconda3/envs/analytics/lib/python3.6/site-packages/pandas/core/series.py", line 1700, in _try_kind_sort
    return arr.argsort(kind='quicksort')
TypeError: '>' not supported between instances of 'Text' and 'Text'

Two more comments:

  • All these apps could do with more comments and docstrings at the top of the file.
  • All these apps are suitable for displaying in the gallery except for player.py and selection_stream.py which are not particularly interesting. Maybe we can list notebooks to exclude from the gallery somewhere so we can leave these apps where they are...

I'll add the taxi datashader app shown in the collage now...

@jlstevens

This comment has been minimized.

Member

jlstevens commented Jun 25, 2017

@philippjfr I just added the economic bars example used in the collage. Two issues:

  • For bokeh I had to use %%opts Bars (color=Cycle(values=bokeh.palettes.Category20[12])) to get enough colors for the countries.
  • Unfortunately, matplotlib doesn't even have 'Category20' so I had to use Palette('tab20') instead.

We really want something that works for both backends independently! The plot option for matplotlib Bars is horrible but that is a separate issue...

@jlstevens

This comment has been minimized.

Member

jlstevens commented Jun 25, 2017

@philippjfr Actually I got confused...it seems that 'tab20' is the same as 'Category20'.

Should we set up some aliases?

@jlstevens

This comment has been minimized.

Member

jlstevens commented Jun 25, 2017

Just some reminders:

  • The colormaps in examples are correct no longer match the one in the tutorial showing the economic data (which is wrong).
  • I need to redo the images in the collage as the colormaps have changed.
@jlstevens

This comment has been minimized.

Member

jlstevens commented Jun 26, 2017

All the examples in the collage are now in the gallery for matplotlib/bokeh.

Note that in the last one (dropdown_economic) I had to use VLines/Text instead of Arrow for bokeh.

@jlstevens

This comment has been minimized.

Member

jlstevens commented Jun 27, 2017

I've stripped out all the kernel metadata fluff using:

find . -name '*.ipynb' -exec /tmp/holoviews/examples/strip_kernel.sh {} \;

Where strip_kernel.sh is:

#!/bin/bash
jq --indent 1 \
    '
    (.cells[] | select(has("outputs")) | .outputs) = []
    | (.cells[] | select(has("execution_count")) | .execution_count) = null
    | .metadata = {"language_info": {"name":"python", "pygments_lexer": "ipython3"}}
    | .cells[].metadata = {}
    ' $1 > /tmp/stripped.ipynb

This is a quick hackish approach for now and we'll find something more robust and useable from holoviews itself in future.

@jlstevens

This comment has been minimized.

Member

jlstevens commented Jun 27, 2017

+14,569 −2,546

We've done a huge amount of work in this PR. Maybe it is time to merge? Just check the notebooks work well for you having stripped out the metadata/kernel spec!

@jlstevens

This comment has been minimized.

Member

jlstevens commented Jun 27, 2017

Phew! Will be very happy to see this on master! :-)

@jlstevens jlstevens merged commit ddd6f07 into master Jun 27, 2017

2 of 4 checks passed

continuous-integration/travis-ci/pr The Travis CI build is in progress
Details
coverage/coveralls Coverage pending from Coveralls.io
Details
continuous-integration/travis-ci/push The Travis CI build passed
Details
s3-reference-data-cache Test data is cached.
Details

@jbednar jbednar deleted the getting_started_guide branch Jun 27, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment