Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Doc cleanup #401

Merged
merged 11 commits into from Jan 16, 2016

Conversation

Projects
None yet
3 participants
@jbednar
Copy link
Contributor

jbednar commented Jan 8, 2016

Partial cleanup, ready for comments (and merging if appropriate) but not yet including edits to the main tutorials. It would be very good if Philipp or Jean-Luc could inspect my changes carefully, looking especially at:

  1. I changed the tagline in the main site to match the one on Github.
  2. Is the tone and level of detail right on Homepage.ipynb? I tried to make it more explicit about the difference in philosophy between HoloViews and other libraries, but it's always hard to explain that.
  3. I added links to the new Bokeh tutorials from the Tutorials index, and references to Bokeh throughout, but there are probably other places to mention it and guide people towards which backend is appropriate for their purposes.
  4. Is the Pandas tutorial out of date? It talks about converting Pandas dataframes, but I thought we were now supporting using them as-is, not copying or converting. Does this need to be clarified somewhere?
  5. Can we simplify the GitHub README.rst? Right now it has a redundant copy of features.rst, which I vote should just be replaced with a link, but in any case at least needs to be updated from features.rst.
  6. There were a few locations, e.g. in the FAQ, where I wanted to document something (e.g. the aliases system), but the only documentation I could find was in a pull request, so I put in a link to the pull request. We can either leave in such links as a pragmatic measure, or pull the information out of the pull request into something we can maintain over time.
  7. I added "bokeh" to the conda install command, so that the result would be more like what happens for matplotlib (things just work). But the right approach is presumably to make the conda package be completely independent of matplotlib (with the default backend falling back to bokeh when mpl is not installed), and then tell people to choose whichever command they wish:
    • conda install holoviews # no plotting backend
    • conda install holoviews matplotlib # default backend is mpl
    • conda install holoviews bokeh # default backend is bokeh
    • conda install holoviews matplotlib bokeh # default backend is mpl
  8. I've run the tutorials, but don't know how to build the web site locally (which is a big pain and really discourages updating the web site!), so I can't tell if everything's still formatted properly.
  9. It needs links to the SciPy paper in appropriate locations, because that has more philosophical discussion that makes good background reading.
  10. I still need to go through the other tutorials, but won't be able to do that today or tomorrow.

jbednar added some commits Jan 8, 2016

@philippjfr

This comment has been minimized.

Copy link
Contributor

philippjfr commented Jan 8, 2016

Thanks for going through it, looks good overall.

(2) Is the tone and level of detail right on Homepage.ipynb? I tried to make it more explicit about the difference in philosophy between HoloViews and other libraries, but it's always hard to explain that.

Looks fine to me.

(4) Is the Pandas tutorial out of date? It talks about converting Pandas dataframes, but I thought we were now supporting using them as-is, not copying or converting. Does this need to be clarified somewhere?

It is yes, large parts of it can be reused as material for either a Transforming Data or Columns Tutorial.

(5) Can we simplify the GitHub README.rst? Right now it has a redundant copy of features.rst, which I vote should just be replaced with a link, but in any case at least needs to be updated from features.rst.

Not sure, it's nice to have a concrete list of features on the GitHub landing page especially since it actually receives more unique visitors than the website.

(6) There were a few locations, e.g. in the FAQ, where I wanted to document something (e.g. the aliases system), but the only documentation I could find was in a pull request, so I put in a link to the pull request. We can either leave in such links as a pragmatic measure, or pull the information out of the pull request into something we can maintain over time.

We should open an issue to collect items that should get some documentation so we can prioritize.

(7) I added "bokeh" to the conda install command, so that the result would be more like what happens for matplotlib (things just work). But the right approach is presumably to make the conda package be completely independent of matplotlib

It shouldn't be too hard to drop the matplotlib dependency for the bokeh backend and fallbacks shouldn't be too hard either. Of course any composite object will continue to simply warn about any unsupported Element types.

(8) I've run the tutorials, but don't know how to build the web site locally (which is a big pain and really discourages updating the web site!), so I can't tell if everything's still formatted properly.

This should do it:

cd doc
conda install sphinx, runipy
make ipynb-doc

Problem the tests require very specific versions of freetype and matplotlib and will only work on Linux so it'll just abort. Should add an option to build the docs without tests.

@jbednar

This comment has been minimized.

Copy link
Contributor Author

jbednar commented Jan 8, 2016

Travis says https://travis-ci.org/ioam/holoviews/jobs/101181829 failed with display output mismatch, but I can't see what the output difference is...

@jbednar

This comment has been minimized.

Copy link
Contributor Author

jbednar commented Jan 8, 2016

BTW, the Introduction tutorial shows that a Numpy array will be preserved inside a HoloViews object; can the same be shown for a Pandas dataframe or other input types? If so we should say that in that location, and show it somewhere (there if appropriate, or elsewhere).

@philippjfr

This comment has been minimized.

Copy link
Contributor

philippjfr commented Jan 8, 2016

BTW, the Introduction tutorial shows that a Numpy array will be preserved inside a HoloViews object; can the same be shown for a Pandas dataframe or other input types? If so we should say that in that location, and show it somewhere (there if appropriate, or elsewhere).

Should presumably go in the notebook about the Columns interface, which I'll adapt from the existing Pandas Conversion notebook.

@philippjfr

This comment has been minimized.

Copy link
Contributor

philippjfr commented Jan 8, 2016

Travis says https://travis-ci.org/ioam/holoviews/jobs/101181829 failed with display output mismatch, but I can't see what the output difference is...

The display tests for two cells in the Elements tutorial are still flakey, if you see a failure in cell 43 or 46 just restart the build.

@jbednar

This comment has been minimized.

Copy link
Contributor Author

jbednar commented Jan 8, 2016

Ok, I managed to cover two more of the tutorials; still lots to go, but the intro ones are the most important and have the most changes...

@jbednar

This comment has been minimized.

Copy link
Contributor Author

jbednar commented Jan 8, 2016

Oh, and I had some changes to About (and maybe others?) stashed away; any idea how to inspect the stash manually to cut and paste those out of there? If not I'll have to recreate them...

@philippjfr

This comment has been minimized.

Copy link
Contributor

philippjfr commented Jan 8, 2016

You can inspect the last stash with:

git stash show -p stash@{0}

I would seriously recommend getting magit for emacs though it makes working with git so much easier.

@jlstevens

This comment has been minimized.

Copy link
Contributor

jlstevens commented Jan 9, 2016

Yes! I highly recommend magit!

Anyway, thank you for doing this - I'll catch up and make some comments.

@jlstevens

This comment has been minimized.

Copy link
Contributor

jlstevens commented Jan 9, 2016

It is really great to see a thorough update of the website!

Here are my replies to your queries:

(1) I changed the tagline in the main site to match the one on Github.

Great!

(2) Is the tone and level of detail right on Homepage.ipynb? I tried to make it more explicit about the difference in philosophy between HoloViews and other libraries, but it's always hard to explain that.

Yes, the tone is right and I like the updated text. It is a bit wordy but I think that is unavoidable as it quite hard to convey what HoloViews offers with text only.

(5) Can we simplify the GitHub README.rst? Right now it has a redundant copy of features.rst, which I vote should just be replaced with a link, but in any case at least needs to be updated from features.rst.

I agree with Philipp. I use the README as the basis of the PyPI page (with minor tweaks) and the Anaconda Page. I also think it is important to have a nice list of features on the GitHub landing page.

(6) There were a few locations, e.g. in the FAQ, where I wanted to document something (e.g. the aliases system), but the only documentation I could find was in a pull request, so I put in a link to the pull request. We can either leave in such links as a pragmatic measure, or pull the information out of the pull request into something we can maintain over time.

Linking to PRs is fine and it is tricky to find a place for the miscellaneous odds and ends. The wiki is still a good place to record various tips and tricks as we encounter them but I do think we want a more visible place for newcomers to find them. I can imagine something in the style of the FAQ (which should be fore higher level questions) that is broken down into sections (rendering, styling, exporting etc). Whatever we do, it should be easily searchable.

(7) I added "bokeh" to the conda install command, so that the result would be more like what happens for matplotlib (things just work). But the right approach is presumably to make the conda package be completely independent of matplotlib.

Are you suggesting the conda package include bokeh as a dependency (already the case I believe) but not matplotlib? I do agree the bokeh backend would ideally not require matplotlib but I have no found a good way to do the equivalent of the extras_require with conda. That is why the conda package includes everything (and conda can install more dependencies a lot quicker than pip anyway).

Anyway, I hope we will be able to shift our attention to improving the website building process now as well as author new tutorials and improve the documentation general. For the time being, I'll have a closer look at the rest of the updated text and make inline comments if I have anything to suggest...


(hv.Image(np.random.rand(10,10), group=al.Spectrum, label=al.Glucose) +
hv.Image(np.random.rand(10,10), group=al.Spectrum, label=al.Water))
```

This comment has been minimized.

@jlstevens

jlstevens Jan 9, 2016

Contributor

I think documenting this in the FAQ isn't too bad in the end. It is probably things like a collection of final_hooks examples that are hard to find a place for...

**Q: How do I create a Layout or Overlay object from an arbitrary list?**

You can supply the list of elements directly to the ``Layout`` and
You can supply a list of ``elements`` directly to the ``Layout`` and

This comment has been minimized.

@jlstevens

jlstevens Jan 9, 2016

Contributor

Shouldn't this be Elements? Bit tricky as only the singular, Element is an actual class name but I think pluralizing it (with a capital) isn't too much of a stretch. To me elements looks a bit odd as I don't see why it should be formatted like code...

This comment has been minimized.

@jbednar

jbednar Jan 9, 2016

Author Contributor

Just me not trying to get into too much explanation; obviously it needs more. It's formatted like code to imply that it's a Python list, which I didn't feel was clear before, but if you didn't catch that, then it's still not clear.

This comment has been minimized.

@jlstevens

jlstevens Jan 9, 2016

Contributor

Ah ok, I didn't get that. The constructor accepts a list of elements but that argument isn't most likely called data because there are multiple ways it can be used.

@@ -17,7 +17,7 @@
"source": [
"import numpy as np\n",
"import holoviews as hv\n",
"%reload_ext holoviews.ipython\n",
"hv.notebook_extension('matplotlib')\n",

This comment has been minimized.

@jlstevens

jlstevens Jan 9, 2016

Contributor

It is nice to put 'matplotlib' in there even it isn't necessary to emphasize we support multiple backends. What would you think about putting a comment here...maybe along the lines of # Or 'bokeh' (if available)?

This comment has been minimized.

@jbednar

jbednar Jan 9, 2016

Author Contributor

Thanks for raising that; I meant to list it as an issue. Yes, the reason I put 'matplotlib' explicitly here on the homepage is to make it clear that (a) we are a way to access plotting libraries, not competing with them, (b) we support multiple backends, not just one, and (c) to imply that a user could type 'bokeh' here to switch the backend. I'd rather avoid putting the comment explicitly, to keep the homepage as simple as possible; I'm pretty sure that if a reader looks at the rest of the page and sees that we support matplotlib and Bokeh, they will guess to try 'bokeh' or 'Bokeh' here, and of course they can also look at the Bokeh tutorials to verify that.

@@ -27,7 +27,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"To use HoloViews, you first wrap your data in a HoloViews component along with optional metadata describing it. It will then display itself automatically on its own or in combination with any other HoloViews component. The separate [matplotlib](http://matplotlib.org) library does the plotting, but none of the data structures depend on the plotting code, so that you can easily create, save, load, and manipulate HoloViews objects from within your own programs. HoloViews objects support arbitrary combination, selection, slicing, sorting, sampling, and animation, to allow you to focus on whatever aspect of your data you wish. Instead of writing or maintaining complex plotting code, just declare what data you want to see, and HoloViews will handle the rest."
"The original data always remains available in its native format (accessible via ``fractal.data`` in this case), but accessing it via the HoloViews object instead lets the data display itself, either alone (just type ``fractal`` in an IPython/Jupyter notebook) or alongside or overlaid with other HoloViews objects as shown above. The actual plotting is done using a separate library like [matplotlib](http://matplotlib.org) or [bokeh](http://bokeh.pydata.org), but all of the HoloViews objects can be used without any plotting library available, so that you can easily create, save, load, and manipulate HoloViews objects from within your own programs. HoloViews objects support arbitrarily high dimensions, using continuous or discrete indexes and values, with flat or hierarchical organizations, and sparse or dense data formats. The objects can then be flexibly combined, selected, sliced, sorted, sampled, or animated, all by specifying what data you want to see rather than by writing plotting code. The goal is to put the plotting code into the background, as an implementation detail to be written once and reused often, letting you focus clearly on your data in daily work."

This comment has been minimized.

@jlstevens

jlstevens Jan 9, 2016

Contributor

Good to mention bokeh here but I think showing how it is enabled in the comment suggestion above might still be worth considering.

@@ -17,7 +17,7 @@ ________

* Supports a truly reproducible workflow by minimizing the code needed for analysis and visualization.
* Already used in a variety of research projects, from conception to final publication.
* All HoloViews objects can be pickled and unpickled.
* All HoloViews objects can be pickled and unpickled, with no plotting-library dependencies.

This comment has been minimized.

@jlstevens

jlstevens Jan 9, 2016

Contributor

@philippjfr Is this always true? Would styles pickle/unpickle if a Palette is used in the style options for instance?

This comment has been minimized.

@philippjfr

philippjfr Jan 9, 2016

Contributor

Don't see any reason why it wouldn't work, Palettes are just fancy cycles. There can be some issues pickling matplotlib objects (e.g. a custom ticker) but I think they were addressing that.

This comment has been minimized.

@jlstevens

jlstevens Jan 9, 2016

Contributor

Ok. So if you use a matplotlib object in style options, you will obviously have trouble unpickling the style if matplotlib is not available. This is a good reason to avoid using matplotlib objects in this way...

This comment has been minimized.

@jbednar

jbednar Jan 9, 2016

Author Contributor

Here I was referring to the objects, not the styles. If we just use unpickle, we'll only get the objects, right? And thus won't have to worry about the plotting library dependencies? The point is to make it clear that the data will always, always be able to be extracted (as long as pickles are supported in general), and thus that it's a safe way to store the data.

This comment has been minimized.

@philippjfr

philippjfr Jan 9, 2016

Contributor

That's a really good point. Maybe our hv.Store.load could somehow have a flag to toggle loading of the style options, afaik it will simply skip and warn if they can't be loaded already.

This comment has been minimized.

@jlstevens

jlstevens Jan 9, 2016

Contributor

I agree that there should never be a case where you can't unpickle the data itself just because a plotting library is missing.

What I wanted to check is that styles (mostly) have the same behavior: although technically styles aren't part of the holoviews elements (so what you say above is true if you replace the word 'object' with 'elements') a user might also expect styles to pickle/unpickle in a way that is agnostic to the plotting library.

This comment has been minimized.

@jbednar

jbednar Jan 9, 2016

Author Contributor

Such a toggle would be a good idea.

This comment has been minimized.

@jbednar

jbednar Jan 9, 2016

Author Contributor

Sure, hopefully styles will normally unpickle also; we just have to make clear that in some cases there might be an object used as a value in those styles whose class is defined by the plotting library, and thus that particular option won't be able to be restored without the plotting library (nor would it presumably make any sense in that case).

@@ -4,6 +4,7 @@
December 22nd 2015: HoloViews 1.4.1 released and now available on
<a href="https://pypi.python.org/pypi/holoviews/">PyPI</a> and
<a href="https://anaconda.org/anaconda/holoviews">Anaconda</a>.
Now includes full support for the <a href="http://bokeh.pydata.org">Bokeh</a> plotting library.

This comment has been minimized.

@jlstevens

jlstevens Jan 9, 2016

Contributor

Not sure if 'full' is the right word as not all elements are supported.

Maybe 'official' support is better? This means that as developers we aim to support bokeh as standard, not that we are necessarily promising anything. :-)

This comment has been minimized.

@jbednar

jbednar Jan 9, 2016

Author Contributor

Well, what defines the list of "all elements"? Can't the list of Bokeh elements be treated as the canonical ones, with the extra ones provided by Matplotlib as bonuses? Seems like it's full support if we say it is. :-) I'm reluctant to say "official", because that sounds like we have a team of paid support professionals at our disposal! But whatever way you and Philipp want to say it is fine. I do want to indicate that the Bokeh support is very different from the Plotly support -- the Plotly backend is not full support, right now at least.

This comment has been minimized.

@jlstevens

jlstevens Jan 9, 2016

Contributor

I don't think the plotly backend is going to be merged any time soon. I would just drop the word 'full' to keep it simple (I agree 'official' commits us to too much) so just 'Now includes support ...'

This comment has been minimized.

@philippjfr

philippjfr Jan 9, 2016

Contributor

Well the Plotly support has not been merged so no one should be under any illusion that it has full support. I did try to raise the issue of what it means for a backend to be ready for inclusion in HoloViews in that PR. As you point out in future there will likely not be one single backend that supports all the Elements. If we ever have cross-backend layout capabilities this won't be such an issue but for now people will have to very deliberately use backends for particular purposes. I think that's reasonable as long as it's easy for people to find out about the drawbacks and benefits of particular backends and what is supported. I don't think we should be overly restrictive about what constitutes a supported backend, since some backends might only support a few Element types but draw them very quickly.

This comment has been minimized.

@jbednar

jbednar Jan 9, 2016

Author Contributor

Ok, I'll remove "full". Looking at it in context, I decided that "extensive" might be a good replacement -- I do want to convey that this is not just a couple of things that are supported, but that there's been reallly a lot of work involved and it's completely viable for use in projects.

This comment has been minimized.

@jbednar

jbednar Jan 9, 2016

Author Contributor

Regarding cross-backend layouts, when Philipp and I were discussing that, I briefly thought that PhosphorJS coupled with some corresponding custom stitching of SVGs into larger SVGs for export might make us be able to solve layout problems once, for all backends. But it's hard to see how we could do that when considering aligning plots by their active area (main plot), not just their bounding boxes (including axes, labels, etc.) Maybe we could extract that information from all backends and supply it to single layout engine, but that seems ambitious. So even if we do have some support for cross-backend layouts, it seems unlikely we'll be able to use that for everything...

@jbednar

This comment has been minimized.

Copy link
Contributor Author

jbednar commented Jan 9, 2016

(7) I added "bokeh" to the conda install command, so that the result would be more like what happens for matplotlib (things just work). But the right approach is presumably to make the conda package be completely independent of matplotlib.

Are you suggesting the conda package include bokeh as a dependency (already the case I believe) but not matplotlib? I do agree the bokeh backend would ideally not require matplotlib but I have no found a good way to do the equivalent of the extras_require with conda. That is why the conda package includes everything (and conda can install more dependencies a lot quicker than pip anyway).

No, I'm suggesting that neither bokeh nor matplotlib should be dependencies of the HoloViews conda package, and that we ask users to specify whichever one or the other they wish to use, recommending that they install both but allowing them to install neither. conda install holoviews matplotlib is not unwieldy, and is explicitly about something the user cares about. (Whereas asking them to do conda install holoviews param would not make sense, as using Param is our own choice, not theirs.)

@jlstevens

This comment has been minimized.

Copy link
Contributor

jlstevens commented Jan 9, 2016

... that we ask users to specify whichever one or the other they wish to use, recommending that they install both but allowing them to install neither.

I am ok with this and it makes sense to me. It does mean we need the user to install the plotting backend themselves but that makes sense if we are going to support more options (e.g plotly) in future.

I can make this change to the conda package for the next minor release i.e 1.4.2 (which will probably happen in a week or two).

@jbednar

This comment has been minimized.

Copy link
Contributor Author

jbednar commented Jan 9, 2016

Changing the conda package spec should be straightforward, and I don't think the change should affect any existing HoloViews users, because they will have already installed matplotlib previously. But of course we do need to make sure that the codebase can run in some meaningful way without either matplotlib or bokeh (presumably a test of just creating, saving, and restoring each Element and Container type, without plotting them?). And we also need to make the Bokeh plotting work when Matplotlib is not installed, which I believe requires making our own copies of a few Matplotlib objects that we use as defaults (e.g. colormaps).

@jbednar

This comment has been minimized.

Copy link
Contributor Author

jbednar commented Jan 9, 2016

BTW, is there already an issue to fix the table of contents from the Elements tutorials online, which are not formatting and linking correctly? They seem formatted fine in my own IPython session, but online are a mess: http://holoviews.org/dev/Tutorials/Elements

@philippjfr

This comment has been minimized.

Copy link
Contributor

philippjfr commented Jan 9, 2016

BTW, is there already an issue to fix the table of contents from the Elements tutorials online, which are not formatting and linking correctly? They seem formatted fine in my own IPython session, but online are a mess: http://holoviews.org/dev/Tutorials/Elements

Not sure what happened there, might be a recent version of nbconvert or Sphinx that broke it. May have to create a pure HTML index.

@jbednar

This comment has been minimized.

Copy link
Contributor Author

jbednar commented Jan 11, 2016

What's the status of the Transforming Data tutorial currently linked from the sidebar? Should I just delete that? Or is there some less-than-perfect version we can slap up there? Anything is better than nothing, even if it's just some code with no text...

@philippjfr

This comment has been minimized.

Copy link
Contributor

philippjfr commented Jan 11, 2016

Yes, please delete it. It doesn't exist and has been subsumed by the Sampling Data and now the Columnar Data tutorial. We do still need a Tutorial about operations but that should probably be called something different.

@jbednar

This comment has been minimized.

Copy link
Contributor Author

jbednar commented Jan 12, 2016

I'm not sure what you mean about a length-1 Element "making sense" -- it's being returned from an indexing operation, not a slice. For slices it would make sense, but not for indexing, right?

@philippjfr

This comment has been minimized.

Copy link
Contributor

philippjfr commented Jan 12, 2016

  • Slices should always return elements (empty, single element..doesn't matter).
  • Concrete indexing should return a value (i.e a scalar) if it is an exact match, IndexError otherwise.

Yes, that's the what the behavior we decided on, will have to fix the length 1 bug for exact indices and add an IndexError for the no match condition.

I'm not sure what you mean about a length-1 Element "making sense" -- it's being returned from an indexing operation, not a slice. For slices it would make sense, but not for indexing, right?

Correct.

@jlstevens

This comment has been minimized.

Copy link
Contributor

jlstevens commented Jan 12, 2016

Ok, we are agreed then! Good that we have decided on what is supposed to be happening and what exactly is a bug and not a feature. :-)

@jbednar

This comment has been minimized.

Copy link
Contributor Author

jbednar commented Jan 12, 2016

Anyway, as you can probably see, I'm working on the Sampling Data tutorial, which I don't think I've ever previously edited, so it's going slowly. I think slicing is making sense mostly, barring the bug above, but I'm having trouble understanding what the different high-level categories of object really are and what is supported for each meaningful category. E.g. the Sampling Data tutorial has a section "Regularly sampled or binned data in 1D", but I can't see anything about that section that requires regular sampling. E.g. the first example in it works fine if I change the sampling to irregular:

image

So I can't figure out what the real breakdown into important categories are -- slicing just works, and so isn't informative of category, discrete vs. continuous seems to be used inconsistently, regular sampling isn't needed for the cases it's said to be needed, etc. -- what's the real story? I'm trying to work it out...

@philippjfr

This comment has been minimized.

Copy link
Contributor

philippjfr commented Jan 12, 2016

So I can't figure out what the real breakdown into important categories are -- slicing just works, and so isn't informative of category, discrete vs. continuous seems to be used inconsistently, regular sampling isn't needed for the cases it's said to be needed, etc. -- what's the real story? I'm trying to work it out...

Histograms are different again since their data is defined in terms of bin boundaries. We ideally also want to switch those to be Columns types eventually. Their current indexing behavior should return the value for the current bin you're in, while slicing has to include the bin boundaries.

The only real distinction that some types are treated as discrete samples of a continuous space and therefore snap. This is currently to restricted 1D Columns types (Curve, Scatter, ErrorBars) and Histograms. All other types are agnostic to their sampling.

@jbednar

This comment has been minimized.

Copy link
Contributor Author

jbednar commented Jan 12, 2016

You're also right that it doesn't really have anything to do with regular sampling, nothing about Columns, Histogram and Path types assumes or enforces anything about the data being regularly sampled. So I guess you should remove references to that.

Ok, will do.

The only real distinction that some types are treated as discrete samples of a continuous space and therefore snap. This is currently to restricted 1D Columns types (Curve, Scatter, ErrorBars) and Histograms. All other types are agnostic to their sampling.

But don't Image and other Raster types also support snapping? In that case the regular grid is important, because it allows you to compute the nearest neighbor in 2D with no search and a well-bounded uncertainty on the point position.

@philippjfr

This comment has been minimized.

Copy link
Contributor

philippjfr commented Jan 12, 2016

But don't Image and other Raster types also support snapping? In that case the regular grid is important, because it allows you to compute the nearest neighbor in 2D with no search and a well-bounded uncertainty on the point position.

True, I was only referring to Columns types and the other types that could eventually become Column like. Raster types are another example of Elements that are represent discrete samples of a continuous space and therefore snap (this time in 2D). QuadMesh is again discrete but not necessarily regular.

Here's a breakdown of the way I see the different Elements:

0D Continuous: Distribution, Spikes*, BoxWhisker*
1D discrete with snapping: Scatter, Curve, Histogram, ErrorBars
1D general: Bars*
2D discrete with snapping: Raster, Image, Surface, QuadMesh
2D general: HeatMap
2D general: Points, Paths, Polygons
3D general: Scatter3D, Trisurface

Note he dimensionality here refers to the dimensionality of the key dimensions, Columns types allow any number of value dimensions to be defined.

* - can also be multi-dimensional

@jbednar

This comment has been minimized.

Copy link
Contributor Author

jbednar commented Jan 12, 2016

Thanks. Is it safe to say this?

All Element types support slicing using a syntax likee[a:b], which will return another Element of the same type, with the data from the specified range [*a*,*b*).
Some Elements also support indexing, using a syntax likee[a], which will return scalar values representing thevdims(value_dimensions) of the nearest data point. Specifically, theScatter,Curve,Histogram, andErrorBars1D Element types, and all of the 2DRaster-based types (Raster,Image,RGB,HSV,QuadMesh) support indexing, because in each case finding the nearest data point to the requested coordinate is straightforward. Other types do *not* support indexing, because doing so would represent a poorly defined search in multidimensional space, rather than just computing bin location in 2D or nearest neighbors in 1D.

@philippjfr

This comment has been minimized.

Copy link
Contributor

philippjfr commented Jan 12, 2016

Thanks. Is it safe to say this?

Yes, looks good. Although indexing isn't actually disabled for the other types, you just won't be able to do it unless you hit the exact sample.

@jbednar

This comment has been minimized.

Copy link
Contributor Author

jbednar commented Jan 12, 2016

I'm also confused -- shouldn't the Sampling Data tutorial also define what .select() does? select is used in only one location, without comment, but the conclusion of the tutorial says that we now know how to select data.

@philippjfr

This comment has been minimized.

Copy link
Contributor

philippjfr commented Jan 12, 2016

I'm also confused -- shouldn't the Sampling Data tutorial also define what .select() does? select is used in only one location, without comment, but the conclusion of the tutorial says that we now know how to select data.

Hmm, wonder if I had something and deleted it. The Exploring Data tutorial discusses it a little bit but we should probably present some simple examples here, showing that it's basically equivalent to indexing and slicing but can be applied to any datastructure however deeply nested it is and is convenient to select by one particular dimension.

@jbednar

This comment has been minimized.

Copy link
Contributor Author

jbednar commented Jan 12, 2016

Oh dear. No wonder I've never been able to keep indexing/sampling/selecting/slicing straight. Ok, is it safe to say this?

In addition to slicing using a syntax likee[a:b](supported for allElementtypes) and indexing using a syntax likee[a](supported for someElementtypes as described above), HoloViews elements also support a separate method.select(), which can do both slicing and indexing but is more verbose.

If .select is supported in some cases that regular [] slicing and indexing is not, I should say what those are. But there do seem to be cases where .select doesn't work where [] does, which implies that .select isn't more general. E.g. .select works for Curves, but doesn't seem to work for Histograms:

image

image

So I'm not sure what I can say about .select.

@jbednar

This comment has been minimized.

Copy link
Contributor Author

jbednar commented Jan 12, 2016

Also, the section on "The .table and .dframe methods" seems out of place in the middle of the Sampling Data tutorial; it doesn't seem to say anything about sampling data, just converting it. It's true that the subsequent section relies on it, but that doesn't mean it can go here. Will it fit into the Pandas or Columnar tutorials?

@philippjfr

This comment has been minimized.

Copy link
Contributor

philippjfr commented Jan 12, 2016

If .select is supported in some cases that regular [] slicing and indexing is not, I should say what those are. But there do seem to be cases where .select doesn't work where [] does, which implies that .select isn't more general. E.g. .select works for Curves, but doesn't seem to work for Histograms:

It isn't more general just more convenient in some deeply nested cases and Histogram should support select. Another bug I'll have to look into.

Also, the section on "The .table and .dframe methods" seems out of place in the middle of the Sampling Data tutorial; it doesn't seem to say anything about sampling data, just converting it. It's true that the subsequent section relies on it, but that doesn't mean it can go here. Will it fit into the Pandas or Columnar tutorials?

True, it's not really about sampling, moving them to Columnar Data might make sense but we should make it clear that most Element types can be converted to a Table.

@philippjfr

This comment has been minimized.

Copy link
Contributor

philippjfr commented Jan 12, 2016

Just a quick side-note I was just looking at some of the new docs for Bokeh and came across their new charts documentation, which has a good section about tall and wide data (see here). We use the same format for our Columns so it might be worth providing a similar explanation.

@philippjfr

This comment has been minimized.

Copy link
Contributor

philippjfr commented Jan 14, 2016

Should we merge soon? We wanted to get the website updated asap, so unless there are any half-finished changes I'd say merge now, I'll update the website and then you can open another PR to go through the rest of the Tutorials.

@jbednar

This comment has been minimized.

Copy link
Contributor Author

jbednar commented Jan 14, 2016

I think it's fine to merge now. Please check my latest commits, and then merge. I'll still have more commits, but nothing else major is ready yet.

@philippjfr

This comment has been minimized.

Copy link
Contributor

philippjfr commented Jan 14, 2016

Something is going wrong in testing the Composing Data tutorial under Python 3, there's 8 of these errors:

======================================================================
ERROR: test_Composing_Data_data_000 (__main__.NBTester)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/travis/build/ioam/holoviews/doc/nbpublisher/nbtest.py", line 437, in data_comparison
    ref_data,  ref_code =  pickle.load(ref_file)
ImportError: No module named 'UserString'
@jbednar

This comment has been minimized.

Copy link
Contributor Author

jbednar commented Jan 14, 2016

That's odd; I don't know what UserString is. I did notice some print statements in your recently added tutorials that lacked () that I meant to fix for Python 3 usage, but I don't recall that being in Composing Data, and in any case I didn't change them...

@philippjfr

This comment has been minimized.

Copy link
Contributor

philippjfr commented Jan 14, 2016

Seeing the same errors on master so it's not your changes. Will have to look into it.

@philippjfr

This comment has been minimized.

Copy link
Contributor

philippjfr commented Jan 16, 2016

Traced down the bugs in the tests. Miniconda got upgraded from Python 3.4.3-2 to 3.4.4, which apparently breaks unpickling, pretty unsettling they're breaking the pickling protocol in bug fix releases. Will have to pin it to 3.4.3-2 for now.

@philippjfr

This comment has been minimized.

Copy link
Contributor

philippjfr commented Jan 16, 2016

Python 2 tests are passing here and Python 3 tests are now passing again on master so I'll go ahead and merge.

philippjfr pushed a commit that referenced this pull request Jan 16, 2016

Philipp Rudiger Philipp Rudiger

@philippjfr philippjfr merged commit 9dbd827 into master Jan 16, 2016

1 of 3 checks passed

continuous-integration/travis-ci/pr The Travis CI build failed
Details
continuous-integration/travis-ci/push The Travis CI build failed
Details
coverage/coveralls Coverage remained the same at 70.185%
Details

@jbednar jbednar referenced this pull request Jan 21, 2016

Merged

Doc cleanup 2 #411

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.