Should default hist() bins be changed in 2.0?

[jakevdp]

I would advocate changing the default histogram bins to a data-adaptive choice in 2.0. For an example of how this might look, see info on the recently-merged astropy hist() function: http://astropy.readthedocs.org/en/latest/visualization/histogram.html#normal-reference-rules

Using the freedman-diaconis rule as the default would be a useful change, IMO (though it would require some modification in the case of weighted samples)

[tacaswell]

Probably not for 2.0, I am pretty adamant about keeping it color/style changes only.

This come up recently (#4316) and it got pushed to the numpy mailing list (but I don't know what happened to it there).

hist is a giant function that already does too many things. I think the future of mpl hist should be to be a dispatch mechanism to the N ways to draw pre-binned histogram data and get out of the computation business all together.

If we really want to maintain the adaptive binning code in mpl we should add a auto_hist function rather than break API.

#4433 makes one common use case (filled steps) a lot easier to do with pre-binned data.

I also have some thoughts that I need to turn into a PR about providing a decorator in pyplot to allow users to more-or-less register pyplot functions which would make writing a small third-party auto_hist library much easier to do.

[jakevdp]

Fair enough – thanks!

[Tillsten]

@tacaswell did you change your opinion, because i would love to see that go into 2.0. Pushing
it to the next major release afterwards would probably mean 2-3 years for matplotlib.

[tacaswell]

Support for automatic bin width selection is on numpy master and will be in numpy 1.11.

If the user has a new enough version of numpy installed ax.hist(..., bins='auto') will work.

I am still skeptical of any 'automatic' data analysis (but if @jakevdp trusts it I should probably stop worrying), however I would not want to bump the mpl default until our min numpy version is > 1.11.

I think adding an rcparam which sets the default values of bins (and hence, can be controlled via the styles module) is a decent compromise and can go in for 2.0.

[nayyarv]

@jakevdp I'm trying to add support for weighted data for the binwidth selection procedures in numpy (which was overlooked in numpy/numpy#6029). See numpy/numpy#6288 for details.

We're currently unsure about how to deal with the various possibilities of the weights, i.e. count-like (n=weights.sum()), probability like (n=a.size), or neither (n=a.size), and was wondering if you had any thoughts or suggestions that might us either automatically guess or get explicit instructions from user.

[tacaswell]

This was addressed by adding an rcparam.

[amueller]

Is the numpy dependency high enough to change the default yet? I don't think an rcparam addresses this. We can try to educate users and should, but people will still call hist without arguments. I had students not do this after I told them for over an hour that they shouldn't use the defaults and should use auto here. Given that sample, I'm pretty sure way too many people use the default unknowingly.

In other words: pretty please change the default for the next major version if possible.

[jklymak]

@amueller can you open a new issue with more details of what you are proposing? As much as possible, I think we should just pass arguments through to np.histogram.

[tacaswell]

Yes, we are are np.1.15 on master so yes. 👍 on a new issue for this. We currently don't have a target date for 4.0 (which is what we are roughly calling the major re-think), but it may be the case that there are enough breaking, but incremental changes, that people have asked for to be worth doing a "small" major release.

[edit to make run on sentence a bit less run-on]