Feature request: change hist bins default to 'auto'

[amueller]

This is a follow-up on a discussion at matplotlib here:
matplotlib/matplotlib#16403

I think 'auto' would be a better default than 10 for 'bins' for visualization purposes. The matplotlib team is hesitant to have a default value different from the numpy one. So I wanted to ask what you think about a change of default here in numpy.

[seberg]

In general, it seems like obviously the better default. I see two things we need to figure out:

1. We have to decide how to do the transition:

   • Make a FutureWarning which may annoy a lot of users that want to just plot quickly (I assume most libraries do not care, because they provide the bins in some form)
   • Accept API breakage, but then we have to discuss whether it is acceptable in a minor release.

2. The current "auto" has some bad corner cases where the number of bins is prohibitively large. It may be good to have some solution for that (I am not sure what, the histogram bin merging, or some "sane" cutoff for a maximum number of bins compared to the number of points).

I will assume we can fix point 2. in some form that is reasonable. So the main point is API stability and how noisy a FutureWarning is. If we can decide that a FutureWarning (forcing users for now to provide bins if they do not want to see the warning) is OK, that is straight forward.
Otherwise we have to decide that we can do this in a minor or major release. (I have been toying with writing a NEP suggesting a fairly regular "scheduled" major release, say every 2 years where we select a small number of such changes each time).

[amueller]

In the other thread @mwaskom mentioned that seaborn uses 50 as a max, so the default is actually min('auto', 50). That might be a reasonable solution for 2).

[seberg]

True, although it is a much more reasonable limitation for plotting than for a generic histogram mechanism... I guess any default must optimize a bit for a specific use-case.

I somewhat read part of the matpltolib thread as: bins=10 is so bad, that most users will realize they should set it manually. Although, not sure I can agree with it. In practice I feel I remember telling people "why don't you increase the number of bins a little, you have enough data?" very often...

[amueller]

My impression and my argument was that people don't realize that they are missing something, so I think similar to yours.
I think the counter-argument was that no default always works and people should experiment.
I don't think I'm a good person to summarize that conversation though ;)

[rossbar]

This has been discussed in a NumPy triage meeting --- in short, changing the default would be an improvement and selecting a default that is better for plotting would be reasonable. More detailed summary below:

The discussion largely revolving around some of the points @seberg brought up above. In general, changing the default number of bins is seen as an improvement as the current default (10) is rarely a good fit for any use-case, neither data analysis or plotting. It will not be possible to satisfy all use-cases, but choosing a default that satisfies a common use-case (e.g. plotting) would be an improvement. The most important consideration is that the new default value is "safe", which, as @seberg mentions above, is not currently the case for auto due to bad failures for some corner cases (see e.g. #10297 , #11879, #8203). In all, the proposed change would be an improvement, but there is not yet consensus on what the default should be.

[seberg]

IIRC, there was a bit of a tendency to view it as a "major" change. I do not think that should slow this down, though. We mostly need a decent proposal probably, after that my guess we get this rolling (although maybe very slowly).

[amueller]

Thank you all for moving this forward!

I think it's tricky to make it safe and not make it arbitrary. I think min('auto', 50) or something similar would be relatively safe but I have no idea how to defend the 50. And would that require adding a new option "safe_auto"? that seems also a bit ugly :-/

[rgommers]

It could be named 'plot' perhaps, that'd be meaningful.