Histograms disappear with logarithmic y-axis

[alexpearce]

Bug report

Bug summary

Setting a logarithmic y-axis with ax.set_yscale('log') causes histograms to disappear from the axis. This occurs in 2.1.0 but not in 2.0.2.

Code for reproduction

import matplotlib
matplotlib.use('Agg')
import matplotlib.pyplot as plt
import numpy as np

xs = np.random.normal(size=int(1e6))
fig, (ax1, ax2) = plt.subplots(1, 2)
ax1.hist(xs, range=(-5, 5), bins=100)
ax2.hist(xs, range=(-5, 5), bins=100)
ax2.set_ylim(ymin=1)
ax2.set_yscale('log')
fig.savefig('histogram.png')

Actual outcome

With 2.1.0:

Expected outcome

I expect to see a logarithm y-axis and the histogram, but I only see the former. With 2.0.2 I see this:

[dstansby]

This came up before here #9247 and was caused by #8836. Since I think a logarithmic histogram with no extra args to set_yscale('log') is a reasonable ask, I'm going to leave this open.

In the meantime a fix is set_yscale('log', nonposy='clip')

[tacaswell]

One of hists many kwargs is log so

import matplotlib
matplotlib.use('Agg')
import matplotlib.pyplot as plt
import numpy as np

xs = np.random.normal(size=int(1e6))
fig, (ax1, ax2) = plt.subplots(1, 2)
ax1.hist(xs, range=(-5, 5), bins=100)
ax2.hist(xs, range=(-5, 5), bins=100, log=True)
ax2.set_ylim(ymin=1)
fig.savefig('histogram.png')

will behave as expected.

[alexpearce]

Clearly my search-foo isn't good enough; thanks for the pointers @dstansby. I would have expected 'mask' to only hide entries with values <= 0, rather than the whole histogram. Is the latter behaviour really the intention?

Thanks @tacaswell, I wasn't aware of that. In my case I need to be able to set the scale after the hist call, but that's still useful to know.

[tacaswell]

It is only hiding the points at 0, but the histogram is drawn as a bunch of rectangles (with their bottom at 0). Once the bottom points are masked out, we don't know how to draw the rectangle (with only 2 points). This is obviously not great for histograms, but it is consistent makes sense in other contexts.

We may want to consider rolling this change back in 2.1.1 and think about it a bit more carefully.

[anntzer]

I guess that "in theory", the solution would be to attach additional information to the rectangles generated by hist, so that a log transform applied to them would use clip mode instead of mask? Looks ugly (and haven't thought about the proper way to implement this), but in truth I think this is somewhat similar to how the sticky-edges mechanism works: a single artist shouldn't get to decide how a whole axis property (in this case, whether to clip or to mask invalid values in log transforms) affects all artists.

[jklymak]

w/o looking at the code, I'm curious how log=True decides on the lowest y-value for the boxes? If its good enough for log=True I'd think that (non-zero) value is good enough for log=False. I appreciate that is a bit hacky...

The OP can of course bypass all this by calculating the histogram values and plotting them w/ fill-between with the step option where they can set the lower edge value themselves and swicth between log and non-log at will.

[anntzer]

Adjustment of ylim is controlled by adjust_ylim in bar() at https://github.com/matplotlib/matplotlib/blob/master/lib/matplotlib/axes/_axes.py#L2188. Basically, on log scale, you can say "ylow is some constant factor below the smallest positive y value (with some clipping)" (so in log scale some constant offset); but you can't afford that in linear scale.

[tacaswell]

Fixed by #9477