Bug with colorbar and low DPI output in PDF

[astrofrog]

The following example (with Matplotlib 2.0.0b3) causes the colorbar inside the colorbar axes to be offset when outputting to PDF with a low DPI:

import matplotlib.pyplot as plt

fig = plt.figure()
ax = fig.add_subplot(1,1,1)
im = ax.imshow([[1]])
fig.colorbar(im)
fig.savefig('colorbar_test.pdf', dpi=7)

This wasn't an issue with Matplotlib 1.4:

and the bug was apparently introduces in Matplotlib 1.5:

[efiring]

The problem is that in the usual case with more than 50 colors in the colormap, the colorbar pcolormesh output is rasterized to eliminate the artifacts that would otherwise appear when the pdf is rendered. Rasterizing at 7 dpi gives terrible resolution, so the colorbar interior doesn't coincide with the vector graphics of its outline.
You can see this with plain pcolormesh:

import numpy as np
import pyplot as plt.
X, Y = np.meshgrid([1, 2], [3, 4])
pc = plt.pcolormesh(X, Y, [[1]])
pc.set_rasterized(True)   # colorbar does this
plt.savefig('test7r.pdf', dpi=7)

test7r.pdf
Some degree of mis-registration is inevitable with this technique of rasterization to avoid the artifacts, but it only becomes a problem with low dpi--which I imagine to be an unusual situation.

[astrofrog]

@efiring - interestingly, this doesn't affect the main image though. Maybe the colorbar could be rasterized in the same way as imshow rather than using pcolormesh?

[astrofrog]

The reason I ran into this problem is that the APLpy package (https://github.com/aplpy/aplpy) basically picks a DPI that matches the imshow resolution to minimize EPS/PDF file size. If this can't be fixed in Matplotlib then the plan B will be to make sure there is a minimum DPI for APLpy figures.

[efiring]

We can't use imshow because the spacing of the color intervals can be nonuniform. Probably we should use image.PcolorImage (directly, or via Axes.pcolorfast) instead of pcolormesh; but a quick check indicates that this doesn't solve the problem. This looks like a difficult problem to solve; one way or another, rasterization of an artist in the vector backends must be restricted to cases where the dpi is reasonably large. Maybe that logic can be put in the backends.

[QuLogic]

Could you use matplotlib.image.NonUniformImage?

[efiring]

No, PcolorImage is a modification of NonUniformImage that has the right property for a colorbar: it colors blocks based on specified boundaries rather than centers. The rendering goes through the same code path.

[tacaswell]

attn @jkseppan @mdboom

I do not understand what is going on here. The artists are rasterized at a DPI which is independent of the vector dpi. This looks like there is a quantization issue going on with placing the raster image back into the vector backend rather than an inherent bug.

[jkseppan]

I agree that this looks like a problem in placing the rasterized image. The code in backend_mixed.py switches around the figure dpi in a way that I find suspicious, but I don't see an obvious bug there either.

[tacaswell]

Could this be yet another integer snapping issue?

[QuLogic]

According to #17103, also affects SVG.

[brunobeltran]

Okay, turns out the issue is rather fundamental to partial rasterization, but I think it can be solved. First, I'll explain the problem in quite some detail to ease discussion (sorry if this is quite stream-of-consciousness-y, feel free to skip to the solution below).

I'll use the following example for reproducibility, since SVG XML is easier to read/debug than PDF output for me.

import matplotlib.pyplot as plt
import matplotlib.cm

fig, ax = plt.subplots(figsize=(2, 1.5))

plt.plot()

sm = matplotlib.cm.ScalarMappable()
sm.set_array([])
plt.colorbar(sm)

plt.savefig('/home/bbeltr1/Downloads/test.svg', dpi=20)

As pointed out by Jouni above, it's important to know that the vector backends draw everything at 72 "dpi", regardless of the actual dpi setting. This is a hack/feature, not a bug. CSS defines the relationships between points, pixels, and inches to be fixed at 96px = 72pt = 1in, so this allows us to use the same internal logic as the rest of the rendering pipeline (if we pretend that points are pixels) and magically end up with our SVG output in units of CSS points.* In what follows, I'll be using "dpi" to refer to ppi (which the mixed mode backend hack-ily refers to as the vector renderer's dpi), and just dpi (no quotes) to refer to the requested dpi.

Anywho, the real issue is how MixedModeRenderer works around this hack to incorporate raster objects. It first draws them at the "true/requested" dpi, then using their equivalent widths in 72"dpi" as the actual "width" when writing the SVG/PDF file. To see how this leads to the error, I'll walk through an example

When plt.savefig is called, nothing of interest to us happens until around MixedModeRenderer.stop_rasterizing. This function is in charge of calling _renderer's draw_image (i.e. RendererSVG.draw_image) to actually create the XML that will hold the colorbar's QuadMesh in the output SVG (i.e.. <svg:path id="...">). MixedModeRenderer.stop_rasterizing determines the width to pass to draw_image by asking _raster_renderer (i.e. RendererAgg) for the bounds of the image. _raster_renderer happily returns these bounds, but does so in units of "pixels at the requested-dpi", which in the above example gives rendered_bounds = (x0=31, y0=4, width=2, height=23).

MixedModeRenderer.stop_rasterizing then correctly gets the position of this rasterized image in "dpi" by just doing *self._figdpi/self.dpi (i.e. *72.0/requested_dpi). RendererSVG.draw_image does the same thing (in this case, *72.0 / self.image_dpi). In our example, this leads to outputting an <svg:image> with x=111.6, y=14.4, width=7.2, height=82.8.

Meanwhile, the patch containing this QuadMesh is made up of a path with bounds given by (x0=112.860, y0=11.880, width=4.158, height=83.160)"dpi". Notice that in requested dpi units, that's (x0=31.35, y0=3.3, width=1.155, height=23.1)dpi. In order for the colorbar to fit correctly, we would have needed this number and rendered_bounds above to match. But rendered_bounds needs to be integers (its a pixel count), so we get the error.

* Note: there is a subtle bug created by pretending points are pixels....but it doesn't affect us here, and I will open a separate issue about it.

[brunobeltran]

In summary, in order to respect the dpi= kwarg, the size of a pixel in figure inches must be fixed. Because the vectorized part of the output is not snapped to these imaginary pixel boundaries, the number of pixels that would be required for the colorbar to exactly fit into its axes is almost always therefore going to be a fraction.

Fundamentally, you can't solve the problem of:

1. acquiescing to a requested dpi
2. render a raster image to an integer number of pixels
3. have that rastered image it exactly fit into its patch.
4. keep the requested "axis" size/tick positions

We can choose 3/4 basically.

Suppose we choose that we want (1), (2) and (4). Then the simplest solution to the current issue seems to be:

1. make sure that the renderer "rounds up" so that the colorbar is always one pixel width/taller (instead of smaller) than the patch that's supposed to contain it
2. position the too-large rendered quadmesh centered on the actual center of its axis, instead of aligned to its bounds from buffer, bounds = self._raster_renderer.tostring_rgba_minimized().
3. set the colorbar's patch's path to clip the colorbar's QuadMesh so that the rendered QuadMesh doesn't overflow the colorbar axes.

Then solves the problem of (3) by simply cropping the color bar by the minimum amount possible.

[jklymak]

I don’t have a ton of sympathy for low dpi plots. However, I actually think a reasonable approach is to use the rasterization dpi as a guide and then make an integer number of pixels fit in the physical space of each artist being rasterized. So if someone requests 100 dpi and has an object that is 1/3 of an inch wide we give them 33 (or 34) pixels So that the resolution for that artist would really be 99dpi. I don’t think anyone using a vector backend needs the rasters to be exactly any dpi whereas they likely would like the objects to line up precisely with non vectorized elements.

[brunobeltran]

However, I actually think a reasonable approach is to use the rasterization dpi as a guide and then make an integer number of pixels fit in the physical space of each artist being rasterized.

I like this solution. To make sure I'm understanding @jklymak, you're suggesting that I basically just take the existing rasterized QuadMesh, and just scale it to fit the actual vectorized patch e.g. via its "transform"? (Easy PR if so).

This would scale all the component pixels up or down a bit, so the dpi would typically be off by 1 or 2 for each object, but not really at all in the high-dpi case...and I agree that low-dpi is not a case we should care as much about.

[jklymak]

dpi need not be an integer, it just happened to be above. If an object is 1.637 inches wide, and dpi is set at 100, then we would use 164 pixels to represent it across 1.637 inches for a practical dpi of 100.18 dpi.

[brunobeltran]

While I'm fixing this, might as well note for posterity that contrary to what was reported originally, this bug is also present for imshow, it's just hidden better because the draw_image pipeline makes sure to clip the image to the axes in that case.

import numpy as np
import matplotlib as mpl
import matplotlib.pyplot as plt

mpl.rcParams['figure.dpi'] = 20
fig, ax = plt.subplots(figsize=(2, 1))

checkers = np.array([[0, 1], [0, 1]])
ax.imshow(checkers)

plt.savefig('/home/bbeltr1/Downloads/test-imshow.svg')

produces a figure that only looks a "little" off,

but if you highlight the boundary of the actual image in Inkscape, you can see the error is just as bad:

The rasterized element <svg:image> has a height and width of 57.6pts, whereas the axes containing it are only 54.36-by-54.36. Since the rounding error more or less aligns the bottom left corner in this case (by chance), this is also noticeable by the fact that the centerline is not quite centered....

[jklymak]

I don't think its by chance that the bottom left corner aligns? The image origin is placed exactly there, isn't it?

[brunobeltran]

Oh I guess in this case it is exactly there?

I just assumed it was subject to the same integer clipping issues as in the colorbar case (where the bottom corner typically does not align).

[jklymak]

I'd have to look at the code, but there is definitely a floating point x and y and a translate command in there...

[brunobeltran]

I see, it's a different bug. Unlike the colorbar, the AxesImage is not processed by MixedModeRenderer.stop_rasterizing (because AxesImage.get_rasterized() is False), so its position could in principle be correct, would have to dig down that code path independently to figure out why its size is wrong when its position isn't.... is correct. PR incoming.

[github-actions]

This issue has been marked "inactive" because it has been 365 days since the last comment. If this issue is still present in recent Matplotlib releases, or the feature request is still wanted, please leave a comment and this label will be removed. If there are no updates in another 30 days, this issue will be automatically closed, but you are free to re-open or create a new issue if needed. We value issue reports, and this procedure is meant to help us resurface and prioritize issues that have not been addressed yet, not make them disappear. Thanks for your help!

[tacaswell]

attn @QuLogic Please re-open if you think this is related to the DPI work you are doing.

[QuLogic]

I'm not sure it's related to my work, but probably to @jklymak's since he also added the keep label.