[Bug]: Subsequent calls of tight_layout() slightly change figure layout

[alexander-held]

Bug summary

Figure layout can change very slightly depending whether tight_layout() is called once or twice. Another presumably equivalent way in which this can show up is with set_tight_layout(True): when saving a figure twice for which this has been called, both figures will slightly differ.

Code for reproduction

import matplotlib.pyplot as plt
from matplotlib.testing.compare import compare_images

# import matplotlib as mpl
# mpl.use("Agg")  # issue also observed with Agg

print("using set_tight_layout(True):")
fig, ax = plt.subplots()
fig.set_tight_layout(True)
fig.savefig("1.png")
fig.savefig("2.png")
fig.savefig("3.png")
print(f"  1 and 2 match? {compare_images('1.png', '2.png', 0) is None}")
print(f"  2 and 3 match? {compare_images('2.png', '3.png', 0) is None}")

print("using tight_layout():")
fig, ax = plt.subplots()
fig.tight_layout()
# fig.tight_layout()  # when uncommenting this, 4/5/6 will match
fig.savefig("4.png")
fig.tight_layout()  # when commenting this out, 4.png and 5.png will match
fig.savefig("5.png")
fig.tight_layout()
fig.savefig("6.png")
print(f"  4 and 5 match? {compare_images('4.png', '5.png', 0) is None}")
print(f"  5 and 6 match? {compare_images('5.png', '6.png', 0) is None}")

print("using constrained_layout=True:")
fig, ax = plt.subplots(constrained_layout=True)
fig.savefig("7.png")
fig.savefig("8.png")
fig.savefig("9.png")
print(f"  7 and 8 match? {compare_images('7.png', '8.png', 0) is None}")
print(f"  8 and 9 match? {compare_images('8.png', '9.png', 0) is None}")

Actual outcome

Output from running the example code:

using set_tight_layout(True):
  1 and 2 match? False
  2 and 3 match? True
using tight_layout():
  4 and 5 match? False
  5 and 6 match? True
using constrained_layout=True:
  7 and 8 match? True
  8 and 9 match? True

When using set_tight_layout(True), the first two figures being saved differ from each other. Subsequent figures look the same, the layout seems to have converged.

When calling tight_layout() once before saving, and again before saving a second time, the resulting figures also differ. Calling it twice before saving for the first time results in consistent figures.

No such behavior is observed with constrained_layout in these tests.

Expected outcome

I would expect all produced figures with tight_layout to be equivalent, similarly to how the layout with constrained_layout is stable.

Additional information

This was tested with matplotlib 3.5.0, but also seems to happen in some earlier versions. 3.3.0 shows the same behavior. I do not know whether there are other versions where the tight layout converges in one step.

The differences between figures in this example are very small, more complicated figures can show slightly larger differences in layout, though in all cases I have seen, the differences tend to be very hard to see by eye unless skipping back and forth. I am not sure whether there are cases where layout convergence happens after more than two iterations, or where things oscillate between layouts.

I tested with MacOSX backend and a Retina display, though I see the same behavior in a container of python:3.9-slim.

Switching to AGG as backend results in the same behavior.

This may be related to #21673. When adding a fig.dpi = 100 after each figure is created, the behavior is also unchanged.

I've opened this as a bug but I am unsure whether it really is a bug, since the documentation for tight_layout does not make any statement about guaranteeing convergence.

Operating system

macOS 11.5.2

Matplotlib Version

3.5.0

Matplotlib Backend

MacOSX

Python version

Python 3.8.10

Jupyter version

6.4.0

Installation

pip

[tacaswell]

Unfortunately I think this may be unavoidable due to the algorithm that tight_layout uses. Constrained layout uses a linear constraints solver internally so I would expect it to be stable.

I think the "fix" here is to document that tight_layout may not converge / is not idempotent.

[alexander-held]

Thank you! It's good to know that this can be expected behavior. I can work around this via fig.set_tight_layout(False) in my actual use case after saving for the first time.

[jklymak]

Well, constrained_layout cheats, in that it does two draws.

The reason for this is relatively prosaic - sometimes if you change the size of an axes, the size of the decorations on it changes - think if you have ticks 2, 4, 6, 8 and your layout manager changes the axes to be larger and now 2, 4, 6, 8, and 10 are the new ticks because there is more space. The bigger 10 might require another adjustment of the layout to make it fit, but hopefully this adjustment is much smaller and the number of ticks doesn't change back. (If you are right on the cusp, it is actually possible for the solution to oscillate back and forth - the only smart way to stop it is to choose the solution with the larger space for the decorations.) Hopefully that explains why tight_layout and constrained_layout are not entirely deterministic and cannot be expected to be.

[alexander-held]

Thanks a lot for the very clear explanation! This sounds like it could be useful to mention in the documentation, but of course also feel free to close this issue since this turned out to not be a bug.

[jklymak]

This is discussed in the constrained_layout tutorial. Given that I don't think people should be using tight_layout 95% of the time, I'm not personally inclined to improve the docs there, but would support a PR to do so ;-)