[Bug]: ax.transData does not honor data limits

[ShawnZhong]

Bug summary

ax.transData does not honor xlim and ylim. The document at https://matplotlib.org/stable/users/explain/artists/transforms_tutorial.html#data-coordinates seems to suggest that data limits are updated automatically when new data are added to the axes, but it's not the case, which breaks ax.transData, since it reads an old version of data limits.

Code for reproduction

from matplotlib import pyplot as plt

fig, ax = plt.subplots(figsize=(10, 10), dpi=100)
print(f"fig size: {fig.get_size_inches() * fig.dpi}")

ax.plot([0, 10], [0, 10], 'o')

print(f"(10, 10) in data coordinates: {ax.transData.transform((10, 10))}")

ax.set_xlim(ax.get_xlim())  # just ax.get_xlim() or ax.viewLim or ax.autoscale_view() is enough
ax.set_ylim(ax.get_ylim())

print(f"(10, 10) in data coordinates: {ax.transData.transform((10, 10))}")

Actual outcome

fig size: [1000. 1000.]
(10, 10) in data coordinates: [7875. 7810.]
(10, 10) in data coordinates: [864.77272727 845. ]

Expected outcome

fig size: [1000. 1000.]
(10, 10) in data coordinates: [864.77272727 845. ]
(10, 10) in data coordinates: [864.77272727 845. ]

Additional information

The wording from https://matplotlib.org/stable/users/explain/artists/transforms_tutorial.html#data-coordinates suggests that data limits are updated automatically when new data are added, but it requires manual trigger of set_xlim/set_ylim (or get_xlim/get_ylim which calls ax.viewLim).

This is quite confusing. I would hope that either the documentation is updated to notify the user to explicitly update the data limits, or even better, let transData/transLimits always use the latest data limits.

Operating system

No response

Matplotlib Version

3.8.3

Matplotlib Backend

No response

Python version

No response

Jupyter version

No response

Installation

None

[rcomer]

This has also confounded me: https://stackoverflow.com/questions/67334384

[jklymak]

Calling plot doesn't recalculate the view limits, it just marks them as "stale". Getting the view limits is expensive, and it's better to do only when needed.

Usually users do not need to directly query ax.transData, though I agree it's useful for debugging sometimes. But if you are going to query it, then you will need to trigger a relim with ax.autoscale_view() (or get_x/ylim, or fig.draw_without_rendering etc).

The todo here may be to make the Transformations tutorial clear that the plt.show has triggered autoscale_view and hence the transform works.

[ShawnZhong]

Calling plot doesn't recalculate the view limits, it just marks them as "stale". Getting the view limits is expensive

I can get the rationale for why the limits are not calculated automatically. This design makes sense to me. Although the wording on the tutorial "Whenever you add data to the axes, Matplotlib updates the datalimits" is a bit confusing.

it's better to do only when needed.

I think it might be desired to update the view limits when ax.transData is used. I couldn't think of a use case for using stale limits to transform the coordinates.

Usually users do not need to directly query ax.transData, though I agree it's useful for debugging sometimes.

My particular use case is that I want to label some numbers on a stacked bar chart, but only for the bars that are long enough so the text can fit within the bars. I cannot directly use height in the "data" coordinate because I have multiple subfigures in different scales. So I use ax.transData to transform the height in "display" coordinate and compare it with the height of the text.

But if you are going to query it, then you will need to trigger a relim with ax.autoscale_view() (or get_x/ylim, or fig.draw_without_rendering etc).

I figured that out after a time of debugging.

The todo here may be to make the Transformations tutorial clear that the plt.show has triggered autoscale_view and hence the transform works.

Do you think it would make sense to call ax._unstale_viewLim whenever ax.transData is accessed? This would be similar to how ax.viewLim is handled:

matplotlib/lib/matplotlib/axes/_base.py

Lines 849 to 852 in 9955a7c

	@property
	def viewLim(self):
	self._unstale_viewLim()
	return self._viewLim

Also cc: @anntzer. Before the commit 160de56, view limits are recomputed automatically after every plotting call.

[anntzer]

That change was intentional, as explained in the linked issues.
I guess it would be possible to likewise hide transData behind a property that likewise unstales the viewlims whenever accessed, though if you want to do that you'd likely need to do the same for transLimits, get_xaxis_transform(), get_yaxis_transform(), ... it's a bit whack-a-mole and I'd be more inclined to just document the behavior.

[tacaswell]

I think that text was written before we made things lazier.

I agree with @anntzer that the best path is to document that derived state may be inconsistent until the figure has been drawn (by rendering it to screen, saving, or fig.draw_without_rendering).

[rcomer]

I was just looking more closely at this tutorial and confusing myself further! The second figure is (I believe) designed to show two annotations in different coordinate systems pointing at the same location. However there are two sizes of the figure generated by sphinx-gallery, and I think which one you get depends on your monitor resolution - is that right?

The lower res image looks as expected:

The higher res image looks like this:

Any suggestions how we get around that?

[rcomer]

I guess one option is to not get around it but use it to illustrate why working in display space might be a bad idea:

[jklymak]

Yes the current example is an example how not to use transforms.

[alex180500]

Due to this problem, I find very hard trying to create some dashes with Line2D using coordinates distances instead of points. My idea was to calculate distances using ax.transData, convert pixels to points and passing them to ax.plot(..., dashes=just_calculated, ...). This is not possible though as the plot rendering changes the limits and the values of ax.transData are just wrong. What can I do to solve this?

[jklymak]

How can just_calculated be correct if the axes doesn't know what the data you are going to plot is yet? ax.transData cannot predict what the user may do in the future.

[tacaswell]

If you want dashes in coordinate distance rather than points I would use LineCollection instead (you will have to chop your lineup your self though).

The core of the issues here is that the transform objects represent computation to be done (they are a form of lazy expressions) and are mutable (e.g. ax.transData knows about the stack of transformation that encode the figure size, dpi, and axes limits, scale, and location) so if you change anything about the layout of the axes, any results of ax.transData.transform(...) need to be invalidated and re-computed.

[alex180500]

You are very right, yes maybe the best choice is to just have either a LineCollection or just various Line2D plots. Thank you a lot!