bug[?]: adjustText doesn't handle FancyArrows correctly

[pank]

I am using networkx and mpl for making graphical representations of graphs. (I can't use graphviz for this project). adjustText seems like a perfect match for networkx as it doesn't adjust labels, see for example this old example where some labels overlap other artists.

My networkx figures use FancyArrowPatch for connecting nodes (not sure if this is always the case; I'm new to networkx). These don't seem to work well with adjustText, presumably because they don't have a get_bbox method.

I will submit a patch addressing this problem that maybe could be added if it passes mustard.

Example:

import matplotlib.pyplot as plt
from matplotlib.patches import FancyArrowPatch

from adjustText import adjust_text

t_config = {"s": "I won't be nudged", "ha": "center"}

fig, ax = plt.subplots(figsize=(3,3))
ax.add_patch(FancyArrowPatch((0, 0), (10, 10)))
t = ax.text(5, 5, **t_config)
original_loc = t.get_position()
ax.autoscale()

# fig.draw_without_rendering()  # not sure if this is needed but just to make sure...
adjust_text([t], objects=[*ax.patches], avoid_self=False, ax=ax)

ax.annotate("(current adjusttext location)", xy=(.5, 0),
            xycoords=t, ha="center", va="top", color="red", fontsize=8)


t_ideal = ax.text(3, 6.5, **t_config, color="C1")
ax.annotate("(ideal-ish location)", xy=(.5, 1),
            xycoords=t_ideal, ha="center", va="bottom", color="red", fontsize=8)

t_bbox_approach = ax.text(10, 10, **t_config, color="C2")
ax.annotate("(bbox location) ", xy=(0, .5),
            xycoords=t_bbox_approach, ha="right", va="center", color="red", fontsize=8)

print(f"Label position: {t.get_position()}")

[Phlya]

Yeah if there is no get_bbox method it might be a problem. Matplotlib is not super consistent about implementing all relevant methods for all objects, unfortunately.
But also avoiding lines is tricky, even if you manage to get a bbox for a FancyArrowPatch, it will be avoiding the bounding box, not the actual arrow (i.e. a rectangle that fits the whole object, which in case of a line is mostly perfectly usable empty space...). Designing an algorithm to avoid lines is a bit trickier, I have not managed to do it, but also didn't spend too much time on it.

But either way, a PR to add a fix of any kind is very welcome!

[pank]

But also avoiding lines is tricky, even if you manage to get a bbox for a FancyArrowPatch, it will be avoiding the bounding box, not the actual arrow

it's not optimal to use a bbox here, but at least for me it seems to work ok-ish.
I guess the optimal solution for the example would be to throw the label sufficiently up and left or down and right to avoid the line. but if using the bbox it would actually be forced to around (10, 10) or (0,0). works for my networkx examples because labels are by constructions at the end of arrows. but it not a general nice solution....

from old code, I can see that I have used something like obj.get_path().intersects_bbox(text_bbox, filled=False) for avoiding some of these issues in my own code for adjusting labels. This might be a better approach?

Re a PR.

the easiest way to fix it (read: the way I had in mind initially) was to normalize object coordinate system in the adjust_text function before they are passed to get_2d_coordinates (here).
Anything with only a window extend could be transformed to data points and passed to the existing infrastructure.
But upon writing the above, it suffers from the above limitations.

Also, I guess this is really the job of get_bbox. but get_bbox is used in a lot of places, including for labels where display cords should be used (as far as I can tell).

here's the simplistic fix I wrote. Not sure I am happy with it anymore...

modified   adjustText/__init__.py
@@ -450,6 +450,32 @@ def remove_crossings(coords, target_coords, step):
     return coords

 
+def normalize_objects(objects, ax):
+    # Normalize object coordinate systems when possible.
+    # The concrete normalization depends on the methods each object has.
+    # The goal is to return all objects in data coordinates.
+
+    try:
+        # objects can also be non-iterable mpl collections like PathCollection
+        iter(objects)
+    except TypeError:
+        return objects
+    normalized = []
+    r = get_renderer(ax.figure)
+    display_to_data = ax.transData.inverted()
+    for obj in objects:
+        if hasattr(obj, "get_bbox"):
+            # Bbox is *often* in data coordinates, but really it
+            # depends on the artist original transform.
+            normalized.append(obj.get_bbox())
+        elif hasattr(obj, "get_window_extent"):
+            # The window extent is in display coords; transform back to data.
+            normalized.append(obj.get_window_extent(r).transformed(display_to_data))
+        else:
+            normalized.append(obj)
+    return normalized
+
+
 def adjust_text(
     texts,
     x=None,
@@ -651,7 +677,8 @@ def adjust_text(
     if objects is None:
         obj_coords = np.empty((0, 4))
     else:
-        obj_coords = get_2d_coordinates(objects, ax)
+        normalized_objects = normalize_objects(objects, ax)
+        obj_coords = get_2d_coordinates(normalized_objects, ax)
         obj_coords[:, [0, 2]] = transform.transform(obj_coords[:, [0, 2]])
         obj_coords[:, [1, 3]] = transform.transform(obj_coords[:, [1, 3]])
     static_coords = np.vstack([point_coords[:, [0, 0, 1, 1]], obj_coords])

[Phlya]

obj.get_path().intersects_bbox(text_bbox, filled=False)

If you mean to use this every time to check for intersections, this would be incredibly slow. That's why I get coordinates of everything in the same coordinate system in the beginning, do all iterative calculations, and then convert back to original coordinate systems... At first I was using the MPL intersection function like above for all calculations and it was super slow.

Actually... the code already tries both get_bbox and get_window_extent methods to get the coordinates

adjustText/adjustText/__init__.py

Line 135 in aab1e19

try:

Is you code actually working in place of the current code for FancyArrowPatches? If so, I don't understand why, or what you are trying to do differently...

[pank]

Actually... the code already tries both get_bbox and get_window_extent methods to get the coordinates

for fancy arrows adjusttext (seems to) gets display coords via get_window_extent, but assume they are data coords. so calculations are wrong. see here.
I have updated the example/image in the first post to hopefully make my point clearer.

Is you code actually working in place of the current code for FancyArrowPatches? If so, I don't understand why, or what you are trying to do differently...

in the example, the black text is the current position after running adjusttext. the label overlaps the arrow. I guess something like the orange/c1 location is (arguably) the ideal location in this case. The green/c2 label is where the bbox location could be.

The code I provided above would move labels to something like the green location. for my networkx application it would be fine. for something like the included example, it's only slightly better than the original location.

I am asking:

1. whether you would find something like the "green solution" acceptable (not sure it is).
2. if you prefer a more "helper" function (like the patch) which works around get_2d_coordinates/get_bboxes which is currently, it seems, using data coords for some things and display coords for other things. OR whether it is better to try and fix get_bboxes knowing that it might mess up other transformation (e.g. it seems text coords are assumed in display coords, skimming the code).

I hope this is clearer. and I may be misunderstanding something so please correct me if I am wrong.

[Phlya]

Right, thank you for the clarification!

I didn't realize it was using a different coordinate system in this case. I think this should warrant a fix regardless of how "ideal" the solution is.
So:

1. It's not ideal, but I think creating a better algorithm to deal with e.g. lines might be out of scope for this PR (it's not trivial).
2. I don't think a new function is necessary, wouldn't just ensuring correct transforms in get_bboxes fix everything? To be honest the whole transforms business is a bit confusing for me and I spent a long time to make it work semi-randomly, so maybe I am missing some details there.