RFC: new function-based API

[tacaswell]

This a compangian to https://paper.dropbox.com/doc/Matplotlib-4.0--Ab_mupLexT4JeesRilNoR2LCAQ-WTYwd0NQaSHTjtLUZwkNx

[anntzer]

hopefully py2 won't really be a thing anymore by the time this is implemented :p

[tacaswell]

indeed! If you dig into the commits on this, I started writing this in 2016...

[anntzer]

Personally I find passing the axes as first argument muuuuuuch, much nicer (well, that may be because I essentially never use the pyplot layer which I understand may not be representative of most users).
Either we could use the "magic" decorator, or alternatively we could just have parallel namespaces plt.somefunc(..., ax=None) (None=gca()) & someothernamespace.somefunc(ax, ...) which would at least have the advantage of keeping reasonable signatures for all functions (with the "magic" decorator, inspect.signature can't represent the signature; which is not nice). Note that one namespace could be autogenerated from the other, e.g. in mod.py

@gen_pyplotlike  # registers to module_level registry
def func(ax, ...): ...

@gen_pyplotlike
def otherfunc(ax, ...): ...

pyplotlike = collect_pyplotlike()

and then one can do import mod; mod.func(ax, ...) or from mod import pyplotlike as mod; mod.func(..., ax=ax).

[tacaswell]

#4488 I tried something similar and it got rejected (and I sadly never followed up on making it its own package).

[timhoffm]

I find all three calling patterns valid approaches:

plt.plot([1, 2, 3])
plt.plot(ax, [1, 2, 3])
plt.plot([1, 2, 3], ax=ax)

and I welcome supporting all of them. Each one has it's use:

• simple interactive use
• interactive use with multiple axes (less to type than ax=ax)
• programmatic use, where the data should be the first arguments for better readability.

I'm against creating multiple namespaces just for the sake of different calling patterns. For one, it's conceptually more difficult to tell people: "Use pyplot.plot() if, or use posax.plot(ax, ...) or use kwargs.plot(..., ax=ax)". Also you would have to create multiple variants of the documentation. While that could be automated, you still have the problem which one to link. It's much easier to once state "axes can be automatically determined, or passed as the first positional arguement, or passed as kwarg."

As @tacaswell has demonstrated that can all be resolved with a decorator.

I'm not quite sure if the actual function definition should be

@ensure_ax
def func(ax, *data_args, **style_kwargs)

or

def func(*data_args, ax=ax, **style_kwargs)

I tend towards the latter because it's the syntactically correct signature for two out of the three cases. And it puts more emphasis on the data_args rather than on the axes. Also it has the big advantage, that it could be build into pyplot in a backward-compatible way. That way, we wouldn't need any new namespace.

Note also, that an axes context could be a valuable addition:

with plt.sca(ax):
    plt.plot([1, 2, 3])
    plt.xlabel('The x')
    plt.ylabel('The y')

(maybe using a more telling name than sca()).

[amueller]

I'm not sure if I understand the goal here.
Is the goal to have libraries add plt.my_new_plotting_thing(...) and ax.my_new_plotting_thing(...)?
That's all about entry points, right? Also: what exactly is the benefit of doing that?

Right now, my pattern is having an axes kwarg and if it's None I do plt.gca().
That's basically a single line, which might be slightly longer than adding the ensure_ax decorator in terms of characters but not by much, and seems much easier to understand.

Right now I'm reasonably happy to do some_plotting(ax=ax). Doing ax.some_plotting instead might be nice, but I'm not entirely sure if that is the main goal of this proposal? Doing plt.some_plotting(...) instead of just some_plotting(...) is just more characters, right? I guess it tells you by convention that if it starts with plt it'll modify the current axes? Though that's not even really true: plt.matshow creates a new figure.

Generally I prefer thinking about what code looks like when I use it first instead of thinking about the implementation first. Usually implementing whatever API we settle on is possible so it's more a question of what we want user code to look like.

[jklymak]

I think the main goal is for matplotlib have plotting libraries that do

import matplotlib.basic as mbasic
import matplotlib.2d as m2d

fig, ax = plt.subplots()
mbasic.scatter(x, y, ax=ax)
m2d.pcolormesh(x, y, z, ax=ax)

so the matplotlib library looks more like what user and third party libraries look like.

I think the goal would then be for ax.scatter to just be a wrapper around mbasic.scatter.

But maybe I've completely misunderstood.

[amueller]

@jklymak Xo you're saying you want to change the matplotlib api to no longer do ax.scatter and plt.scatter but do scatter(ax=ax).
That is very different from what I understood, but I'm also very confused ;)

[jklymak]

I think its meant to be a third way. Ahem, I don't particularly want this, but I think the point is to make third-party libraries more plug-and-play with the main library. It also would allow us to have more domain-specific sub libraries without polluting the ax.do_something name space. But maybe it has some deeper advantages as well?

[amueller]

Ok yeah I follow your interpretation. Let's see if that's what the others meant ;)

[jklymak]

This could be made significantly more clear by using ax.scatter (or your choice) as a concrete example.

[amueller]

I'm not sure if this is discussed somewhere, but is there a suggestion on what to do if a function wants to own a figure?

I feel the convention is to call gcf() but it's a bit unclear to me in what situations you'd call gcf() vs create a new figure, and if it ever makes sense to get a figure as an argument (and then clear it?).

[edit: replaced gca by gcf which is what I meant]

[timhoffm]

I don't think this is discussed somewhere.

Generally, I discourage using pyplot in functions (except for figure creation via plt.subplots() or plt.figure()). The problem with other functions is that they implicitly rely on a global state, so the result of your function would depend on what the user has done before.

Instead, either create the figure and axes you need inside the function, or pass them in as parameters (depends a bit what you your function should do).

[amueller]

@timhoffm can you give an example where passing in makes sense? I'm thinking about a function that creates several axes and does other things to the figure.
If the user plotted anything into a figure beforehand that might interact. If you require the user to pass an empty figure, why not create it yourself?

But basically you're saying not to use gcf. Your argument would also apply to gca, though, and axis-level functions, right?
For those I feel using gca is a reasonable thing to do and what the functions in matplotlib and pandas do. So if I want my axes-level function to feel like matplotlib I'd have to use gca.

[timhoffm]

If you're fully controlling the figure, then you can create it within your function. A typical pattern would be:

def my_figure(data, fig_kw):
     fig, axs = plt.subplots(1, 2, **fig_kw)
     # plot data into axs
    axs[0].plot(data)
    axs[1].bar(data)
    return fig, axs

Whether to pass fig_kw as a single dict or as keyword arguments is up to you.

You are only using pyplot to create a new figure. You don't use it's notion of current figure or axes (gcf/gca).

Passing the figure in as an argument makes sense when you don't necessarily control the full layout of the figure. An example is https://matplotlib.org/api/_as_gen/mpl_toolkits.axes_grid1.axes_grid.ImageGrid.html. ImageGrid needs adds a specific layout of Axes to the figure. But you might still want to be able to combine these with regular Axes. Of course, the caller is responsible to provide a reasonable figure and rect. Otherwise, he might get overlap of the ImageGrid and other parts of the figure.

The other case for passing in a figure is when you cannot rely on pyplot for figure creation. For example if your function should be able to be used for drawing a figure in a GUI Application. Such figures need a different set up for binding to the canvas and backend (pylot figure creation hides these details from you to make simple popup or notebook figures simple).

[tacaswell]

You are only using pyplot to create a new figure. You don't use it's notion of current figure or axes (gcf/gca).

But you will tap into it as the newly created figure will be the 'current figure' .

[anntzer]

But you will tap into it as the newly created figure will be the 'current figure'.

Actually I think it should not, there should be some way of saying "create a new pyplot-managed figure but don't touch the current figure/axes", otherwise, say your 3rd-party plotting function creates a multi-axes plot; now which axes is the current axes becomes deeply coupled with the internals of the function or that function needs to be careful to set the current axes at the end (or perhaps even place all its axes where it wants in the current-axes stack...)

[amueller]

@timhoffm @tacaswell @anntzer thank you for your input.

In some sense I feel that it might be nice for a library to control the state of the axes stack. But I'm also not sure what the expected behavior of a function should be.
Maybe it's best to discourage users from using gca in most cases, unless they really know what's going on. I see no way that a user could benefit from using gca without knowing the internals of a third-party plotting function, even if the function author had full control.

[anntzer]

I briefly thought about this again (due to the mention of function-based APIs in #9629 (comment)) and I think I realized another reason why I'm uncomfortable with function-based APIs which "promote" the use of the current-axes state (even if one can explicitly pass ax whether as first or as last argument):

Fairly often, an issue is reported in the tracker of people mixing pyplot and embedding in a GUI, and nearly always, the resolution is "don't mix pyplot and embedding". Which, in fact, is very easy to check: one can just see whether pyplot is imported. On the contrary, with function-based APIs where ax is optional, this becomes much more difficult to verify: one needs to painstakingly check that each call to a plotting function correctly passes ax in; in other words, it becomes much easier to accidentally rely on global state (which I think we agree is bad when embedding).

[amueller]

@anntzer this seems like a good argument to me, but it addresses a use-case that's quite far removed from my usual use of matplotlib (which is interactive use in jupyter).
I wonder if there's a way to make it easy to write interfaces for both communities.

Is your preferred solution never to use the current-axes state, so basically make ax a required argument?
I think you'll have a hard time convincing the data science community of that because they do a lot of interactive work (and it's not how any of the APIs there work right now, afaik).

[anntzer]

I agree the use cases are fairly orthogonal (but I basically end up never using pyplot even for interactive work; ax = plt.figure().subplots() is not that onerous to type anyways) and we're clearly never getting rid of pyplot (no matter how much I would like it).
My point is rather that I think any API should make it "easy" to check that a program does not accidentally rely on the global state, because that can be an annoying source of bugs, which does show up regularly on the issue tracker (so it's not purely theoretical or purely in my twisted mind). One proposal I put forward (not necessarily the best solution, of course) was to autogenerate separate namespaces, per #14058 (comment).

[amueller]

ax = plt.figure().subplots() is not that onerous, but if you call many plotting functions in Jupyter, you'd have to copy and paste that in every cell, and then pass it to the plotting function.

Basically I think of a notebook in which I call a bunch of pandas plotting functions, like df.hist(), pd.plot.pairplot(df) and so on. It's very common to have dozens of cells with single line plot commands for EDA.

The namespace doesn't really work for methods attached to objects, though, which is very common in pandas and now also present in sklearn.

And I have no doubt that this is a real issue and making it easy to check for the use of global state is a goal I'd be totally on-board with.

[github-actions]

Since this Pull Request has not been updated in 60 days, it has been marked "inactive." This does not mean that it will be closed, though it may be moved to a "Draft" state. This helps maintainers prioritize their reviewing efforts. You can pick the PR back up anytime - please ping us if you need a review or guidance to move the PR forward! If you do not plan on continuing the work, please let us know so that we can either find someone to take the PR over, or close it.