Allow options for non-enabled backends

[douglas-raillard-arm]

An attempt at fixing: #3587

This allows passing options for backends that have not been enabled, so that it becomes possible for a library to return plots that have been styled for multiple backends, leaving the choice of what backend to use to the end user.

This change seems to work, but options passed to non-enabled backends are discarded. If the user subsequently changes the backend and tries to plot the object, the styling for that backend is just the default. However, if multiple backends are already enabled when the .options() is called, everything works as expected.

Note that I'm not very familiar with the internals of the Options system of holoviews, so this PR was hacked together until it worked, rather than being really planned by knowing how it should work.

[douglas-raillard-arm]

This was tested in jupyterlab with:

# Cell 1
import holoviews as hv
hv.extension('bokeh')
hv.extension('matplotlib')
x = hv.Curve([0,1])

# Cell 2
y = x.options(color='green', backend='matplotlib').options(color='red', backend='bokeh')
y

# Cell 3
x.options(color='red', backend='bokeh').options(color='green', backend='matplotlib')

# Cell 4
# Switch to the other backend and re-display an already-built plot. As long as the backend was enabled (even if not the one in use), this works
hv.extension('bokeh')
y

Note that the code in the original issue #3587 still fails, I'll see if I can fix that as well:

import holoviews as hv
from holoviews import opts

hv.extension('matplotlib')

hv.Curve([1,2])\
 .opts(opts.Curve(backend='matplotlib', fig_size=400)) \
 .opts(opts.Curve(backend='bokeh', height=300, width=400))

[douglas-raillard-arm]

Updated PR with fixes for the original issue as well

[philippjfr]

This seems like a fabulous idea if we can make it work.

[douglas-raillard-arm]

Updated with PR rebased on main and fixed (and simplified) to pass the test suite.

[TheoMathurin]

+1 for this feature when you want to delay the choice of backend to later stages for different contexts.

[philippjfr]

This looks right to me, but the proof is really in the pudding. A few things we need to check here:

1. Will validation of options still work after the backend is imported (I think so)
2. What happens to invalid options that were set when validation wasn't available
3. How do we test this? Unloading a backend is difficult, so we may have to test it with a fake custom backend.

[douglas-raillard-arm]

Will validation of options still work after the backend is imported (I think so)

From my experiment, if you set an option for a backend that is not imported yet, the option is just discarded, e.g. if you use .options(foo=1, backend='matplotlib') when the matplotlib backend is not imported, then it's as good as doing nothing.

What happens to invalid options that were set when validation wasn't available

They are not set, they are discarded. If you enable the backend in question later on, the plot will just have the default styling, none of your previous options will have been saved.

Maybe it would be possible to save the unvalidated options, but I'm not sure how useful that would be. Our jupyter use-case is like that:

import holoviews as hv
hv.extension('bokeh')

# the_library_func() returns a plot styled for all backends,
# so the user can choose any backend and don't need to
# import backends they won't use.
plot = the_library_func()
plot

In that scenario, it's not hard for the user to just pick a backend before calling any library code.

How do we test this? Unloading a backend is difficult, so we may have to test it with a fake custom backend.

I was also thinking it would be a problem. As it stands I see two solutions:

1. Use multiprocessing or concurrent.futures.ProcessPoolExecutor() to run the test in a fresh process. The ProcessPoolExecutor is the best option as it transparently re-raises exceptions in the calling process so very little boilerplate.
2. A terrible hack: make a context manager that removes some modules from sys.modules, then run the code, then restores sys.modules. This should work in terms of providing a "fresh set of modules", but any object produced this way will be subtly incompatible with the ones created in normal conditions, e.g. hv.Curve instances will not pass isinstance(x, hv.Curve) outside of the context manager since they will belong to different classes, even if those classes have the exact same name and code.

I'd say that 2. is only reasonable if it saves a lot of re-import of other modules that can be kept as they are. Otherwise, 1. is much likely to work without troubles. I tried 2. with polars and it failed pretty badly, so really not the most robust approach.

[hoxbro]

For checking, I would propose having the test as strings and then running the command with a subprocess, like what I have done in this test.

[philippjfr]

I would just register a new backend and test with that. We already have other tests doing that.

Maybe it would be possible to save the unvalidated options, but I'm not sure how useful that would be.

I would say that I would expect them to persist but for them to be validated and perhaps raise warnings if it detects invalid options.

[douglas-raillard-arm]

What area should host that feature ? I'm not familiar with the rendering infrastructure, so it's easy to discard what cannot be validated, and it would be rather easy to hang on to the unexpanded options, but I have no idea where these options should be actually processed