events refactor

[kushalkolar]

From today's meeting, superseeds #345, #182, #100. Much of the ideas from those issues either belong in examples or a neuro-widget library @clewis7

1. I think we will keep a very simple event system in fastplotlib. I will explore observ and see whether it makes sense to incorporate it within fastplotlib or not. Otherwise we will make a very simple event system in fastplotlib and give users examples of how to use observ or other libs for more complex event handling
2. Using what Almar and Korijn call a store model is probably a better paradigm than circular events when those types of interactions are required. Have a concept of a "state", for example "selected cell" and events modify that state. This way you don't have to worry about circular events. For example, if you click on a cell it modifies "selected cell", if you click on a temporal component it modifies "selected cell" etc.
3. Pull out the Interaction class from fastplotlib and put it into a new neurowidgets that Caitlin will work on.

Proposed basic API for adding event handlers suggested by Almar:

graphic.add_event_handler("feature", callable)

# could also be a pygfx event and use the same API?
graphic.add_event_handler("click", callable)

[clewis7]

Just to clarify...

Under this model we are completely revamping the interactivity abstraction layer to handle all interactivity using the add_event_handler function? A user will always have to pass a callable defining the behavior they want?

What about for something where you are linking graphic features together? You would still need to pass a callable?

For example:

xs = np.linspace(-10, 10, 100)
# sine wave
ys = np.sin(xs)
sine = np.dstack([xs, ys])[0]

# cosine wave
ys = np.cos(xs) + 5
cosine = np.dstack([xs, ys])[0]

plot_l = Plot()

# plot sine wave, use a single color
sine_graphic = plot_l.add_line(data=sine, thickness=5, colors="magenta")

# you can also use colormaps for lines!
cosine_graphic = plot_l.add_line(data=cosine, thickness=12, cmap="autumn")

sine_graphic.link(
    "click",
    target=cosine_graphic,
    feature="colors", 
    new_data="white", 
)

We would be removing the link function in favor of:

def callable():
   cosine_graphic.colors = "white"

sine_graphic.add_event_handler("click", callable)

[kushalkolar]

We would completely remove the link functionality from fastplotlib in favor of just keeping the graphic-features events system (and pygfx events system), in favor of a state-management model instead of circular events.

Rudimentary ideas, the rest is up to you 😄 . Can probably be made more abstract and use arrays instead of lists to make it faster (instead of the for loop in reset()), but then you'll have to keep track of the GraphicCollection size and change the size of the arrays in the state instance.

You can play with this idea with some viz, and if it's fully generalizable we can just have it in fpl as a general "CollectiveGraphicsState" object that works for all graphic features.

# let's say we have a few graphic collections, this could work with just regular Graphics too 
# but you can worry about that later

image_graphic
contours_graphic  # LineCollection
temporal_stack  # LineStack

GraphicCollectionColorState:
  """stores state of a graphic collection's colors"""
  def __init__(self, collection: GraphicCollection):
    self._collection = collection

    # used to store state of graphic prior to being selected
    self._previous_states = [None] * len(self._collection)

  def set_selected(self, index: int):
    self.reset()

  def add_to_selected(self, indices: np.ndarray[int]):
    # not sure if this will work but we can make it work
    self._previous_states[indices] = self._collection[indices].colors

    # "highlight" these graphics
    self._collection[indices].colors = "w"

  def remove_from_selected(self, indices: np.ndarray[int]):
    # reset them back to the original state
    self._collection[indices].colors = self._previous_states[indices]
    self._previous_states[indices] = None

  def reset(self):
    for i in range(len(self._previous_states)):
      if self._previous_states[i] is not None:
        self._collection[i].colors = self._previous_states[i]
        self._previous_states[i] = None

contours_state = GraphicCollectionColorState(contours_graphic)
temporal_state = GraphicCollectionColorState(temporal_stack)

# simple 
def image_clicked(ev):
  # if shift key is held
  if "Shift" in ev.modifiers
    contours_state.add_to_selected(index_from_euclidean)
    temporal_state.add_to_selected(index_from_euclidean)
  else:
    contours_state.set_selected(index_from_euclidean)
    temporal_state.set_selected(index_from_euclidean)

# the repetitive code where the same state is set for contours and temporal could be removed by having 
# GraphicCollectionColorState manage a list of GraphicCollection as well instead of just a single one

def temporal_stack_clicked(ev):
  if "Shift" in ev.modifiers
    contours_state.add_to_selected(index)
    temporal_state.add_to_selected(index)
  else:
    contours_state.set_selected(index)
    temporal_state.set_selected(index)

A nice thing is this will also ultimately allow create a resizable box to select graphics and set their state (like clicking your mouse and selecting icons in a file manager).