Improve support for reactive programming

[marcushellberg]

When I build a view, I want it to be a function of the current state.

Currently, Flow only supports data binding to Fields. This is restrictive and leads to a boilerplate code. I believe that improving support for reactive UI building would help increase developer productivity and happiness.

Scenario:
I have a master-detail view of patient data. Selecting a patient should display or update a view of patient details based on a Patient object. I do not want to recreate the entire view on each change for performance reasons. The view contains a picture and both editable and non-editable information on the patient as well as buttons for saving or discarding changes. The save button is enabled/disabled based on form state and includes the patient's name, like "Save NNN's record", as a safeguard against updating the wrong patient's records.

Current state
✔️Binder can be used to bind the Patient object to the editable fields
✖️I need to manually call Button's setEnabled() from a listener on Binder
✖️I need to create and keep references to each Span used for non-editable info and call setText() on each every time the Patient object changes
✖️I need to keep a reference to the Image and manually set the image source every time the Patient object changes
✖️I need to keep a reference to the Button and manually set the text every time the Patient object changes

Desired outcome
I can bind any property of Components/Elements to functions of the state: in this case, my Patient object. Below are some non-working examples to show the idea. Something would need to observe the state object and trigger these functions to be re-evaluated.

new Span(patient::getSSN)

Button saveButton = new Button(
  patient -> "Save "+patient.getLastName()+"'s records", 
  click-> save());
saveButton.setEnabled(Patient::isValid));

new Image(Patient::getImgUrl, Patient::getFullName);

The same could probably also be achieved, perhaps easier for us, by making Binder more generic by allowing developers to bind any property on the bean to a function.

Span firstNameSpan = new Span();
...
binder.bind(Person::getFirstName).to(firstNameSpan::setText);

[jojule]

Thought on the implementation side on the above example for saveButton.setEnabled(Patient::isValid): We need to somehow propagate the changes from Patient to saveButton enabled state. For that I would propose that we provide the following 2 means for this as a part of this proposal:

1) Manually asking saveButton to refresh.

This could be something like having the Button implement

interface Refreshable {
   void refresh();
   void refresh(Method toRefresh);
}

With this one could call saveButton.refresh() to update button or if one wants to be more efficient saveButton.refresh(Patient::isValid). One could also route events from custom listeners by saying something like patient.addChangeListener(e -> saveButton.refresh())

2) We should have a way of saveButton to automatically listen to changes.

This could be achieved by having the implementation of saveButton.setEnabled(Patient::isValid) check if Patient implements an interface for publishing change events. This could be something like

interface ChangeNotifier {
   void addChangeListener(Refreshable toBeNotified);
   void addChangeListener(Refreshable toBeNotified, Method whatWasChanged);
   void removeChangeListener(Refreshable toBeNotified);
}

[jojule]

Furthermore I would like to see us implement build on top of this proposal something along these lines:

var t = new PolymerTemplate(
   "<div class$={1}>Medication: ${0}</div>", 
   patient::getMedication, 
   e -> MedicationClassiffication.isCritical(patient.getMedication()) ? "critical" : "");

While the inlined templates in Java are awkward, it would allow languages like Kotlin and Scala build React-like inlined templates.

This should probably be split to a separate ticket.

[marcushellberg]

That looks like TemplateRenderer in Grid. Maybe that could be made to work outside Grid.

[Legioth]

I see several different directions we could take when it comes to "reactive" support:

1. Pseudo reactive with additional Binder functionality
2. "Classical" reactive programming
3. Reactive with automatic listener registration (similar to Meteor)
4. Reactive-ish by re-running initialization and finding differences from that (similar to React)

The thing that all of these basically do is that they eliminate the need for separately defining how to set initial values and how to react to incremental updates.

Binder enhancements

There have been numerous suggestions to somehow make Binder support something like "readonly HasValue", i.e. cases when a value derived from the edited item should be displayed, but there won't be any changes to the value through that binding.

This approach has two major limitations:

1. It's only applicable for cases when Binder , even though there's no reason why Binder couldn't also be used outside of form binding if it would have this functionality.
2. It's only feasible to react to setBean/readBean. Binder doesn't directly support any convenient way of making a certain callback run when a specific property changes.

Classical reactive programming

The basic idea is that you can connect any getter to any setter (optionally with a transformation, filter or asynchronous delimiter in between). When the value that the getter would return changes, the setter is automatically run with the new value.

The main challenge here is that regular Java POJOs don't fire events when their setter is run (i.e. when the value returned by the getter changes). Beans would instead have to wrapped in proxies that make setters fire events, or alternatively data would be stored in a generic key-value store instead.

Reactive with automatic listeners

This is small addition on top of the classical reactive approach. Instead of explicitly declaring which property a specific action depends on, the framework would automatically register which property values are read when running a specific action, and then automatically registering that action to be rerun when any of those properties are updated.

React-style reactiveness

The React framework basically turns reactiveness upside down. Application logic only defines how to "initialize" (i.e. render) a component based on specific data, and then the framework takes care of comparing the newly initialized representation with the previously rendered version to figure out how to update the actual DOM.

This approach has the benefit that it decouples the state of the application from the component tree, which in turn means that only the actual state needs to be stored in memory or serialized between requests. This approach might thus open the door for stateless or even paritally offline operation.

[marcushellberg]

Great overview @Legioth!

I like what you're saying with decoupling the state from the component tree. I've been thinking about that for a while. Having Flow move from managing individual component states to managing application state would give us more freedom in where the logic resides and, like you said, could enable some interesting offline and stateless options in the future.

[eoliphan]

+1 for the React-style approach. I do a lot of Clojure(Script) programming, and libraries like this (https://github.com/Day8/re-frame) really make for a cleaner, more comprehensible experience.

[heruan]

I'm currently using java.util.concurrent.Flow interfaces to implement a (very) similar behavior, where basically most of my components are Subscribers and I subscribe them to services which are Publishers.

The advantage of adopting core reactive Java APIs is that it comes free to have Vaadin components talk the same language with non-Vaadin ones, e.g. I have services providing a GraphQL API (via SPQR), so when an update happens both API subscribers via GraphQL subscriptions and UI users are pushed with the updates.

I can share a sample of this if it may be of any interest.

[dohnala]

Hello everyone, some time ago, I was experiencing with combination of Vaadin and reactive programming and the result is and addon (https://github.com/dohnala/reactive-vaadin). It was just a proof of concept to see how this could work. There is also a demo with all features and code deployed at Heroku https://reactive-vaadin-demo.herokuapp.com/. Unfortunately, it is not completed, because lack of time. Documentation and support for Vaadin Flow is missing. It isn't published, too.

The reason why I am writing here is that I am very interested in this combination and I would be very excited to get some feedback, critics, new ideas or even some people who would like to continue with this with me in any form.

[Legioth]

Introducing React-style or Meteor-style reactivity would most likely require quite significant changes, so I haven't really investigated those routes at this point. The other concepts can on the other hand be introduced cleanly on top of current concepts.

Readonly bindings

One very easy thing is to add readonly binding support for Binder. There are two different aspects of this that would most likely need separate APIs.

Readonly property

The item doesn't have a public setter, or alternatively the setter is never run while an item is being edited. The binding only needs to update when running setBean or readBean, and in that case the new value to show can be calculated directly based on the item. An example of this is cases when the user only has permissions to edit some of the fields.

This can be supported with any of the API suggestions from in #4980.

Computed property

The computation needs to be re-run as the user makes changes through other bindings. In this case, the use of buffered mode (i.e. readBean instead of setBean) means that the inputs for the computation might not yet be present in the bean itself. One typical example is a summary label that e.g. shows the number of days for a reservation based on editable start and end dates.

The generic case where multiple properties affect the computed result is doable already today using Binder.addValueChangeListener and manually extracting values from individual fields. There are some opportunities to improve DX with some helpers:

• For potentially expensive computations, there could be a helper that allows defining which fields or bindings to listen to and running the listener only once per round trip (i.e. beforeClientResponse) even if multiple bindings have changed.
• Accessor in Binding for retrieving the value that would currently be written to the bean if writeBean would be run.
• For cases when the value is computed based on a limited number of bindings (e.g. two or three), the two previous cases could be combined into a helper that takes a number of bindings and a consumer with the corresponding types. This would eliminate the need to manually extract the relevant values.
• For cases when the value is computed based on only one binding, something along the lines of peek from Stream might be appropriate. The peek consumer could be added at any point in the binding builder chain and thus give freedom on whether to inspect the value before or after any specific converter or validator.

Value subscriptions

To better support reactive functionality in other contexts, any part of the framework that "publishes" a value could expose a generic API for creating one-way subscriptions to the value. This could be a fluent API where one could chain in various operations such as map, filter and accept. The prime candidate example of something that publishes values is of course HasValue, but there are also some other relevant examples such as the validation status of Binder. One could also see individual bindings in Binder as value publishers, but I don't know whether that would be of any practical use compared to readonly binding support.

This is conceptually slightly similar to the Stream API or the various reactive streams APIs that have java.util.concurrent.Flow as a unifying factor. At the same time, there are enough differences to warrant a separate API. The main differences compared to Stream are that there are no terminal actions and no parallel processing. The main difference to reactive streams is that concepts such backpressure and handling updates on different threads are irrelevant and would only add complexity. Some kind of bridge to reactive streams might still make sense, but that would be an advanced feature and not a central aspect of the API.

One example of how this could be used is a situation where the choice in one dropdown changes the options in another dropdown. A basic example could be along these lines:

Select<ProductCategory> categories = ...;
Select<Product> products = ...;
categories.useValue().filter(Objects::nonNull).map(category::getProducts).accept(products::setItems);

Another quite typical example is a checkbox that controls visibility of some other part of the UI. This could be supported by a shorthand that immediately accepts the published value.

Checkbox advancedMode = ...;
Div advancedOptions = ...;
advancedMode.useValue(advancedOptions::setVisible);

Yet another relevant example is to automatically enable and disable a submit button based on whether a form is valid (assuming #4988 has been fixed).

binder.useStatus().map(ValidationStatus::isValid).accept(submitButton::setEnabled);

[Legioth]

There is one challenging aspect with the suggested useValue design, which is cases when multiple values are used as input for one computation. As an example, there might be a preview label that shows the number of days of a reservation based on the current value in two date pickers. Without any special support, the easiest way of dealing with that might be to extract the actual logic into a separate method and then trigger that method from the useValue of each input field (with a formatDuration helper that handles null values and such):

private void updateDuration() {
  durationLabel.setText(Util.formatDuration(startField.getValue(), endField.getValue()));
}

private void bindDuration() {
  startField.useValue(value -> updateDuration());
  endField.useValue(value -> updateDuration());
}

One thing that can be done to slightly simplify this particular case is if the return value of useValue() would contain a method for chaining in a reducer.

startField.useValue().reduce(endField.useValue(), Util::formatDuration).accept(durationLabel::setText);

That approach works quite smoothly when there's only two values to combine, but it's not pretty for multiple values. What would be needed in those cases are instead a helper that runs a callback whenever any dependency has changed, accepting one callback and a varargs array of useValue() instances.

DependencyHelper.whenAnyChanged(this::updateDuration, startField.useValue(), endField.useValue());

Taking this one step further, we could remove the need of passing the dependencies to whenAnyChanged if that method would automatically detect which getters with a corresponding useValue() method has been run by the callback. For this to work, whenAnyChanged must set a thread local that is inspected by any call to such getters.

The example could then be simplified to

DependencyHelper.whenAnyChanged(this::updateDuration);

This could be encapsulated in a ReactiveValueHolder helper that would encapsulate everything needed both for implementing useValue() and for making the getter register itself as a dependency. In that way, a simplified field implementation could look like this:

public class SimplifiedFieldExample {
  private final ReativeValueHolder<String> valueHolder = new ReactiveValueHolder<>();

  public String getValue() {
    return valueHolder.get();
  }

  public void setValue(String value) {
    valueHolder.set(value);
  }

  public ReactiveValue<String> useValue() {
    return valueHolder.useValue();
  }

  public Registration addValueChangeListener addValueChangeListener(ValueChangeListener<String> listener) {
    valueHolder.addValueChangeListener(listener);
  }
}

One final tweak could be to add a more discoverable shorthand for this pattern in association with setters that are typically expected to be used this this. This would increase discoverability over using a static method on an unrelated class.

durationLabel.setTextComputation(() -> 
  Util.formatDuration(startField.getValue(), endField.getValue());

[Legioth]

I decided to take a quick look on what these ideas could mean in practice, and then I got slightly carried away. I ended up with a partial reference implementation and functional demo at https://github.com/Legioth/reactivevaadin and an explainer document for one of the most central concepts in https://vaadin.com/labs/viewmodel. You can see those as one design suggestion for this ticket.

[Legioth]

My latest thoughts revolve around using explicitly defined UI state as a way of also significantly reducing server-side session sizes to the point where all state could optionally be passed back and forth with each request and response so that a Flow application could even be deployed as FaaS.

This concept is described in https://github.com/orgs/vaadin/discussions/3471