Timeout in BufferUtil

[georgweiss]

A user reports that the Tank widget sometimes does not render at all when starting Phoebus with an OPI containing Tank widgets. Cause of this I think I have narrowed down to BufferUtil.getBufferedImage() where - if not already on JavaFX thread - the call is wrapped in Platform.runLater() and a CompletableFuture. Looks like the CompletableFuture.get() times out (default 500ms) when BufferUtil.getBufferedImage() is called initially (from a non-JavaFX thread).

BufferUtil calls AWT APIs to create objects, but why is it taken for granted that those calls must happen on the JavaFX thread? Reason I ask is that simply returning a new BufferUtil object seems to not trigger a warning related to execution on non-JavaFX thread, and also resolves the observed issue. Another change to resolve the issue is to not instantiate BufferUtil in the JavaFX-thread, but to still use a CompletanbleFuture, like so:

 final CompletableFuture<BufferUtil> result = new CompletableFuture<>(); 
 result.complete(new BufferUtil(width, height));
  try
  {
      return result.get(500, TimeUnit.MILLISECONDS);
  }
...

Am I missing something?

[kasemir]

but why is it taken for granted that those calls must happen on the JavaFX thread?

There's nothing about that in the AWT javadoc, and for the most part drawing into a buffer off the UI thread is fine, which is why we're able to create images in the background.

In practice, however, I saw hangups in BufferedImage.createGraphics, see comment in https://github.com/ControlSystemStudio/phoebus/blob/master/core/ui/src/main/java/org/phoebus/ui/javafx/BufferUtil.java:

 *  At least on Mac OS X, <code>BufferedImage.createGraphics()</code>
 *  has deadlocked when the UI thread is concurrently creating a buffered
 *  image.

Creting the BufferedImage is fine but pretty useless unless you also have the graphics to draw into the buffer, and BufferedImage.createGraphics() is what needed to happen on the UI thread.
This was discovered in 2017 when the display builder was in a separate repo,
kasemir/org.csstudio.display.builder@6df9647

The timeout is there to avoid long UI hangups, assuming one missing update of a widget is better than handling the whole UI.
Plots sometimes print that one update was skipped, same idea.
The real problem here seems to be that the tank never gets updated again, so you're stuck looking at nothing.

[georgweiss]

BufferedImage.createGraphics() is what needed to happen on the UI thread

In my test it works off the UI thread.

The real problem here seems to be that the tank never gets updated again, so you're stuck looking at nothing.

Exactly, like in the editor or for a PV that does not update for a while after connection.

[georgweiss]

An ugly fix may be to set a longer timeout when RTTank calls BufferUtil.getBufferedImage() for the first time.

[kasemir]

In my test it works off the UI thread.

Yes, it would work many times, but eventually the complete GUI hung up.

But go ahead, change it, maybe that was a quirk in the AWT vs. JavaFX threading implementation on the Mac, how they each tried to access graphics hardware, who knows. Good luck.

[georgweiss]

Thing is I have reproduced the reported issue on both Mac and Windows...

[kasemir]

I don't doubt that you can reproduce the tank skipping an update.
Reproducing the hangup is harder, but if that happens, people are very unhappy.

In the RTPlot, if an update fails, another is scheduled soon afterwards:

phoebus/app/rtplot/src/main/java/org/csstudio/javafx/rtplot/internal/PlotCanvasBase.java

Lines 202 to 216 in 1a41496

	update_throttle = new UpdateThrottle(200, TimeUnit.MILLISECONDS, () ->
	{
	if (need_update.getAndSet(false))
	{
	in_update = true;
	final BufferedImage latest = updateImageBuffer();
	in_update = false;
	if (latest == null)
	// Update failed, request another
	requestUpdate();
	else
	plot_image = latest;
	}
	if (!pending_redraw.getAndSet(true))
	Platform.runLater(redraw_runnable);

You could try something like that for the tank's buffer timeout.