Issues with fl_overlay_rect on Linux and Mac

[dannye]

Describe the bug
Apologies if this ends up being a bit all over the place.
Note: Putting the videos in fullscreen mode makes the issues much clearer.

tl;dr, fl_overlay_rect()'s dotted border drawn incorrectly on Linux. Crashes when drawn too far outside the window on Mac.

In my application, I changed my Fl_Double_Window to a Fl_Overlay_Window so that I could implement a draw_overlay() in order to draw a fl_overlay_rect().
But there was an unbearable flicker:

overlay-rect-windows-flicker.mp4

I was able to avoid the flicker by abandoning Fl_Overlay_Window and sticking with a Fl_Double_Window and simply putting fl_overlay_rect() in draw() instead of in draw_overlay() (although I may be assuming incorrectly that fl_overlay_rect() may be called in any general purpose draw()).

This now works fine on Windows and is much more responsive than draw_overlay():

overlay-rect-windows.mp4

However on Linux, the dotted line for the overlay rectangle is often corrupt and uneven, especially on the top border, although all 4 borders may be affected:

overlay-rect-linux.mp4

And on Mac, it crashes if I drag the rectangle a little too far outside the window:

overlay-rect-mac.mp4

To Reproduce
I may try to find time to create a minified repro program that demonstrates the issue. (EDIT: done, check comments)
If you're really interested, the repo where this issue exists is here: https://github.com/dannye/crystal-tracker
Although I don't really expect you to go so far out of your way to build the thing.

Expected behavior
I would expect Fl_Overlay_Window/draw_overlay()/fl_overlay_rect() to not have terrible flicker/lag (at least on Windows). (Full disclosure, I didn't even test Fl_Overlay_Window and draw_overlay() on Linux or Mac.)

On Linux, I would expect fl_overlay_rect() to not have a corrupt, uneven dotted line.

On Mac, I would expect fl_overlay_rect() to not crash if used to draw a rect that extends beyond the client rect.

Screenshots
See video clips above.

FLTK Version

• Version: 1.3.8 (I apologize if some or all of these issues are moot in the upcoming 1.4.0)
• Downloaded and self-built from: Git
• If from Git, tag: release-1.3.8

FLTK Configure / Build Options
Windows:

git clone -b release-1.3.8 https://github.com/fltk/fltk.git lib/fltk

Added #define FL_ABI_VERSION 10308 in abi-version.ide.
Opened ide\VisualC2010\fltk.sln.
Built the following .lib files: fltk, fltkimages, fltkjpeg, fltkpng, and fltkzlib

Linux and Mac:

git clone -b release-1.3.8 https://github.com/fltk/fltk.git lib/fltk
pushd lib/fltk
./autogen.sh --prefix="$(realpath "$PWD/../..")" --with-abiversion=10308
make
make install
popd

Operating System / Platform:

• OS: Linux, macOS, Windows
• OS Version: Peppermint OS 10, macOS 11.4, Windows 11

[Albrecht-S]

@dannye Thank you for the detailed report and the videos. As you may know, FLTK 1.3 is at its end of life and will only get fixes for major bugs. We are planning to release 1.3.9 soon, very likely as the last 1.3.x release. Many issues have been fixed in 1.4.0, and some of the issues you mention appear very familiar.

I'm not going into all details of your report right now, but looking at the FLTK test and demo code, the only place where we're using fl_overlay_rect() is in test/mandelbrot(.cxx), hence I tried this with FLTK 1.4.0 (from Git master) on Windows, macOS, and Linux - and I couldn't confirm any of the issues you report. Some details follow:

1. The dotted rectangle drawing code on Linux has been fixed. Part of this issue was (is) caused by a graphics driver issue but we found a way to work around it. As a proof I used test/mandelbrot built with FLTK 1.3 (git branch branch-1.3) and I can confirm the issue on my system with 1.3. test/mandelbrot built with FLTK 1.4 works fine on my system.

2. I tested test/mandelbrot (FLTK 1.4) on macOS and Windows as well, and everything looked fine.

That said: please give FLTK 1.4 a try and run test/mandelbrot on your system.

• Do you see the issue in FLTK 1.4?
• To verify, can you confirm the issue in test/mandelbrot on your 1.3 build?

3. Regarding the crash you reported on macOS: I can't reproduce this with the test/mandelbrot demo program (using 1.4). Caveat: this may be due to the fact that dragging the mouse beyond the window borders and drawing the overlay is limited to the window width/height in this demo. This is just a guess, I didn't verify it, and I didn't bother to build 1.3.x on macOS.

If your program uses coordinates beyond the window borders then this could be called a programming error but I agree that FLTK should not crash. To verify this we really need a short test case (a complete program source code in one file) that shows the issue so we can confirm and fix it. Can you provide such a short program with code similar to what your own program does? Thanks in advance.

[dannye]

That said: please give FLTK 1.4 a try and run test/mandelbrot on your system.

• Do you see the issue in FLTK 1.4?

• To verify, can you confirm the issue in test/mandelbrot on your 1.3 build?

@Albrecht-S So far, I've just tested mandelbrot on Linux.

Here is what I found.

On branch: branch-1.3 (not tag release-1.3.8 that I've been using up until now):

fltk-1.3-mandelbrot.mp4

Not sure why the drawing area gets stale.

And on branch master (ie, 1.4):

fltk-1.4-mandelbrot.mp4

The uneven dotted lines seem to be significantly worse on my system in version 1.4.

I will continue to comment here if I test mandelbrot on Windows or Mac, or if I create a repro for the fl_overlay_rect() crash on Mac.

[dannye]

Testing on Mac

branch-1.3: (has a very strange "smearing" issue)

fltk-1.3-mandelbrot-mac.mp4

master (ie, 1.4): (works fine)

fltk-1.4-mandelbrot-mac.mp4

[dannye]

Testing on Windows

branch-1.3: (has the egregious delay/flickering)

fltk-1.3-mandelbrot-windows.mp4

master (ie, 1.4): (seemingly has the same behavior/performance as 1.3):

fltk-1.4-mandelbrot-windows.mp4

[dannye]

Reproducing the crash on Mac in 1.3

The crash can easily be reproduced in 1.3 using the mandelbrot demo by simply commenting out the two if statements which perform the clamping of x2 and y2:

fltk-1.3-fl_overlay_rect-crash-mac.mp4

The crash does not happen in 1.4:

fltk-1.4-fl_overlay_rect-no-crash-mac.mp4

However, you can see in both of these videos that 1.3 and 1.4 on Mac have a weird smearing issue if the rect is partly offscreen (different from the smearing issue I mentioned in my previous comment):

Of course, the smearing outside the mandelbrot drawing area is pretty much expected and not a real problem. But notice all the seemingly random smearing inside the mandelbrot drawing area. Probably an uneducated statement.

You said:

If your program uses coordinates beyond the window borders then this could be called a programming error but I agree that FLTK should not crash.

I would disagree that it could be described as a programming error.
I think the aesthetic of an overlay rect with 3 visible edges and 1 offscreen edge (as opposed to all visible, clamped edges) should be completely valid.
Additionally, in my own application's case (not mandelbrot), the call to fl_overlay_rect() is inside the draw() of a Fl_Group, and the group is a child of a Fl_Scroll, so the group shouldn't be too concerned if certain parts of itself are visible or not, since most of the group is usually scrolled way off screen and not within the presently-visible region of the Fl_Scroll.
Obviously we're in agreement that FLTK shouldn't crash, so no issue, just adding my 2 cents.

[dannye]

I may be too un-knowledgeable here, but isn't it strange that in the mandelbrot demo fl_overlay_rect() is being called in handle() instead of draw() or draw_overlay()?

fltk/test/mandelbrot.cxx

Line 219 in 7d7edcf

fl_overlay_rect(sx,sy,sw,sh);

I could be barking up the wrong tree, but I wonder if some of the issues are due to a malformed demo program rather than a bug with FLTK.

[Albrecht-S]

Thanks for further testing and your videos, and for your comments. I believe that fl_overlay_rect() is pretty old and not much tested in the FLTK test and demo programs. My spare time is currently rather limited and it would need some investigation (including code and documentation) to find out how it is (was) intended to be used and how it can be used correctly.

I would disagree that it could be described as a programming error.

That's why I used "could". ;-)

I think the aesthetic of an overlay rect with 3 visible edges and 1 offscreen edge (as opposed to all visible, clamped edges) should be completely valid.

Thanks for your opinion, point taken; this should be considered.

I may be too un-knowledgeable here, but isn't it strange that in the mandelbrot demo fl_overlay_rect() is being called in handle() instead of draw() or draw_overlay()?

Unfortunately the documentation is pretty sparse on how to use fl_overlay_rect(). However, it says "Using these functions is tricky." See chapter "Drawing Things in FLTK", section "Drawing Overlays".

[ManoloFLTK]

@dannye Function fl_overlay_rect does not need to use an Fl_Overlay_Window object. That function is to be used with a normal Fl_Double_Window or Fl_Window. Its only purpose is to draw a dotted rectangle that can be easily deleted by any of 2 means :

1. draw another overlaid rectangle elsewhere (the previous one will be deleted)
   or
2. call fl_overlay_clear().

That kind of operation is visible in the mandelbrot test program of the FLTK source code.

[ManoloFLTK]

I may be too un-knowledgeable here, but isn't it strange that in the mandelbrot demo fl_overlay_rect() is being called in handle() instead of draw() or draw_overlay()?

An important element here is the statement before the call to fl_overlay_rect:
window()->make_current();

This statement directs all future drawing statements to draw to the active Fl_Window. That's the reason why it's possible to use function fl_overlay_rect out of a widget's draw function.

I don't succeed to reproduce on my MacBook Pro the smear you mention with macOS, FLTK 1.4 and the (modified) mandelbrot test program. Could you describe with more detail how to get it?

[dannye]

I don't succeed to reproduce on my MacBook Pro the smear you mention with macOS, FLTK 1.4 and the (modified) mandelbrot test program. Could you describe with more detail how to get it?

@ManoloFLTK Sure, here's a longer video demonstrating when exactly the smearing does and does not happen:

fltk-1.4-mandelbrot-smear-mac.mp4

This was from commit 7d7edcf on master, with the only code change being commenting out if (x2>=x()+w()) x2=x()+w()-1; and if (y2>=y()+h()) y2=y()+h()-1; on lines 213 and 214 of test/mandelbrot.cxx and rebuilding mandelbrot with make.

If I keep the overlay rectangle within the window, the smearing does not happen. But as soon as the overlay rect is dragged outside the window to the right, the "smearing" appears as horizontal streaks (presumably because the previous overlay rect is not being "erased" correctly).

In the second half of the video I show that vertical streaks also appear if dragging the overlay rect below the bottom edge of the window.

[MatthiasWM]

tl;dr: fl_overlay_rect() is very specific in what it does. It's a hack for old slow computers and should probably not be used anymore unless the user know its limitations. If teh devs are ok with it, I will update the docs and leave it at that.

Ok, sorry, I completely missed this thread when it was created. I can give a little history, and maybe we can come up with a solution.

The original target platform of FLTK, SGI computers, had a hardware overlay plane in X11. Fl_Overlay_Window used that feature, and in draw_overlay(), you could draw anything without the need to call refresh()/draw(). This was important because draw() calls could take much more than the 50th f a second it takes on 2023 machines. We used the overlay plane to visually select nodes in our GUI.

When we ported FLTK to MSWindows, there was no longer a hardware overlay, so Fl_Overlay_Window was modified to draw the regular graphics first and then the overlay on top. This takes significantly longer than just redrawing the overly plane.

So we came up with a special use case when all we needed in the overlay was a single rectangle, but there was no time to refresh the entire window: we simply XOR a line over the existing drawing to draw the rectangle. fl_draw_overlay() was designed as a quick fix for missing hardware. The intention is to avoid Fl_Overlay_Window and instead draw something that looks like a rectangle while the actual graphics must(!) remain static. The idea was to use FL_PUSH, FL_DRAG, and FL_RELEASE to initiate, draw, and clear the rectangle, while avoiding refresh. The original implementation simply XOR'd a white line over the existing drawing to draw, and XOR'd again to erase the rect.

Now, with computers getting faster and users expecting instant refresh, fl_overlay_rect is too limited. The correct solution here is to use Fl_Overlay_Window and implement draw_overlay() using the original drawing functions fl_color(), fl_rect(), etc. . If the content of the window changes, call redraw(), if the overlay changes, call redraw_overlay(. FLTK will take care of the rest.

As for the smearing on macOS, that is my fault. Eventually, the XOR drawing method was no longer adequate, so I decided to implement fl_overlay_rect by reading the current screen content into four skinny images and then drawing the dotted line, disturbing the original content. If the rect is moved or erased, the old content is restored by drawing the skinny images where the dotted lines were. This has the same limitations (the original content must no change), but allows for a beautiful dotted line (no clue why X11 doesn't do that correctly).

Why does that smear on MacOS? Well, Apple introduced Retina displays, and the original code reads the screen at 100%, but Retina renders at 200%, so reading the screen effectively runs a blur filter over the restored screen content. So dragging the rectangle around a lot will continuously blur the content until it's refreshed the next time.

[ManoloFLTK]

Why does that smear on MacOS? Well, Apple introduced Retina displays, and the original code reads the screen at 100%, but Retina renders at 200%, so reading the screen effectively runs a blur filter over the restored screen content. So dragging the rectangle around a lot will continuously blur the content until it's refreshed the next time.

That used to be true, but is no longer. Retina displays are now correctly handled by fl_overlay_rect on macOS: 2-pixel wide or high images are read from the screen and drawn back to screen to erase the dotted rectangle.

[dannye]

@MatthiasWM Thank you for that useful info.

I'll be honest, my desire to use fl_overlay_rect first and foremost is for the dotted rectangle style. Like you were saying, a full redraw is fast enough that I'm not worried about speed problems.

I first tried to use draw_box but none of the default Fl_Boxtypes provide a dotted selection rectangle that looks similar to the fl_overlay_rect, so that's why I began experimenting with the overlay rect.

I'm aware that custom box type routines can be provided, but I haven't experimented with that yet.
Would you assume that mimicking the dotted style of fl_overlay_rect would be straightforward enough with a custom Fl_Box_Draw_F?

[MatthiasWM]

Try this inside Fl_Overlay_Window::draw_overlay():

  fl_color(FL_WHITE);
  fl_line_style(FL_SOLID);
  fl_rect(px, py, pw, ph);
  fl_color(FL_BLACK);
  fl_line_style(FL_DOT);
  fl_rect(px, py, pw, ph);
  fl_line_style(FL_SOLID);

[dannye]

Wow, thanks, that fixes all my problems at once. I should have taken note of fl_line_style() sooner!

I just had to account for negative width and/or height since fl_rect() doesn't handle that automatically in the same way that fl_overlay_rect() does.

This also allows me to customize the style and only use FL_DOT for dark themes and instead use FL_DASH for light themes, which I think provides better contrast. (dannye/crystal-tracker@4eaae39)

[MatthiasWM]

Nice. I am glad that this works for you ;-) and thanks for the feedback.

[MatthiasWM]

Updated documentation in c5f5973.

[Albrecht-S]

Thanks to @MatthiasWM for the improved documentation.

Edit: ... and thank you very much for the "history" comments. This is always good to know.

Yet another commit (157d276) fixes some typos and improves doxygen links of function names.

[MatthiasWM]

Thanks. Sorry for the typos.