Overlay flickers on VTK cells

[remram44]

(originally TRAC#21, 2013-03-20)

What have been tried:

• Setting WA_TransparentForMouseEvents on the containedWidget as well
• Replacing the cellWidget with an image during a drag, like QCellPresenter does in the spreadsheet

---

comment by @remram44:

Okay I really can't figure this one out. VTK cells seem to behave very differently from other cell types and many of my attempts to work around this simply segfault...

---

comment by @remram44:

I've tried rewriting my 'cellcontainer' thing from scratch, using an approach similar to what the spreadsheet does (e.g. a 'Presenter' class, which gets put as a Cell in the spreadsheet) but to no avail, VTK still breaks it. Using VTK also breaks the other cell types for unknown reasons. I don't know how to make this work.

---

comment by dakoop:

Is it possible that Qt5/PyQt4.10 may fix this? I think VTK doesn't yet support Qt5, but there was a change in how things are drawn in QT5 that may impact this.

[remram44]

From yesterday's email:

• Works fine on Linux, VTK 5.8.0
• Transparent VTK cell for @rexissimus on Mac
• Crashes on my Mac, VTK 5.10.1:

X Error of failed request:  BadWindow (invalid Window parameter)
  Major opcode of failed request:  2 (X_ChangeWindowAttributes)
  Resource id in failed request:  0x75351c90
  Serial number of failed request:  7
  Current serial number in output stream:  9

[rexissimus]

Using CDMSCell and VTK 6 I still get a transparent cell, but lots of these errors:

ERROR: In /Users/doutriaux1/build/build/VTK/Rendering/OpenGL/vtkOpenGLRenderer.cxx, line 1202
vtkOpenGLRenderer (0x7f9199a47aa0): failed after Clear 1 OpenGL errors detected
0 : (1286) Invalid framebuffer operation

ERROR: In /Users/doutriaux1/build/build/VTK/Rendering/OpenGL/vtkOpenGLPolyDataMapper2D.cxx, line 442
vtkOpenGLPolyDataMapper2D (0x7f91997f5220): failed after RenderOverlay 1 OpenGL errors detected
0 : (1286) Invalid framebuffer operation

ERROR: In /Users/doutriaux1/build/build/VTK/Rendering/OpenGL/vtkOpenGLDisplayListPainter.cxx, line 188
vtkOpenGLDisplayListPainter (0x7f9198ef9a30): failed after RenderInternal 1 OpenGL errors detected
0 : (1286) Invalid framebuffer operation

[rexissimus]

This patch makes the VTK scene render correctly for me, but the overlay is not visible.

+++ b/dat/gui/cellcontainer.py
@@ -215,12 +215,10 @@ def setWidget(self, widget):
                 self.toolBar = None
             if widget:
                 widget.setParent(self)
-                widget.show()
             self.containedWidget = widget

         if widget is None:
             return
-        widget.raise_()
         self._set_toolbar_buttons(True)

         self.contentsUpdated()

[remram44]

Well yes, the cell should not be raise_d over the overlay.

[rexissimus]

I am not sure it is possible to use a Qt overlay with VTK cells. OpenGL may take ownership of that part of the screen. This may differ between platforms as you say it works on Linux. It looks like UV-CDAT is working around this by using either OpenGL-based configuration overlays and standalone qt-based configuration windows.

We could remove the cell while showing the overlay like is done with the patch below. We could have a flag on the cells telling DAT if it supports overlays?

diff --git a/dat/gui/cellcontainer.py b/dat/gui/cellcontainer.py
index e1b09ac..92461ef 100644
--- a/dat/gui/cellcontainer.py
+++ b/dat/gui/cellcontainer.py
@@ -215,12 +215,10 @@ def setWidget(self, widget):
                 self.toolBar = None
             if widget:
                 widget.setParent(self)
-                widget.show()
             self.containedWidget = widget

         if widget is None:
             return
-        widget.raise_()
         self._set_toolbar_buttons(True)

         self.contentsUpdated()
@@ -273,7 +271,7 @@ def contentsUpdated(self):
         self._set_overlay(None)

     def _set_overlay(self, overlay_class, **kwargs):
-        if overlay_class is None:
+        if overlay_class is None and not self._execute_pending:
             # Default overlay
             if self._plot is not None and self.has_error():
                 self._set_overlay(VariableDroppingOverlay, overlayed=False)
@@ -302,13 +300,18 @@ def _set_overlay(self, overlay_class, **kwargs):
             # Now that we are done with the overlay, we can go on with a
             # deferred execution
             if self._execute_pending:
-                self.update_pipeline()
+                self.update_pipeline(True)
                 self._execute_pending = False
         else:
             self._overlay = overlay_class(self, **kwargs)
             self._overlay_scrollarea.setWidget(self._overlay)
             self._overlay.show()
             self._overlay_scrollarea.raise_()
+            if self.containedWidget:
+                self.containedWidget.setParent(None)
+                self.containedWidget.deleteLater()
+                self.containedWidget = None
+                self._execute_pending = True
             self.do_layout()
             self._set_toolbar_buttons(None)

[remram44]

There is no need for the cell to be interactive while it is behind the overlay. Removing the cell is totally acceptable, in fact it is done on the branch use-pixmaps-when-dragging; unfortunately that didn't fix the flickering last I checked.

[remram44]

My goal was to do something similar to what is done in the spreadsheet overlay (QCellPresenter, used to move/copy cells).

[rexissimus]

I did not get hiding the cell to work before, but I now see this is working on use-pixmaps-when-dragging.
I was able to modify use-pixmaps-when-dragging to also use pixmaps when clicking "show overlay". It works great! I see some flickering when moving the mouse while dragging over the cell but I am not sure this is what you meant? If you got constant flickering (which I have not seen), I have seen a suggestion to disable double buffering? But that may impact the rendering?

[remram44]

Did you push your changes? I can try it on both Linux and Mac and report back

[rexissimus]

I pushed the changes to use-pixmaps-when-dragging. It is working for both vtk and vcs cells (which is based on VTK cells). It can probably be improved to remove the flickering while dragging.

[remram44]

I still get a crash on OSX:

creates overlay PlotPromptOverlay
adds overlay_scrollarea
deletes overlay
removes overlay_scrollarea
X Error of failed request:  BadWindow (invalid Window parameter)
  Major opcode of failed request:  2 (X_ChangeWindowAttributes)
  Resource id in failed request:  0xda5e9200
  Serial number of failed request:  7
  Current serial number in output stream:  9

[rexissimus]

Which VTK version are you running? One solution I found here: http://stackoverflow.com/questions/6700615/issue-getting-vtk-to-work-with-qt

[remram44]

I'm using 5.10.1 on Mac; I also filed VisTrails/VisTrails#1082 which does seem similar (same abort from VTK).

[remram44]

Yep, that fixed it on my Mac! It does make the overlay code very ugly though. The branch also has to be rebased, it is based pretty far behind.

[rexissimus]

Great! Yes, we should de-duplicate the pixmap code and fix the drag flickering. I did not want to change too much before the rebase though. Maybe we can just cherry-pick the pixmap stuff into the updated branch?

[remram44]

use-pixmaps-when-dragging-rbs should be the correct rebase

[remram44]

Hmm no, I got it wrong... Not sure what is going on in there anymore.

[remram44]

Ok I don't have this. Desperately needed though.

I'm really not sure what is the issue here, it only affects VTK; is it because the viewport becomes invisible? The VisTrails spreadsheet does something similar with QCellPresenter.

[rexissimus]

a90de6a fixes this for me.

I got the flickering while moving the mouse. It also happens with non-vtk cells, but it is so fast it is not visible. Does this solve the problem? It is not clear from your video if you move the mouse during the flickering.