Qt screenshots and scaling

[hmaarrfk]

On our qt application, we would like to take a "screenshot" of the window.

However, wgpu seems to draw directly onto the surface and bypasses qt's paintEvent so qt can't do it all on its own.

I've modified the triangle_qt_embed.py demo to allow it to fire off a request for a screenshot using Qt's grab() method.
hmaarrfk#1

(PS. I'll continue this draft of an issue in a bit, I'm going to try to provide an example of working code to "pull a screenshot" forcifully from the renderer's internal buffer)

[Korijn]

You might want to compare your approach to solutions for Qt with pyopengl as well

[hmaarrfk]

Hmm. Great tip!

Truthfully I had a much harder translating my pygfx demo into a wgpu demo......

For the onlookers browsing. The symptom are that the widget on which the rendering is happening in wgpu is just blank.

My "fix" is to grab the rendered texture on the GPU and then rescale it myself prior to pasting it into the rest of the buffer provided by Qt.

It's pretty tricky because on pygfx at least, the internal texture can be of a different size than the Qt widget. So you have to make sure to resize things correctly (at least in my strategy)

[almarklein]

The best solution would be to take the GPU screenshot from the canvas' frame buffer. However, that texture has a lifetime that is bound to a "draw event". Or maybe it lives until the next draw event? I'd have to try/check.

Another possible route might be to have a public method to draw (I think this came up somewhere else to), and perhaps such a method could provide a custom texture. Basically a method to take a screenshot (to a texture) of any size you want.

[hmaarrfk]

I think this came up somewhere else to

I saw this issue today, pygfx/pygfx#754 so it encouraged me to post my qt issue here to give an other perspective.

I updated my example above to generate:

The red and blue rectangles are there to give you a sense that if we can somehow get the texture (correctly scaled for qt ....) then we can place it in the right position ourselves manually as a workaround. Of course, the best way would be to play nice with Qt, but sometimes thats just hard....

[hmaarrfk]

In pygfx, I have access to renderer.snapshot() which makes it possible to get the right data in there. I just don't know how to do it (yet) with wgpu.

[panxinmiao]

The usage flag of the surface texture (obtained from the context.get_current_texture() method) does not include COPY_SRC, so the texture data cannot be read directly from it.

If you need to capture a screenshot of the rendered scene, you should create a render target texture yourself with the usage flag set to COPY_SRC | RENDER_ATTACHMENTS and use it as the color attachment in the render pass. Then, use command_encoder.copy_texture_to_texture() to copy the data to the surface texture.
When you call your screenshot() method, you can read the data from the target texture you created.

For QT applications, if you want to capture the entire GUI interface, not just the rendered scene, the best approach is to customize a QT widget by overriding its paintEvent method. Offscreen rendering with wgpu-py can easily assist you in achieving this.
Here is a sample code snippet (modified from triangle_qt_embed.py):

import importlib

# For the sake of making this example Just Work, we try multiple QT libs
for lib in ("PySide6", "PyQt6", "PySide2", "PyQt5"):
    try:
        QtWidgets = importlib.import_module(".QtWidgets", lib)
        break
    except ModuleNotFoundError:
        pass


from wgpu.gui.offscreen import WgpuCanvas

from triangle import main


from PySide6.QtWidgets import QWidget
from PySide6.QtGui import QPainter, QImage

class RenderableCanvas(QWidget):
    def __init__(self, parent=None):
        super().__init__(parent)
        self._canvas = WgpuCanvas()

    def paintEvent(self, event):
        main(self._canvas) # do the animation and render logic
        frame = self._canvas.draw()
        painter = QPainter(self)
        width, height = frame.shape[1], frame.shape[0]
        img = QImage(frame, width, height, 4 * width,  QImage.Format_RGBA8888)
        painter.drawImage(event.rect(), img)


    def resizeEvent(self, event):
        self._canvas.set_logical_size(event.size().width(), event.size().height())


class ExampleWidget(QtWidgets.QWidget):
    def __init__(self):
        super().__init__()
        self.resize(640, 480)
        self.setWindowTitle("wgpu triangle embedded in a qt app")

        splitter = QtWidgets.QSplitter()

        self.button = QtWidgets.QPushButton("screnshot", self)
        self.button.clicked.connect(self.screenshot)
        self.canvas1 = RenderableCanvas(self)
        self.canvas2 = RenderableCanvas(self)

        splitter.addWidget(self.canvas1)
        splitter.addWidget(self.canvas2)

        layout = QtWidgets.QHBoxLayout()
        layout.addWidget(self.button, 0)
        layout.addWidget(splitter, 1)
        self.setLayout(layout)

        self.show()

    def screenshot(self):
        self.grab().save("screenshot.png")


app = QtWidgets.QApplication([])
example = ExampleWidget()

# Enter Qt event loop (compatible with qt5/qt6)
app.exec() if hasattr(app, "exec") else app.exec_()

[hmaarrfk]

Thank you for the detailed response, it will take me time to digest it.