Memory corruption when using Marco with compositing on AMD RX 550 + DVI-D

[AsciiWolf]

Expected behaviour

Marco continues to work fine with compositing effects enabled.

Actual behaviour

See this screenshot:

Steps to reproduce the behaviour

1. Have a system with AMD Radeon RX 550 (the 4 GB version) and DVI-D display.
2. Boot Linux Mint 20 Mate (it happens even on the live system, Marco + Compositing is default there) on such system.

MATE general version

1.24.0

Package version

1.24.0-1ubuntu1

Linux Distribution

Linux Mint 20 Mate

Link to downstream report of your Distribution

linuxmint/mintdesktop#40

Additional information

I have tried different compositors like Metacity + Compositing and Marco + Compton and they all worked fine.
Marco + Compositing worked fine on old Nvidia GPU (with both Mesa and proprietary drivers), but not on this new AMD one.

My Mesa version is: 20.0.8

Unfortunately, I was not able to get any relevant system logs.

[lukefromdc]

What is odd is that this would even be possible. Your screenshot proves something is going on, but Marco's compositor is CPU only and does not use GPU acceleration at all unless via Compton (which worked in your test). Note that Marco and Metacity are very unusual in supporting straight CPU compositing.

I wonder if you are getting a driver bug in which the display is normally corrupted but GPU compositing bypasses the problem? Yearts ago I had some corruption issues in the panel on an early Intel Atom netbook that were fixed by a driver update-even though I was not using GPU acceleration.

Also wonder if this relates to the Xpresent work from a while back( maybe a bug triggered by or bypassed by that extension?) Note that Marco can be built with or without Xpresent support. A difference there might explain why you get this with Marco but not Metacity.

I can't duplicate this, having neither that card nor that monitor connection.

[AsciiWolf]

I wonder if you are getting a driver bug in which the display is normally corrupted but GPU compositing bypasses the problem?

I forgot to mention that Marco without any compositing works fine.

Also, it's really weird that it worked fine on Nvidia GPU, but is broken on AMD one. Even when both were using the same Mesa drivers (I later moved to a proprietary Nvidia driver, but was running Mesa before and had Marco with compositing enabled).

[muktupavels]

If you can / know how to then can you try metacity from git? Or do it when 3.38.0 (not released yet) will be available in your distro?

Metacity now can use Xpresent, but it is experimental and can be enabled only with environment variable. Use META_COMPOSITOR=xpresent metacity --replace. I would like to know if it will have same problem.

[AsciiWolf]

Is it possible to run the compiled metacity without actually installing it into system? If so, I will try testing the git version with xpresent, however the next time I will be using that computer will probably be in a month or two.

[muktupavels]

There are changes to GSettings that will probably cause problems to run without installing... But I think it should be possible to update schema with extra/new key and then it should work. You would need to make two changes in /usr/share/glib-2.0/schemas/org.gnome.metacity.gschema.xml file. First you would need to add new extra enum:

  <enum id='org.gnome.metacity.MetaCompositorType'>
    <value nick='none' value='0'/>
    <value nick='xrender' value='1'/>
  </enum>

and new key that can be added after compositing-manager:

    <key name="compositor" enum="org.gnome.metacity.MetaCompositorType">
      <default>'xrender'</default>
      <summary>Compositor</summary>
      <description>
        Compositor that Metacity will use for compositing. Possible values
        are “none” and “xrender”.
      </description>
    </key>

After that you would need to recompile schemas with sudo glib-compile-schemas /usr/share/glib-2.0/schemas. Then you should be able to run/use compiled version without installing it.

[lukefromdc]

Note that it now looks like we've got TWO problems related to XPresent:
#578 which again is driver dependent, AMDGPU driver has an issue with it according to that report

[AsciiWolf]

Still the same issue with marco 1.24.0 on Linux Mint 20 with kernel 5.8.0-33-generic and mesa 20.2.6-0ubuntu0.20.04.1.

[lu-ccc]

I have that issue on ubuntu-MATE 20.04
turning compositor off, fixes it

[AsciiWolf]

Are you also using DVI-D?

[lu-ccc]

No, just HDMI but it seems to change with resolution
but i also use amd