Some devices can't be rotated in the left or right directions

[ruineka]

Apparently Alder Lake intel chips don't have hardware support to rotate displays to the right or left and I'm told that the compositors will need to support the rotating of the display. I did some testing and it seems that the legacy steamcompmgr and the internal one in gamescope both have the same issues with this device. I talked quite a bit about this here over at the gitlab. https://gitlab.freedesktop.org/drm/intel/-/issues/8278#note_1813580

[steamcmpmgr] << xrandr is unable to rotate the display

[gamescope] << gamescope works when ran with the normal or upsidedown orientations, but left and right crash the internal steamcmpmgr

If I run gamescope using mutter I am able to get the screen to work correctly.
This is using mutter --wayland -- gamescope -e -f -w 1280 -h 800 --xwayland-count 1 -- steam -steamos3 -steampal -steamdeck -gamepadui (I had to use a modified gamescope in order to rotate the DSI-1 connector. It needed to be treated as internal display and I wasn't able to use --xwayland-count 2 because steam would run, but the actual frames of the screen wouldn't update).

I'm also able to rotate the screen without any issues using Gnome X11 and Gnome Wayland. For the record the GPD Win 3 also has issues where you are unable to rotate the display to the left or the right, but additionally stability is another problem for that device.

[ruineka]

@emersion While I was looking to see if it was more of a wlroots problem than a gamescope issue I noticed you mentioned what appears the be the same problem I'm having here https://gitlab.freedesktop.org/wlroots/wlroots/-/issues/1058

[ruineka]

I think what needs to happen is the ctrc planes need to be rotated instead of the primary drm if hardware rotation is not supported. I think the devices that work with the current method of rotating the display have a MiPi DSI to eDP adapter that adds support for this. If a device doesn't have such adapter you are working with a DSI connector that is likely to fail rotating 90 or 270 degrees on the primary plane. I'm doing research to learn how all this works so I may be completely off base on this.

[emersion]

That wlroots issue is unrelated. gamescope has its own KMS code.

ATM gamescope requires KMS rotation support. It should be possible to patch gamescope to remove this requirement and rotate in the shaders instead.

[ruineka]

That wlroots issue is unrelated. gamescope has its own KMS code.
ATM gamescope requires KMS rotation support. It should be possible to patch gamescope to remove this requirement and rotate in the shaders instead.

Interesting, it seems that is what KDE did to handle the rotations too. https://phabricator.kde.org/D25907

I'm not sure this is something I'd be capable of doing a PR for though. I'll keep tinkering to see what I can accomplish anyway. Thanks for the insight!

[ruineka]

That wlroots issue is unrelated. gamescope has its own KMS code.

ATM gamescope requires KMS rotation support. It should be possible to patch gamescope to remove this requirement and rotate in the shaders instead.

I've been studying vulkan and I'm starting to understand the graphics pipeline a bit more, but I have a question when it comes to how vertex shaders should be loaded.

I was following the vulkan tutorial and got to the part where they create vkCreateGraphicsPipelines and noticed gamescope itself has nothing for this. Instead gamescope compiles the vkCreateComputePipelines using VK_SHADER_STAGE_COMPUTE_BIT and has nothing for VK_SHADER_STAGE_FRAGMENT_BIT or VK_SHADER_STAGE_VERTEX_BIT.

Is there a reason why gamescope doesn't do this? With my current understanding it looks like wlroots handles the graphics pipeline when the display and renderer are created. It seems that compilePipeline() would need to take an additional parameter to check the desired pipeline type to handle graphics and compute pipes.

I know I have a lot more to learn and this is probably a clueless question, so I apologize in advance. Any pointers is extremely helpful so I know what to focus on to hopefully be able to tackle this. ;)