OpenGL severe display problem on macOS 11 Big Sur

[DominusExult]

SDL 1.2x apps compiled against SDK 10.15 or SDK 11.x have severe display problems when started on macOS 11. All is fine on macOS < 11.x or when compiled against SDK 10.14.
The window is as big as it should be but the actual video display is in the lower left corner, as if it only renders to the lower left quarter of the screen. Observed with DOSBox but reproduced with SDL's testgl program.

It seems that it doesn't matter against which SDK you compiled SDL 1.2x.

Tested with current git and also 5d44141 (as the next commit 718b76b added some macOS OpenGL fixes, I thought I'd try the commit before).

Screenshot of testgl compiled against SDK 10.15 on macOS 11

Screenshot of testgl compiled against SDK 10.14 on macOS 11

Edit: with testdyngl, you can see that it really renders to the lower left quarter of the window

[DominusExult]

in the repository of DOSBox-x there is a fix for this, I think at joncampbell123/dosbox-x@0c87b06#diff-652a270a42d971e3094d3410653821f6d4dd312cc006a9ac3e101b0fa64baee9 but as there has been more added to the code, it doesn't apply and fix it right away.

[sezero]

AFAICS, that patch requires applying joncampbell123/dosbox-x@c924e71 first.

[sezero]

The two patches combined and applies-cleanly-against-current-git
is like the following (not even compile-tested). Does it help?

diff --git a/src/video/quartz/SDL_QuartzVideo.m b/src/video/quartz/SDL_QuartzVideo.m
index 6e71fbe..1228bfb 100644
--- a/src/video/quartz/SDL_QuartzVideo.m
+++ b/src/video/quartz/SDL_QuartzVideo.m
@@ -1078,6 +1078,18 @@ static SDL_Surface* QZ_SetVideoWindowed (_THIS, SDL_Surface *current, int width,
 
         window_view = [ [ NSView alloc ] initWithFrame:contentRect ];
         [ window_view setAutoresizingMask: NSViewWidthSizable | NSViewHeightSizable ];
+
+        NSRect contentRectOrig = NSMakeRect (0, 0, width, height);
+
+#if MAC_OS_X_VERSION_MAX_ALLOWED >= 101500
+        /* OpenGL in Catalina apparently wants to match pixels on Retina displays. Without this the OpenGL
+           display will only use 1/4th the display in the lower left corner. This seems to be an API issue
+           with XCode as binaries previously compiled on Mojave don't have this issue on Catalina. */
+        contentRectOrig = [ window_view convertRectFromBacking:contentRectOrig ];
+#endif
+
+        [ window_view setBoundsSize: contentRectOrig.size ];
+
         [ [ qz_window contentView ] addSubview:window_view ];
         [ gl_context setView: window_view ];
         [ window_view release ];

[DominusExult]

Thanks.
Yes, both commits apply and do fix the problem!

[sezero]

Thanks.
Yes, both commits apply and do fix the problem!

@icculus, @slouken: OK to apply?

[icculus]

This is probably correct, please apply.

(Usual disclaimer applies, @DominusExult: please move to SDL2 or at least sdl12-compat and let's figure out how to make that happen if something is preventing either solution!).

But also: we ran into this in ioquake3 and I believe the fix we ended up with was calling glViewport() with the results of SDL_GL_GetDrawableSize(). Obviously that fix doesn't apply to SDL 1.2, where SDL_GL_GetDrawableSize doesn't exist, but I'm saying this out loud to ponder if SDL2 also should have gotten a similar fix internally. It would need some research.

[DominusExult]

(Usual disclaimer applies, @DominusExult: please move to SDL2 or at least sdl12-compat and let's figure out how to make that happen if something is preventing either solution!).

I'm just the build maintainer of DOSBox and I am hoping that the main dev makes the plunge soon :)
But I really need to try out sdl12-compat. I'm always getting side tracked.

[icculus]

But I really need to try out sdl12-compat. I'm always getting side tracked.

It's made a lot of improvements recently! If you make the plunge, I won't be offended if you come directly to me with issues.

[sezero]

Patch applied as 89fc1b1

[DominusExult]

Something I "just now" realized something is still wrong there.
The display is ok but the mouse is not correctly tracked in the window/screen relation. It seems as if the cursor is only recognized in the lower left quarter and outside of this a click is not registered in the SDL window but to anything that is below the window.
I hope that makes sense.

(btw. this works correctly with SDL12compat)

[sezero]

Re-opening

[sezero]

@icculus : Any ideas?

[icculus]

I can't think of anything obvious right now. I'd have to look more deeply.

[sezero]

I'd have to look more deeply.

Yes please :)

[DominusExult]

It's all so very interesting this bug...
Only the lower left quarter of the DOSBox screen is capturing the mouse.
If I keep SDL 1.2 git compiled against SDK 10.15 or SDK11.x but compile DOSBox against SDK 10.14 (which doesn't need the fix) I end up with DOSBox window only showing the lower left quarter of the screen it should show.
Compiling SDL 1.2 git and DOSBox against SDK 10.14 everything works fine.

I wonder if @joncampbell123 has this already fixed in his fork of SDL12

[DominusExult]

As you can see by my mentions in Dosbox-X issues, there is something bad going on that also affects SDL2 as @icculus hinted at here in an earlier comment.

[kjliew]

I can't think of anything obvious right now. I'd have to look more deeply.

It is very obvious for me.
https://www.vogons.org/viewtopic.php?p=983850#p983850

[slouken]

Does this happen when using sdl12-compat?

[icculus]

(Sorry for the late reply, I just noticed there was exactly one open bug on the 1.2 repo)

that also affects SDL2 as @icculus hinted at here in an earlier comment.

We made fixes for HighDPI stuff since then. :)

But yeah, SDL 1.2 isn't highdpi aware at all. The best bet, if this is still a going concern, is to move to sdl12-compat, which as far as I know works with DosBox-X, and gets you some useful improvements, like moving software rendering to the GPU, etc.

[DominusExult]

Does this happen when using sdl12-compat?

very late reply, sorry.

This does not happen with current sdl12-compat and I've now changed my whole snapshot builds to only use sdl12-compat and be done with it :)

[icculus]

Yeah, I'm inclined to say this should be closed with no fix planned and just tell people to use sdl12-compat, as 1.2 is going to continue to bitrot across all platforms as time goes on.