New High DPI Support API is needed

[SDLBugzilla]

This bug report was migrated from our old Bugzilla tracker.

These attachments are available in the static archive:

• windows highdpi patch (windows-highdpi-2017-10-26.diff, text/plain, 2017-10-26 23:04:06 +0000, 57916 bytes)

Reported in version: HG 2.1
Reported for operating system, platform: All, All

Comments on the original bug report:

On 2016-03-01 19:08:34 +0000, Yuya Kumagai wrote:

SDL's High DPI support is already available for Mac OS X and iOS (marked as DRAFT). But there is no Windows support. So, windows are scaled by Windows.

Win32 API has functions to specify a process's dpi awareness they named SetProcessDPIAware and SetProcessDpiAwareness. If a process is specified as dpi aware, windows are not scaled.(SFML and GLFW calls them).

Currently, SDL can create a high dpi window (by SDL_WINDOW_ALLOW_HIGHDPI when SDL_HINT_VIDEO_HIGHDPI_DISABLED is set to 0). But, Windows has no function to specify dpi awareness per-window.

And, SDL_HINT_VIDEO_HIGHDPI_DISABLED is set to 0 (so, high dpi window is allowed) by default. This cause breaking change if High DPI support on Windows is added.

So, I think we should create a new API for High DPI support.

The same opinion is commented on here by Eric Wasylishen: https://bugzilla.libsdl.org/show_bug.cgi?id=2713#c5

On 2017-08-12 04:34:18 +0000, Sam Lantinga wrote:

Hey Eric, what's the current state of your Windows high DPI patch? Have you and Alex sorted out the semantic differences between Windows and Mac implementations?

Ryan and I discussed it a bit last week and agreed that having the API work in virtual screen points and then having rendering contexts have a separate physical pixel size made sense for SDL.

On 2017-08-12 04:40:58 +0000, Alex Szpakowski wrote:

Eric's changes seem good to me, but I haven't actually built and tested them on a real Windows system yet – I definitely plan on doing so soon because I want to use the changes in an upcoming version of my own project that uses SDL, though.

On 2017-08-12 20:00:57 +0000, Eric Wasylishen wrote:

I think it's in pretty good shape.

Ryan and I discussed it a bit last week and agreed that having the API work in
virtual screen points and then having rendering contexts have a separate
physical pixel size made sense for SDL.

Yeah, agreed. The patch emulates a virtual screen coordinate system in points on top of Windows' virtual screen coordinates, which are in pixels when the app is highdpi aware.

The outstanding things to do for my patch:

• rebase it to to account for https://hg.libsdl.org/SDL/rev/0060bcf673e8 (should be easy / doable today)

• check that it sends a window resize event when the dpi changes

• decide on the API changes. What I currently do is deprecate the SDL_WINDOW_ALLOW_HIGHDPI window flag and add a SDL_HINT_VIDEO_HIGHDPI_ENABLED hint, so if your SDL app is high-dpi aware, you would set the hint in your app's startup code before initializing SDL. Maybe "SDL_HINT_VIDEO_HIGHDPI_ALLOWED" would be a clearer name? Alex also suggested we could merge the enable/disable hints into one hint.

The other option is, if we limit highdpi support to Windows 10 Anniversary Update and above, I think we could avoid this API change and stick with the window flag, since MS added the ability to enable highdpi on a per-window basis in that update. (otherwise, with my patch as-is, we get highdpi support on Vista and up).

A disadvantage of going with a hint is, you lose the ability to change DPI awareness while the app is running.

Also I'm not sure about the situation on x11, wayland, etc., whether they allow DPI awareness to be set per-window.

• regarding SDL_DisplayMode, Alex mentioned he was working on adding an API to get the pixel dimensions of a mode. I'm still not totally sure I understand having the w/h fields of display modes in points, at least, for games using SDL I'm pretty sure they will only care about pixels (e.g. in the game's settings menu).

BTW, here is the link where Alex and I were discussing the patch:
ericwa/SDL-mirror@8870be8

On 2017-08-27 23:14:23 +0000, Alex Szpakowski wrote:

I wonder if it's worth it to have both the global app-startup DPI hint and the per-window DPI flag, and a new API to query whether the latter is supported? That way macOS/iOS/Win10-anniversary could enjoy the benefits of per-window high DPI support, and stock Win10 and older can still have global high DPI support.

On 2017-08-27 23:16:57 +0000, Alex Szpakowski wrote:

Side note: I haven't looked into what kind of high DPI / retina support is available on Linux, so I don't know if it's capable of per-window high DPI or not (although I do know SDL's linux backends don't have any code for highdpi support at all, currently).

On 2017-09-04 07:25:37 +0000, Alex Szpakowski wrote:

I tested the latest code from the branch for a few minutes today, some things I noticed:

• SDL_GetDesktopDisplayMode seemed to return the size in raw pixels rather than DPI-scaled points of the desktop (which doesn't play well with the window position, which is in points). It behaved like that regardless of whether I had the high DPI hint enabled.

• Similarly, enumerating available display modes seemed to give back sizes in pixels.

• Creating an exclusive-fullscreen window and then calling SDL_SetWindowFullscreen caused weird issues with the window not sizing correctly / getting stuck in limbo.

On 2017-09-05 07:27:45 +0000, Eric Wasylishen wrote:

Thanks for testing it out!

• SDL_GetDesktopDisplayMode seemed to return the size in raw pixels rather than DPI-
  scaled points of the desktop (which doesn't play well with the window position, which
  is in points). It behaved like that regardless of whether I had the high DPI hint
  enabled.

I just pushed a commit that should make SDL_GetDesktopDisplayMode return points, consistent with macOS:
https://github.com/ericwa/SDL-mirror/commits/windows-highdpi

• Similarly, enumerating available display modes seemed to give back sizes in
  pixels.

The API's for enumerating and changing display modes seem to be exclusively pixels -
EnumDisplaySettings ( https://msdn.microsoft.com/en-us/library/windows/desktop/dd162611(v=vs.85).aspx ) and ChangeDisplaySettingsEx ( https://msdn.microsoft.com/en-us/library/windows/desktop/dd183413(v=vs.85).aspx ).

Did a bit of searching, and it looks like the scale factor is stored in the registry on a per-monitor basis: https://stackoverflow.com/questions/35233182/how-can-i-change-windows-10-display-scaling-programmatically-using-c-sharp

So, it's not like macOS where there the OS's mode list also includes scale factors (unless I've overlooked something).

• Creating an exclusive-fullscreen window and then calling SDL_SetWindowFullscreen
  caused weird issues with the window not sizing correctly / getting stuck in limbo.

I'm not seeing any issues with Ctrl+Enter in testgl2, which toggles between these:
SDL_SetWindowFullscreen(window, SDL_WINDOW_FULLSCREEN);
SDL_SetWindowFullscreen(window, SDL_FALSE);

Are those the SDL_SetWindowFullscreen calls that were causing issues for you?

I wonder if it's worth it to have both the global app-startup DPI hint and the per-
window DPI flag, and a new API to query whether the latter is supported? That way
macOS/iOS/Win10-anniversary could enjoy the benefits of per-window high DPI support,
and stock Win10 and older can still have global high DPI support.

Yeah - I'm thinking the same thing, and going to have a shot at implementing that.

One thing that might be possible is getting rid of the messiest / most fragile part of my patch, the WIN_PhysicalToVirtual_ScreenPoint function, and getting Windows to do the conversion - see the "Windows Placement" section of this blog post:

https://blogs.windows.com/buildingapps/2016/10/24/high-dpi-scaling-improvements-for-desktop-applications-and-mixed-mode-dpi-scaling-in-the-windows-10-anniversary-update/#SYriggHty8TDC6jJ.97

On 2017-10-26 23:04:06 +0000, Eric Wasylishen wrote:

Created attachment 3045
windows highdpi patch

Here's an update to my patch.

I tried, and gave up on, doing per-window DPI awareness on Windows 10 Anniversary (what Windows docs call "mixed-mode" DPI Awareness; see: https://msdn.microsoft.com/en-us/library/windows/desktop/mt744321(v=vs.85).aspx ). It creates a confusing situation where the units returned by API's (e.g. GetClientRect) depend on when you call them (whether inside a window's WindowProc or outside). The bigger problem is it doesn't seem to interact well with OpenGL, at least with Nvidia drivers: what happens is, if you have the process set as DPI unaware, but create a DPI-aware window, OpenGL can only render in 1/4 of the window (with the monitor using 200% scaling). I'm guessing that something in the WGL internals is calling GetClientRect to determine the framebuffer size, but it's not set up to handle the case when the thread's DPI awareness differs from the window's.

So, I'm back to suggesting deprecating the window flag and adding the hint. I renamed the hint to SDL_HINT_VIDEO_ALLOW_HIGHDPI.

Other changes:

• standardized the terminology in the code to "Windows" and "SDL" coordinates, it was previously all over the place (points/pixels/unscaled/scaled). I went with that instead of "points" and "pixels" because Windows coordinates are only pixels if the process is DPI aware, and also the "SDL" coordinate system is a weird hybrid of points and pixels (the monitor top-left corners are in pixels, but everything else is in points).

• implement WM_GETDPISCALEDSIZE

• add documentation for the SDL_HINT_VIDEO_ALLOW_HIGHDPI hint and lots of comments on the confusing bits.

• figured out how Windows handles display mode changes and high-dpi. I added some explanation in WIN_SetDisplayMode. Short version: changing the screen resolution resets the monitor DPI to 96... unless you change to whatever the current desktop resolution is (in pixels), which changes the monitor DPI to the value saved in Windows's Display Settings.

The main thing I am unsure about in the patch is the change to CreateWindow, now it creates every window as 0x0 initially, then moves/sizes it to the desired location in SetupWindowData. I haven't seen an problems from this, but I can probably modify this to use the WIN_ScreenRectFromSDL function if this looks like a bad idea.

On 2018-10-24 11:13:08 +0000, Ellie wrote:

Ok, so I haven't looked at the patch yet, but:

But can you PLEASE for the love of all things, can you PLEASE CHANGE that SDL_GetWindowPos(), SDL_GetWindowSize() and others do sometimes NOT return native hardware pixels?? (which seems to be the case according to the libSDL wiki)

This is kind of a coordinate nightmare, because 1.) the wiki is absolutely unclear which functions are truly affected, 2.) some parts like the renderer always expect native coordinates anyway, 3.) the application needs to be high dpi aware in any case for things to work correctly, so why do this weird half-baked sometimes-coordinates-are-scaled-up-oddly thing?? It just doesn't make any sense.

Maybe for the original underlying OS API this made sense for some cases, but SDL should really abstract this away. The current behavior is just a mess, hard to understand and doesn't help anyone. Can you please remove this?

I propose adding new flag to SDL_Init to switch to the new behavior. I can also make a separate ticket for this if required.

On 2018-10-24 11:16:26 +0000, Ellie wrote:

I am referring to this paragraph by the way:

"The window size in screen coordinates may differ from the size in pixels, if the window was created with SDL_WINDOW_ALLOW_HIGHDPI on a platform with high-dpi support (e.g. iOS or OS X). Use SDL_GL_GetDrawableSize() or SDL_GetRendererOutputSize() to get the real client area size in pixels. "

For me quite a nightmare, because on Windows with HighDPI as it is now, I can't test this, so how am I even supposed to make sure my application works with it? I tested HighDPI, but with sane pixel coordinates, not whatever that paragraph suggests might be going on sometimes. So I propose that this simply shouldn't be happening, I can't even see why that would ever be useful...

On 2018-10-24 11:27:43 +0000, Ellie wrote:

I just read more in the comments above, the "points" coordinate system is what I am referring to. I would LOVE a way to just get rid of it, some "I am properly DPI aware, just give me the pixels and stop scaling coordinate systems around" flag.

On 2019-10-15 04:19:51 +0000, Anthony Pesch wrote:

Hi everyone,

Is there any interest in getting Eric's patch into master?

I've just recently started using SDL_WINDOW_ALLOW_HIGHDPI for https://redream.io on Mac, but Windows is still problematic.

I'm game for rebasing Eric's patch, but I'm having a hard time figuring out what other work is required / what blocked it from getting in ~2 years ago.

On 2019-10-15 23:45:20 +0000, Alex Szpakowski wrote:

I haven't had time to look at this in a while, but in general I think I'd prefer if per monitor v2 DPI awareness is supported first (since it's more flexible, more modern, and theoretically matches up with SDL's existing APIs a lot better than the older versions), and then the older versions of Windows DPI awareness can be added after if there's a high demand.

On 2019-10-15 23:47:34 +0000, Alex Szpakowski wrote:

(In reply to Alex Szpakowski from comment # 13)

in general I think I'd prefer if per monitor v2 DPI awareness is supported first

Also for selfish reasons - my current monitor setup at my job has two monitors at 150% DPI scaling, and one at 100%, and most Windows apps are horrible at supporting scaling in that situation. I don't want SDL to be added to that pile.

On 2019-10-19 20:06:40 +0000, Sam Lantinga wrote:

Agreed.

Does anyone have a design that we can point to and say "let's do that?" If not, let's coordinate an actual meeting to talk through the issues and figure out what design makes sense for SDL.

On 2019-10-20 11:47:54 +0000, Ellie wrote:

Is this the right time and place to note my suggestion that I think using points instead of just the true pixels anywhere does more harm than good? https://bugzilla.libsdl.org/show_bug.cgi?id=4423 It's ok if this is eventually decided against, I am just suggesting it would be useful to at least consider changing this inconsistency

On 2019-10-20 20:47:06 +0000, Alex Szpakowski wrote:

(In reply to Jonas Thiem from comment # 16)

Is this the right time and place to note my suggestion that I think using
points instead of just the true pixels anywhere does more harm than good?

This is your fourth post about it in this bug report, plus the three other posts you made in the other bug report you linked. I think we understood your opinion in your first post.

I get that it's simpler as a developer to only care about raw pixels and nothing else, but operating systems, display hardware, and user expectations don't work like that anymore. They will continue to work less and less like that going into 2020 and the future. Going down the route of "literally everything is in pixels" results in apps that end up looking broken on various display/OS configurations. I explained a bit in my post in your other thread.

SDL currently has consistent high-dpi support on macOS, iOS, and Wayland, where all windowing and mouse coordinates are in DPI-scaled units, and SDL_GL_GetDrawableSize and SDL_GetRendererOutputSize are in pixels. SDL doesn't have any knowledge about DPI scaling on Windows, X11, and Android (I'm not sure about other platforms such as emscripten), so using those as examples of what to do doesn't make sense. Your other thread talked about consistency and I agree that's important - and SDL on macOS etc. behaves consistently with respect to DPI scales.

---

In terms of API/design changes going forward, IMO the current system works pretty well on the operating systems it's implemented in (except I'd like a way to determine the DPI scale of a particular display or display mode before creating a window with it), I dunno if I like the idea of drastic changes to the way SDL exposes DPI scale.

If Windows' pmv2 can work with OpenGL drivers (does it work to enable DPI awareness globally and then selectively disable it per window, instead of the opposite?) then the only big question in my mind is whether to expose windowing/monitor/input coordinates on Windows as DPI-scaled points (consistent with SDL on other platforms, and consistent with other platforms in general) or as raw pixels (sort of consistent with Windows right now I think, but who knows in the future).

If windowing/monitor/input coordinates are exposed as pixels instead of DPI-scaled points on Windows, then there also needs to be new APIs to get the DPI scale factor for a given window and a given screen index (and maybe display mode), because dividing the drawable size by the window size won't work anymore.

I suppose a third option is to add new APIs for everything that returns window/monitor/input coordinates, to allow SDL users to choose whether to get the coordinates in DPI-scaled units or pixels, but it also sounds easy for users to use the wrong function accidentally, and it would be a lot of new APIs to add.

On 2019-10-21 07:52:30 +0000, Eric Wasylishen wrote:

Hey, sorry for dropping this. I'm game to help get the patch updated and work out the details.

The last time I updated my patch was Feb 2018: https://github.com/ericwa/SDL-mirror/tree/windows-highdpi

If Windows' pmv2 can work with OpenGL drivers (does it work to enable DPI
awareness globally and then selectively disable it per window, instead of the
opposite?)

Agreed, this is the key issue, because if it works we can implement the current SDL API on Windows (SDL_WINDOW_ALLOW_HIGHDPI window flag), otherwise it's impossible and we would need to go the hint route instead. Good idea to try enabling it and then selectively disabling.

Other API concerns that come to mind:

• It's unfortunate the SDL mouse events are ints. So if these are in DPI-scaled points on macOS/iOS/Wayland, you'd get mouse movement in 2px steps if you have a 2x scale factor.

• SDL should report relative mouse movement (xrel/yrel) as raw USB data with no scaling, not points, right?

• Having SDL_DisplayMode use width/height in points only, with no mention of the pixel size, has some issues.

e.g. consider the retina MBP returning these modes: (hypothetical, not sure what macOS actually returns)
a) 2880x1800 pixels @1x (2880x1800 points), so the "Everything is tiny" mode
b) 2880x1800 pixels @2x (1440x900 points)
c) 1440x900 pixels @1x (1440x900 points), things are visually the same size as b but the game is rendering 1/4 the pixels.

If the only value you can see in the SDL_DisplayMode is points, i.e. you're given the scale factor and pixel size multiplied together, you can't distinguish between b) and c), which are very different display modes.

(On Windows, last I checked, there is no notion of scale factors for display modes other than the desktop one, and changing the display mode to something other than the what the Windows desktop is configured to use resets the scale factor to 1x (96DPI))

• Lastly there's SDL_GetDisplayDPI. There's the issue where 1x scaling is called 96 DPI on Windows, whereas 1x is called 72DPI on macOS (+wayland?). We could always rescale the Windows values by 72/96 and just document that in SDL 1x = 72DPI on all platforms. Also I remember something about one of the x/y/diagonal DPI components reported by SDL are meant to be the actual physical "pixels / inches" measurement, and others are the scale factor * 72DPI.

On 2019-10-21 14:46:46 +0000, Alex Szpakowski wrote:

I agree with pretty much everything you said. :)

(In reply to Eric Wasylishen from comment # 18)

• It's unfortunate the SDL mouse events are ints. So if these are in
  DPI-scaled points on macOS/iOS/Wayland, you'd get mouse movement in 2px
  steps if you have a 2x scale factor.

Ah, thanks for bringing this up, I forgot about that. I definitely think non-integer mouse coordinate APIs are needed in the future, if we continue with DPI-scaled units.

(In reply to Eric Wasylishen from comment # 18)

• SDL should report relative mouse movement (xrel/yrel) as raw USB data with
  no scaling, not points, right?

That's an interesting question.. intuitively I'd say it should be points to be consistent with other mouse movements, but I'm not too familiar with developer and user expectations around relative mouse mode.

[DanielGibson]

As far as I can tell, SDL2 on Windows still has no HighDPI support - does anyone (@ericwa or @slouken ?) know what happened to this?

Is there any chance projects using SDL will ever be able to get rid of their custom workarounds?

[ericwa]

I'll take a look at dusting off my PR this weekend, bringing it up to date with current sdl2 etc.

[DanielGibson]

Awesome, thanks a lot!

[PJB3005]

@ericwa Have any advancements been made on this front yet? I'm willing to try dusting the patch off too if necessary.

[ericwa]

@PJB3005 Sorry for the delay, here's the current status:

My previous work is in a branch called windows-highdpi2, I'm keeping notes on its status in a PR in my SDL repo: ericwa#1

Overall, it's working OK, I think I've ironed out most of the issues. My plan for a while has been to split it into 2 separate patches which I'm hoping to make PR's for soon:

1. the first PR will introduce a new SDL hint (tentatively calling it SDL_HINT_WINDOWS_DPI_AWARENESS) which makes SDL call the Windows API's to declare the process DPI aware. Alongside this, I'll make SDL work properly if the process is "per-monitor V2" DPI aware - which could either be done through that hint, or in external code. This should fix Windows bug: window size continues to increase or shrink across monitors of different magnifications #4712 (which I think is due to external code enabling "per-monitor V2" DPI awareness while SDL doesn't (yet) support it).

   Current status of this patch is here: https://github.com/ericwa/SDL/commits/windows-dpi-awareness2

   This is just about ready to make a PR for, just needs a last bit of cleanup.

   For SDL apps that just want "1 SDL unit = 1 pixel", this hint would be all they need.

2. the second PR will be the rest of Windows highdpi2 ericwa/SDL#1 rebased onto the above patch. This will be another SDL hint which change the SDL coordinate system to DPI-scaled points (matching SDL's highdpi behaviour on macOS/Wayland). This one requires more invasive changes to SDL, but basically all of that work is done in Windows highdpi2 ericwa/SDL#1 and I'm hoping splitting it from the first patch (SDL_HINT_WINDOWS_DPI_AWARENESS) will make it more manageable.