Idea to improve high DPI handling on OS X

[jqdg]

• Forum posts: 1, 2, 3. Issues and PRs: Support Retina Displays and other High DPI Displays #353, Support high DPI displays on OS X #388
• Does the code follow the SFML Code Style Guide?
• Have you provided some example/test code for your changes?

---

Problem Description

SFML doesn't really have complete support for DPI awareness (it is my understanding that this is planned for SFML version 3).

This can be an issue on MacBooks with retina displays (among other situations). There are two possible cases:

1. NSHighResolutionCapable is set to true, and all windows appear to be half the size in each dimension as expected.
2. NSHighResolutionCapable is set to false in the Info.plist file, and the whole app renders in low resolution.

Both of these cases are highly problematic.

In case 1, by default, all windows and graphics will be drawn far too small to be usable. In order to render at the correct size, the application developer must scale all of the window and graphics sizes by the correct scale. However, SFML does not expose an API to identify if the display is high DPI, and to return the correct scaling. This is a fairly invasive change to the application code, and will require platform-specific code to identify the display scaling.

In case 2, everything draws at the correct size, but the window decorations and fonts are blurry. Furthermore, if one uses a raw executable (rather than application bundle) then it is not possible to set this option using Info.plist.

NSHighResolutionCapable true	NSHighResolutionCapable false

---

Proposed Solution

I believe that the default behavior should be as follows:

• The apparent window sizes should be the same on high DPI displays and low DPI displays
• Window decorations and fonts should be rendered in high resolution
• Graphics should be rendered at the same apparent size on high DPI and low DPI displays (without making SFML DPI-aware, which is a much bigger project, this means that graphics will be slightly pixelated on high DPI displays)

If the user wants to avoid the scaled rendering, and instead put in the extra effort to support high DPI rendering, then they can opt-in by setting a high DPI window style.

This PR accomplishes this by adding a new window style HighDPI (which has an effect only on Mac OS, but this could potentially be extended to other platforms). This style is not set by default. This will cause the window and graphics to render at the same apparent size on high DPI and low DPI displays, while retaining high-resolution window decorations and fonts. This is because if the style is not set, then setWantsBestResolutionOpenGLSurface is set to NO in the SFOpenGLView.

On the other hand, if the application developer wants to avoid possible pixelation from scaling the window, they can set the HighDPI window style, which will call setWantsBestResolutionOpenGLSurface to YES, and also enable scaling of window dimensions by the display factor. The developer will then need to manually scale all of the windows and graphics sizes in their rendering code to render everything at 2x.

Tasks

• Tested on Linux
• Tested on Windows
• Tested on macOS
• Tested on iOS
• Tested on Android

How to test this PR?

#include <SFML/Graphics.hpp>

int main()
{
    sf::Uint32 style = sf::Style::Default;
    // Uncomment the next line to enable high DPI mode:
    // style |= sf::Style::HighDPI;
    const char *title = (style & sf::Style::HighDPI) ? "High DPI" : "Default";
    sf::RenderWindow window(sf::VideoMode(300, 300), title, style);
    window.setFramerateLimit(60);
    sf::CircleShape circle(150, 100);
    while (window.isOpen())
    {
        sf::Event event;
        while (window.pollEvent(event))
        {
            if (event.type == sf::Event::Closed)
                window.close();
        }
        window.clear();
        window.draw(circle);
        window.display();
    }
}

Default mode	High DPI mode	Low DPI mode (previous)

From the above screenshots, we can see that the "default mode" preserves the advantages of setting NSHighResolutionCapable to be false (everything renders at the correct size), while avoiding the blurry window decorations and fonts. Additionally, the application developer still retains the option to render everything in high resolution if desired.

[eXpl0it3r]

Maybe taking one step back again, I think there are essentially two issues here:

1. The drawing canvas/buffer needs to be able to differentiate/scale between the full DPI (high DPI) and the configured DPI scaling
2. The window styling needs to either handle high DPI and scaled DPI, but not just by rescaling a low-res version

I'm not as deep into macOS nor do I have a retina display to fully understand the defaults and possible options.
Do you think my assumption is correct, when I believe that if you have enabled DPI scaling, you'd expect every application to uphold that scaling? Or at least for the window styling?
With that in mind, I'd say that we shouldn't provide a window style for high DPI.

For the buffer/canvas itself, the idea was to provide an API which gives not just access to the window size, but also the non-scaled canvas size. That way you should be able to calculate the wanted difference yourself and if wanted render at "high DPI" while the window style itself is at scaled DPI.

From my limited understanding, I'd suggest the following changes:

• Always respect DPI scaling for the window styling
• As long as we don't have a way to provide the non-scaled DPI resolution, I'd force the canvas size to match the DPI scaling

Do you think this makes sense?

[jqdg]

Thanks for the feedback!

1. The drawing canvas/buffer needs to be able to differentiate/scale between the full DPI (high DPI) and the configured DPI scaling
2. The window styling needs to either handle high DPI and scaled DPI, but not just by rescaling a low-res version

I'm not as deep into macOS nor do I have a retina display to fully understand the defaults and possible options.
Do you think my assumption is correct, when I believe that if you have enabled DPI scaling, you'd expect every application to uphold that scaling? Or at least for the window styling?

Agreed. In particular, every Mac application should display high-resolution window decorations and fonts on a retina display. GUI applications with native controls should display high-resolution UI elements. Some legacy application (i.e. those built before high DPI support was widespread) do not do this, and instead have pixelated/scaled low-res UI elements (like in some of the screenshots I posted). These applications generally stick out like a sore thumb (Mac users tend to notice details like this), and this situation should be avoided if possible. That is the reason that the current solution/workaround of setting NSHighResolutionCapable to false is not really satisfactory.

The problem with the current SFML functionality is that if you use high-res window decorations, you get a tiny window size. If you want a full-sized window (without manually scaling everything in your rendering code, keeping in mind that the display scale is not accessible with an SFML API), then the window decorations and fonts will be low-res.

What this PR does is make the default mode for windows have high-resolution decorations and fonts, but keep the scaled window size, and simply scale the contents of the OpenGL view.

• Always respect DPI scaling for the window styling
• As long as we don't have a way to provide the non-scaled DPI resolution, I'd force the canvas size to match the DPI scaling

Do you think this makes sense?

If I understand you correctly, that is what the new "default mode" of this PR does.

Now, one potential issue with this approach is that graphics within the OpenGL view will not be rendered at high-resolution. That is the reason I introduced the HighDPI window style: if the application developer wants to implement themselves high-resolution rendering (for example, getting the display scale factor from platform-specific APIs), then they can choose to do so by enabling this window style. In other words, the extra work of supporting high DPI rendering becomes opt-in.

If you believe that this window style adds extra complexity for little benefit, then I am completely fine getting rid of it. The consequence of that would mean that rendering high DPI graphics within the OpenGL view would not be possible.

[yuupsup]

I currently don't know anything about high DPI, but I'm willing to do research into whatever is needed. This issue has been the reason I have not been able to dive into SFML on my Macbook Pro. 😭

[jqdg]

I currently don't know anything about high DPI, but I'm willing to do research into whatever is needed. This issue has been the reason I have not been able to dive into SFML on my Macbook Pro. 😭

Hi @yuupsup, would you mind giving this PR a try and let me know if it works according to your needs? Thanks!

[yuupsup]

@jqdg Sorry, I never got a notification that you responded. PR means "pull request", correct? Do you mind working with me on how I can try the request? How do I pull the changes you've made?
Also, I assume I'll need to build the code again, correct?

[Bromeon]

@yuupsup Yes, "PR" means "pull request".

You can checkout this request locally by doing this:

# Clone SFML (you can skip if you have done this already)
git clone https://github.com/SFML/SFML.git
cd SFML

# Check out branch of this PR locally
git fetch origin pull/1817/head:high-dpi
git checkout high-dpi

In the future, you can overwrite your local branch with newest PR changes like this:

git fetch origin pull/1817/head:high-dpi -uf

Alternatively, directly track the remote branch in the original repo: https://github.com/jqdg/SFML/tree/osx-high-dpi.

[yuupsup]

@Bromeon Thanks I was able to get it. :)

@jqdg I'm amazed haha, it worked. Using the default (the only one I have currently tested) style is displaying the correct window and image size.

Edit
I hope this message will be seen, but I'm looking at the list of expected checks in order for the pull request to get accepted.. these are just all the other machines (and compilers) that need to be tested, correct?

[jqdg]

@Bromeon Thanks I was able to get it. :)

@jqdg I'm amazed haha, it worked. Using the default (the only one I have currently tested) style is displaying the correct window and image size.

Thanks for checking! Glad it looks like it's working for you. I suspect that the default style should work well in almost all cases (I only introduced the HighDPI style so that people who want to manually implement high-DPI rendering themselves will still have the option to opt-in should they want to).

Edit
I hope this message will be seen, but I'm looking at the list of expected checks in order for the pull request to get accepted.. these are just all the other machines (and compilers) that need to be tested, correct?

There are some more CI checks that are needed. I think the maintainers would also need to decide that they want to adopt this design. I would definitely like to make whatever modifications are needed to get this PR accepted. It would be great to make working with high DPI displays easier on Mac OS.

[eXpl0it3r]

Now, one potential issue with this approach is that graphics within the OpenGL view will not be rendered at high-resolution. That is the reason I introduced the HighDPI window style: if the application developer wants to implement themselves high-resolution rendering (for example, getting the display scale factor from platform-specific APIs), then they can choose to do so by enabling this window style. In other words, the extra work of supporting high DPI rendering becomes opt-in.

But isn't this also possible by adjusting the view after the fact? Say you have a 150% DPI scaling, if you as a library user somehow retrieve this scaling information, you could just zoom out the view by a factor of 1/1.5, no?

I still believe the HighDPI as a style is not the way to go, but stick to DPS scaling as default and later down the line provide a function that returns the "unscaled" canvas size, so the user can get that factor on their own.

[jqdg]

But isn't this also possible by adjusting the view after the fact? Say you have a 150% DPI scaling, if you as a library user somehow retrieve this scaling information, you could just zoom out the view by a factor of 1/1.5, no?

No, I don't think so. If the high-DPI mode is not enabled, the entire OpenGL view will be rendered at 2x. If you use a view to zoom out, you can make sure that everything is rendered at the same apparent size as it would with high-DPI enabled, but it will still be pixelated. Compare the following:

HighDPI style

Default style with scaling

I still believe the HighDPI as a style is not the way to go, but stick to DPS scaling as default and later down the line provide a function that returns the "unscaled" canvas size, so the user can get that factor on their own.

I would also be happy to just remove the HighDPI style from this PR, in which case there would be no (easy) option for users to render with high DPI.

I think the default behavior covers 90+% of the use-cases, I imagine most users of SFML are not handling high-DPI rendering themselves. In which case, the new default behavior introduced in this PR is a big improvement compared with the status quo, which is a big stumbling block for those wanting to use SFML on Macs with retina displays.

[jqdg]

I still believe the HighDPI as a style is not the way to go, but stick to DPS scaling as default and later down the line provide a function that returns the "unscaled" canvas size, so the user can get that factor on their own.

@eXpl0it3r, what do you think about the option to just remove the HighDPI style and always render scaled? Or is there some other API that would be better to expose this option in your opinion?

Perhaps as a first step, we could do the first (render everything scaled), and then separately (in a different PR) consider different options for public APIs to enable high DPI rendering?

[eXpl0it3r]

I think it's the way to go, unfortunately, I'm somehow having a hard time to fully understand what you answered to my question (it's me not you 😅).

I mean the "Default style with scaling" looks kind of bad, but it's also not really representative of how it actually looks, since nobody would scale that much, correct?

[jqdg]

I think it's the way to go, unfortunately, I'm somehow having a hard time to fully understand what you answered to my question (it's me not you 😅).

I mean the "Default style with scaling" looks kind of bad, but it's also not really representative of how it actually looks, since nobody would scale that much, correct?

Yes, you're exactly right. I zoomed in the previous screenshots just to emphasize the differences. Normally, the difference would be subtle:

High DPI	Scaled

The point I was making is that the "scaled" approach will not be able to reproduce the "high DPI" approach pixel-for-pixel. So if a user wants to properly implement high DPI rendering, and if they care about these subtle differences, then there needs to be some way for them to enable high DPI mode for the OpenGL view.

On the other hand, I believe that the "scaled" rendering is almost always good enough (and is certainly better than having all the graphics show up too small), and so I think that changing this behavior would be an improvement, even if (for now) there is not an easy way to enable true high DPI mode.

[eXpl0it3r]

Okay, then my assumptions were correct so far.

But then doesn't my point still hold true, that you could expand/zoom out in the scaled view and render things at a higher resolution?
For example if I have an texture that is draw scaled as 100x100px and now I zoom out the view by a factor of 0.5f and then render a higher resolution texture of 200x200px, which visually will be shown as the previous 100x100px. Will the end result not be of higher quality? Or does the scaled version prevent any usage of "more pixels"?

[jqdg]

But then doesn't my point still hold true, that you could expand/zoom out in the scaled view and render things at a higher resolution? For example if I have an texture that is draw scaled as 100x100px and now I zoom out the view by a factor of 0.5f and then render a higher resolution texture of 200x200px, which visually will be shown as the previous 100x100px. Will the end result not be of higher quality? Or does the scaled version prevent any usage of "more pixels"?

Right, the scaled version prevents the use of more pixels. In the scaled version, every pixel rendered to the OpenGL view will be blown up to become a 2x2 block of pixels.

That is exactly what is happening in the two screenshots that I posted previously. The SFML logo (included in the repo at examples/tennis/resources/sfml_logo.png) is 373×113 pixels. On a 2x retina display, displaying this image without any scaling, it will appear to be the same size as a 186x56 pixel image on a non-high DPI display (just with higher pixel density). In high DPI mode, if I draw this image without scaling, I indeed obtain an image that is 186x56 points (i.e. half size when compared with non-high DPI), where each point consists of 2x2 pixels. This is expected. The result is the "high DPI" screenshot from the previous post.

Now, I try to replicate this behavior using the scaling, without high DPI mode. I use the new default window style from this PR, and create a half-sized window, and render the texture and 0.5 scale. What happens is that the texture is down-sampled to 186x56 pixels, and then drawn to the OpenGL view, which (as I mentioned above) will draw each pixel of the down-sampled texture as a block of 2x2 pixels. So I get an image that's the same size as the high DPI case, but pixelated.

There is no way (that I am aware of) to obtain the same result as the "high DPI" screenshot using only the scaled mode.

[eXpl0it3r]

Okay, thank you very much to have all this patience with my stupid questions, just wanted to make sure I understand it fully.

I am still for removing the HighDPI style and use the default DPI scaling. As a user, you enabled DPI scaling, so you should expect to have your applications scaled.

Have support of High DPI will look like API wise, should be discussed at a later stage.

Any objections to this, @SFML/sfml team?

[jqdg]

I got rid of the proposed HighDPI window style, so now the window will always render scaled. Let me know what you think.

[eXpl0it3r]

FYI: I've squashed the commits, so it looks a bit nicer in the git history.

Thanks a lot for this suggestion! I hope, we'll get some more positive feedback as it makes it to various macOS users! 🙂

[eXpl0it3r]

First success story: TankOs/SFGUI#88 🙂

Btw. do you want to continue the conversation on how to allow the user to decide whether they want to use high-dpi on the forum thread?

[williamhCode]

I have a proposal, is it possible to make the HighDPI flag make everything render in high resolution, but keep the window size and coordinates the same? This is the default behavior in most other libraries such as SDL, and I'm confused as to why nobody pointed this out.

[yuupsup]

I have a proposal, is it possible to make the HighDPI flag make everything render in high resolution, but keep the window size and coordinates the same? This is the default behavior in most other libraries such as SDL, and I'm confused as to why nobody pointed this out.

I've been looking for this issue haha. I'm glad to see everything turned out well.
Sorry to revive this thread, but is the request @williamhCode mentioned possible?

Thanks again @jqdg for initially creating this pull request.