Scaling on Retina displays

[tanis2000]

I'm trying to understand what's happening with scaling and Retina displays (but not just them, even resizing windows is having some issues).

I noticed that you're assigning the scale coming from SDL (I double checked that value and it's correct) to the window.scale property, but then that property isn't used by anything. Where am I wrong?

https://github.com/openfl/lime/blob/master/lime/_backend/native/NativeRenderer.hx#L48

[tanis2000]

I think I fixed this with the following PRs:
#609
openfl/openfl#873

[jgranick]

Ouch, I had an improvement in OpenFL before, but this appears to have not been committed somehow?!

We need to apply the window.scale to incoming stage size, mouse event coordinates, etc, on the OpenFL side. Otherwise, tweaking the sizes in Lime still results in all our mouse events (etc) being wrong. Let me take a look

[tanis2000]

I already added it to stage size and resize I think .. I don't know about mouse events as they seems to work but that might be because I'm tapping in the correct half of the screen :)

[jgranick]

Okay, could you take a look at that change? I believe this is what I had working and tested before on iOS.

I'll try and take another look here, but it helps to get more feedback from others as well. Are you testing on a retina Mac laptop/desktop, or a device?

[larsiusprime]

Here's an example from prime31 on gitter chat (uses latest changes):
https://vid.me/e/suFy

[tanis2000]

I've been testing on iOS devices. I have a Retina Macbook Pro as well where I can try this out. I'll have a look at it tomorrow if I can find some time during the weekend. I'll keep you posted.

[mrcdk]

@larsiusprime this may be related https://github.com/openfl/openfl/pull/828/files#diff-850c104eddc0405ea6d18a7b67318b35R1039 The change wasn't merged.

[tanis2000]

@mrcdk I tried but it looks it's not directly related to this. At least it's not part of the Retina problem.. might be for the resize, though. I'll have another look at it after I get Retina working right.

On Mac and iOS the stage and window size at startup is being set to exactly 2x what it should be (so the window goes off the screen and the stage is scaled 2x more than what it should be).
The problem is that it happens with just Lime as well, so it's something that has to be fixed on the Lime side.

@jgranick can you find me a sample using only Lime that draws a colored rectangle that fills the whole screen, with a line all along the border? I fear I'm messing up with matrices and stuff when I try on my own and eventually the window might be bigger than it should be and I can't see it as the SampleImage only draws a bitmap that is smaller than the screen.

[tanis2000]

Here's what I understand of what's going on between SDL and Retina displays.

On iOS the size of the window is exactly the size in points (not pixels) and the scale property that we calculate is the multiplier to obtain the size of the screen in pixels.

On OSX the size of the window is the size in pixels. The scale is being calculated the same way, so it ends up having a window that is double the size that it should actually be.
And of course all of the calculations involving the scale are wrong.

I don't know what's the SDL best practice, but we should either differentiate between iOS and Mac or find out how to get both the scale and window size with homogeneous values.
Any suggestions?

[tanis2000]

Reading around the web, it looks like SDL is working this way:

Window size, window position, display mode size, and mouse position are all in �points� (as iOS and OS X defines them) rather than pixels. If the mouse position isn�t in that coordinate space then there�s a bug somewhere. 
SDL�s touch coordinates are always in the range of [0, 1]. On touch screen devices that�s relative to the SDL window�s position. 

Here�s some more explanations: 
https://developer.apple.com/library/mac/documentation/GraphicsAnimation/Conceptual/HighResolutionOSX/Introduction/Introduction.html 
https://developer.apple.com/library/mac/documentation/GraphicsAnimation/Conceptual/HighResolutionOSX/Explained/Explained.html 

The drawable size is in pixels rather than points when high-dpi mode is enabled on a retina display. The idea is to have significantly more pixel density while taking up the exact same amount of area and with content displaying at the same sizes and relative positions as when high-dpi mode isn�t enabled. 

which suggests that the correct way to handle those things is to work in points just like they do.

[jgranick]

I believe that we should be handling this in points now. On iOS, the window size, resize, and mouse coordinates are in points. By reading the scale value, we can extrapolate to pixels.

Does it seem like SDL is doing this wrong on OS X? Or that Apple handles this differently?

[tanis2000]

It looks like it's SDL that's wrong. SDL_GetRendererOutputSize() gives the size in pixels instead of points