Fix/document emscripten_set_fullscreenchange_callback behaviour

[floooh]

The current behaviour of the elementWidth and elementHeight members in EmscriptenFullscreenChangeEvent (html5.h) produces strange values and differs between browsers (see discussion thread here: https://groups.google.com/forum/#!topic/emscripten-discuss/mc3T1SYycdw)

The documentation for these elements is currently:

// The new pixel size of the element that changed fullscreen status.
int elementWidth;
int elementHeight;

It would be at least a good idea to state there that this currently doesn't work as expected in some/all browsers, and the screenWidth/screenHeight members should be used if the actual display resolution of fullscreen is required.

From the view of a game engine I would expect that switchting to fullscreen has the effect that rendering happens in a decoration-less fullscreen window at the system's native fullscreen resolution covering the entire display (although the engine should be able to intercept and set the canvas resolution to (for instance) half the native display resolution), in this case I would expect that the canvas is stretched/upscaled to cover the entire screen.

As a workaround, I'm currently using the screenWidth/screenHeight members for switching to fullscreen, and store the previous canvas size when switching back from fullscreen, for example: https://github.com/floooh/oryol/blob/f19b1e4a943618e6028dfcbdf14000df61ed77cb/code/Modules/Render/egl/eglDisplayMgr.cc#L193

Live demos with console messages when switching to/from fullscreen:

http://floooh.github.io/oryol/

[juj]

I've been working on this now to debug the behavior in different browsers. Here's my findings:

• When an element is transitioned to fullscreen mode in Firefox, the browser stretches the CSS size of that element to cover the whole screen. IE11 does the same.
• When an element is transitioned to fullscreen mode in Chrome, the browser doesn't change the CSS size of the element at all, but simply displays it centered in the original size.

As a result, in Firefox and IE11, the browser sends the DOM 'resize' events along with the 'fullscreenchange' events when the transition occurs, and both FF and IE11 even have bugs that they send resize events with the CSS size of the target element being bogus, reported at:

• https://bugzilla.mozilla.org/show_bug.cgi?id=1088196
• https://connect.microsoft.com/IE/feedbackdetail/view/1009458/bogus-js-resize-events-are-fired-when-transitioning-a-canvas-between-fullscreen-modes

To unify the behavior between browsers, it's necessary to manually resize the CSS size of the element to cover the full screen, and use a padding style to implement aspect ratio control. Different projects have different needs: some are able to render to only a single fixed resolution, others are capable of rendering to an arbitrary native resolution, and some may or may not require a certain aspect ratio. Some projects may already be deeply integrated to the web page using their own means. Because of that, my current thinking is to implement a companion API to the existing emscripten_request_fullscreen, which allows specifying a preset or a strategy to follow. The ones that I can think of currently are

• none: the current behavior which doesn't adjust the DOM elements in any way, and just requests fullscreen
• native+stretch: resize the WebGL canvas to cover the whole screen, and resize WebGL render target to the same resolution.
• native+stretch+hidpi: Same as above, but render to high-dpi resolution instead of CSS resolution.
• native+aspect: resize the WebGL canvas to cover the whole screen and resize the render resolution as well, but retain aspect ratio that the canvas currently has and add black letterboxes.
• native+aspect+hidpi: Same as above, but render to high-dpi resolution instead of CSS resolution.
• fixed+stretch: keep the current rendering resolution, but stretch the canvas to cover the whole screen, without regards to aspect ratio. (current Firefox default behavior)
• fixed+aspect: keep the current rendering resolution and scale it to cover the full screen, retaining aspect ratio and adding letter boxes.
• center: Move the canvas to the center of the screen completely unresized, add black borders (current Chrome default behavior)

I have a test page which implements those already, and they look like a nice fit and a set of default strategies that should help people quickly get the behavior working. In general I prefer that the html5 api is a very "bare" api which just routes the existing browser apis to the user, but here it looks like the per-browser behavior is too different that too many developers need to independently research the per-browser differences, so having a (opt-in) default scheme sounds like an useful thing. Does that sound like a reasonable solution to you @floooh ?

[floooh]

That sounds very convenient, but see below for an 'advanced mode' idea :) I agree that the html5 api has to find the right balance in some cases between 'unfiltered' access to underlying browser APIs, but at the same time try to clean up different browser behaviours. It's a surprisingly complex topic...

I'm a bit concerned about 'upscale artefacts' when stretching with uneven scaling factors, and the fillrate/resolution ratio differences between mobile and desktop devices... on desktop, a 1:1 ratio is usually good, while on mobile upscaling may be needed. For convenience and quick results your solution looks very good, but I think better control over the WebGL render target size in relation to the actual fullscreen resolution would be a good thing...

But as far as I see there's no way to query the native display resolution in the HTML5 fullscreen API before actually switching to fullscreen...

What about a 2-step-'advanced'-mode (not sure if that is even possible): after requestFullscreen has switched the HTML page to fullscreen mode, invoke a C callback which is called with the native fullscreen resolution and the 'strategy flags' as input. The callback has to provide (1) the canvas rectangle in 'fullscreen space' where the emscripten canvas should be placed, and (2) the new WebGL render target resolution. Apps with advanced requirements would then have full control over placement, aspect and upscaling ratios by overriding the default callback (the default callback would implement the fullscreen strategies as you described above).

As I said, not sure if this would even work with the HTML5 API.

I actually have a little sample game which fits the 'keep aspect ratio' requirement:

http://floooh.github.io/oryol/Paclone.html

Previously this was stretching to fullscreen mode which of course isn't right. I fixed this with glViewport() to define the rendering area when running in fullscreen mode, this is portable between platforms, but makes gl_FragCoord useless in fullscreen-quad shaders.

[juj]

I think for advanced uses the best way is to manually set the WebGL render target size and CSS size, and then call emscripten_request_fullscreen(); without specifying a strategy. This should serve all advanced cases, since it gives full control over everything.

To compute the native display resolution in fullscreen before switching to fullscreen, one can do

int fullscreenWidth = (int)(emscripten_get_device_pixel_ratio() * EM_ASM("screen.width") + 0.5);
int fullscreenHeight = (int)(emscripten_get_device_pixel_ratio() * EM_ASM("screen.height") + 0.5);

The rounding there is just to avoid floating point errors when one is an epsilon away of the correct integer, and not to round 0.25 or 0.5 fractions. This is guaranteed to work if devicePixelRatio is an integer. If devicePixelRatio is a fraction, there does not exist a way to get the full screen size before one first transitions to full screen and then uses canvas.getBoundingClientRect() to measure it. This case can occur on mobile devices. The problem is that both screen.width and screen.height are specified to return integers, even if the screen size was a fraction when measured in CSS pixels. I've previously called that out as a browser bug, however there's apparently no interest to change this behavior in the fear of breaking existing pages.

One should never cache the values of screen.width and screen.height, because those are overloaded to return the size of the monitor that the browser is currently in. In dual-monitor setup the values returned by screen.width and screen.height change when the browser window is hovered over to another monitor.

[juj]

I've now got a demo up at https://dl.dropboxusercontent.com/u/40949268/emcc/bugs/fullscreen_canvas.html . That shows the different usage scenarios I can think of being useful or interesting. Tested to work on current Firefox, Chrome, IE11 and Safari 8. I'm thinking of adding a new function emscripten_request_fullscreen_preset(enum preset) which allows requesting a fullscreen mode in one of the strategies enumerated in that page. Certain functionality (autohide elements, esc keydown handler) in the soft mode can be viewed intrusive in some usages, so those would be only optionally enabled. Also cc @azakai , how does this look to you?

[floooh]

Ah ok, thanks for the info re full control over the render target and CSS size, very helpful. Your demo and the implemented fullscreen scenarios look good to me :) I tested on a mid-2012 MBP with OSX 10.10 in public and canary Chrome, and Firefox Nightly and Safari, and on Windows7: current Chrome Canary, IE11 and FF Nightly.

[juj]

Thanks for testing! I wrote a braindump of the differences I can recall while writing that test page to https://www.w3.org/Bugs/Public/show_bug.cgi?id=27162 , perhaps there's a chance to clean up some of these observed differences in the spec itself so that this might become easier in the future.

[kripken]

Looks nice and comprehensive.

[juj]

Thanks @kripken. I'll follow up with pull requests once I've got the same in the browser environment.

With the help of Boris Zbarsky, I've reported bugs of wrong behavior to different bug trackers:

Chrome:

• https://code.google.com/p/chromium/issues/detail?id=427179: Fullscreen mode is rendered with incorrect scaling.
• https://code.google.com/p/chromium/issues/detail?id=427181: Element margin is incorrectly applied to the content displayed in fullscreen mode.
• https://code.google.com/p/chromium/issues/detail?id=427185: Element padding is incorrectly applied to content displayed in fullscreen mode.
• https://code.google.com/p/chromium/issues/detail?id=427186: Default padding color is incorrect for an element displayed in fullscreen mode.
• https://code.google.com/p/chromium/issues/detail?id=427189: Transitioning between displaying an element in fullscreen mode displays flickering artifacts during the transition.

IE11:

• https://connect.microsoft.com/IE/feedback/details/1009458/bogus-js-resize-events-are-fired-when-transitioning-a-canvas-between-fullscreen-modes
• https://connect.microsoft.com/IE/feedbackdetail/view/1010959/element-margin-is-incorrectly-applied-to-the-content-displayed-in-fullscreen-mode
• https://connect.microsoft.com/IE/feedbackdetail/view/1010979/element-padding-is-incorrectly-applied-to-the-content-displayed-in-fullscreen-mode
• https://connect.microsoft.com/IE/feedbackdetail/view/1011000/setting-document-body-style-backgroundcolor-does-not-always-update
• https://connect.microsoft.com/IE/feedbackdetail/view/1011008/transitioning-between-displaying-an-element-in-fullscreen-mode-displays-flickering-artifacts-during-the-transition
• https://connect.microsoft.com/IE/feedbackdetail/view/1009458/bogus-js-resize-events-are-fired-when-transitioning-a-canvas-between-fullscreen-modes

Firefox:

• https://bugzilla.mozilla.org/show_bug.cgi?id=1089147: Transitioning between displaying an element in fullscreen mode displays flickering artifacts during the transition.
• https://bugzilla.mozilla.org/show_bug.cgi?id=1088196: Bogus JS 'resize' events are fired when transitioning a canvas between fullscreen modes.

Safari issues still need reporting, which I'll do later once I'm back to my OSX box.

[kripken]

wow, nice work!

[juj]

After the above pull request was merged, the html5.h api for fullscreen management should now be ok., so closing this.