Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix/document emscripten_set_fullscreenchange_callback behaviour #2556

Closed
floooh opened this Issue Jul 22, 2014 · 10 comments

Comments

Projects
None yet
3 participants
@floooh
Copy link
Contributor

floooh commented Jul 22, 2014

The current behaviour of the elementWidth and elementHeight members in EmscriptenFullscreenChangeEvent (html5.h) produces strange values and differs between browsers (see discussion thread here: https://groups.google.com/forum/#!topic/emscripten-discuss/mc3T1SYycdw)

The documentation for these elements is currently:

// The new pixel size of the element that changed fullscreen status.
int elementWidth;
int elementHeight;

It would be at least a good idea to state there that this currently doesn't work as expected in some/all browsers, and the screenWidth/screenHeight members should be used if the actual display resolution of fullscreen is required.

From the view of a game engine I would expect that switchting to fullscreen has the effect that rendering happens in a decoration-less fullscreen window at the system's native fullscreen resolution covering the entire display (although the engine should be able to intercept and set the canvas resolution to (for instance) half the native display resolution), in this case I would expect that the canvas is stretched/upscaled to cover the entire screen.

As a workaround, I'm currently using the screenWidth/screenHeight members for switching to fullscreen, and store the previous canvas size when switching back from fullscreen, for example: https://github.com/floooh/oryol/blob/f19b1e4a943618e6028dfcbdf14000df61ed77cb/code/Modules/Render/egl/eglDisplayMgr.cc#L193

Live demos with console messages when switching to/from fullscreen:

http://floooh.github.io/oryol/

@juj juj self-assigned this Jul 28, 2014

@juj juj added the HTML5 API label Jul 28, 2014

@juj

This comment has been minimized.

Copy link
Collaborator

juj commented Oct 23, 2014

I've been working on this now to debug the behavior in different browsers. Here's my findings:

  • When an element is transitioned to fullscreen mode in Firefox, the browser stretches the CSS size of that element to cover the whole screen. IE11 does the same.
  • When an element is transitioned to fullscreen mode in Chrome, the browser doesn't change the CSS size of the element at all, but simply displays it centered in the original size.

As a result, in Firefox and IE11, the browser sends the DOM 'resize' events along with the 'fullscreenchange' events when the transition occurs, and both FF and IE11 even have bugs that they send resize events with the CSS size of the target element being bogus, reported at:

To unify the behavior between browsers, it's necessary to manually resize the CSS size of the element to cover the full screen, and use a padding style to implement aspect ratio control. Different projects have different needs: some are able to render to only a single fixed resolution, others are capable of rendering to an arbitrary native resolution, and some may or may not require a certain aspect ratio. Some projects may already be deeply integrated to the web page using their own means. Because of that, my current thinking is to implement a companion API to the existing emscripten_request_fullscreen, which allows specifying a preset or a strategy to follow. The ones that I can think of currently are

  • none: the current behavior which doesn't adjust the DOM elements in any way, and just requests fullscreen
  • native+stretch: resize the WebGL canvas to cover the whole screen, and resize WebGL render target to the same resolution.
  • native+stretch+hidpi: Same as above, but render to high-dpi resolution instead of CSS resolution.
  • native+aspect: resize the WebGL canvas to cover the whole screen and resize the render resolution as well, but retain aspect ratio that the canvas currently has and add black letterboxes.
  • native+aspect+hidpi: Same as above, but render to high-dpi resolution instead of CSS resolution.
  • fixed+stretch: keep the current rendering resolution, but stretch the canvas to cover the whole screen, without regards to aspect ratio. (current Firefox default behavior)
  • fixed+aspect: keep the current rendering resolution and scale it to cover the full screen, retaining aspect ratio and adding letter boxes.
  • center: Move the canvas to the center of the screen completely unresized, add black borders (current Chrome default behavior)

I have a test page which implements those already, and they look like a nice fit and a set of default strategies that should help people quickly get the behavior working. In general I prefer that the html5 api is a very "bare" api which just routes the existing browser apis to the user, but here it looks like the per-browser behavior is too different that too many developers need to independently research the per-browser differences, so having a (opt-in) default scheme sounds like an useful thing. Does that sound like a reasonable solution to you @floooh ?

@floooh

This comment has been minimized.

Copy link
Contributor Author

floooh commented Oct 24, 2014

That sounds very convenient, but see below for an 'advanced mode' idea :) I agree that the html5 api has to find the right balance in some cases between 'unfiltered' access to underlying browser APIs, but at the same time try to clean up different browser behaviours. It's a surprisingly complex topic...

I'm a bit concerned about 'upscale artefacts' when stretching with uneven scaling factors, and the fillrate/resolution ratio differences between mobile and desktop devices... on desktop, a 1:1 ratio is usually good, while on mobile upscaling may be needed. For convenience and quick results your solution looks very good, but I think better control over the WebGL render target size in relation to the actual fullscreen resolution would be a good thing...

But as far as I see there's no way to query the native display resolution in the HTML5 fullscreen API before actually switching to fullscreen...

What about a 2-step-'advanced'-mode (not sure if that is even possible): after requestFullscreen has switched the HTML page to fullscreen mode, invoke a C callback which is called with the native fullscreen resolution and the 'strategy flags' as input. The callback has to provide (1) the canvas rectangle in 'fullscreen space' where the emscripten canvas should be placed, and (2) the new WebGL render target resolution. Apps with advanced requirements would then have full control over placement, aspect and upscaling ratios by overriding the default callback (the default callback would implement the fullscreen strategies as you described above).

As I said, not sure if this would even work with the HTML5 API.

I actually have a little sample game which fits the 'keep aspect ratio' requirement:

http://floooh.github.io/oryol/Paclone.html

Previously this was stretching to fullscreen mode which of course isn't right. I fixed this with glViewport() to define the rendering area when running in fullscreen mode, this is portable between platforms, but makes gl_FragCoord useless in fullscreen-quad shaders.

@juj

This comment has been minimized.

Copy link
Collaborator

juj commented Oct 24, 2014

I think for advanced uses the best way is to manually set the WebGL render target size and CSS size, and then call emscripten_request_fullscreen(); without specifying a strategy. This should serve all advanced cases, since it gives full control over everything.

To compute the native display resolution in fullscreen before switching to fullscreen, one can do

int fullscreenWidth = (int)(emscripten_get_device_pixel_ratio() * EM_ASM("screen.width") + 0.5);
int fullscreenHeight = (int)(emscripten_get_device_pixel_ratio() * EM_ASM("screen.height") + 0.5);

The rounding there is just to avoid floating point errors when one is an epsilon away of the correct integer, and not to round 0.25 or 0.5 fractions. This is guaranteed to work if devicePixelRatio is an integer. If devicePixelRatio is a fraction, there does not exist a way to get the full screen size before one first transitions to full screen and then uses canvas.getBoundingClientRect() to measure it. This case can occur on mobile devices. The problem is that both screen.width and screen.height are specified to return integers, even if the screen size was a fraction when measured in CSS pixels. I've previously called that out as a browser bug, however there's apparently no interest to change this behavior in the fear of breaking existing pages.

One should never cache the values of screen.width and screen.height, because those are overloaded to return the size of the monitor that the browser is currently in. In dual-monitor setup the values returned by screen.width and screen.height change when the browser window is hovered over to another monitor.

@juj

This comment has been minimized.

Copy link
Collaborator

juj commented Oct 24, 2014

I've now got a demo up at https://dl.dropboxusercontent.com/u/40949268/emcc/bugs/fullscreen_canvas.html . That shows the different usage scenarios I can think of being useful or interesting. Tested to work on current Firefox, Chrome, IE11 and Safari 8. I'm thinking of adding a new function emscripten_request_fullscreen_preset(enum preset) which allows requesting a fullscreen mode in one of the strategies enumerated in that page. Certain functionality (autohide elements, esc keydown handler) in the soft mode can be viewed intrusive in some usages, so those would be only optionally enabled. Also cc @azakai , how does this look to you?

@floooh

This comment has been minimized.

Copy link
Contributor Author

floooh commented Oct 24, 2014

Ah ok, thanks for the info re full control over the render target and CSS size, very helpful. Your demo and the implemented fullscreen scenarios look good to me :) I tested on a mid-2012 MBP with OSX 10.10 in public and canary Chrome, and Firefox Nightly and Safari, and on Windows7: current Chrome Canary, IE11 and FF Nightly.

@juj

This comment has been minimized.

Copy link
Collaborator

juj commented Oct 24, 2014

Thanks for testing! I wrote a braindump of the differences I can recall while writing that test page to https://www.w3.org/Bugs/Public/show_bug.cgi?id=27162 , perhaps there's a chance to clean up some of these observed differences in the spec itself so that this might become easier in the future.

@kripken

This comment has been minimized.

Copy link
Member

kripken commented Oct 25, 2014

Looks nice and comprehensive.

@juj

This comment has been minimized.

Copy link
Collaborator

juj commented Oct 25, 2014

Thanks @kripken. I'll follow up with pull requests once I've got the same in the browser environment.

With the help of Boris Zbarsky, I've reported bugs of wrong behavior to different bug trackers:

Chrome:

IE11:

Firefox:

Safari issues still need reporting, which I'll do later once I'm back to my OSX box.

@kripken

This comment has been minimized.

Copy link
Member

kripken commented Oct 25, 2014

wow, nice work!

@juj juj added the IE Support label Mar 13, 2015

@juj

This comment has been minimized.

Copy link
Collaborator

juj commented Mar 18, 2015

After the above pull request was merged, the html5.h api for fullscreen management should now be ok., so closing this.

@juj juj closed this Mar 18, 2015

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.