Handle HiDPI displays in WebAgg/NbAgg backends

[mdboom]

Fix #5381.

Tested on Firefox/Chrome/Safari on Linux/Mac. Anyone able to do Windows testing?

[QuLogic]

Should there be this hard-coded 2 here, not self.canvas._dpi_ratio?

[mdboom]

Indeed. Good catch.

[mdboom]

Rebased

[WeatherGod]

Does this work correctly if the user changes the DPI (not even really sure if that is possible anyway). The way I see it, the original dpi could never get updated?

[mdboom]

With Chrome, I can't find a way to change the "devicePixelScale" dynamically. It can be set with a commandline argument, but that implies restarting the browser.

With Firefox, it can be set dynamically through about:config:layout.css.devPixelsPerPx. In that case, the plot scales up with pixelation, which is fine. It's theoretically possible to catch that and start regenerating the images at a higher resolution, but that seems like a real corner case given that the user has to visit the "void your warranty" about:config page.

This also performs fine under Ctrl + and Ctrl - scaling, though the plot is pixelated at the higher scales. That might be something worth trying to fix, but it doesn't exactly feel broken as it is, either.

[jenshnielsen]

If I move a window from my external Non high dpi to my internal Retina monitor
window.devicePixelRatio changes from 1 to 2 as expected I guess you would need to regenerate the image if you change monitor. I.e. unplug external monitor

[mdboom]

@jenshnielsen: That's a good point. I only have HiDPI monitors. I'd be curious what happens for you if you're able to test that. My best guess is that if you move from lodpi to hidpi the image will be pixelated. If you move from hidpi to lodpi the image will be downsampled (but appear at a reasonable total size). Neither of which are the end of the world.

I guess we could try to trap the change and adapt on the fly. It seems at least theoretically possible: https://stackoverflow.com/questions/28905420/window-devicepixelratio-change-listener

[jenshnielsen]

If I understand the code correctly it will get the DPI every time a new figure is created?
I think that recreating the figure to get the correct dpi when changing is perfectly fine as long as it doesn't require restarting the notebook or some such thing. Unfortunately I only have a non hiDPI monitor in the office so I will have to wait with testing that till tomorrow.

Otherwise this is really nice 👍 however there seems to be a bug in the conversion from low to high when zooming on a highDPI screen. When drawing a zoom box it seems to get the coordinates in the mixed up and zoom to the corner. I guess there is a multiplication by dpi missing somewhere

[jenshnielsen]

This also manifests it self in that the coordinates in the lower right corner are wrong.
Also when stopping a figure on a highDP screen the static figure is much larger than the original

[mdboom]

I've fixed the issues with the coordinates and the rubberband.

This still has potential issues when moving the browser from a lo to hi-res screen, but I don't have a way of testing that.

[jenshnielsen]

Thanks I will test out the moving when I'm back with my external monitors

[jenshnielsen]

Did some testing.

• When you move a non highdp figure to a high dp display it is non highdp until the figure is recreated.
• When you move a highdp figure to a non highdp display it is slightly more fuzzy until the figure is recreated

I think this is fine and there is no need to do anything more elaborate.

Did you push the fix for rubberband? I dont see the any commits newer than d0b16a1 which is 12 days old

[mdboom]

Sorry -- recent changes pushed now.

[jenshnielsen]

The rubberband and coordinates look right now.

I am still seeing the issue where the figure grows in size when stopped and converted to a static figure. I guess the right dpi is not set for the static figure.

[mdboom]

Ah -- See that DPI change now, too. Let me see what can be done. At least for me, the static figure is scaled up by the browser (i.e. not using the full DPI of the display). We should fix that, but I suspect that might require tweaking something in IPython.

[jenshnielsen]

IPython at least used to have some hidpi support ipython/ipython#3381 not sure where that lives now

[mdboom]

I see -- maybe we can add that flag for the user, since we already know they are retina. (Of course, multiple clients connected to the same kernel will be a problem -- but we have that problem in NbAgg anyway).

[jenshnielsen]

I was playing around with this and it seems to me that we can solve it by doing something like
changing handle_close to:

mpl.figure.prototype.handle_close = function(fig, msg) {
    var width = fig.canvas.width/mpl.ratio
    fig.root.unbind('remove')

    // Update the output cell to use the data from the current canvas.
    fig.push_to_output();
    var dataURL = fig.canvas.toDataURL();
    // Re-enable the keyboard manager in IPython - without this line, in FF,
    // the notebook keyboard shortcuts fail.
    IPython.keyboard_manager.enable()
    $(fig.parent_element).html('<img src="' + dataURL + '" width="' + width + '">');
    fig.close_ws(fig, msg);
}

I.E explicitly setting the width according to the dpi ratio. This obviously needs some more work

[jenshnielsen]

And a similar modification in push_to_output

mpl.figure.prototype.push_to_output = function(remove_interactive) {
    // Turn the data on the canvas into data in the output cell.
    var width = this.canvas.width/mpl.ratio
    var dataURL = this.canvas.toDataURL();
    this.cell_info[1]['text/html'] = '<img src="' + dataURL + '" width="' + width + '">';
}

[jenshnielsen]

I put in a pull request with the changes in mdboom/matplotlib#15

With those changes I think this is ready to go

[jenshnielsen]

Im happy to merge this now. It is tagged agains 2.0 but I'm not sure if we want to backport it?

[jenshnielsen]

Happy to backport if anyone thins we should

[mdboom]

I can see this going either way for backporting. @tacaswell: Wanna be the tie-breaker?

[tacaswell]

👍 to backporting. It isn't a new plotting feature, it is fixing what is arguably a bug in nbagg on retina

[QuLogic]

Backported to v2.x via #6956.