Renderer is blurry when window zoom level is changed

[Tyriar]

#985 (comment)

Right now the canvas size is changed based on window.devicePixelRatio, one idea is to disable this type fo scaling (see how VS Code's minimap doesn't change when zooming) and then scale manually by applying a multiplier to relevant numbers.

Applies to WebGL and canvas renderers.

[yoctozepto]

Just a small comment - this affects HiDPI settting in Windows 10 as well. OTOH, DOM is also affected but looks different (no terminal renderer is able to provide the crispiness of the main editor though :-( ).

EDIT: Forgot I got redirected to xterm.js, I am referring to usage in VSCode as terminal.

EDIT 2: False alarm with DOM, it appears that the scaling is just unfortunate for bold fonts.

[AddisonG]

Getting this with vscode 1.49.2 on Ubuntu 20.04, 1920x1080, with Nvidia GPU + 450.66 drivers. This is with "window.zoomLevel": -1, and no sidebar open (search/explorer).

This is the same row, at the start and at the middle of a line. The middle is always blurry.

[Tyriar]

Some reports in VS Code that it happens without a changes zoom level.

[carlfriedrich]

I have cloned the current master and run the demo like described in the wiki. This is how it looks like with Firefox on Windows 10 with a DPI scaling factor of 125%:

It might seem okay on first sight, but the OOO sections are rendered differently depending on the horizontal position in the terminal. This is an enlargement of the three portions:

You can see that the leftmost characters are rendered way more clearly than the ones farther on the right side.

This happens only with the canvas renderer. Switching to the dom renderer leads to much crisper and consistent visualization:

With the dom renderer, the three OOO sections appear nearly identical:

[carlfriedrich]

Update: the above observations were taken with a Windows scaling factor of 125%. When I set this to 100%, the canvas renderer is consistent as well:

So obviously the problem occurs because the canvas renderer does not handle DPI scaling correctly.

[Tyriar]

the above observations were taken with a Windows scaling factor of 125%.

@carlfriedrich good observation, I guess this is just another case of devicePixelRatio not being 1 or 2, so the same issue as window.zoomLevel not being 0.

[carlfriedrich]

I took some time to dig into this a bit deeper. @Tyriar Please excuse if I'm repeating what you possibly already know. However, I think this might help other people understand why the issue still exists.

General observations

1. The issue appears only when zooming is applied to the terminal.
2. It does not matter where the zooming comes from (can be the Windows DPI setting or the browser/parent window, e.g. VS Code).
3. Zooming is a necessary, but not a sufficient condition, i.e. there are cases where the rendering is crisp despite of an applied zoom factor.

The last point was the reason for why it was so hard to make the issue reproducible. I had to research the basics about how the renderer works, so let's take a short tour about:

The canvas element

Both the Canvas and the WebGL renderer rely on HTML's canvas element. This is an area of a webpage which a script can draw content on. Drawing means that the resulting object is a pixel-based image. Thus both renderers produce a pixel image of the terminal. This technology has been chosen for performance reasons.

Zooming the canvas element: problem and workaround

When a canvas object is zoomed, it produces aliasing effects, due to the nature of pixel graphics. It's just the same as zooming into a photo.

The xterm.js renderer works around this using the following steps:

• Detect whether a zoom is applied to the page.
• If no:
  • Just draw everything on the canvas normally.
• If yes:
  • Increase the canvas size by the zoom factor.
  • Draw everything on the canvas in the zoomed size.
  • Reduce the canvas to its original (non-zoomed) size.
  • The browser then increases the canvas to the zoomed size again, as it applies the zoom factor to the complete webpage.

Let's use the following symbols to describe the behaviour mathematically:

w: canvas width
z: zoom factor

The scaled canvas size w' is then caluclated as:

w' = w * z

So the terminal is painted on w' and then it is visually scaled down, resulting in its original size

w' / z = w

The browser then zooms in:

w * z = w'

i.e. the width that we see is exactly the width that we painted.

Why the workaround does not guarantee a crisp presentation

In theory and in a continouus scale the above workaround works great. However, since a screen has a limited number of pixels, we only have a discrete number of possible values for w and w', which makes us eventually stumble upon rounding errors.

Let's see two examples:

1. No rounding error

   Let's assume a scaling factor of 125% and a canvas width of 100px:

   w' = w * z
w' = 100 * 1.25
   w' = 125

   So the terminal is drawn to a 125px wide canvas. Afterwards it is zoomed down:

   w' / z = w
   125 / 1.25 = 100

   So the canvas is included into the HTML document at a width of 100px. Note that internally the canvas still has its drawn size of 125px, we're just displaying it with a reduced width.

   The browser then scales the whole site:

   w * z = w'
   100 * 1.25 = 125

   So in the end the canvas is displayed in the same size it was drawn to, everyhing looks crisp.

2. Rounding error

   This time, let's assume the same scaling factor of 125% but a canvas width of 95px:

   w' = w * z
w' = 95 * 1.25
   w' = 118.75

   Since the canvas has to have a discrete size, we have to round it:

   ⌊w'⌉ = 119

   In the HTML document, we're styling the canvas with it original width of 95, so:

   w = 95

   The browser, however, does not scale each item individually but the complete webpage in one piece. Thus individual items are not rounded to pixel boundaries, only the complete page is. This makes the resulting width of our canvas after the browser zoom in fact:

   w' = w * z
w' = 95 * 1.25
   w' = 118.75

   Result: We have drawn the terminal to a 119px wide canvas, which is now displayed over the width of 118.75px. So even though we tried to make canvas size and resulting display size identical, in fact they are not.

I came across this by analyzing the canvas element in the DOM, where the canvas size and the display size can be directly seen:

<canvas width="119" height="119" style="width: 95px; height: 95px;"></canvas>

(Disclaimer: This is an artificial example in order to match my mathematical example above. In the real world I wasn't able to produce these exact sizes with xterm.js, but I had several other combinations of canvas width and style width leading to the same result.)

The issue now gets even more clear when we calculate the zoom factor z' between the canvas width and the style width:

z' = w' / ⌊w'⌉
z' = 119 / 95
z' ≈ 1,2526315789 ≠ 1.25

It is obvious that the scaling factor which we actually used to downscale the canvas to its CSS width is not equal to the scaling factor the browser uses to upscale the site.

It's exactly these cases where we're getting the blurry terminal lines. I checked cases without rounding errors (like the 100px and 125% zoom above) and in those cases all characters are displayed equally throughout the whole line.

Possible solution

We could choose the CSS width w in a way that the canvas width w' does not have to be rounded. If it has to be rounded, we have to add some padding around the DOM element in order to reduce its size slightly until w * z is an integer.

This will result in blank spaces around the terminal, the maximum size of which depends on the denominator of the zoom factor z.

Example:
The zoom factor 125% from the above examples can be written as 5/4, so the denominator is 4. The CSS width then has to be integrally divisible by 4 in order to make the canvas width integer as well.

1. w = 100px
   100 is integrally divisible by 4. No adjustments necessary.
2. w = 95px
   95 is not integrally divisible by 4. Next lower number is 92 -> difference d = 3.

So if we reduce our visual representation of the canvas in the latter example from 95px to 92px and add a 3px wide empty space on one side, we would be able to paint on a canvas of 92px * 1.25 = 115px, which is an integer and thus does not lead to a slightly off zoom factor.

Drawbacks of the solution

The difference calculated in the above example is the maximum difference that can occur with this given zoom factor, since the next higher number would be 96, which is again integrally divisible by 4.

Generally speaking:

d max = denom(z) - 1

So with a zoom factor of 125%, the empty border will never be wider than 3px. Personally I would trade off 3px of my monitor for a crisp presentation.

With more sophisticated zoom levels this might get worse, though. If someone sets a zoom level of 101% for example (that would be 101/100 in fractional), we have a denominator of 100, meaning that in the worst case we would add 99px of empty space in order to have a crisp presentation. I'm not sure if this is still a good tradeoff.

Bottom line

A good compromise might be to define a maximum width of empty space we would accept when aiming for a crisp presentation. If we would have to add more than this maximum, then fall back to the whole width with blurry rendering.
Windows has support for only a limited number of zoom levels:

125% (=5/4), 150% (=3/2), 175% (=7/4), 200% (=2/1)

Browsers generally allow any zoom level, but pre-defined values e.g. in Firefox are limited as well:

30% (=3/10), 50% (=1/2), 67% (=2/3), 80% (=4/5), 90% (=9/10), 110% (=11/10), 125% (=5/4), ...

So perhaps we can agree that we would cover the vast majority of use cases by supporting the above listed zoom levels. That would mean that the maximum supported denominator would be 10, resulting in a maximum addition of empty space of 9px.

I would assume that everybody would trade 9px of their monitor space if the alternative is a blurry font. Still the 9px are not lost, you could resize your terminal accordingly so that other parts around it get more space. However, I am not sure about this (there's always someone to complain ;-)), so we might make this a configurable option.

Let me know what you think.

[carlfriedrich]

Addendum: I don't know if it's particularly reproducible in other environments, but in my Firefox running the xterm.js demo with the default settings (font Fira Code, size 15) I get non-rounded values (canvas width: 1320px, display width: 792px) on an overall zoom level of 167% (which is 125% DPI factor and 133% browser zoom). In this setting the font is totally crisp, characters rendered exactly identical throughout the line.
Changing the font type (e.g. to Consolas) or size leads to slightly different canvas and display sizes, resulting in a non-exact zoom factor and therefore a blurry font with characters rendered differently depending on their position.

[Eugeny]

IMO trading a few pixels of padding for clear font rendering is a perfectly reasonable thing to do.

[psqli]

We could choose the CSS width w in a way that the canvas width w' does not have to be rounded. If it has to be rounded, we have to add some padding around the DOM element in order to reduce its size slightly until w * z is an integer.

@carlfriedrich Awesome work! Are you preparing a PR?

[carlfriedrich]

@pasqualirb Thank you. I'm actually not familiar with the code, I just analyzed the problem to have it reproducible. So I'm hoping for some profound opinion from @Tyriar on this topic.

[psqli]

ping @Tyriar

[Tyriar]

Sorry about the delay, I haven't reviewed the proposal thoroughly as I'm a bit swamped after a long break from work, but I agree 100% with @Eugeny if it does indeed fix the problem.

IMO trading a few pixels of padding for clear font rendering is a perfectly reasonable thing to do.

[carlfriedrich]

@Tyriar Thanks for your comment. If you like, I can offer to help with the implementation, but I would need some guidance for the code base first.

[Tyriar]

@carlfriedrich sure!

Each renderer has a method to calculate the dimensions. Here's the canvas renderer:

xterm.js/src/browser/renderer/Renderer.ts

Lines 195 to 206 in cae9f44

	// Recalculate the canvas dimensions; scaled* define the actual number of
	// pixel in the canvas
	this.dimensions.scaledCanvasHeight = this._bufferService.rows * this.dimensions.scaledCellHeight;
	this.dimensions.scaledCanvasWidth = this._bufferService.cols * this.dimensions.scaledCellWidth;
	// The the size of the canvas on the page. It's very important that this
	// rounds to nearest integer and not ceils as browsers often set
	// window.devicePixelRatio as something like 1.100000023841858, when it's
	// actually 1.1. Ceiling causes blurriness as the backing canvas image is 1
	// pixel too large for the canvas element size.
	this.dimensions.canvasHeight = Math.round(this.dimensions.scaledCanvasHeight / window.devicePixelRatio);
	this.dimensions.canvasWidth = Math.round(this.dimensions.scaledCanvasWidth / window.devicePixelRatio);

And webgl renderer:

xterm.js/addons/xterm-addon-webgl/src/WebglRenderer.ts

Lines 456 to 467 in 1e89016

	// Recalculate the canvas dimensions; scaled* define the actual number of
	// pixel in the canvas
	this.dimensions.scaledCanvasHeight = this._terminal.rows * this.dimensions.scaledCellHeight;
	this.dimensions.scaledCanvasWidth = this._terminal.cols * this.dimensions.scaledCellWidth;
	// The the size of the canvas on the page. It's very important that this
	// rounds to nearest integer and not ceils as browsers often set
	// window.devicePixelRatio as something like 1.100000023841858, when it's
	// actually 1.1. Ceiling causes blurriness as the backing canvas image is 1
	// pixel too large for the canvas element size.
	this.dimensions.canvasHeight = Math.round(this.dimensions.scaledCanvasHeight / this._devicePixelRatio);
	this.dimensions.canvasWidth = Math.round(this.dimensions.scaledCanvasWidth / this._devicePixelRatio);

I think those are the main places you would need, let me know if you need any other pointers.

[Tyriar]

@carlfriedrich thanks a lot for the investigation, I spent quite a while adapting your proposal and it turns out it didn't end up working. However, I did find another solution using a fairly recent web API that wasn't around when the renderers were initially created! #3926