Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

WebGL Renderer #1790

Merged
merged 111 commits into from Jun 23, 2019
Merged

WebGL Renderer #1790

merged 111 commits into from Jun 23, 2019

Conversation

@Tyriar
Copy link
Member

Tyriar commented Nov 20, 2018

This PR introduces the 'webgl' renderer which is basically canvas but much lower level and much faster. I see this eventually replacing the canvas renderer but it will need some time before that happens.

But there's already a canvas renderer?

To illustrate the difference when drawing, the canvas renderer basically does this:

  1. Tell the GPU to draw character 0,0 from texture a
  2. Tell the GPU to draw character 1,0 from texture a
  3. Tell the GPU to draw character 2,0 from texture a
  4. and so on.

These individual instructions are costly, the webgl renderer instead builds a Float32Array containing all the data needed to draw and a "webgl program" (a vertex shader and a fragment shader) that knows how to draw things from the Float32Array is uploaded to the GPU which does the actual drawing. Of course the details are much more involved than this, but that's the basics of why the webgl renderer will be superior to the canvas renderer.

Features

  • It's super fast, it also scales much better with really large viewports.
  • All characters are cached in the atlas, including unicode, combined chars (emoji), etc.
  • The atlas trims glyphs to their minimal rectangles and stores that in the texture, this means better space utilization of the texture. Here's a smaller 256x256 example illustrating this:
    image
  • Selection is now drawn underneath the text (#720), using a uniform selection fg for accessibility reasons:
    screen shot 2018-11-20 at 8 34 18 am
  • Since the buffer is redrawn every frame, characters are no longer trimmed at the top and the bottom (#1138).
    screen shot 2018-11-20 at 8 36 41 am
    screen shot 2018-11-20 at 8 37 00 am
  • Renderer pausing when the canvas is offscreen is supported

Notes

  • WebGL2 is used, this reduces browser compatibility a little but allows using some niceties like vertex array objects and drawing using drawElementsInstanced.
  • Render layers still work, the webgl renderer actually still utilizes the CursorRenderLayer and LinkRenderLayer in order to simplify the rendering logic (and because they're pretty fast anyway).
  • When the texture atlas has reached its capacity, it will currently just clear itself and restart. This could be improved but this route was chosen due to the relatively inexpensive cost of starting over and so we don't need to juggle multiple textures.
  • I experimented with disabling double buffering and drawing the minimal set of changes, it didn't appear to be worth the effort (at least given the time I had to work on this).

Issues

馃洃= merge blocker

  • Currently you cannot customize the selection color. This is because cached glyphs do not currently support true color, this should be trivial to fix once true color is added.
  • 馃洃 Character joiners are not supported yet.
  • 馃洃 allowTransparency is not supported yet.
  • Rendering seems to become offset on my macbook when rows are increased after 300x91, I'm guessing this is related to maximum canvas dimensions?

Comparing to canvas renderer

Summary

This table shows the average time to draw the frame before and after, see below for more details.

Benchmark Canvas/dynamic (avg ms/frame) WebGL (avg ms/frame) Change (c/w-1)
Macbook 87x26 4.80 0.69 596% faster
Macbook 300x80 15.28 3.69 314% faster
Windows 87x26 7.31 0.73 901% faster
Windows 300x80 19.34 2.06 839% faster
Macbook 87x26 CJK 14.63 5.93 147% faster
Macbook 87x26 Emoji 27.47 19.28 42% faster

Methodology

The tests are done by running various commands in the build xterm.js repo directory and the following measuring code is injected to get the report:
(<any>window).frames = [];
(<any>window).report = () => {
  const frames = (<any>window).frames as number[];
  const average = Math.round(frames.reduce((p, c) => p + c, 0) / frames.length * 100) / 100;
  console.log(`...: frames ${frames.length}, average ${average}ms`);
};

Wrapping _renderRows of Renderer.ts (canvas) and WebglRenderer.ts:

const startTime = performance.now();
...
(<any>window).frames.push(performance.now() - startTime);

Parts of the chrome timelines are also presented. Note that the injected code is necessary because it's difficult to measure otherwise as the number of frames and average frame time isn't captured by the timeline.

The Macbook Pro is mid 2014 with an Intel Iris Pro 1536 MB, the Windows box has a GTX 760.

Macbook Viewport 87x26 (demo default), ls -lR .

Canvas Renderer (dynamic atlas)

Canvas: frames 69, average 4.8ms screen shot 2018-11-20 at 8 07 28 am

WebGL Renderer

WebGL: frames 80, average 0.69ms screen shot 2018-11-20 at 8 05 24 am

Macbook Viewport 300x80, ls -lR .

Canvas Renderer (dynamic atlas)

Canvas: frames 30, average 15.28ms screen shot 2018-11-20 at 8 09 28 am

WebGL Renderer

WebGL: frames 43, average 3.69ms screen shot 2018-11-20 at 8 10 27 am

Windows Viewport 87x26 (demo default), tree

Canvas Renderer (dynamic atlas)

Canvas: frames 54, average 7.31ms

image

WebGL Renderer

WebGL: frames 60, average 0.73ms

image

Windows Viewport 300x80, tree

Canvas Renderer (dynamic atlas)

Canvas: frames 59, average 19.34ms

image

WebGL Renderer

WebGL: frames 68, average 2.06ms

image

Macbook Viewport 87x26, cat zh_wcag_100.txt

zh_wcag_100.txt contains this contents of https://www.w3.org/Translations/WCAG20-zh/ 100 times to help test throughput for wide characters.

Canvas Renderer (dynamic atlas)

Canvas: frames 40, average 14.63ms screen shot 2018-11-19 at 5 06 10 pm

WebGL Renderer

WebGL: frames 45, average 5.93ms screen shot 2018-11-20 at 8 13 02 am

Macbook Viewport 87x26, cat emoji_100.txt

emoji_100.txt contains this contents of https://getemoji.com/ times to help test throughput for combined characters.

Canvas Renderer (dynamic atlas)

Canvas: frames 13, average 27.47ms screen shot 2018-11-20 at 8 22 03 am

WebGL Renderer

WebGL: frames 16, average 19.28ms screen shot 2018-11-20 at 8 20 22 am

Fixes #720
Fixes #1138 (microsoft/vscode#35901)
Fixes #1170 (I think)
Obsoletes a bunch of issues related to the canvas renderer: #941, #955, #1389, #1614

@Tyriar Tyriar self-assigned this Nov 20, 2018
@Tyriar Tyriar requested a review from xtermjs/core Nov 20, 2018
@jerch

This comment has been minimized.

Copy link
Member

jerch commented Nov 21, 2018

@Tyriar Have not yet looked at the code but could not resist to do a few tests on my own. Therefore first a few notes about the perf results.

I see about the same perf boost (rendering being 3-5x faster) on Linux with either intel graphics or nvidia GT520M (very old) and GTX 960. With my typical ls -lR /usr/lib benchmark I also hit the system throughput, means the terminal has to wait in between for the pty to deliver more data, and is idle the rest of the time (both systems with SSDs lol). That are really great results!
Your timelines show weak performance for windows systems, really slow incoming data, hmm. This might be worth to be addressed upstream in win-pty/conpty.

Your results also show a big perf decrease on the MacBook for the bigger terminal view. I wonder how fullscreen would perform. Note that I cant resemble this decrease with Linux (even tried 500x100), I see only slightly worse runtime for intel or nvidia gc. Maybe thats a resolution problem (retina?) and could be tweaked further?

What I see now for my benchmark in the demo:

  • total runtime: ~1800 ms (seems this cannot go faster due to system limits)
  • ~ 120 ms render time (was 500 ms before) 馃槏
  • ~ 900 ms for the input chain (parser, buffer inserts, state handling) 馃槖
  • ~ 200 ms JS overhead (function calls, GC etc.)
  • ~ 600 ms idle 馃槏

Last but not least: since many peeps nowadays are on laptops - does anyone know if the gl renderer would drain more power than the canvas renderer? Note that the canvas renderer also relies on hardware acc, still unsure whether this might trigger a more costly power profile.

Tyriar added 2 commits Nov 22, 2018
@Tyriar

This comment has been minimized.

Copy link
Member Author

Tyriar commented Nov 22, 2018

On support, according to https://webglstats.com/ 68% of desktops support WebGL2. I looked into downgrading to WebGL1 (98% support) but there doesn't appear to be an extension for layout. I'm thinking we should push forward with replacing the canvas renderer with the WebGL2 implementation once stable and improve the DOM renderer a little so it has more reasonable performance, that way the majority of consumers will be on the best implementation, we don't muck the code up with WebGL1 messiness (+slowness?) and there is a viable alternative for the rest.

https://caniuse.com/#search=webgl2 says that Chrome, Firefox, Opera support WebGL2.

@jerch

This comment has been minimized.

Copy link
Member

jerch commented Nov 22, 2018

@Tyriar Yeah seems only Safari cant do Webgl2, I am not sure if Safari will ever see this as Apple is pushing WebGPU based on Metal/Vulkan. Webgl1/2 is based on OpenGL which got deprecated on macOS.

Tyriar added 4 commits Nov 22, 2018
@Tyriar Tyriar closed this Nov 24, 2018
@Tyriar Tyriar reopened this Nov 24, 2018
@Tyriar

This comment has been minimized.

Copy link
Member Author

Tyriar commented Nov 24, 2018

Currently experimenting with minimizing the amount of data uploaded to the GPU via bufferData, results looks favorable so far, especially when scrolling.

Tyriar added 2 commits Nov 25, 2018
@Tyriar Tyriar mentioned this pull request Nov 28, 2018
@Tyriar

This comment has been minimized.

Copy link
Member Author

Tyriar commented Jun 17, 2019

This is ready for review/merge, it's still a work in progress as these won't work:

  • Underline attribute
  • Selection color
  • Character joiners
  • allowTransparency
  • Handle context loss when there are too many contexts (see Tyriar#5)
  • True color
  • Move CharData usage over to CellData and remove CharDataCompat.ts
  • WebglAddon.dispose is not implemented

However it's a standalone addon now, the only thing that affects the core is the very small diff outside of the addons/ directory. Once merged in we'll get the build and tests in CI to prevent future breakages and we can create issues to finish off the above issues.

@Tyriar Tyriar added this to the 4.0.0 milestone Jun 17, 2019
Copy link
Member

jerch left a comment

I cannot not really review this monster :P - still I have a few questions/remarks.

@Eugeny

This comment has been minimized.

Copy link

Eugeny commented Jun 17, 2019

Could you please republish the addon with the fixed typings file?

@Tyriar

This comment has been minimized.

Copy link
Member Author

Tyriar commented Jun 17, 2019

@Eugeny done

@Eugeny

This comment has been minimized.

Copy link

Eugeny commented Jun 21, 2019

Looks like emojis are not considered to be double width chars anymore?
image

@Tyriar

This comment has been minimized.

Copy link
Member Author

Tyriar commented Jun 22, 2019

@Eugeny I didn't think that ever worked because we don't detect unicode version we have to guess?

Tyriar added 5 commits Jun 22, 2019
@Tyriar

This comment has been minimized.

Copy link
Member Author

Tyriar commented Jun 23, 2019

Let's do this! 馃殌

I've created a bunch of issues tracking the remaining issues under the addon/webgl label. Community welcome to help finish it off 馃槂. If I missed something please create another issue.

@Tyriar Tyriar merged commit 99a31b3 into xtermjs:master Jun 23, 2019
5 checks passed
5 checks passed
xtermjs.xterm.js Build #20190623.12 had test failures
Details
xtermjs.xterm.js (IntegrationTests) IntegrationTests succeeded
Details
xtermjs.xterm.js (Linux) Linux succeeded
Details
xtermjs.xterm.js (Windows) Windows succeeded
Details
xtermjs.xterm.js (macOS) macOS succeeded
Details
@Tyriar Tyriar deleted the Tyriar:webgl2 branch Nov 9, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
9 participants
You can鈥檛 perform that action at this time.