Skip to content
Please note that GitHub no longer supports Internet Explorer.

We recommend upgrading to the latest Microsoft Edge, Google Chrome, or Firefox.

Learn more
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Experimental WebGL terminal renderer #84440

Merged
merged 6 commits into from Nov 16, 2019
Merged

Experimental WebGL terminal renderer #84440

merged 6 commits into from Nov 16, 2019

Conversation

@Tyriar
Copy link
Member

Tyriar commented Nov 11, 2019

Fixes #35901
Fixes #47523
Fixes #84949

Font rendering

Notice how in the canvas line many characters get their right-side cut off (eg. ~, m, e, r):

image

Canvas glyphs are much more restricted because drawing them and invalidating stale glyphs is a lot more expensive. This is even worse vertically as all glyphs are clipped to remain within a row, this means some fonts don't render underscores (#35901) and some tall characters can be cut off at the top.

WebGL doesn't have this problem as every character is drawn on every frame. Currently glyphs are drawn to a canvas twice as wide as the cell with some additional vertical and horizontal padding (source) but this can be extended in the future if necessary.

Speed

Since we're leveraging the GPU at a much lower level, rendering in distributed across the GPU's many cores which results in a far lower CPU load, more responsive UI, less skipped frames and faster parsing of program output as a result. Additionally care was taken in assembling the buffers that get uploaded to the GPU to make them very fast, it's pretty much all just dealing with array buffers and numbers.

Compare the following timelines which were recorded when running ls-lR . in the vscode dir (WSL, GTX 760).

Canvas:

image

WebGL:

image

The actual rendering takes significantly less time in WebGL, notice also that the 3.96ms frame is the frame that took the longest to render in the whole timeline, the average is < 1ms.

I did some in depth measurements some time ago so they'll be a little stale, but the order of the improvement should remain around the same:

Benchmark Canvas/dynamic (avg ms/frame) WebGL (avg ms/frame) Change (c/w-1)
Macbook 87x26 4.80 0.69 596% faster
Macbook 300x80 15.28 3.69 314% faster
Windows 87x26 7.31 0.73 901% faster
Windows 300x80 19.34 2.06 839% faster
Macbook 87x26 CJK 14.63 5.93 147% faster
Macbook 87x26 Emoji 27.47 19.28 42% faster

Latency between keystrokes and the graphical response should also be improved, I have not measured this though.

Improved glyph caching

Glyph caching had many drawbacks in the canvas renderer, basically only the standard 16 colors and common ascii. Anything beyond that including any combination of unicode, emoji, 256 color, true color cannot be cached and need to be drawn directly to the canvas every time. This is a particularly big deal for supporting other languages and programs that choose to draw a background color.

Canvas texture atlas after running 256 color test:

image

WebGL texture atlas after running 256 color test:

image

Additionally you will notice that there is a strategy for compressing glyphs with the WebGL renderer instead of all glyphs being the static size of a cell, allowing many more glyphs to be packed into the texture before needing to start evicting them.

Selection drawn underneath text

It's bugged me for some time that both the DOM and canvas renderers draw the selection color on top of the foreground text (#47523), there are a few reasons for this but basically performance is why they don't support this. The WebGL renderer works, additionally we can do cool things like allow a theme to set a static selection foreground color for contrast or even better support a minimum contrast level like iTerm2 (xtermjs/xterm.js#1514) which is a great accessibility feature.

Canvas (notice the blue is different in the selection):

image

WebGL:

image

WebGL has some smarts to blend the selection color into the background color if it's transparent so that existing themes won't break, now that opaque selections work. This also allows using much stronger colors if desirable since you no longer need to worry about high alpha values lowering the contrast of text:

image

Reliability

This can only really be verified after being out in the wild for a while but I expect WebGL to be a much better from a reliability standpoint. The canvas renderer suffers from many different issues such as several Electron/Chromium regressions in drawImage, certain configurations slowing to a crawl when rendering the text.

Less reliance on trusting the browser to do the right thing between VS Code and the GPU should improve reliability, especially since we essentially never want the browser to drop down to CPU-based canvas rendering if GPU acceleration is disabled as the DOM renderer is a far better option than that. My hope is that if webgl2 is said to be supported, it should just work and otherwise hard fail (throw) which would allow us to fallback to the DOM renderer.

Lazy loading/code splitting

One of the biggest blockers that I forced upon myself while working on this was that I didn't want the WebGL renderer to be included in the core of xterm.js, instead we worked on a new addon system that became stable in v4 and the WebGL renderer got moved into an addon. What this means is that if you do not set the renderer type to webgl then the code does not get loaded.

Early testing

Both Hyper and Terminus have been including the WebGL renderer in their products for some time. Hyper has used it as the default renderer since v3 was released back in May, Terminus has it as opt-in until transparency and ligatures work (which vscode doesn't support).

Known issues

https://github.com/xtermjs/xterm.js/issues?q=is%3Aissue+is%3Aopen+label%3Aarea%2Faddon%2Fwebgl

@Tyriar Tyriar added this to the November 2019 milestone Nov 15, 2019
@Tyriar Tyriar merged commit 3ad68c3 into master Nov 16, 2019
18 checks passed
18 checks passed
linux
Details
windows
Details
darwin
Details
VS Code Build #20191115.4 had test failures
Details
VS Code #20191115.86 succeeded
Details
VS Code (Compile) Compile succeeded
Details
VS Code (Linux) Linux succeeded
Details
VS Code (LinuxAlpine) LinuxAlpine succeeded
Details
VS Code (LinuxArm64) LinuxArm64 succeeded
Details
VS Code (LinuxArmhf) LinuxArmhf succeeded
Details
VS Code (LinuxSnap) LinuxSnap succeeded
Details
VS Code (LinuxWeb) LinuxWeb succeeded
Details
VS Code (Mooncake) Mooncake succeeded
Details
VS Code (Windows) Windows succeeded
Details
VS Code (Windows32) Windows32 succeeded
Details
VS Code (macOS) macOS succeeded
Details
VS Code Distro Sync & Merge Check #20191115.63 succeeded
Details
license/cla All CLA requirements met.
Details
@Tyriar Tyriar deleted the tyriar/webgl2 branch Nov 16, 2019
@Tyriar Tyriar mentioned this pull request Nov 16, 2019
@Remzi1993

This comment has been minimized.

Copy link

Remzi1993 commented Nov 17, 2019

@Tyriar Thank you very much for fixing this. Do you know when this gets released?

@Tyriar

This comment has been minimized.

Copy link
Member Author

Tyriar commented Nov 17, 2019

@Remzi1993 you can try it out in tomorrow's insiders by setting terminal.integrated.rendererType to experimentalWebgl

@Fmstrat

This comment has been minimized.

Copy link

Fmstrat commented Nov 19, 2019

@Tyriar What about timeframe for full release? This has been plaguing us Ubuntu users for a while now ;)

@Tyriar

This comment has been minimized.

Copy link
Member Author

Tyriar commented Nov 19, 2019

@Fmstrat depends how many problems there are with it when it goes out, I know of one major problem that I hope to fix before this release is out #85048. I'd expect it to get more and more polished until we're ready to call it stable and remove the "experimental" part (and then look at removing the canvas renderer all together). In the meantime you can just have that setting if it's working for you, I highly doubt it will get removed once it hits a stable release.

@Tyriar Tyriar mentioned this pull request Dec 2, 2019
3 of 3 tasks complete
@warpdesign

This comment has been minimized.

Copy link
Contributor

warpdesign commented Dec 5, 2019

I downloaded the latest 1.41 insiders on Windows, set terminal.integrated.rendererType to experimentalWebgl but it doesn't seem to be enabled:

image

I tried re-starting vscode but it didn't help.

@Tyriar

This comment has been minimized.

Copy link
Member Author

Tyriar commented Dec 5, 2019

@warpdesign what's $0? There are multiple canvases that are used still, the layout should look like this where the highlighted one is the one with the webgl context:

Screen Shot 2019-12-05 at 2 18 00 PM

@warpdesign

This comment has been minimized.

Copy link
Contributor

warpdesign commented Dec 5, 2019

@Tyriar This canvas doesn't have the webgl context:
image

@Tyriar

This comment has been minimized.

Copy link
Member Author

Tyriar commented Dec 6, 2019

@warpdesign it's webgl2

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants
You can’t perform that action at this time.