Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Hosting Avalonia in WebAssembly #1387

Open
vectorix opened this issue Feb 24, 2018 · 44 comments

Comments

Projects
None yet
@vectorix
Copy link

commented Feb 24, 2018

Would it be possible to host Avalonia with WebAssembly and .NET in a browser?

Then we could write the same C# code and run it native in all browser and on all desktops.

@kekekeks

This comment has been minimized.

Copy link
Member

commented Feb 24, 2018

It would be possible and we are currently considering our options. The major blocker is that HTML5 doesn't have a proper 2D drawing API, so Skia needs to be compiled to wasm and somehow linked to resulting application. The problem with linking is that Mono doesn't currently provide P/Invoke support for wasm target. Another problem is that P/Invoke means indirect calls (i. e. calls via pointer), while WebAssembly doesn't support indirect calls between modules. That means that Skia needs to be linked statically, so we'll need a custom Mono build.

It would be easier with CoreRT, since it supports static linking out of the box, but they are still implementing MSIL instructions for wasm target, so it's not ready even for an experimental port.

@vectorix vectorix closed this Mar 3, 2018

@legistek

This comment has been minimized.

Copy link

commented Mar 7, 2018

Sorry if this is an obviously terrible idea, but why wouldn't you use WebGL?

@danwalmsley

This comment has been minimized.

Copy link
Member

commented Mar 8, 2018

@kekekeks

This comment has been minimized.

Copy link
Member

commented Mar 9, 2018

@pmoorelegistek We will be using WebGL at some point. The problem is that WebGL by itself doesn't support drawing 2D vector graphics. So we need Skia anyway.

@legistek

This comment has been minimized.

Copy link

commented Mar 9, 2018

@legistek

This comment has been minimized.

Copy link

commented Mar 9, 2018

Hmm. The problem with compiling Skia to WASM is you'll lose any benefits of hardware graphics acceleration on the device. Browser will be doing all the math, painting pixels, etc. I don't know that you'd ever get anything close to 60fps with a complex UI.

Have you tried just using the HTML5 Canvas or SVG?

@kekekeks

This comment has been minimized.

Copy link
Member

commented Mar 9, 2018

Skia can use OpenGL ES profile for hardware-accelerated rendering. Which is basically what is provided by WebGL.
HTML5 Canvas and SVG are not sufficient to render Avalonia.

@legistek

This comment has been minimized.

Copy link

commented Mar 9, 2018

You're saying Skia / OpenGL will be able to access hardware acceleration (i.e. the GPU) after compiled to WASM? Are you certain about that? Everything I understand about WASM would lead me to think not.

I would think you'd be better off using three.js - or better yet porting it to C# - wrapped around WebGL.

I'd be happy to look into it some more and experiment. Do you have a working prototype using Mono-to-WASM at the moment with any rendering at all?

@kekekeks

This comment has been minimized.

Copy link
Member

commented Mar 10, 2018

You're saying Skia / OpenGL will be able to access hardware acceleration (i.e. the GPU) after compiled to WASM

Emscripten toolchain has complete support for WebGL. It might be a bit difficult to make it work with Mono since it has it's own js glue code, but other than that I don't see any issues.

@danwalmsley

This comment has been minimized.

Copy link
Member

commented Mar 15, 2018

@danwalmsley danwalmsley reopened this Mar 15, 2018

@kekekeks

This comment has been minimized.

Copy link
Member

commented Mar 16, 2018

@danwalmsley it uses HTML. Avalonia can't be rendered with HTML.

@adoris

This comment has been minimized.

Copy link

commented Mar 16, 2018

Is there are any try to use Skia/WebGL? Any branch for this experiment?

@legistek

This comment has been minimized.

Copy link

commented Mar 16, 2018

if I understand right, what @kekekeks is saying is that it's not feasible at the moment because Mono can't use P/invoke to call the Skia functions. So while they can compile Skia to WASM, there would be no way for the .NET code to access it.

What I don't understand fully, though, is why it has to be Skia. There are Javascript libraries that provide 2D drawing function wrappers over WebGL (e.g., https://threejs.org/). I'd think it at least worth an experiment to make a DrawingContext that wraps three.js and go from there. If that works, porting threejs to C# and simply including it in the Avalonia build would not be too bad of a project and would probably get you performance on par with Skia.

@kekekeks

This comment has been minimized.

Copy link
Member

commented Mar 17, 2018

@pmoorelegistek The main problem is the complete lack of a proper text measurement API. We can't get our TextBox to work if we don't have an accurate position of each individual symbol. Even for our TextBlock we need to be able to measure and lay out text lines properly. There is no way we can do that with the APIs available to JavaScript.

@yowl

This comment has been minimized.

Copy link

commented Apr 4, 2018

I added an issue for pinvoke support on mono so this can be tracked. mono/mono#8007

@TonyHenrique

This comment has been minimized.

@legistek

This comment has been minimized.

Copy link

commented Apr 13, 2018

@kekekeks

This comment has been minimized.

Copy link
Member

commented Apr 13, 2018

Noesis has it's own rendering engine, if I recall correctly. A proprietary one. We don't have the resources to create a whole new 2D graphics library, so we are using an existing one, which is Skia. Until we have a way of linking it in on wasm target (be it Mono or CoreRT), we won't be able to get Avalonia working in the browser.

@legistek

This comment has been minimized.

Copy link

commented Apr 13, 2018

Hey Nikita - since this thread keeps coming alive -
I looked deep into three js and the code for text rendering could easily be adapted for text measuring. I'd be willing to make the modifications if you're willing to consider something besides skia.

And beyond that, I don't think a port of three js to C# - or at least enough of it for our purposes - would be at all that difficult.

This is my company, and as you can see from our product we have a vested interest in your project: www.legistek.com

@kekekeks

This comment has been minimized.

Copy link
Member

commented Apr 13, 2018

Well, you see, the only text measurement API that I've found for HTML5 Canvas is measureText call which returns a TextMetrics object. The problem is that it doesn't support anything but text width:

default

As you can see, "advanced" properties are only supported on Google Chrome and need to be explicitly enabled in browser settings. Getting text width is unfortunately not enough for implementing our IFormattedText interface.

@kekekeks

This comment has been minimized.

Copy link
Member

commented Apr 13, 2018

Does three.js have it's own rendering engine that can draw text using TrueType/Freetype fonts with proper subpixel rendering?

@kekekeks

This comment has been minimized.

Copy link
Member

commented Apr 13, 2018

If it does, it might be worth to create some kind of a wrapper, but currently calling JS code from WASM is very inconvenient.

This is currently the only way of calling JS code from C#, which is basically an eval call.

There is some existing glue code here (JS part here) which uses the internal call from an alternative mono wasm target implementation, which is basically an eval too, so it could be adapted.

@legistek

This comment has been minimized.

Copy link

commented Apr 13, 2018

https://github.com/mrdoob/three.js/blob/f81506e172571ab106d0164530bbc1a4802fc2d4/src/extras/core/Font.js

They are going through the font glyphs and turning them into path geometries and then feeding them to WebGL. If they can do that then they could have measured them. It looks like nothing more than an oversight that they didn't include measuring functions. (I'm not yet clear on what font format they're expecting but my assumption would be WOFF).

re JS interop, again I would assume the endgame would be porting what we need from three.js to C# so that all the calculating code would be done in WASM and the only time you'd need to interop would be the final render into WebGL. But why not try a POC at least?

@kekekeks

This comment has been minimized.

Copy link
Member

commented Apr 13, 2018

@pmoorelegistek It seems that Mono WebAssembly SDK is providing libmono and an example C source file with the entry point. That means that we can link additional libraries to the resulting runtime mostly without issues. mono-dl-wasm.c file still needs to be changed for supporting __Internal but at least we could use existing build scripts

@jmacato

This comment has been minimized.

Copy link
Member

commented Apr 13, 2018

@kekekeks I'd like to bring this lib to our attention: SharpFont; could be useful for text rendering, alongside the PixelFarm renderer that i linked before.

I just noticed that PixelFarm is already using this particular library, so yeah

@kekekeks

This comment has been minimized.

Copy link
Member

commented Apr 15, 2018

NanoVG is a C library. So we are back to the square one where we need to somehow enable native code interop. I'll try to get Skia# working on top of WebAssembly first before trying managed alternatives (which will be super-slow because Mono currently runs managed code using interpreter)

@kekekeks

This comment has been minimized.

Copy link
Member

commented Apr 15, 2018

BTW, we can't use LGPL code (nanovgdotnet is LGPL-licensed) for WASM target since everything will be linked to a single file once Mono gets a proper compiler instead of interpreter.

@PerArneng

This comment has been minimized.

Copy link

commented Apr 16, 2018

This would be so cool. I think it could be the killer app since there are not alot of options today for doing that.
Some demos from other UI toolikts running in webassembly in the browser:

@jpbruyere

This comment has been minimized.

Copy link

commented May 9, 2018

canva and SVG are the browser's 2d api's.

@galvesribeiro

This comment has been minimized.

Copy link

commented May 17, 2018

There are some people working Skia for wasm https://github.com/Zubnix/skia-wasm-port but like @kekekeks said, you need a custom built mono liked with skia... I have canvas APIs wrapped for Blazor https://github.com/BlazorExtensions/Canvas using the CanvasRenderingContext2D APIs. So, it wouldn't be hard to use it but, the subpixel measurement required by Avalonia text renderer would probably not work with it...

@kekekeks

This comment has been minimized.

Copy link
Member

commented May 17, 2018

@galvesribeiro I had some success with building libSkiaSharp last month (I even have some code that actually creates some skia types from C#) and encountered two major problems:

  1. The current Mono interpreter has a hardcoded list of function signatures that can be P/Invoked. If I try to call something else it fails with errors like CANNOT HANDLE COOKIE VIIFFI. We need make that call pluggable, analyze the list of imported function signatures used by SkiaSharp and generate needed trampolines

  2. WASM implementations in Chrome/Firefox have an arbitrary limit on the number of functions declared in the binary fine. I was actually hitting that limit even with incomplete (no libwebp, pdf, svg, xml) libSkiaSharp and getting "functions count of 1589554 exceeds internal limit of 1000000" error. To make it load I had to remove GPU support which will hurt performance by a lot. The problem might be caused by the fact that I'm currently adding all libSkiaSharp exports to the mono.wasm binary. We might need to integrate a linker step that removes unused DllImports, reanalyze the SkiaSharp.dll to get the list of actually used imports and only then generate a separate C file. However I'm afraid that without stripping GPU support the function count will stay way to huge.

The first problem is solvable, but we might be hitting a somewhat "fundamental" limit with the second one. I'm not sure if it's possible to reduce the function count without removing the needed features.

@kekekeks

This comment has been minimized.

Copy link
Member

commented May 17, 2018

We might also try to resort to WASM dynamic linking to make libSkiaSharp live in a separate wasm file. That should increase the total function count limit.

@galvesribeiro

This comment has been minimized.

Copy link

commented May 17, 2018

We might also try to resort to WASM dynamic linking to make libSkiaSharp live in a separate wasm file. That should increase the total function count limit.

I'm not sure with the current wasm implementations we can do that... Last time I checked, you can't dynamic load anything neither can you link another .wasm.

@kekekeks

This comment has been minimized.

Copy link
Member

commented May 17, 2018

According to this doc dynamic linking should be possible in MVP. We can't make DllImport work with it out of the box and have to resort to C codegen, but it should be possible.

@galvesribeiro

This comment has been minimized.

Copy link

commented May 17, 2018

Yeah, what I meant was that I'm not sure the browsers support that yet. There are discussions in some other frameworks where they can't so some stuff just because of that same problem. I may be wrong, its being a while since I don't check it...

@birbilis

This comment has been minimized.

Copy link

commented May 19, 2018

Can't a wasm file talk to javascript? If it can, that one could load other wasm(s) and act as a bridge between them

@kekekeks

This comment has been minimized.

Copy link
Member

commented May 20, 2018

We need to at least somehow setup a shared linear memory for multiple wasm files.

@kekekeks

This comment has been minimized.

Copy link
Member

commented Jun 26, 2019

FYI: I've spent a weekend tinkering with WASM target and SVG render target about a month ago and decided that the current performance of Mono interpreter is way too slow. It also might be a fault of our layout algorithms or me missing something since the debugging story on WASM is currently not existing

You can see the incomplete demo here: http://testapp.keks-n.net/ don't even try to open calendar or datagrid pages, they take ages to load.

@ghuntley

This comment has been minimized.

Copy link

commented Jul 3, 2019

Nice work @kekekeks!

@ghuntley

This comment has been minimized.

Copy link

commented Jul 3, 2019

@kekekeks you might enjoy this blog post re: skia and perf. Hopefully it gives you some ideas you can incorporate. https://platform.uno/skiasharp-support-for-webassembly-via-uno-platform/ :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.