Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Is there a way I can compile small piece of C# code to wasm #46344

Closed
1 task done
anirugu opened this issue Jan 31, 2023 · 34 comments
Closed
1 task done

Is there a way I can compile small piece of C# code to wasm #46344

anirugu opened this issue Jan 31, 2023 · 34 comments
Milestone

Comments

@anirugu
Copy link

anirugu commented Jan 31, 2023

Is there an existing issue for this?

  • I have searched the existing issues

Describe the bug

I have developed a Chrome extension using jQuery and then converted it to vanilla JavaScript. I am now seeking a way to improve its performance by using Blazor. Is it possible to write a basic C# code, compile it to WebAssembly (.wasm), and import and execute it in a JavaScript file?

Expected Behavior

I am looking for result similar to this post https://nodejs.dev/en/learn/nodejs-with-webassembly/

Steps To Reproduce

I discovered a project on Github (https://github.com/mingyaulee/Blazor.BrowserExtension) that has caught my interest. However, I am looking for a way to compile a single file into a single WebAssembly (.wasm) file, similar to the approach used in Angular and React projects. Is there a method I am unaware of that can achieve the same result? I prefer to avoid deploying multiple DLLs for a simple task. Is there a way in .NET that can serve this purpose?

Exceptions (if any)

NA

.NET Version

7.0.200-preview.22628.1

Anything else?

No response

@davidfowl
Copy link
Member

cc @SteveSandersonMS

@danmoseley danmoseley added the area-blazor Includes: Blazor, Razor Components label Jan 31, 2023
@SteveSandersonMS
Copy link
Member

@anirugu There are ways of doing this, but neither of them involve Blazor. Blazor is a UI framework, but you're just looking for a way to run arbitrary .NET code on WebAssembly. The two approaches I can suggest are:

  1. Using the wasm-tools workload

    • See https://devblogs.microsoft.com/dotnet/use-net-7-from-any-javascript-app-in-net-7/
    • This lets you create a C# project that compiles to a set of assets that can be run inside a JavaScript environment, such as a Chrome extension. You do not get a single file. Instead you get a dotnet.wasm file that loads your .NET assemblies (.dll files) and executes them.
    • It supports AOT compilation for improved performance
    • It contains built-in JS interop APIs that will simplify interacting with the host browser
  2. Using the experimental .NET WASI SDK

    • See https://github.com/dotnet/dotnet-wasi-sdk
    • This produces a single, standalone .wasm file that can run in any WebAssembly environment, without requiring any JavaScript support
    • This experimental SDK does not yet support AOT compilation, so you will likely not get better performance than your JavaScript implementation until we do add AOT support
    • Since this SDK doesn't assume any JavaScript environment exists, there are no built-in JS interop APIs, so you would have to wire up your own way of calling into the host browser to interact with the Chrome extension APIs

Option 1 is probably more practical for your scenario, since you will be running inside a browser.

@SteveSandersonMS SteveSandersonMS removed the area-blazor Includes: Blazor, Razor Components label Jan 31, 2023
@SteveSandersonMS SteveSandersonMS added this to the Discussions milestone Jan 31, 2023
@danmoseley
Copy link
Member

@mkArtakMSFT do we need a new area label for issues like this one?

@SteveSandersonMS
Copy link
Member

SteveSandersonMS commented Jan 31, 2023

It should have gone to the runtime repo really. But in any case it's in the Discussions milestone so will be auto-closed when it becomes inactive.

@RReverser
Copy link

My scenario is similar to the original question, but what I'm trying to get is a single Wasm that is also not tied to either browser or WASI - akin to the wasm32-unknown-unknown target available in Clang and Rust.

That is, C# compiled to Wasm shouldn't be dependant upon any host functions except those explicitly provided via custom DllImport. I understand this would limit its functionality in the same way as wasm32-unknown-unknown target does (e.g. no access to console, filesystem, timers, etc), but it would allow it to be used with custom runtimes.

Is there any way to achieve that with either of Wasm targets currently available in .NET? Everything I find, even in experimental repos, seems to assume either Web or WASI hosts.

@SteveSandersonMS
Copy link
Member

Sorry for the slow response on this. The technique I've used in the past to eliminate all the WASI imports from the wasm file is to stub them out. For example:

I think that covers all the WASI imports but if any remain you can add similar stubs for those too, then the resulting wasm file will require no imports at all.

@RReverser
Copy link

RReverser commented Apr 21, 2023

@SteveSandersonMS Right, thanks, that covers removing imports, but how about adding imports/exports to custom runtime?

Since asking that question I switched to NativeAOT-LLVM for now, which allows to just declare UnmanagedCallersOnly for exports and DllImport for imports and it takes care of passing them all the way to the linker, but when using Wasi.Sdk those attributes seem to be ignored and it still contains only WASI imports/exports.

@RReverser
Copy link

Since asking that question I switched to NativeAOT-LLVM for now, which allows to just declare UnmanagedCallersOnly for exports and DllImport for imports and it takes care of passing them all the way to the linker, but when using Wasi.Sdk those attributes seem to be ignored and it still contains only WASI imports/exports.

Looks like those 2 are currently TODO items in dotnet/runtime#65895:

  • create WASM module import from any [DLLImport] which doesn't match known static native symbols
  • maybe create WASM module export from [UnmanagedCallersOnly]

So I guess no way to do that with Wasi.Sdk yet.

@SteveSandersonMS
Copy link
Member

SteveSandersonMS commented Apr 21, 2023

@RReverser It is possible to define imports/exports with this SDK but currently you have to do that in C and use the Mono hosting APIs to interop with C# code. It’s inconvenient.

I switched to NativeAOT-LLVM for now

Interesting. Have you been able to run the result in non-browser/Node environments, or does your scenario allow for depending on JS? Last time I checked that was using Emscripten and hence involved a JS requirement. If it now supports non-JS host environments that would be extremely interesting and I’d love to know how you got that to work!

@RReverser
Copy link

RReverser commented Apr 21, 2023

It is possible to define imports/exports with this SDK but currently you have to do that in C and use the Mono hosting APIs to interop with C# code. It’s inconvenient.

I'd love to see an example of both imports & exports even if it's not very convenient at the moment, just to have something to compare with.

Since I couldn't find a way to do that myself (and didn't get a response at the time), I started trying other solutions and .NET NativeAOT LLVM seemed most promising as imports/exports worked out of the box and, well, I know Emscripten quite well so figured might try to patch over the missing parts.

Interesting. Have you been able to run the result in non-browser/Node environments, or does your scenario allow for depending on JS? Last time I checked that was using Emscripten and hence involved a JS requirement. If it now supports non-JS host environments that would be extremely interesting and I’d love to know how you got that to work!

It doesn't quite support execution w/o JS (STANDALONE_WASM is somewhat limited in what it can do) but I'm currently working on an idea that might allow it. If all goes well, I'll blog about it!

Exceptions support is the most tricky bit as neither of major WASI runtimes (Wasmtime & Wasmer) support Wasm exceptions yet and neither does .NET AOT LLVM, so that currently has to go via JS - that's the part I'm trying to eliminate.

I'm guessing that one is not a problem for Wasi.Sdk as it bundles the .NET bytecode + interpreter so exceptions "just work"?

@SteveSandersonMS
Copy link
Member

See examples in the dotnet-wasi-sdk repo, e.g.

I'm currently working on an idea that might allow it

That would be the holy grail for NativeAOT-LLVM so if you do manage to make it work please let me know! It could unlock a lot of very interesting scenarios.

@RReverser
Copy link

See examples in the dotnet-wasi-sdk repo, e.g.

That's actually not too bad. And then these two lines https://github.com/dotnet/dotnet-wasi-sdk/blob/2dbb00c779180873d3ed985e59e431f56404d8da/src/Wasi.AspNetCore.Server.Atmo/build/Wasi.AspNetCore.Server.Atmo.targets#L4-L5 are enough to include those in the build?

That would be the holy grail for NativeAOT-LLVM so if you do manage to make it work please let me know! It could unlock a lot of very interesting scenarios.

To be fair, it's more of a hack rather than a long-term solution, I'd still love for NativeAOT-LLVM to properly support raw WASI like was started in dotnet/runtimelab#1850, but yeah, I will!

@RReverser
Copy link

@SteveSandersonMS Is the list of available mono_* functions documented anywhere?

@RReverser
Copy link

I guess it's just this? https://github.com/dotnet/runtime/blob/d11ed579a34b9650faba9275054e5990587c17f7/src/mono/wasi/mono-include/driver.h

But if so, it seems pretty limited in terms of objects it can manipulate.

@RReverser
Copy link

And then these two lines dotnet/dotnet-wasi-sdk@2dbb00c/src/Wasi.AspNetCore.Server.Atmo/build/Wasi.AspNetCore.Server.Atmo.targets#L4-L5 are enough to include those in the build?

Hmm no that doesn't seem to help...

@SteveSandersonMS
Copy link
Member

I guess it's just this?

No, there's a lot more. Once you have the wasm-tools workload installed, look in your .NET SDK's dotnet\packs\Microsoft.NETCore.App.Runtime.Mono.browser-wasm\8.0.0-preview.3.23174.8\runtimes\browser-wasm\native\include for a large collection of .h files.

And then these two lines

Yes that is the case. Not sure why it wouldn't be working in your project.

@RReverser
Copy link

Yes that is the case. Not sure why it wouldn't be working in your project.

Sorry, something changed (I'm not sure what, I did a bunch of experiments in the last few minutes) and it builds now. It complained about unknown attribute Include on WasiNativeFileReference for some reason.

Anyway, it builds now and I can see imports added correctly, but, no matter what I do, a native export from the C file doesn't seem to be exposed. I tried:

  • adding __attribute__((used)) (equivalent of EMSCRIPTEN_KEEPALIVE that should preserve symbols all the way to the linker)
  • adding <WasiSdkClangArgs>$(WasiSdkClangArgs) -Wl,--export-dynamic</WasiSdkClangArgs> (just in case linker is configured to ignore those annotations)
  • even adding explicit <WasiSdkClangArgs>$(WasiSdkClangArgs) -Wl,--export=helloworld</WasiSdkClangArgs>, but it seems to be just ignored

Those examples you linked seem to only use custom imports, any tips on how to export functions as well?

@RReverser
Copy link

Sorry, something changed (I'm not sure what, I did a bunch of experiments in the last few minutes) and it builds now. It complained about unknown attribute Include on WasiNativeFileReference for some reason.

Ah this one was because I tried putting it into PropertyGroup not ItemGroup.

So only the problem with exports remains. (and, well, digging through headers to find the necessary mono C functions)

@RReverser
Copy link

So I figured since WasiSdkClangArgs seems to be ignored at project level, but WasiNativeFileReference works fine and in https://github.com/SteveSandersonMS/dotnet-wasi-sdk/blob/e1d138717589c397ea7ecb2fd7015799b44e8bee/src/Wasi.Sdk/build/Wasi.Sdk.targets gets passed to the same clang args, I could try...

<ItemGroup>
  ...
  <WasiNativeFileReference Include="-Wl,--export=helloworld" />

and it worked 😂

This feels extremely hacky, but I guess it at least unblocks me for now... Please let me know if there's a better way to pass extra Clang args.

@RReverser
Copy link

Now I'm getting some failed assertion...

[wasm_trace_logger] * Assertion at /home/runner/work/dotnet-wasi-sdk/dotnet-wasi-sdk/modules/runtime/src/mono/mono/metadata/loader.c:1817, condition `mono_metadata_token_table (m->token) == MONO_TABLE_METHOD' not met

@RReverser
Copy link

Ohh okay that one is just because I had OutputType set to Library, I guess I'll have to use Exe even for libraries for now.

@SteveSandersonMS
Copy link
Member

Ohh okay that one is just because I had OutputType set to Library, I guess I'll have to use Exe even for libraries for now.

@RReverser Just guessing but calling invoking __wasm_call_ctors before your library export might fix that. That keeps tripping me up.

<WasiNativeFileReference Include="-Wl,--export=helloworld" />

You can mark a C function as exported or imported without needing any clang args at all:

__attribute__((export_name("my_function")))
void my_function() {
    ...
}

It doesn't quite support execution w/o JS (STANDALONE_WASM is somewhat limited in what it can do) but I'm currently working on an idea that might allow it. If all goes well, I'll blog about it!

I got something that does work in my situation, but no promises about yours! https://gist.github.com/SteveSandersonMS/f1a54f033d7fb78fd9c409d406398fa2 I was able to run a NativeAOT-LLVM build under wasmtime with this.

@RReverser
Copy link

RReverser commented Apr 25, 2023

You can mark a C function as exported or imported without needing any clang args at all:

Huh, odd that __attribute__((used)) didn't result in the same. Good to know an export with explicit name works, thanks.

I got something that does work in my situation, but no promises about yours! gist.github.com/SteveSandersonMS/f1a54f033d7fb78fd9c409d406398fa2 I was able to run a NativeAOT-LLVM build under wasmtime with this.

Heh yeah that's another idea I considered, but I'm currently working on a solution that wouldn't require runtime imports at all, that is, it would work with wasmer/wasmtime out of the box :)

We just don't really want to expose those imports to arbitrary users, as they're somewhat unstable plus the list of imports required by .NET might change over time.

@RReverser
Copy link

It's pretty cool that you got it to work though!

@RReverser
Copy link

No, there's a lot more. Once you have the wasm-tools workload installed, look in your .NET SDK's dotnet\packs\Microsoft.NETCore.App.Runtime.Mono.browser-wasm\8.0.0-preview.3.23174.8\runtimes\browser-wasm\native\include for a large collection of .h files.

How do you specify include paths for those (without hardcoding the full path)?

If I try to include like #include <mono/metadata/object.h>, compilation fails with includes not found.

@RReverser
Copy link

...which is weird because I can see -I"C:\Users\me\.nuget\packages\wasi.sdk\0.1.3-preview.10012\build\..\packs\wasi-wasm\\native\include" in the Clang flags... Maybe that \\ is throwing it off? But then mono-wasi/driver.h wouldn't work either...

@RReverser
Copy link

Ah looks like I was missing something, perhaps because I had wasm-tools-net7 installed. Installed wasm-tools and it compiles now.

@RReverser
Copy link

Okay, it all comes together nicely and much quicker than with NativeAOT. The only problem I'm running into is that it seems WasiNativeFileReference et al. are not propagated from dependencies to the main project - that is, if they declare custom imports to implement a library, then those imports appear when library is built separately to .wasm but not in the main app's .wasm.

Is such propagation expected to work or is it currently a known limitation?

@SteveSandersonMS
Copy link
Member

Is such propagation expected to work or is it currently a known limitation?

This is a general MSBuild thing, not something specific to dotnet-wasi-sdk. It's inconvenient, but to solve it you can:

  • Create a file build/YourPackageName.targets that adds the WasiNativeFileReference itemgroup entry. This will then run in anything that consumes your library as a package.
  • For things that consume your library as a project reference (e.g. in your own source repo), they will have to do <Import Project="../../path/to/that/file/above" /> to import it manually

There's an example of this in https://github.com/dotnet/dotnet-wasi-sdk/tree/main/src/Wasi.AspNetCore.Server.Atmo/build

@RReverser
Copy link

  • For things that consume your library as a project reference (e.g. in your own source repo), they will have to do <Import Project="../../path/to/that/file/above" /> to import it manually

Hmm thanks but TBH if path needs to be hardcoded anyway, at that point it feels easier to leave user setting the WasiNativeFileReference manually.

This is a general MSBuild thing, not something specific to dotnet-wasi-sdk.

I guess I was hoping that if I add Wasi.Sdk as a dependency of the library as well, it could compile & save any referenced C files in some well-known directory inside output dirs, and then Wasi.Sdk in the main project would pick those up.

@RReverser
Copy link

Just guessing but calling invoking __wasm_call_ctors before your library export might fix that. That keeps tripping me up.

Forgot to respond to this btw - doesn't look like this is exported even in Library mode, it's still only _start export but attempting to invoke it results in infinite recursion (that eventually fails due to stack overflow).

In Exe it's also _start export but it works 🤷‍♂️ Not sure what makes the difference.

@zxyao145
Copy link

Just guessing but calling invoking __wasm_call_ctors before your library export might fix that. That keeps tripping me up.

Forgot to respond to this btw - doesn't look like this is exported even in Library mode, it's still only _start export but attempting to invoke it results in infinite recursion (that eventually fails due to stack overflow).

In Exe it's also _start export but it works 🤷‍♂️ Not sure what makes the difference.

It's the same for me, but I got error:

Assertion at /home/runner/work/dotnet-wasi-sdk/dotnet-wasi-sdk/modules/runtime/src/mono/mono/metadata/assembly-load-context.c:81, condition `default_alc' not met

I must first call the c# Main function (_start function) before I call my custom exported function:

  • This will throw the exception
// var main = instance.GetFunction("_start");
// main.Invoke();

var r = instance.GetFunction("run");
r.Invoke();
  • This is working well
var main = instance.GetFunction("_start");
main.Invoke();

var r = instance.GetFunction("run");
r.Invoke();

@SteveSandersonMS
Copy link
Member

@zxyao145 Yes, that's expected. The runtime does have to be started before it's valid to call .NET methods.

@zxyao145
Copy link

@zxyao145 Yes, that's expected. The runtime does have to be started before it's valid to call .NET methods.

Got it, thank you very much for your reply

@dotnet dotnet locked as resolved and limited conversation to collaborators Jun 17, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

6 participants