Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Plans about WebAssembly #829

Open
sergey-kucher opened this issue Jun 19, 2015 · 58 comments

Comments

Projects
None yet
@sergey-kucher
Copy link
Contributor

commented Jun 19, 2015

Do you have any plans about WebAssembly support?

@vendethiel

This comment has been minimized.

Copy link
Contributor

commented Jun 19, 2015

LLVM already has a branch for that, so Crystal should be covered already ;)

@asterite

This comment has been minimized.

Copy link
Member

commented Jun 21, 2015

Maybe when there's a WebAssembly specification we can start discussing this, but right now Crystal is more geared towards server and desktop applications, not the web. I'll close this until then.

@asterite asterite closed this Jun 21, 2015

@Svenskunganka

This comment has been minimized.

Copy link

commented Jun 7, 2016

A year later, now that a spec is in the works I figured that we could open discussion for this again? Chrome Canary and Firefox Nightly has experimental support for WebAssembly.

@asterite I understand that this is probably not a priority right now - but is this something you guys might be interested in the future?

@lbguilherme

This comment has been minimized.

Copy link
Contributor

commented Jun 7, 2016

I believe there is not much to be done from crystal side until both browsers and the llvm target get mature enough. Clang is not ready for this yet.

We can for sure make a proof of concept that works, but not maintain it or port the whole stdlib to work on it yet. I suggest creating a milestone "post-1.0" and leaving this issue open there.

@vendethiel

This comment has been minimized.

Copy link
Contributor

commented Jun 7, 2016

Clang is not ready for this yet.

Not sure how Clang is related to Crystal? Once it gets down to LLVM IR, does it matter?

@asterite

This comment has been minimized.

Copy link
Member

commented Jun 7, 2016

@Svenskunganka I guess so, if you could write Crystal to target the browser it would be awesome. I know nothing about WebAseembly, and just a bit about asm.js, but I don't know what kind of things you can do... dom manipulation? Or just games? In any case, I think we should wait a bit.

@lbguilherme

This comment has been minimized.

Copy link
Contributor

commented Jun 7, 2016

@vendethiel It is not in any way related or needed for crystal to work. But it is the one compiler that is working most towards targeting wasm. In fact, this is the goal of the first release of wasm: Have clang targeting wasm and running it in multiple browsers. It is safer and less work for us to let clang test the ground, solve most bugs, and get there before us.

@asterite It let's you access all standard web apis thought some kind of "LibJS", that includes DOM manipulation and everything else. All the stdlib (libc included, I think) would have to be implemented on top of that.

@jwaldrip

This comment has been minimized.

Copy link

commented Jun 7, 2017

@waj It may be a time to revisit/reopen this issue.

@RX14

This comment has been minimized.

Copy link
Member

commented Jun 7, 2017

@jwaldrip why?

@pannous

This comment has been minimized.

Copy link

commented Jun 7, 2017

because the standards are now fixed and the toolchain is working nicely.
full support in most browsers and node.js

There is still a big opportunity to become the language of the binary Internet, racing against c++, rust and typescript.

@Svenskunganka

This comment has been minimized.

Copy link

commented Jun 7, 2017

Today WebAssembly is available in all the major stable browser releases, however its features are limited. It currently has no way to access the DOM or the Web APIs except if you're using JavaScript as the middle man. The WebAssembly Roadmap looks very promising however. Planned features are to allow wasm to directly access and modify the DOM, and call Web APIs without needing any JS at all.

Wasm has an (experimental) LLVM backend, and since Crystal uses LLVM I can imagine it would ease the work required to compile to wasm. Wasm post-MVP roadmap shows plans to implement access to low-level GC, however I'm not sure how this would interact with the Boehm GC that Crystal currently uses. I imagine that a new GC would have to be built specifically for wasm, or that the Boehm GC is adjusted to support the wasm "runtime". But this is only speculation, I'm not knowledgeable enough in that area to draw a conclusion.

Looking at these in progress features, the possibilities are very promising. Being able to compile to wasm would allow the community to create "isomorphic, fullstack" Meteor-like frameworks that shares the same code on both server & client (browser). To simplify, it would allow Crystal to do everything that JS currently does, like having a React-like library with full server-side rendering support. Other examples would be a Slack alternative that compiles to both the browser and regular platforms like Linux, Windows and Mac. I would imagine that Wasm would also be available on smartphones like iOS and Android through WKWebView and the like, so that Slack client could perhaps also be able to run on those platforms using the Wasm code.

It's worth mentioning though that Wasm will not be able to do things that JS currently cannot, like opening a raw TCP socket. We would have to use the alternatives the Web APIs provide like WebSocket and WebRTC.

@drhuffman12

This comment has been minimized.

Copy link

commented Jun 8, 2017

:+1

@akzhan

This comment has been minimized.

Copy link
Contributor

commented Jun 9, 2017

@ysbaddaden what about wasm LLVM target? looks like it's a time to do.

@rhysd

This comment has been minimized.

Copy link
Contributor

commented Jun 9, 2017

As long as I know, LLVM 4.0.0 does not support wasm. LLVM HEAD already supports it. So it may be available in 4.0.1 but I'm not sure. I remember LLVM is now in semantic versioning. So I guess it comes with 5.0.0 for stable release.

@ysbaddaden

This comment has been minimized.

Copy link
Member

commented Jun 9, 2017

Same as other arch's: LLVM does the underlying work, that is generate the binary code (it's awesome), but there is far more to Crystal than just the language: it's library.

Without an official libc (the musl port is unofficial and experimental, advised not to use), we're left with the same state than embedded systems: most of the core and stdlib won't be available, the GC will likely not work, ... not even thinking of the fiber context switches ASM (cold sweat).

WASM may be fun, but it's far off the Crystal roadmap. If someone is courageous (crazy?) enough to work on this, please try, you'll have fun (maybe) :-)

@pannous

This comment has been minimized.

Copy link

commented Jun 12, 2017

isn't libc provided by emscripten? how else would it run hello_world.c ?

Quote:

Emscripten lets you "link" with JavaScript libraries to do useful things, like render WebGL, run a browser main loop, handle a filesystem, provide bindings to JS so it's easy to call into your compiled code, etc., as well as a full set of compiled libraries like libc and so forth.'

GC can be added later.

@pannous

This comment has been minimized.

Copy link

commented Jun 12, 2017

Here is how rust does it https://github.com/brson/mir2wasm/

@RX14

This comment has been minimized.

Copy link
Member

commented Jun 12, 2017

@pannous rust doesn't need a gc.

If someone can demonstrate libgc compiled and working on wasm then this endeavour becomes actually credible.

@pannous

This comment has been minimized.

Copy link

commented Jun 12, 2017

can't we start without GC? should be fine for demo code which doesn't run for extended periods of time?

@akzhan

This comment has been minimized.

Copy link
Contributor

commented Jun 12, 2017

BDWGC on Emscripten (LLVM to WebAssembly backend) now fails on MacOS hosts (but should be OK on others - ivmai/bdwgc#163 (comment)).

I suppose it will be fixed with ivmai/bdwgc#163.

@RX14

This comment has been minimized.

Copy link
Member

commented Jun 12, 2017

So it looks like wasm doesn't have a normal stack at all. I have no idea how we would implement fibers in such an architecture. The note on the GC issue saying that the GC can't walk the stack to find roots is basically a show stopper too. Implementing crystal on wasm looks like it'd require a great deal of thought.

@pannous

This comment has been minimized.

Copy link

commented Jun 12, 2017

yep, no GC and fibres/threads in first version:
http://webassembly.org/docs/future-features/

I'm fine with those crystal features missing in the first adaptation.

@akzhan

This comment has been minimized.

Copy link
Contributor

commented Jun 12, 2017

Looks like Crystal cannot be provided until EH, GC and coroutines landed to WebAssembly.

Just wait for these features (they will be added to browsers fast, I suppose).

@RX14

This comment has been minimized.

Copy link
Member

commented Jun 12, 2017

@pannous I doubt most people would be OK with that or call it a "complete" crystal port so I don't think it's worth the effort at this point in crystal's development. If you or anyone else wishes to work on it i'm sure the core team would be happy to help or merge PRs.

@RX14

This comment has been minimized.

Copy link
Member

commented Jun 12, 2017

@akzhan I doubt fibers are impossible in webassembly as-is or would get any kind of special support in the future. It's just a matter of finding out how to do it. GC and direct DOM access are things we should wait for.

@catmando

This comment has been minimized.

Copy link

commented Jun 23, 2017

if anybody else is interested in collaborating on a crystal -> JS transpiler let me know. I think it would have to be a subset without fibers initially. My gut feeling is that we be best to leverage the work down by the opal (ruby -> js) transpiler. I am guessing the major problem is that you have to map crystal class structures, method calls, etc -> JS equivilents, rather than trying to do some lower level emulation. This is what opal does.

@vendethiel

This comment has been minimized.

Copy link
Contributor

commented Jun 23, 2017

if anybody else is interested in collaborating on a crystal -> JS transpiler let me know.

wouldn't it be easier to find a way to get a GC/wait for GC?

My gut feeling is that we be best to leverage the work down by the opal (ruby -> js) transpiler.

maybe parts of the parser can be reused, but i don't see anything else

@badlee

This comment has been minimized.

Copy link

commented Sep 9, 2017

👍 +1

@larubujo

This comment has been minimized.

Copy link
Contributor

commented Dec 17, 2017

jack of all trades, master of none

@RX14

This comment has been minimized.

Copy link
Member

commented Dec 18, 2017

@larubujo that's a great reason for not implementing or focussing on implementing wasm now or before 1.0, but I see no harm in keeping this issue open as a place to discuss possibilities. I've made it clear I'm not going to put any effort into wasm personally.

@catmando

This comment has been minimized.

Copy link

commented Dec 18, 2017

I don't get it... sorry.
WASM provides a way to get crystal running in the browser
Crystal running in the browser would greatly speed adoption by making it a full-stack language.
Crystal in browser would allow for a crystal React, DSL and a system similar to ruby-hyperloop to exist.

Understand that you can't do everything at once, but this does seem like it would greatly speed adoption.

What am I missing

@straight-shoota

This comment has been minimized.

Copy link
Member

commented Dec 18, 2017

Widespread adaption is not an immediate goal (for the core team) - getting a stable 1.0 release of the language is. And even that is conceiveably not really immediate.

As I understand it, WASM support is not considered a critical feature for 1.0, and that seems logical. In the end, it is just another backend Crystal can compile to.

Is WASM really going to be that huge deal? I don't know.
Windows support seems certainly much more important and is a requirement for 1.0.

Now, this should not hold anyone off from working on WASM support. If you manage to accomplish something in that direction it will certainly be appreciated. And if it get's done before 1.0 release all the better.

But it's not a high priority for core developers currently. There are more than enough things to work on.

@pannous

This comment has been minimized.

Copy link

commented Dec 18, 2017

"Windows support seems certainly much more important"
An external observer (and fan of the language) dares to strongly disagree here.

  • if you compile to wasm, crystal programs can automatically run on all platforms including windows. without installation!
  • this is huge! wasm is going to be huge. prediction: it will replace the JVM as the most popular 'portable' platform.
  • there are already hundreds of languages running on windows
  • right now there is almost no language compiling to wasm

Don't miss the opportunity guys.

If you target wasm you get Windows for free. Also with Ubuntu subsytem you can already compile crystal on Windows? So why waste your valuable time on something almost obsolete, when you can change the world?

@ylluminarious

This comment has been minimized.

Copy link

commented Dec 18, 2017

Can't agree with @pannous enough. It's really sad to see how many open-source projects dwindle because of a lack of proper marketing. Crystal would be very astute to take advantage of wasm and not just scoff at it as a far-off milestone.

Crystal certainly is at a point where it could take programming languages by storm. It's even higher than Rust and several others on the TIOBE index right now. That could all change in an instant, though. And I bet it will have something to do with whether Crystal runs on wasm.

@ylluminarious

This comment has been minimized.

Copy link

commented Dec 18, 2017

@catmando is also very on-point and I share his confusion at the resistance to this issue.

@ylluminate

This comment has been minimized.

Copy link

commented Dec 18, 2017

So we're going to get this done. We have several commitments now from people. If you'd like to get involved, shoot me a note at my username at gmail and we'll get the collaboration going.

@RX14

This comment has been minimized.

Copy link
Member

commented Dec 19, 2017

The problem I forsee is the standard library. We haven't got most of the stdlib portable to non-posix OSes, let alone the browser. The browser isn't a traditional OS, it has no filesystem and wasm's memory model is pretty wacky compared to the tradtional one.

It'll be extremely hard but if we can port crystal to wasm it'll mean that we know that crystal is extremely portable.

@ylluminate

This comment has been minimized.

Copy link

commented Dec 19, 2017

@RX14 this is precisely where I see empscripten coming in strong since it provides those for us. Then it will just be a matter of testing and benchmarking to see if we need to do something different, etc.

@RX14

This comment has been minimized.

Copy link
Member

commented Dec 19, 2017

Oh, emscripten provides a libc I guess. Fair enough, thats probably easier than I thought.

@ghost

This comment has been minimized.

Copy link

commented Dec 19, 2017

kotlin (native) has wasm support and has garbage collection as well, maybe their approach helps

@catmando

This comment has been minimized.

Copy link

commented Dec 19, 2017

Stable 1.0 release vs. Wide Spread Adoption.

These are not contradictory. Wide Spread Adoption means you have more people interested in contributing, you have more people using in "anger", you get more realistic use cases, and find those edge case flaws you didn't think of until you get wider usage. This is all builds momentum towards the 1.0 release.

Huge advantage of crystal that everybody talks about is SPEED. Well if you can offload computation to the client's browser, that is really going to give you speed.

@theodorton

This comment has been minimized.

Copy link

commented Dec 20, 2017

Code sharing between client and server application is IMO a huge argument for targeting wasm. It seems like game development is one of the big applications on WebAssembly and from my little experience with Crystal it seems like it would be a great platform for developing games.

I did some proof-of-concept work on a Crystal to Javascript transpiler last year. My original roadmap was to have a transpiler as 1.0 and convert to tooling for wasm with a 2.0 release as the only goal of the project was to write Crystal code that could be executed one way or another in a browser environment. I'd be happy to help with any work needed. Although my experience with low-level compiler stuff is very limited I have a lot of experience with web applications and requirements for code sharing between server and client.

@ylluminate

This comment has been minimized.

Copy link

commented Dec 20, 2017

Great stuff @theodorton. I was also previously playing with the direct to JS route a la Opal, which I quite like the idea, BUT it really does seem that by piping things through Emscripten we can do this a lot more quickly and with markedly less effort. You might want to pop over here to chime in and chat with us.

@catmando

This comment has been minimized.

Copy link

commented Dec 22, 2017

I have a longer conversation here: https://gitter.im/crystal_wasm/implementation
The more I think about it, the more I think @theodorton 's approach (Crow?) is correct.
Why? Because at the moment WASM is just not mature enough. I.e. lacks GC, and as a result (and more critically) objects can't be shared with JS land.
This means that even if you can get a full crystal compiler going, you have to rewrite every JS lib in crystal. React for example.

I think if a bunch of us help what @theodorton has done so far, it might be quicker.

I am just playing devils advocate here, and can be convinced that I am wrong of course!

@andriytyurnikov

This comment has been minimized.

Copy link

commented Jan 26, 2018

It looks like few blockchain projects considering WASM as their primary or secondary VM, so browsers are not only runtime for WASM.

@asterite

This comment has been minimized.

Copy link
Member

commented Jan 26, 2018

But blockchains are a waste of energy and resources... let's not hurry all together to destroy this planet faster, please?

@andriytyurnikov

This comment has been minimized.

Copy link

commented Jan 26, 2018

@asterite not those blockchains :)
Blockchains with wasm are environment friendly, and they need your support :)

@faustinoaq

This comment has been minimized.

Copy link
Contributor

commented Mar 24, 2018

Q: Don't you need features like garbage collection and threading added to WebAssembly to make this work?

No, WebAssembly in its current state is sufficient. The .NET runtime handles its own garbage collection and threading concerns.

^ Interesting 😅

^ https://github.com/aspnet/Blazor/wiki/FAQ#q-dont-you-need-features-like-garbage-collection-and-threading-added-to-webassembly-to-make-this-work

@Svenskunganka

This comment has been minimized.

Copy link

commented Mar 28, 2018

A design document from the Go core developers are available here: https://docs.google.com/document/d/131vjr4DH6JFnb-blm_uRdaC0_Nv3OUwjEY5qVCxCup4/edit

Looks like their implementation is under review and possibly being merged soon: https://go-review.googlesource.com/c/go/+/102835

Posting it in case anyone wants to take a bite at this in the future. It covers some of the questions already asked here.

@RX14

This comment has been minimized.

Copy link
Member

commented Mar 28, 2018

And as we can see from that document, go's port to wasm is a bunch of hacks to work around missing features in wasm. If someone wants to make a PR hacking those in, go for it, but until then I'll be waiting for proper GC and coroutine support.

@osarrouy

This comment has been minimized.

Copy link

commented Jun 6, 2018

Hi everyone,

I'm a big fan of crystal and also happen to be a blockchain developer so i'd just like to know if there is any news about this issue and if i could help in any way ... ?

Thanks!

@rdp

This comment has been minimized.

Copy link
Contributor

commented Sep 5, 2018

I guess you could target "javascript" (not wasm) via emscriptem and go single threaded. Though that's not what the title of this issue is about :) I assume that's how mruby in the browser works [?] http://qiezi.me/projects/mruby-web-irb/mruby.html (just uses the javascript built-in GC I presume?). Watch also https://github.com/WebAssembly/gc/ (WIP gc proposal). Other options: Add a "new gc" option ("reference counting with cycle checking" JetBrains/kotlin-native#1170 (comment) or the "double copy stack to the heap" trick RX14 mentions). Or manage stack manually if you want to keep fibers. Doesn't sound trivial...

@sam0x17

This comment has been minimized.

Copy link

commented Oct 22, 2018

This provides some very useful and up-to-date high-level information on the web assembly feature set and roadmap:

https://hacks.mozilla.org/2018/10/webassemblys-post-mvp-future/

@straight-shoota @RX14

@bew

This comment has been minimized.

Copy link
Contributor

commented Oct 30, 2018

Interesting read about threading implementation for wasm: https://rustwasm.github.io/2018/10/24/multithreading-rust-and-wasm.html
There might be some useful things for us ;)

@parruda

This comment has been minimized.

Copy link

commented Feb 18, 2019

I just stumbled upon this issue. I’ve been thinking about adding Opal support to FaaStRuby, and if we could compile Crystal to Wasm... Full stack Crystal development in a serverless platform. Think about the possibilities!

I wish I could contribute to this effort, but I don’t know enough about Wasm or LLVM. But you guys can be sure I would contribute to make it popular by pushing the support to the platform as soon as you guys release it!

Thanks for the hard work.

@sam0x17

This comment has been minimized.

Copy link

commented Feb 19, 2019

Yeah, I imagine this is going to be one of the first big pushes after 1.0, and by then other languages will have figured out some of the do's and dont's of web assembly support, so it will be that much easier

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.