New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Emscripten 1.37.16 to 1.37.36 #37291

Merged
merged 12 commits into from Mar 18, 2018

Conversation

Projects
None yet
4 participants
@qknight
Copy link
Member

qknight commented Mar 18, 2018

Motivation for this change

this extends the PR from #35636 which got reverted and does many cleanups.

  • the emscriptenVersion is a global attribute which is used for emscripten and emscripten-fastcomp (this was a requirement from kripken)

  • made some emscripten tests working so we know the toolchain is working on nixpkgs

  • fix emscriptenPackages.libxml2

  • fix emscriptenPackages.zlib

  • fix emscriptenPackages.xmlmirror, seems libz can't be found ... and the cause was the order of libraries supplied to the dynamic linker... stupid!

  • updated to 1.37.36, fully working for emscriptenPackages now

  • rewrite that python must not be included in each emscriptenPackage manually, but instead add it from pkgs/development/em-modules/generic/default.nix

  • force users to implement tests & make sure users understand why this is important

  • add a test to json_c

  • add a test to zlib

  • refactor the code: check dependencies once more, might have added unneded stuff

  • add documentation

    • document the .emscripten behaviour
    • add section on how to use nix-shell with emscripten and how to use emcc there to build the test and debug the stuff
    • add HOME=$TMPDIR explanation
    • add export EMCC_DEBUG=2 explanation

@qknight qknight requested a review from joachifm Mar 18, 2018

@qknight qknight merged commit f41a3e7 into NixOS:master Mar 18, 2018

8 checks passed

grahamcofborg-eval ^.^!
Details
grahamcofborg-eval-check-meta config.nix: checkMeta = true
Details
grahamcofborg-eval-nixos-manual nix-instantiate ./nixos/release.nix -A manual
Details
grahamcofborg-eval-nixos-options nix-instantiate ./nixos/release.nix -A options
Details
grahamcofborg-eval-nixpkgs-manual nix-instantiate ./pkgs/top-level/release.nix -A manual
Details
grahamcofborg-eval-nixpkgs-tarball nix-instantiate ./pkgs/top-level/release.nix -A tarball
Details
grahamcofborg-eval-nixpkgs-unstable-jobset nix-instantiate ./pkgs/top-level/release.nix -A unstable
Details
grahamcofborg-eval-package-list nix-env -qa --json --file .
Details
@Ericson2314

This comment has been minimized.

Copy link
Member

Ericson2314 commented Mar 19, 2018

@qknight I must say... emscripten seems awful requiring all this extra stuff per package. Let me or @ElvishJerricco if you are interested in leveraging the cross stuff and use LLVM directly to try to avoid needing such special-casing.

@ElvishJerricco

This comment has been minimized.

Copy link
Contributor

ElvishJerricco commented Mar 19, 2018

@Ericson2314 Unfortunately, the wasm backend in LLVM isn't nearly mature enough for most people's use cases. The compiler is fine, but it lacks critical components like libc. I've been getting by without by just porting stuff myself, but it's all very thrown-together and I probably wouldn't recommend it to be anyone's default at the moment. Emscripten is a reasonably production ready toolchain, plus it supports compiling to JS. As much as I dislike Emscripten, I think it'll be here for a while until the wasm story gets more fleshed out as a proper cross target.

That said, I'm not super happy seeing all kinds of special cases for Emscripten in nixpkgs. As a toolchain, Emscripten is really poorly designed.

@Ericson2314

This comment has been minimized.

Copy link
Member

Ericson2314 commented Mar 19, 2018

I mean we could piecemeal rewrite and repackage enscripten, slowly turning things into pain WASM-via-LLVM.

@qknight

This comment has been minimized.

Copy link
Member

qknight commented Mar 20, 2018

as long as there is no prototype code or even documentation on how to get rid of emscripten i don't see how that should be done.

emscripten works actually pretty well as a standalone and combined with nixpkgs i think it can be automated even futher. that said, over time we can move away from it but there need to be examples working and i don't see any.

@ElvishJerricco

This comment has been minimized.

Copy link
Contributor

ElvishJerricco commented Mar 20, 2018

@qknight wasm-cross builds C packages with LLVM + musl. It's not perfect but it's much more in line with a proper cross toolchain and works more easily with nixpkgs. EDIT: I would love to steal Emscripten's libc implementation, but that proved rather difficult last time I tried.

@qknight

This comment has been minimized.

Copy link
Member

qknight commented Mar 20, 2018

@ElvishJerricco so do we have a prototype somewhere?

@ElvishJerricco

This comment has been minimized.

Copy link
Contributor

ElvishJerricco commented Mar 20, 2018

@qknight What do you mean? wasm-cross is the prototype. To be clear: I'm not suggesting getting rid of Emscripten, or merging wasm-cross into nixpkgs (yet).

@qknight

This comment has been minimized.

Copy link
Member

qknight commented Mar 20, 2018

@ElvishJerricco can't find wasm-cross in nixpkgs. so this is more of a vision than an actual thing we can do. so when you got something to look at, i'm eager to review it.

in general, don't underestimate the amount of work emscripten does behind the scenes. i've spent at least 3 full weeks (nerd weeks, not employe weeks) in emscripten documentation and examples, talked to several developers while writing fastxmllint.js and nix-instantiate.js.

but i've not yet used WASM but instead asm.js.

@ElvishJerricco

This comment has been minimized.

Copy link
Contributor

ElvishJerricco commented Mar 20, 2018

@qknight my first comment about wasm-cross was a link. https://github.com/WebGHC/wasm-cross.

in general, don't underestimate the amount of work emscripten does behind the scenes

I don't. And this is exactly why it's terrible. C toolchains should not have so much work to do. wasm-cross certainly doesn't.

@Ericson2314

This comment has been minimized.

Copy link
Member

Ericson2314 commented Mar 20, 2018

@qknight in particular https://github.com/WebGHC/nixpkgs has yet to be merged back. That may be what you are looking for.

@ElvishJerricco

This comment has been minimized.

Copy link
Contributor

ElvishJerricco commented Mar 20, 2018

@Ericson2314 probably not. The nixpkgs fork just has some miscellaneous things necessary for cross, most of which has been merged upstream (not all). The meat of the wasm work is in wasm-cross.

@Ericson2314

This comment has been minimized.

Copy link
Member

Ericson2314 commented Mar 20, 2018

@ElvishJerricco Well then you need to start leveraging new nixpkgs more :P At least after #36867

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment