Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Will you make tags in this repo again which are in sync with emscripten? #264

Open
svenstaro opened this issue Jun 17, 2019 · 26 comments

Comments

@svenstaro
Copy link

commented Jun 17, 2019

emscripten, emscripten-fastcomp, emscripten-fastcomp-clang used to all carry the same tags at the same time but this is not the case anymore. For instance, what tag of emscripten-fastcomp am I supposed to be using to go along with emscripten 1.38.36? Will you sync the tags up again?

@kripken

This comment has been minimized.

Copy link
Member

commented Jun 17, 2019

I updated those tags now.

Docs for how to figure out the revisions are here: https://github.com/emscripten-core/emscripten/blob/incoming/docs/process.md#minor-version-updates-1xy-to-1xy1

Note that we are going to switch away from fastcomp as our default backend soon, which is why there isn't any development going on there, and I forgot to update the tags. I suppose we can still tag versions there for now. But we encourage people to try to upstream llvm wasm backend at this point (blogpost coming soon with more details!).

@svenstaro

This comment has been minimized.

Copy link
Author

commented Jun 17, 2019

Thanks for the tags. I'd generally be happy to lose the fastcomp part but I'd need instructions on how to use the other backend. Happy to wait for the blog article for that, though. As context: I package emscripten for Arch Linux.

@kripken

This comment has been minimized.

Copy link
Member

commented Jun 18, 2019

It's pretty simple, and should work right now (except for a few features specific to fastcomp). You can get it with emsdk install latest-upstream and likewise add -upstream for the activate command. Or for building manually (which maybe you're doing as a packager?) you just need to get very recent llvm, and build it (it will build the wasm backend by default, but must also make sure it builds clang and wasm-ld, which I'm not sure are on by default). Then just point emscripten to that llvm build.

@kripken

This comment has been minimized.

Copy link
Member

commented Jun 18, 2019

Oh, and also this requires a very recent emscripten.

Docs on how we tag releases are here. That explains how you can find which emscripten commit goes with which LLVM commit etc. https://github.com/emscripten-core/emscripten/blob/incoming/docs/process.md#minor-version-updates-1xy-to-1xy1

@svenstaro

This comment has been minimized.

Copy link
Author

commented Jun 19, 2019

Thanks for the description but that's still too vague for me. Basically I'd be really happy if you could make a short compile script so that I can see how everything fits together and what gets cloned where.

@kripken

This comment has been minimized.

Copy link
Member

commented Jun 19, 2019

The main thing to compile is LLVM. Here is where the waterfall bots build it: https://github.com/WebAssembly/waterfall/blob/master/src/build.py#L790 The only other thing you need to build is binaryen, which is done here: https://github.com/WebAssembly/waterfall/blob/master/src/build.py#L932

In general though, building them is basically following the upstream project's normal build instructions, links to them are here, https://emscripten.org/docs/building_from_source/index.html

@svenstaro

This comment has been minimized.

Copy link
Author

commented Jun 19, 2019

Oh, and also this requires a very recent emscripten.

Docs on how we tag releases are here. That explains how you can find which emscripten commit goes with which LLVM commit etc. https://github.com/emscripten-core/emscripten/blob/incoming/docs/process.md#minor-version-updates-1xy-to-1xy1

I really can't find out how to see which LLVM commit goes into with which emscripten tag. Mind shedding some light on that?

Apart from that, I had to hack settings_template_readonly.py a bit to make sure it'll work with my custom LLVM for emscripten. I just took the current master HEAD of LLVM. Hopefully it'll work.

@kripken

This comment has been minimized.

Copy link
Member

commented Jun 19, 2019

@svenstaro I opened a PR now with more docs to answer that question, emscripten-core/emscripten#8829 Is that ok, or is there more I should clarify?

@svenstaro

This comment has been minimized.

Copy link
Author

commented Jun 19, 2019

Seems like a great start! There are some details with installation that I'd like clarified. For instance, which files from LLVM do you actually require? Currently I'm installing lld, clang and clang++ as can be seen in the PKGBUILD.

Does that suffice?

@kripken

This comment has been minimized.

Copy link
Member

commented Jun 19, 2019

I believe a few more are needed: wasm-ld (just a symlink for lld I think?), llvm-nm, etc. I added a commit to that PR now with what I hope is the full list.

@svenstaro

This comment has been minimized.

Copy link
Author

commented Jun 21, 2019

From my view, it's now almost sufficient to provide a good user experience as a maintainer. However, I'm also doing this to make sure the paths are correct for a user to start with so that might be a good thing to suggest to other maintainers as well:

sed -i 's|EMSCRIPTEN_ROOT.*|EMSCRIPTEN_ROOT = "/usr/lib/emscripten"|g' tools/settings_template_readonly.py
sed -i 's|LLVM_ROOT.*|LLVM_ROOT = "/usr/lib/emscripten-llvm"|g' tools/settings_template_readonly.py
@svenstaro

This comment has been minimized.

Copy link
Author

commented Jun 23, 2019

By the way, it seems that those binaries are not sufficient. Could you check that out in depth and write down the entire list of binaries required?

@kripken

This comment has been minimized.

Copy link
Member

commented Jun 23, 2019

@svenstaro Thanks, I updated that PR now with llc which was indeed missing, sorry about that (did you see anything else? if you can't tell exactly what's missing, the command that failed for you might help). I also added docs for the .emscripten file for packagers, let me know what you think.

@svenstaro

This comment has been minimized.

Copy link
Author

commented Jun 23, 2019

At least llvm-link and opt are also missing.

@kripken

This comment has been minimized.

Copy link
Member

commented Jun 24, 2019

Oh, those are needed for fastcomp - the docs I wrote were for the wasm backend. I added docs for fastcomp now, thanks.

@svenstaro

This comment has been minimized.

Copy link
Author

commented Jun 25, 2019

My non-fastcomp emcc complained about those binaries missing so I don't know. :P

@kripken

This comment has been minimized.

Copy link
Member

commented Jun 25, 2019

Thanks @svenstaro, I forgot about the sanity warnings - those were wrong. Fix in #8853

kripken added a commit to emscripten-core/emscripten that referenced this issue Jun 26, 2019
@Beuc

This comment has been minimized.

Copy link

commented Aug 19, 2019

Hi. I'm still using fastcomp because I'm relying on emterpreter (I know about the new experimental async but I can't start investigating it right now), so when I'm compiling the latest fastcomp it would help to have up-to-date tags (current 1.38.37 vs. 1.38.41). Would that be possible?

Beuc added a commit to renpy/renpyweb that referenced this issue Aug 19, 2019
@kripken

This comment has been minimized.

Copy link
Member

commented Aug 19, 2019

Sure, I pushed 1.38.39+ now.

Sorry for forgetting - there is literally no code landing there, so all the tags are just the same...

@svenstaro

This comment has been minimized.

Copy link
Author

commented Aug 19, 2019

@Beuc

This comment has been minimized.

Copy link

commented Aug 19, 2019

A bit of rationale: I have a compilation script that ensures that all the tags match, which broke, so I had to hard-code 1.38.37 in a couple places; plus when I saw tags missing I had to spent time checking if the reference repository changed or was already deprecated (which was not the case as it's still emsdk's default). So tags are still useful :)

@kripken

This comment has been minimized.

Copy link
Member

commented Aug 19, 2019

@svenstaro - We probably should, yeah. In practice there isn't a script to tag, just a few manual steps that I do each time, so this isn't an automated process.

@Beuc - Yeah, makes sense, I'll make sure to make tags. I just tagged 1.38.42 and updated all repos with them.

@svenstaro

This comment has been minimized.

Copy link
Author

commented Aug 19, 2019

When will fastcomp be entirely deprecated? The problem will then go away by itself. :P

@kripken

This comment has been minimized.

Copy link
Member

commented Aug 19, 2019

@svenstaro yes, that's the goal :)

No specific date yet for removing fastcomp. First we want to switch the default to upstream, for which we need to fix some blockers. Then we need to get feedback and start to plan fastcomp's removal.

@Beuc

This comment has been minimized.

Copy link

commented Sep 18, 2019

Hi,

Tag 1.38.45 is missing - my colleague got puzzled when upgrading our build environment and supposed the new version wasn't published yet :)

@kripken

This comment has been minimized.

Copy link
Member

commented Sep 18, 2019

Thanks @Beuc, looks like we forgot that tag, fixed now!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.