Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

-s USE_XXX should accept -lXXX #8650

Closed
Beuc opened this issue May 22, 2019 · 22 comments
Closed

-s USE_XXX should accept -lXXX #8650

Beuc opened this issue May 22, 2019 · 22 comments

Comments

@Beuc
Copy link
Contributor

@Beuc Beuc commented May 22, 2019

Hi,

I get a lot of errors like in my builds with 1.38.32 such as:

emcc -s USE_ZLIB=1 -o python Modules/python.o libpython2.7.a -ldl -lz -lm  
shared:ERROR: emcc: cannot find library "z" (`-s ERROR_ON_MISSING_LIBRARIES=0` to disable this error)

This is quite puzzling.
It would make sense for emcc to accept -lXXX when using a built-in port -s USE_XXX.

@sbc100
Copy link
Collaborator

@sbc100 sbc100 commented May 23, 2019

The -s USE_FOO flags for libraries implicitly add the include paths and link flags that you need in order to use the library. You don't need to add additional flags. They would be definition be redundant. I don't love the USE_FOO system for including ports, but that is how it currently works.

I recently re-worked the ports so at least they have the correct names (libfoo.a).. but the name of the actual library produced by the port is still currently and implementation default and passing -lz implies that you know that filename is libz.a under the hood somewhere.

I kind of agree with your though, it would be nice some day to be able to consume ports simply by passing -lz. I imagine this being separate to the -s USE_FOO flags though. We could have embuilder be the way to build the libraries and have -lz be the way to consume them. Currently we have USE_ZLIB which means "build and consume".

@sbc100
Copy link
Collaborator

@sbc100 sbc100 commented May 23, 2019

Are you saying that it used to work to pass -lz? Perhaps it was just failing silently prior to #8461?

@Beuc
Copy link
Contributor Author

@Beuc Beuc commented May 23, 2019

From a porter PoV, I ./configure the target project with appropriate CFLAGS/LDFLAGS, which include -L and -s XXX options. The target project's build system will then expect my configuration to satisfy its link dependencies, which it unconditionally specifies as -lXXX.

That's what I think it makes sense for -s USE_XXX to provide a high-precedence equivalent to -L/hidden-emscripten-path -lXXX. Otherwise I have to patch the target build system, which would be emscripten-specific, time consuming and error-prone.

The issue is indeed all the more present with #8461 - as emcc used to warn in the past, and now fails. I can pass -s ERROR_ON_MISSING_LIBRARIES=0 everywhere but that sounds like a work-around and not a proper fix.

@sbc100
Copy link
Collaborator

@sbc100 sbc100 commented May 23, 2019

Yes, its seems that this has always been an issue just hidden by the fact that ERROR_ON_MISSING_LIBRARIES used default to false. I suggest you add ERROR_ON_MISSING_LIBRARIES=0 to get the old behaviour until we can come up with a more elegant solution.

@Beuc
Copy link
Contributor Author

@Beuc Beuc commented Aug 25, 2019

ERROR_ON_MISSING_LIBRARIES=0 is not not available anymore - any progress on the more elegant solution?
I'm starting to implement silly work-arounds like creating dummy libXXX.a so the build system can work, which is probably a bad sign ;)

@kripken
Copy link
Member

@kripken kripken commented Aug 26, 2019

Looks like ERROR_ON_MISSING_LIBRARIES was removed in #9223.

@sbc100 maybe we removed it too early? The workarounds @Beuc describes do sound bad :(

@sbc100
Copy link
Collaborator

@sbc100 sbc100 commented Aug 26, 2019

No I still don't think ERROR_ON_MISSING_LIBRARIES is the right solution here.

@Beuc, why can't you remove -lz to you link flags? That seems like the correct solution for now.

@sbc100
Copy link
Collaborator

@sbc100 sbc100 commented Aug 26, 2019

In the long run it would be nice to make -l work with emscripten ports. Today -s USE_ZLIB is the way to use the port. I'll leave this bug open so we get it fixed, but simply removing the offending argument for the link line seems like a good enough workaround until this is fixed.

@Beuc
Copy link
Contributor Author

@Beuc Beuc commented Aug 27, 2019

@Beuc, why can't you remove -lz to you link flags?

That's because, as I explained above, this is not a trivial project, but I'm interacting with moderately complex build systems, this concerns -lz but also -lSDL2* and -lfreetype, and it's more of a work-around to patch-up the various build systems than it is to make dummy .a libs.

@Beuc
Copy link
Contributor Author

@Beuc Beuc commented Sep 4, 2019

By the way, my idea of an elegant solution would be a simple include/ & lib/ hierarchy where ports would be installed (or symlinked/copied on demand). Something like https://mxe.cc/ does for cross-compiling to windows. It would silently add -I and -Lflags and so would work as the rest of my dependencies, could be easily overridden with a patched/in-progress port, etc.
For the record :)

@sbc100
Copy link
Collaborator

@sbc100 sbc100 commented Sep 4, 2019

I think we are already moving in exactly that direction: #9353. Hopefully we will get there sooner rather than later.

@sbc100
Copy link
Collaborator

@sbc100 sbc100 commented Sep 4, 2019

There are two problems emscripten faces that not all crosstools face:

  1. How to handle the case of a read-only emscripten install (i think in this case we have to fall back to some kind $HOME-based solution for usr/local).
  2. How do the auto-built ports interact with user-built ports (we never want to clobber things the user has installed by hand). We can hopefully handle this by having the ports system target an non-obvious location (e.g. usr/ports/lib rather than just usr/lib).

@Beuc
Copy link
Contributor Author

@Beuc Beuc commented Sep 4, 2019

Currently the ports do require write access to build and install the port, I wouldn't change that. Similarly MXC builds in $(checkout-path)/usr/bin. It doesn't matter to me if the ports are installed in a specific location, as a porter/cross-compiler I'm used to that. If we want to enable/disable them on demand the ports would probably reside not in Emscripten's global sysroot, but in a throw-away root (though it seems you may not want to keep that).

Your second point makes rethink my workflow the other way around: currently my root issue is I'm stuck with the SDL2 port because the official SDL2 version sadly lags behind (I used to install the official SDL2 in my build root, until it became incompatible with the current emscripten).
So if I can build an Emscripten port, with the same (ad-hoc) build-system from tools/ports/xxx.py, from a separate command line, but installing in my build root, that would suit my needs. (I want to keep the same build instructions so I can patch and contribute to the port. If the ports switch back to ./configure&&make that works too.)

@hikari-no-yume
Copy link

@hikari-no-yume hikari-no-yume commented Oct 15, 2019

This seems to be causing me trouble with Rust. I want to use a Rust SDL2 wrapper crate (see https://puddleofcode.com/story/definitive-guide-to-rust-sdl2-and-emscriptem, I'm roughly copying that guide), but it causes -lSDL2 to be passed to emcc and so building always fails with = note: shared:ERROR: emcc: cannot find library "SDL2", regardless of whether I have EMMAKEN_CFLAGS="-s USE_SDL=2" set 😢

Of course, that crate has not been designed with Emscripten in mind, so the solution may just be to modify that crate (I may well do so at some point!). But it is a shame that Emscripten can't “just work” here.

@brion
Copy link
Collaborator

@brion brion commented Oct 15, 2019

As a workaround, you should be able to create an empty (or mostly-empty) libSDL2.a of your own and stick it in the library path. This'll get linked in harmlessly while the -s USE_SDL=2 links in the real one... I think. :)

@hikari-no-yume
Copy link

@hikari-no-yume hikari-no-yume commented Oct 15, 2019

@brion Hey, I think that worked (— at least, it made it compile!), thanks so much for the suggestion!

@tlively
Copy link
Member

@tlively tlively commented Oct 16, 2019

@hikari-no-yume I'd like to play around with this and see if I can improve this experience. Can you share your project or describe how to reproduce the issue?

@hikari-no-yume
Copy link

@hikari-no-yume hikari-no-yume commented Oct 16, 2019

@tlively Unfortunately I am not able to share the code right now (much as I'd like to), but I was roughly following this guide: https://puddleofcode.com/story/definitive-guide-to-rust-sdl2-and-emscriptem

Basically, use the Rust sdl2 crate and try building with Cargo for wasm32-unknown-emscripten (with EMMAKEN_CFLAGS='-s USE_SDL=2) and it should fail because that crate currently makes Cargo pass -lSDL2 to emcc. This is a problem with the crate of course (will probably eventually fix it myself if I can), but there's an opportunity here for emscripten to be helpful and just ignore the flag.

@Beuc
Copy link
Contributor Author

@Beuc Beuc commented Mar 17, 2020

It looks like #9983 fixes this issue!

@sbc100
Copy link
Collaborator

@sbc100 sbc100 commented Mar 17, 2020

Yup! Thanks.

@sbc100 sbc100 closed this Mar 17, 2020
@hikari-no-yume
Copy link

@hikari-no-yume hikari-no-yume commented Mar 17, 2020

Oh, cool, thank you! But why does it fix it?

@sbc100
Copy link
Collaborator

@sbc100 sbc100 commented Mar 17, 2020

Because it puts the directory where the ports get installed to on the library path and include path.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
6 participants