Passing '--bind' (and thus setting -std=c++11) causes 'unresolved symbol: exp2f' error

[weitjong]

I have a project which built fine using Emscripten compiler. I want to use embind to do some binding works. However, I already encounter a build failure simply by passing --bind compiler flag before actually adding any binding code. My project uses expf function from Box2D and Bullet libs. I cannot find any references to exp2f, so I am not sure why I got this unresolved symbol error.

More interestingly, I do not get this unresolved symbol error when I build my project with Release build configuration (-O3 and -s AGGRESSIVE_VARIABLE_ELIMINATION=1) for both with and without --bind flag. The llvm-nm produces this similar output for both.

$ llvm-nm ../emscripten-Build/lib/libUrho3D.a |grep -P exp.?f
         U exp2f
         U expf

This is the project library which is used when linking the final html.

In Debug build configuration (no optimization flag and -g4), both with and without --bind flag produce library without the exp2f symbol.

$ llvm-nm ../emscripten-Build/lib/libUrho3D.a |grep -P exp.?f
         U expf

However, the build without --bind still ends in a success, perhaps it does not look for exp2f symbol during linking. Only the Debug build with --bind flag fails with the unresolved symbol error.

Is this a normal behavior? Why would Release configuration has more symbols than Debug configuration in this particular case? And why would --bind (or -std=c++11) cause the linker to look for different symbol?

[kripken]

cc @chadaustin

[chadaustin]

That's weird. I have no explanation for this behavior. cc @waywardmonkeys Bruce: any ideas?

[juj]

I do not know why expf and exp2f would go missing - expf is available from
JS Math.exp function at
https://github.com/kripken/emscripten/blob/incoming/src/library.js#L3977
and exp2f is available from musl via
https://github.com/kripken/emscripten/blob/incoming/system/lib/libc/musl/src/math/exp2f.c
.

However it is normal behavior that in debug mode there might be more
symbols present compared to release mode. This is because for example the
following code

float x = expf(42.f);

computes something that is effectively a compilation time constant
operation, so when optimizations are enabled, LLVM is eager to precompute
the results during the compilation step, and directly expand the results of
those computations in the output. That means that in release mode the
dependency to that function might go missing.

In particular the reason why --bind affects this, my guess is that since
--bind has a hidden behavior of setting -std=c++11, this triggers a mode in
LLVM that allows it to do that optimization, and in older C++ compilation
models it won't do it.

Assuming that is what is happening, the real question is then why expf and
exp2f symbols would not be found. At least compiling a simple test app of
form

#include <math.h>
#include <stdio.h>
#include <stdlib.h>

int main()
{
float e = expf(rand());
float e2 = exp2f(rand());
printf("%f %f\n", e, e2);
}

does properly find the functions and succeed. If you are able to tone it
down to a small test case, that would be helpful.

2015-03-15 21:08 GMT+02:00 Chad Austin notifications@github.com:

That's weird. I have no explanation for this behavior. cc @waywardmonkeys
https://github.com/waywardmonkeys Bruce: any ideas?

—
Reply to this email directly or view it on GitHub
#3259 (comment).

[weitjong]

Thanks all for your fast reply.

@juj, I think you misunderstood me. I agree with you that both math functions are available during compile time. It is just that exp2f is never referenced explicitly in my project but yet the symbol ends up in the intermediate library archive *.a.

My intuition also tells me that in debug mode there might be more symbols present than in release mode. However, what I observed here is exactly the other way round, which is interesting.

[chadaustin]

If you look in the generated code, can you see where it's being called or referenced?

[weitjong]

Unfortunately no. In debug mode with --bind, the build failed so there is no generated code to analyze. I have performed a thorough scan and I am quite confident that my project code base does not reference exp2f. The command git grep exp2f on my project root gave no output. I then did a search using this command find emsdk_portable -type f -exec grep exp2f {} /dev/null \; |grep -v Binary on Emscripten SDK and find a number of references there in the 'emscripten' and 'clang/fastcomp' subdirs. Not sure how this is related to my issue though.

Although in release mode with --bind flag the build was successful and produced generated code, however, I don't see how I can read the generated JS file in release mode or to verify whether it uses exp2f or not.

More frustratingly I was not able to reproduce the issue with a much simplified code base, such as a simple test lib that just calls expf() and then a simple main.cpp linking with that simple test lib. In this simplified case, I was not able to produce an archive *.a containing the exp2f symbol no matter how hard I try to combine the Debug|Release + with|without --bind flags. But I can definitely see the expf symbol in the llvm-nm output for the test lib as expected.

Any other idea how to proceed to troubleshoot this? Thanks.

[waywardmonkeys]

I haven't found anything that would cause this.

One thing that might help is to look at your intermediate files and see when / where the exp2f call is getting inserted.

Is it in the LLVM IR? At which point does it enter the JS?

Using EMCC_DEBUG might help with that (by keeping a copy of the intermediate files in /tmp/emscripten_temp (or /private/tmp/emscripten_temp).

I think that, given your llvm-nm output above, it might be happening LLVM-side ... and if that's the case, debugging it is a bit more difficult, but still possible.

[waywardmonkeys]

Also, is this problem something that can be reproduced with a public branch of your Urho3D engine or is it in code that you just have locally?

[weitjong]

Thanks for the hint with EMCC_DEBUG, finally I see some light! I have also switched to use RelWithDebugInfo build configuration which has the same issue as Debug build configuration but minus the -s ERROR_ON_UNDEFINED_SYMBOLS=1, so the compiled-code is generated despite the missing exp2f symbol.

Based on the generated JS file (in the output dir or in the emscripten_temp dir), I am almost certain now that the exp2f reference actually came from powf() function in this native C++ code. https://github.com/urho3d/Urho3D/blob/master/Source/Urho3D/Scene/Scene.cpp#L629

With the same RelWithDebugInfo build configuration but without the --bind (or -std=c++11), the generated JS file uses Math_pow instead of exp2f. So, this explains why the build was successful even though the exp2f symbol is not present in the lib archive.

As reported earlier, in Release build configuration the exp2f symbol is always present in the lib archive whether with or without --bind flag. So, it does not matter at the final linking phase whether exp2f is used or not because the symbol is available in the libUrho3D.a.

In conclusion, with --bind flag the exp2f is indeed required. But why it does not present in Debug build configuration when the lib archive is being built?

[weitjong]

Also, is this problem something that can be reproduced with a public branch of your Urho3D engine or is it in code that you just have locally?

This problem cannot be reproduced with our public https://github.com/urho3d/Urho3D master branch without some manual change at the moment. The embind binding work is only available in my local workspace. You could, however, clone from the public master branch and use CMake to generate an Emscripten build tree then carefully inserting the --bind flag manually to the CXX compiler flags in the CMake-generated project file.

[weitjong]

Good news. I think I have narrowed down the root cause to a line in the emcc script. https://github.com/kripken/emscripten/blob/master/emcc#L693. Commenting out the offending line fixes my build failure when using Debug configuration. Without the LTO enabled, the exp2f is not emitted while linking my target html against the debug library archive which also does not have the exp2f emitted.

IMHO, to permanently fix this issue then the conditional check in the emcc script needs to be revised so that it only enables LTO when there is no optimization flag or -O0 is in effect. That is, if no optimization is requested (such as in Debug build configuration for my particular case) then Link Time Optimization is also off consistently.

[waywardmonkeys]

I wonder if that "pointer issue" is still around ... the commit comment for that change doesn't yield many clues:

commit 681ab3e9157f15c8276695413648b2ba9775894b
Author: Alon Zakai <alonzakai@gmail.com>
Date:   Thu Jun 27 11:26:46 2013 -0700

    always use lto in embind, workaround for a pointer issue

[waywardmonkeys]

@weitjong I've submitted a pull request (#3264) that just removes the auto-enabling of LTO when --bind is passed.

It shouldn't be needed and is a strange special case that is apparently there for an old, long-dead bug.

Doing so broke one of the tests, so I fixed that there as well.