Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

error: undefined symbol: htons / ntohs / rand due to deps_info.json not seeing files from .a's #9566

Closed
Beuc opened this issue Oct 2, 2019 · 8 comments · Fixed by #9602

Comments

@Beuc
Copy link
Contributor

commented Oct 2, 2019

Hi,

In the final link of my RenPyWeb project and its ~10 dependencies, since upgrading 1.38.46 I now get:

$ emcc ... *.o ... -o index.html
error: undefined symbol: htons
warning: To disable errors for undefined symbols use `-s ERROR_ON_UNDEFINED_SYMBOLS=0`
error: undefined symbol: ntohs
error: undefined symbol: rand
Error: Aborting compilation due to previous errors

This reminds me of #9154 though that one remains fixed.

Let me know how I can further diagnose the issue.

EDIT: reported by compiler.js:

shared:ERROR: '.../emsdk/node/12.9.1_64bit/bin/node .../emsdk/upstream/emscripten-git/src/compiler.js /tmp/tmpcnboo9.txt .../emsdk/upstream/emscripten-git/src/library_pthread_stub.js' failed (1)
@Beuc

This comment has been minimized.

Copy link
Contributor Author

commented Oct 2, 2019

In addition, passing -s ERROR_ON_UNDEFINED_SYMBOLS=0 triggers ReferenceError: __get_timezone is not defined at runtime (not sure it's related).

@kripken

This comment has been minimized.

Copy link
Member

commented Oct 2, 2019

If it's hard to reduce this, if you can provide the final emcc link command + input files, that should be enough for us to debug it.

@Beuc

This comment has been minimized.

Copy link
Contributor Author

commented Oct 2, 2019

Here's another simpler one (Python web prompt) which seems related:

+ emcc -o t/webprompt.html webprompt-main.c t/emscripten.c -I/home/me/workdir/emtests/renpyweb-asyncify/python-emscripten/2.7.10/destdir/include/python2.7 -L/home/me/workdir/emtests/renpyweb-asyncify/python-emscripten/2.7.10/destdir/lib -lpython2.7 -s EMULATE_FUNCTION_POINTER_CASTS=1 -s USE_ZLIB=1 -s TOTAL_MEMORY=256MB --preload-file /home/me/workdir/emtests/renpyweb-asyncify/python-emscripten/2.7.10/destdir/lib/python2.7@/lib/python2.7 --shell-file webprompt-shell.html -s 'EXPORTED_FUNCTIONS=['\''_main'\'', '\''_malloc'\'', '\''_Py_Initialize'\'', '\''_PyRun_SimpleString'\'']' -s 'EXTRA_EXPORTED_RUNTIME_METHODS=["ccall", "cwrap"]'
wasm-ld: error: /home/me/workdir/emtests/renpyweb-asyncify/python-emscripten/2.7.10/destdir/lib/libpython2.7.a(posixmodule.o): undefined symbol: environ

Do you want a tar archive?

@kripken

This comment has been minimized.

Copy link
Member

commented Oct 2, 2019

Sure, a tar is ok.

@Beuc

This comment has been minimized.

Copy link
Contributor Author

commented Oct 2, 2019

There it is (lighter than my full ~2GB RenPyWeb build environment...)

emcc -o t/webprompt.html webprompt-main.c t/emscripten.c -Ipython2.7 -L. -lpython2.7 -s EMULATE_FUNCTION_POINTER_CASTS=1 -s USE_ZLIB=1 -s TOTAL_MEMORY=256MB -s 'EXPORTED_FUNCTIONS=['\''_main'\'', '\''_malloc'\'', '\''_Py_Initialize'\'', '\''_PyRun_SimpleString'\'']' -s 'EXTRA_EXPORTED_RUNTIME_METHODS=["ccall", "cwrap"]'

https://www.beuc.net/tmp/link.tgz

There's a difference though, the OP is reported in compiler.js, this one in wasm-ld.

@kripken

This comment has been minimized.

Copy link
Member

commented Oct 2, 2019

Thanks!

Ok, I think I understand the underlying problem. Here is a reduced testcase:

  # in tests/test_other.py
  def test_gethostbyname_in_a(self):
    # handle singleton archives
    run_process([PYTHON, EMCC, path_from_root('tests', 'sockets', 'test_gethostbyname.c'), '-o', 'a.o'])
    run_process([LLVM_AR, 'r', 'liba.a', 'a.o'])
    create_test_file('empty.c', 'static int foo = 0;')
    run_process([PYTHON, EMCC, 'empty.c', '-la', '-L.'])
    self.assertContained('success', run_js('a.out.js'))

The issue is that we use deps_info.json to handle linking in C code that JS will need, and we do implement the sockets code in JS, which ends up wanting htons etc. This normally works, but I bisected to #9436, and from that point on we are broken. I didn't look into that PR's code but looking on incoming, the issue is that for deps_info.json purposes we only look at the immediately compiled files... and not files in a library that is linked in. So we fail to see we will need things. My guess is that that PR changed which files get looked at there.

@kripken kripken changed the title error: undefined symbol: htons / ntohs / rand error: undefined symbol: htons / ntohs / rand due to deps_info.json not seeing files from .a's Oct 2, 2019
@kripken kripken added this to Blocker in LLVM Upstream Backend Oct 2, 2019
@sbc100

This comment has been minimized.

Copy link
Collaborator

commented Oct 8, 2019

@kripken are you looking into this already? Want me to take a look?

@kripken

This comment has been minimized.

Copy link
Member

commented Oct 8, 2019

@sbc100 I'm not looking into it, no - was hoping you'd have an idea from the investigation I did + your better understanding of the PR where this broke? Would be great if you can take a look!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
3 participants
You can’t perform that action at this time.