Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[TODO] [dynlib] Nim doesn't use default dynamic library loader, we need a replacement for standard tools, eg: ldd, otool, LD_PRELOAD, rpath #9203

Closed
timotheecour opened this issue Oct 5, 2018 · 3 comments

Comments

Projects
None yet
3 participants
@timotheecour
Copy link
Contributor

commented Oct 5, 2018

  • what were the reasons why Nim chose to not use the default (platform specific) dynamic library loader?
    was that to ease portability and provide same interface across platforms (especially on windows)?
    was that to allow more flexibility?
    IIRC there was a forum post about it but couldn't find it because of nim-lang/nimforum#180

  • in any case, maybe these are valid reasons, but because of that, we can't use existing infrastructure built around default dynamic library loader, eg:

  • otool -L (OSX) or ldd (linux) are very useful to list the library dependencies of a binary/shared library, eg:
otool -L someBinary
/pathto/app:
        /System/Library/Frameworks/OpenGL.framework/Versions/A/OpenGL (compatibility version 1.0.0, current version 1.0.0)
        /Users/timothee/homebrew/opt/cairo/lib/libcairo.2.dylib (compatibility version 11403.0.0, current version 11403.12.0)
        /Users/timothee/homebrew/opt/ffmpeg/lib/libavutil.56.dylib (compatibility version 56.0.0, current version 56.14.100)
        /Users/timothee/homebrew/opt/ffmpeg/lib/libavfilter.7.dylib (compatibility version 7.0.0, current version 7.16.100)
        ...
        /usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 400.9.0)
        /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1252.50.4)

this won't work in a Nim binary as it'll only list /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1252.50.4) , omitting all the other libraries that are needed to run a Nim program (mentioned via dynlib eg:
proc myExtFunc*() {.importc: "myExtFunc", dynlib: "libfoo.dylib", cdecl.}

we need a replacement for that in Nim: a Nim program (say nimldd) that can obtain the same information from a Nim binary/shared library.
The information could be saved in a file section of the Nim binary/shared library.

note

in discussion above I'm referring to fact that in Nim, instead of -lfoo to link against libfoo, Nim uses {.dynlib: "libfoo.dylib"} which causes genereted C code to call dlopen at program start

@alaviss

This comment has been minimized.

Copy link
Contributor

commented Oct 5, 2018

You can just use --dynlibOverrideAll then pass the link flags yourself.

Possible duplicate of https://github.com/nim-lang/Nim/issues/8655

@Araq

This comment has been minimized.

Copy link
Member

commented Oct 9, 2018

Duplicate of #8655 and all of your tooling is already broken anyway, dlopen is the standard mechanism used by all dynamic languages out there.

@Araq Araq closed this Oct 9, 2018

@timotheecour timotheecour changed the title [dynlib] Nim doesn't use default dynamic library loader, we need a replacement for standard tools, eg: ldd, otool, LD_PRELOAD, rpath [TODO] [dynlib] Nim doesn't use default dynamic library loader, we need a replacement for standard tools, eg: ldd, otool, LD_PRELOAD, rpath Oct 9, 2018

@timotheecour

This comment has been minimized.

Copy link
Contributor Author

commented Jan 12, 2019

EDIT: #8655 was moved to nim-lang/RFCs#58 ; I don't think that RFC is a duplicate unless nim changes the way it registers and loads dynamic libraries (so that ldd/otool-L etc would work out of the box); if it doesn't, than this issue is still valid.

unfortunately otool -L doesn't work for nim binaries because of (#9203 or nim-lang/RFCs#58)

Oh not that again, so use OSX's strace variant.

here's what I tried:

  • OSX's strace variant is dtruss
  • this requires sudo whereas ldd/otool -L don't (eg won't work for users wo sudo access at work for eg)
  • sudo -E maybe be required if your program access environment variables
  • not sure what's unavailable here: dtrace: system integrity protection is on, some features will not be available
import dynlib
proc main()=
  let status = loadLib("libmyplugin.dylib")
  echo status != nil
main()
nim c main.nim
sudo -E dtruss -t open ./main
SYSCALL(args)            = return
true
open("/dev/dtracehelper\0", 0x2, 0xFFFFFFFFE9XXXAE0)             = 3 0
open("libmyplugin.dylib\0", 0x0, 0x0)              = 3 0
open(".\0", 0x0, 0x1)            = 3 0
open(".\0", 0x0, 0x1)            = 3 0

so with a bit of setup we can get to what is loaded at runtime; it's definitely useful although not quite a replacement for otool -L etc (eg with https://github.com/jtanx/lddx we can get the tree of dynamic library dependencies, which we can't with dtruss etc)

note

seems quite doable to add the functionality to binaries (lib/exe) compiled by nim:
something similar to debugger.dbgRegisterGlobal can be done

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.