-
Notifications
You must be signed in to change notification settings - Fork 4.7k
-
Notifications
You must be signed in to change notification settings - Fork 4.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
lldb-3.9 cannot load libsosplugin.so #8292
Comments
@micli that's expected. The lldb plugin that's present in the package is for lldb 3.6 and it cannot work with other versions since it is statically linked to the lldb libraries of that specific version. You'll need to build coreclr yourself with lldb-3.9 dev package installed to get plugin that works with that. |
@janvorli, I actually get the same error with lldb-3.6 on my Ubuntu 16.10 VM. |
I have looked into that and the issue is that the libsosplugin.so in our portable package is not portable itself due to the different SONAME of liblldb.so on Fedora based distros and others. Since out portable build is built on Fedora based distro, the plugin cannot resolve the dependency. |
Assigning to @mikem8361 who owns SOS. CC @lt72 |
It seems it is actually even more complicated. The SONAME of the liblldb on Fedora 25 is liblldb.so.3.9.1 while on CentOS 7 it is just liblldb.so and on ubuntu 14.04 it is liblldb-3.9.so.1, so it looks like there are possibly many variants of the SONAME. |
I actually think that the best solution would be to stop packaging the libsosplugin.so in the dotnet core runtime / sdk and provide it in a separate package or tarball for each [DISTRO, LLDB_VERSION] path. This plugin is independent of the coreclr version, so we could basically build it once, publish it and forget. The build of that plugin would not have to happen on our build machines, any time a new distro / lldb version needs to be supported, we could do a one-off build of that in a docker container for that distro and publish that. |
It is true that the libsosplugin.so does really change that much and the interface between it and sos should always be backward/forward compatible. Putting it in a separate distro specific package/zip/etc. will make using SOS a lot harder. I understand that the "portable" build will require us to do something like this but it just adds to overhead of getting SOS working. |
I thought the portable builds will built on RHEL. On Ubuntu 14.04, the lldb 3.9 install does create links for "liblldb.so" to "liblldb-3.9.so.1", etc. I'll have to investigate all the other distros to see if they all have "liblldb.so". If so, then maybe we can force the portable build to find/use "liblldb.so" SONAME to build libsosplugin.so against instead of "liblldb.3.9.so.1". Jan can you look at your Fedora and any other distro you have handy? Building with liblldb.so might just be getting the CMakeLists.txt logic right. |
@mikem8361 the loader doesn't care about the filenames. It uses the SONAME stored in the libraries. So for example, if you have libssl.so.1.0.2 which has SONAME libssl.so.1.0.0 and your binary references libssl.so.1.0.0, then it loads the file libssl.so.1.0.2.
|
Building for each distro and packaging libsosplugin separately may be the only choice but it is going to be a big hassle. This would mean we still have to build the coreclr repo for each distro (the build system could be changed to just build the sos plugin). The sos plugin and sos have dependencies on each (a COM like interface) that would make it inconvenient to move to a separate repo. And we want to try to do this for 2.0.0? I don't think so. |
@mikem8361 I don't think we can just leave the plugin broken. Currently, it doesn't work on anything else than the Centos 7 / RHEL 7 where we build the portable build. So we need to do something about it. I can see the following options:
Actually, now that I think about it, the last described approach might give us a libsosplugin.so that would be independent of the lldb version, which would be really cool. But that really depends on the stability of the liblldb interface over the lldb versions. |
I agree this needs to be fixed - and building it for various distros (other than RHEL7 where portable build happens) would require bringing back distro specific builds that we intentionally removed. @mikem8361 Can you please look into determining what is the feasibility of building it portable and if not, how should this be supported? Unless the work required is minimal (I doubt it), this will need to be done post 2.0. For 2.0, I would suggest we build these manually for the key distros and make them available. |
Removing the explicit reference to liblldb. Since the lldb program has already loaded this lib, our will now load regardless of the distro and version of lldb. Issue #12098.
I need to do some more testing but I have a simple fix for this that removes the explicit reference to the troublesome liblldb* (simple CMakeList.txt change). Because lldb itself has the correct liblldb already loaded this works. So far it works on Ubuntu 14.04 with lldb 3.9 and 3.8 on the machine the libsosplugin was built on and my Centos VM under lldb 3.8. |
Removing the explicit reference to liblldb. Since the lldb program has already loaded this lib, our will now load regardless of the distro. Issue #12098.
* Fix portable build sos plugin problems. Removing the explicit reference to liblldb. Since the lldb program has already loaded this lib, our will now load regardless of the distro and version of lldb. Issue #12098. * Fix OSX build.
Removing the explicit reference to liblldb. Since the lldb program has already loaded this lib, our will now load regardless of the distro. Issue #12098.
Fixed in master and 2.0.0. |
From the user perspective, how can I know the version of lldb that should be installed on the system to be compatible with the libsosplugin.so?
|
@raffaeler CentOS was really let me sad that a lot of default packages include LLDB and CMake are out of date. A way to install LLDB 3.6 on CentOS is download source code and compile by yourself. I cannot understand that Be a mirror of commercial copy of Red Hat, how could it be like this? I can share you some commands to compile LLVM, Clang and LLDB 3.6/3.9 on your CentOS. Compile and install CMake 3.8 first. [centos-linux-7 ~]$ wget Get LLVM source code [centos-linux-7 ~]$ wget http://llvm.org/releases/3.6.0/llvm-3.6.0.src.tar.xz Get Clang [centos-linux-7 ~]$ cd llvm/tools Get LLDB [centos-linux-7 tools]$ wget http://releases.llvm.org/3.6.0/lldb-3.6.0.src.tar.xz Compile and Install [centos-linux-7 llvm]$ mkdir llvmbld After a long time waiting(maybe some hours), you will get your LLDB 3.6 environment. Please notice that, you'd better have more than 4GB memory if you want to compile LLDB 3.9 in same way, or you will get some linking error at 64% and 78% on code compiling. |
@micli wow, a lot of work then. |
Hi! Trying to use lldb for netcoreapp2.0 debugging on odroid-xu3 device (linux, armv7). Which version of lldb should I use? The problem is when Ive tried to use 3.9 got a segfault, when Ive tried to use 3.6 got a message: |
If you are using version 2.0.x of coreclr/.NET Core, then the libsosplugin.so will only work/load with lldb 3.6. If you are using/building 2.1 of coreclr (from the master branch) or one of the 2.1.0-preview's, then it you need to use lldb 3.9. What command exactly are you using to load sos? libsos.so is loaded by libsosplugin.so which should be loaded with the "plugin load libsosplugin.so" command? If that is how you loaded sos, then what sos command did you execute to get the above error? It is very strange and doesn't make a lot of sense. |
@mikem8361 I'm using netcore 2.0.2 Here is how I'm running lldb with a dump from my app:
|
Can you zip up your “MyApp” directory and post it to this issue? It looks like it has the core dump and the 2.0.2 runtime binaries like libsos.so and libmscordaccore.so.
|
@mikem8361 here is my app: 7z archive on dropbox |
Very easy way to reproduce netcore 2.0.3 on Stretch. Application is not needed.
|
@sherlock1982 3.9? version 3.9 works only from 2.1 on. Compiling lldb3.6 on CentOS failed, but this is another problem. |
Yes, as I mentioned before. Net Core libsosplugin.so supports LLDB 3.9 start from 2.1.x,not 2.0.3. |
@sherlock1982 in my case I'm trying to find out why my ASP.NET core app fails with segmentation fault on linux-arm (while simple console app works ok) and why I can't use sos commands. So it differs from your example. |
@Happi-cat I had segmentation faults in the past (unrelated to lldb or sos) on ARM because of wrong packages. Also verify the bitness (arm32 vs arm64). |
@raffaeler thanks for advice, but there are few odd things I've faced:
|
@Happi-cat you are welcome. In my experience, I had a lot of trouble on ARM cards that use an old-fashioned style development stack like Yocto. The libraries they use are typically old (ehm ... consolidated). |
After allocation objects in the large object heap needs to be published for some cleanups related to the background gc. The constant value for this limit (RH_LARGE_OBJECT_SIZE) was not loaded correctly in the register. This caused that the upper 32 bits of the register were in an undefined state. Therefore the check for large objects did practically always fail and the objects were never published. Therefore the cleanup never happened and the background GC did fail.
Environment
dotnet --info
.NET Command Line Tools (2.0.0-preview1-005867)
Product Information:
Version: 2.0.0-preview1-005867
Commit SHA-1 hash: 37d826d763
Runtime Environment:
OS Name: debian
OS Version: 8
OS Platform: Linux
RID: debian.8-x64
Base Path: /opt/dotnet/sdk/2.0.0-preview1-005867/
Microsoft .NET Core Shared Framework Host
Version : 2.0.0-preview1-002091-00
Build : fa62d01
lldb-3.9 which was installed from apt.llvm.org
Repo steps
using offical .NET Core 2.0 Preview
cd /opt/dotnet/shared/Microsoft.NETCore.App/2.0.0-preview1-002091-00
lldb-3.9
(lldb) plugin load libsosplugin.so
error: this file does not represent a loadable dylib
using custom build .NET Core 2.0 source code clone from github.com/dotnet/coreclr
cd /home/micl/dotnet/coreclr/bin/Product/Linux.x64.Debug
lldb-3.9
(lldb) plugin load libsosplugin.so
Segmentation fault
Neither of them was not loaded correctly.
The text was updated successfully, but these errors were encountered: