Skip to content
This repository has been archived by the owner on Dec 18, 2018. It is now read-only.

Issues running Kestrel on Linux: System.DllNotFoundException: libdl #78

Closed
danroth27 opened this issue Mar 9, 2015 · 28 comments
Closed
Assignees
Milestone

Comments

@danroth27
Copy link
Member

Looks like folks are having issues running Kestrel on Linux: dotnet/aspnetcore#172

@alexwahl
Copy link

Confirming that.. i am not able to run kestrel on debian linux. Libuv is present in version v1.0.0-rc1 but still i get an exception when running kestrel :

Mono: DllImport unable to load library 'libapi-ms-win-core-file-l1-2-0.dll: Kann die Shared-Object-Datei nicht öffnen: Datei oder Verzeichnis nicht gefunden'.
Mono: DllImport unable to load library 'libapi-ms-win-core-file-l1-2-0.dll: Kann die Shared-Object-Datei nicht öffnen: Datei oder Verzeichnis nicht gefunden'.
Stacktrace:

  at <unknown> <0xffffffff>
  at (wrapper managed-to-native) Microsoft.AspNet.Server.Kestrel.Networking.PlatformApis/LinuxApis.dlopen (string,int) <0xffffffff>
  at Microsoft.AspNet.Server.Kestrel.Networking.PlatformApis/LinuxApis.LoadLibrary (string) <0x0001b>
  at Microsoft.AspNet.Server.Kestrel.Networking.Libuv.Load (string) <0x0002e>
  at Microsoft.AspNet.Server.Kestrel.KestrelEngine..ctor (Microsoft.Framework.Runtime.ILibraryManager) <0x003db>
  at Kestrel.ServerFactory.Start (Microsoft.AspNet.Builder.IServerInformation,System.Func`2<Microsoft.AspNet.FeatureModel.IFeatureCollection, System.Threading.Tasks.Task>) <0x00107>
  at Microsoft.AspNet.Hosting.HostingEngine.Start (Microsoft.AspNet.Hosting.HostingContext) <0x001a8>
  at Microsoft.AspNet.Hosting.Program.Main (string[]) <0x00333>
  at (wrapper runtime-invoke) <Module>.runtime_invoke_void__this___object (object,intptr,intptr,intptr) <0xffffffff>
  at <unknown> <0xffffffff>
  at (wrapper managed-to-native) System.Reflection.MonoMethod.InternalInvoke (System.Reflection.MonoMethod,object,object[],System.Exception&) <0xffffffff>
  at System.Reflection.MonoMethod.Invoke (object,System.Reflection.BindingFlags,System.Reflection.Binder,object[],System.Globalization.CultureInfo) <0x000ca>
  at System.Reflection.MethodBase.Invoke (object,object[]) <0x00046>
  at Microsoft.Framework.Runtime.Common.EntryPointExecutor.Execute (System.Reflection.Assembly,string[],System.IServiceProvider) <0x000f7>
  at Microsoft.Framework.ApplicationHost.Program.ExecuteMain (Microsoft.Framework.Runtime.DefaultHost,string,string[]) <0x004af>
  at Microsoft.Framework.ApplicationHost.Program.Main (string[]) <0x00347>
  at (wrapper runtime-invoke) <Module>.runtime_invoke_object__this___object (object,intptr,intptr,intptr) <0xffffffff>
  at <unknown> <0xffffffff>
  at (wrapper managed-to-native) System.Reflection.MonoMethod.InternalInvoke (System.Reflection.MonoMethod,object,object[],System.Exception&) <0xffffffff>
  at System.Reflection.MonoMethod.Invoke (object,System.Reflection.BindingFlags,System.Reflection.Binder,object[],System.Globalization.CultureInfo) <0x000ca>
  at System.Reflection.MethodBase.Invoke (object,object[]) <0x00046>
  at Microsoft.Framework.Runtime.Common.EntryPointExecutor.Execute (System.Reflection.Assembly,string[],System.IServiceProvider) <0x000f7>
  at dnx.host.Bootstrapper.RunAsync (System.Collections.Generic.List`1<string>) <0x00393>
  at dnx.host.RuntimeBootstrapper.ExecuteAsync (string[]) <0x00633>
  at dnx.host.RuntimeBootstrapper.Execute (string[]) <0x0004f>
  at EntryPoint.Main (string[]) <0x00247>
  at (wrapper runtime-invoke) <Module>.runtime_invoke_int_object (object,intptr,intptr,intptr) <0xffffffff>

Native stacktrace:

        mono() [0x80fcaa3]
        mono() [0x814b23c]
        mono() [0x806f30f]
        [0xb77d840c]
        /lib/ld-linux.so.2(+0xe9f6) [0xb77e79f6]
        /lib/ld-linux.so.2(+0x11ba8) [0xb77eaba8]
        /usr/lib/i386-linux-gnu/libdl.so(+0xc2b) [0xb3cd5c2b]
        /lib/ld-linux.so.2(+0xdde6) [0xb77e6de6]
        /usr/lib/i386-linux-gnu/libdl.so(+0x10bc) [0xb3cd60bc]
        /usr/lib/i386-linux-gnu/libdl.so(dlopen+0x41) [0xb3cd5b61]
        [0xb3e0dcac]
        [0xb3e0dc34]
        [0xb3e0d8bf]
        [0xb3e0d17c]
        [0xb3e0cb10]
        [0xb5633a29]
        [0xb57da4ac]
        [0xb57da14f]
        mono() [0x8073be1]

Debug info from gdb:


=================================================================
Got a SIGSEGV while executing native code. This usually indicates
a fatal error in the mono runtime or one of the native libraries
used by your application.
=================================================================

any helping ideas?

@acastaner
Copy link

I'm running into a similar issue on CentOS 6.5 with Mono 3.12.0:

[root@localhost HelloWeb]# 
[root@localhost HelloWeb]# dnx . kestrel
System.InvalidOperationException: Unable to load libuv. Make sure libuv is installed and available as libuv.so.1
  at Microsoft.AspNet.Server.Kestrel.Networking.Libuv.Load (System.String dllToLoad) [0x00000] in <filename unknown>:0 
  at Microsoft.AspNet.Server.Kestrel.KestrelEngine..ctor (ILibraryManager libraryManager) [0x00000] in <filename unknown>:0 
  at Kestrel.ServerFactory.Start (IServerInformation serverInformation, System.Func`2 application) [0x00000] in <filename unknown>:0 
  at Microsoft.AspNet.Hosting.HostingEngine.Start () [0x00000] in <filename unknown>:0 
  at Microsoft.AspNet.Hosting.Program.Main (System.String[] args) [0x00000] in <filename unknown>:0 
  at (wrapper managed-to-native) System.Reflection.MonoMethod:InternalInvoke (System.Reflection.MonoMethod,object,object[],System.Exception&)
  at System.Reflection.MonoMethod.Invoke (System.Object obj, BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[] parameters, System.Globalization.CultureInfo culture) [0x00000] in <filename unknown>:0 
--- End of stack trace from previous location where exception was thrown ---
  at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw () [0x00000] in <filename unknown>:0 
  at Microsoft.Framework.Runtime.Common.EntryPointExecutor.Execute (System.Reflection.Assembly assembly, System.String[] args, IServiceProvider serviceProvider) [0x00000] in <filename unknown>:0 
  at Microsoft.Framework.ApplicationHost.Program.ExecuteMain (Microsoft.Framework.Runtime.DefaultHost host, System.String applicationName, System.String[] args) [0x00000] in <filename unknown>:0 
  at Microsoft.Framework.ApplicationHost.Program.Main (System.String[] args) [0x00000] in <filename unknown>:0 
--- End of stack trace from previous location where exception was thrown ---
  at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw () [0x00000] in <filename unknown>:0 
  at Microsoft.Framework.Runtime.Common.EntryPointExecutor.Execute (System.Reflection.Assembly assembly, System.String[] args, IServiceProvider serviceProvider) [0x00000] in <filename unknown>:0 
  at dnx.host.Bootstrapper.RunAsync (System.Collections.Generic.List`1 args) [0x00000] in <filename unknown>:0 

But libuv.so.1 is present in /usr/lib:

[root@localhost HelloWeb]# ll /usr/lib/ | grep libuv 
lrwxrwxrwx.  1 root root   25 Apr 15 17:10 libuv.so.1 -> /usr/local/lib/libuv.so.1

@alexwahl
Copy link

Did you try to build libuv by yourself? i did it this way : https://github.com/w41dev/ASPNet5_MonoSetup/blob/master/SetupLibuv.sh

on Ubuntu 14.04.2 LTS kestrel works, i got the Problem only on Debian.

@acastaner
Copy link

Yes I built it myself. I found the issue. Kestrel looks for /usr/lib64/libuv.so.1 but when you build it on CentOS it's stored in `/usr/local/lib/libuv.so``. The solution was to add a symbolic link, but I wonder by maybe Kestrel should also look under /usr/local/lib ?

@Tragetaschen
Copy link
Contributor

@acastaner Kestrel itself is not concerned with actually finding the library. Kestrel asks libdl's dlopen to find "libuv.so.1" and this in turn follows documented rules.

One of the largest issues with Linux in general is the idiosyncrasies of distributions when it comes to the actual semantics of those rules. If you build libuv yourself, please make sure you follow your distribution's rules of where to store the binaries. There is nothing Kestrel can do to help you with that.

@faviocb
Copy link

faviocb commented May 18, 2015

Hi @acastaner
How did you find out Kestrel was looking for /usr/lib64/libuv.so.1?
Cheers

@Tragetaschen
Copy link
Contributor

@thekane Have a look at Mono's interop documentation. Setting the environment variables also works for the dnx commands: MONO_LOG_LEVEL="debug" MONO_LOG_MASK="dll" dnx . kestrel

@faviocb
Copy link

faviocb commented May 18, 2015

Thanks @Tragetaschen I will have a look

@faviocb
Copy link

faviocb commented Jul 7, 2015

I think this may be an issue in mono (any versions) running on Debian 32-bit.

Florian Kummer found out https://bugzilla.xamarin.com/show_bug.cgi?id=4190

I isolated part of the kestrel source code that called libdl

For example, the following C# code fails in mono: (it uses [DllImport("libdl")])

using System;
using System.Reflection;
using System.Runtime.InteropServices;
using System.Text;

namespace TestLibdl
{
    public static class PlatformApis
    {
        public static class LinuxApis
        {

            [DllImport("libdl")]
            public static extern IntPtr dlopen(String fileName, int flags);

                public static void Main()
                {
                dlopen("libuv.so.1", 2);
                        Console.WriteLine("open :)");

                }   
        }
    }
}
Stacktrace:

  at <unknown> <0xffffffff>
  at (wrapper managed-to-native) TestLibdl.PlatformApis/LinuxApis.dlopen (string,int) <0xffffffff>
  at TestLibdl.PlatformApis/LinuxApis.Main () <0x0001b>
  at (wrapper runtime-invoke) object.runtime_invoke_void (object,intptr,intptr,intptr) <0xffffffff>

Native stacktrace:

    mono() [0x81027e3]
    mono() [0x814ea14]
    mono() [0x806d977]
    linux-gate.so.1(__kernel_rt_sigreturn+0) [0xb77ce40c]
    /lib/ld-linux.so.2(+0xf0f6) [0xb77de0f6]
    /lib/ld-linux.so.2(+0x12245) [0xb77e1245]
    /usr/lib/i386-linux-gnu/libdl.so(+0xcbc) [0xb71b4cbc]
    /lib/ld-linux.so.2(+0xe716) [0xb77dd716]
    /usr/lib/i386-linux-gnu/libdl.so(+0x136c) [0xb71b536c]
    /usr/lib/i386-linux-gnu/libdl.so(dlopen+0x41) [0xb71b4d71]
    [0xb7392f8c]
    [0xb7392e34]
    [0xb7392ef7]
    mono() [0x8072101]

Debug info from gdb:


=================================================================
Got a SIGSEGV while executing native code. This usually indicates
a fatal error in the mono runtime or one of the native libraries 
used by your application.
=================================================================

Aborted

I followed Florian's instructions and I created a wrapper for libdl in C, and named it libdl-debian-32-bit. I changed my C# code to use the wrapper ([DllImport("libdl-debian-32-bit")]) and "it worked".

using System;
using System.Reflection;
using System.Runtime.InteropServices;
using System.Text;

namespace TestLibdl
{
    public static class PlatformApis
    {
        public static class LinuxApis
        {

            [DllImport("libdl-debian-32-bit")]
            public static extern IntPtr dlopen(String fileName, int flags);

                public static void Main()
                {
                dlopen("libuv.so.1", 2);
                        Console.WriteLine("open :)");

                }   
        }
    }
}

This is how I created the wrapper libdl-debian-32-bit.c :

#include <stdlib.h>
#include <stdio.h>
#include <dlfcn.h>

void* dlopen(const char* file, int flag) {
    void* ret;
    ret = dlopen(file,flag);
    return ret;
}

char* dlerror() {
    char* err;
    err = dlerror();
    return err;
}

void* dlsym(void* handle, const char* symbol) {
    void* ret;
    ret = dlsym(handle,symbol); 
    return ret;
}

int dlclose(void* handle) {
    return dlclose(handle);
}
gcc -Wall -fPIC -c libdl-debian-32-bit.c 
gcc -shared -o libdl-debian-32-bit.so libdl-debian-32-bit.o
sudo cp libdl-debian-32-bit.so /lib

I said "it worked" because it didn't crash when opening libuv.
I guess I have to find out how to test it in Microsoft.AspNet.Server.Kestrel/Networking/PlatformApis.cs

Any help will be appreciated. I am a newbie and I have been enjoying Debian.

@DamianEdwards
Copy link
Member

We'll investigate this in beta8 and see if it's still an issue.

@glennc glennc modified the milestones: 1.0.0-rc1, 1.0.0-beta8 Sep 1, 2015
@halter73 halter73 assigned troydai and unassigned halter73 Oct 20, 2015
@troydai
Copy link
Contributor

troydai commented Oct 26, 2015

I can't repo this scenario on Ubunto 14.04 with Mono 4.0.4 and our 1.0.0-beta8 bits.

@troydai
Copy link
Contributor

troydai commented Oct 26, 2015

@w41dev @thekane @Tragetaschen @acastaner this issue was filed a few months back. I retried the scenario following our Guide for installing ASP.NET 5 on Linux would you please, give it a try?

I can't repo this issue with

  • Ubuntu 14.04
  • Mono 4.0.4
  • AspNet 5 1.0.0-beta8
  • Libuv 1.4.2 build from code

If you find this issue to be reproducible on a specific platform, please feel free to reopen this issue and let me know the config.

@faviocb
Copy link

faviocb commented Nov 8, 2015

Hi @troydai

I gave it a try yesterday.

This is my configuration:

  • Debian 8 "Jessie" 32 bits
  • Mono 4.0.4
  • ASP.Net 5 1.0.0-beta8
  • libuv-1.4.2 build from code

I got the same result as described on July 8 (running dnx + kestrel on mono).

suarez@pluto:~/git-repos/Home/samples/1.0.0-beta8/HelloWeb$ dnx kestrel
Stacktrace:

  at <unknown> <0xffffffff>
  at (wrapper managed-to-native) Microsoft.AspNet.Server.Kestrel.Networking.PlatformApis/LinuxApis.dlopen (string,int) <0xffffffff>
  at Microsoft.AspNet.Server.Kestrel.Networking.PlatformApis/LinuxApis.LoadLibrary (string) <0x0001b>
  at Microsoft.AspNet.Server.Kestrel.Networking.Libuv.Load (string) <0x0002e>
  at Microsoft.AspNet.Server.Kestrel.KestrelEngine..ctor (Microsoft.Dnx.Runtime.ILibraryManager,Microsoft.AspNet.Server.Kestrel.ServiceContext) <0x002b3>
  at Microsoft.AspNet.Server.Kestrel.ServerFactory.Start (Microsoft.AspNet.Http.Features.IFeatureCollection,System.Func`2<Microsoft.AspNet.Http.Features.IFeatureCollection, System.Threading.Tasks.Task>) <0x0030f>
  at Microsoft.AspNet.Hosting.Internal.HostingEngine.Start () <0x00223>
  at Microsoft.AspNet.Hosting.Program.Main (string[]) <0x001c0>
  at (wrapper runtime-invoke) <Module>.runtime_invoke_void__this___object (object,intptr,intptr,intptr) <0xffffffff>
  at <unknown> <0xffffffff>
  at (wrapper managed-to-native) System.Reflection.MonoMethod.InternalInvoke (System.Reflection.MonoMethod,object,object[],System.Exception&) <0xffffffff>
  at System.Reflection.MonoMethod.Invoke (object,System.Reflection.BindingFlags,System.Reflection.Binder,object[],System.Globalization.CultureInfo) <0x000c0>
  at System.Reflection.MethodBase.Invoke (object,object[]) <0x00046>
  at Microsoft.Dnx.Runtime.Common.EntryPointExecutor.Execute (System.Reflection.Assembly,string[],System.IServiceProvider) <0x00103>
  at Microsoft.Dnx.ApplicationHost.Program.ExecuteMain (Microsoft.Dnx.ApplicationHost.DefaultHost,string,string[]) <0x00283>
  at Microsoft.Dnx.ApplicationHost.Program.Main (string[]) <0x0042b>
  at (wrapper runtime-invoke) <Module>.runtime_invoke_object__this___object (object,intptr,intptr,intptr) <0xffffffff>
  at <unknown> <0xffffffff>
  at (wrapper managed-to-native) System.Reflection.MonoMethod.InternalInvoke (System.Reflection.MonoMethod,object,object[],System.Exception&) <0xffffffff>
  at System.Reflection.MonoMethod.Invoke (object,System.Reflection.BindingFlags,System.Reflection.Binder,object[],System.Globalization.CultureInfo) <0x000c0>
  at System.Reflection.MethodBase.Invoke (object,object[]) <0x00046>
  at Microsoft.Dnx.Runtime.Common.EntryPointExecutor.Execute (System.Reflection.Assembly,string[],System.IServiceProvider) <0x00103>
  at Microsoft.Dnx.Host.Bootstrapper.RunAsync (System.Collections.Generic.List`1<string>,Microsoft.Dnx.Runtime.IRuntimeEnvironment,string,System.Runtime.Versioning.FrameworkName) <0x0035f>
  at Microsoft.Dnx.Host.RuntimeBootstrapper.ExecuteAsync (string[],Microsoft.Dnx.Host.BootstrapperContext) <0x0085b>
  at Microsoft.Dnx.Host.RuntimeBootstrapper.Execute (string[],Microsoft.Dnx.Host.BootstrapperContext) <0x00027>
  at EntryPoint.Main (string[]) <0x00483>
  at (wrapper runtime-invoke) <Module>.runtime_invoke_int_object (object,intptr,intptr,intptr) <0xffffffff>

Native stacktrace:

    mono() [0x8102e03]
    mono() [0x814f0b4]
    mono() [0x806dc77]
    linux-gate.so.1(__kernel_rt_sigreturn+0) [0xb7726d00]
    /lib/ld-linux.so.2(+0xf0f6) [0xb77380f6]
    /lib/ld-linux.so.2(+0x12245) [0xb773b245]
    /usr/lib/i386-linux-gnu/libdl.so(+0xcbc) [0xb4a02cbc]
    /lib/ld-linux.so.2(+0xe716) [0xb7737716]
    /usr/lib/i386-linux-gnu/libdl.so(+0x136c) [0xb4a0336c]
    /usr/lib/i386-linux-gnu/libdl.so(dlopen+0x41) [0xb4a02d71]
    [0xb42727a4]
    [0xb427272c]
    [0xb42917bf]
    [0xb42873fc]
    [0xb42862e0]
    [0xb54d7fbc]
    [0xb5b1f8a9]
    [0xb5b1f525]
    mono() [0x8072421]

Debug info from gdb:


=================================================================
Got a SIGSEGV while executing native code. This usually indicates
a fatal error in the mono runtime or one of the native libraries 
used by your application.
=================================================================

Aborted
suarez@pluto:~/git-repos/Home/samples/1.0.0-beta8/HelloWeb$ 

@troydai
Copy link
Contributor

troydai commented Nov 8, 2015

/cc @glennc @muratg i reopen this bug since customer can repo this issue on another platform.

@troydai troydai reopened this Nov 8, 2015
@troydai troydai modified the milestones: 1.0.0-rc2, 1.0.0-rc1 Nov 8, 2015
@muratg muratg changed the title Issues running Kestrrel on Linux: System.DllNotFoundException: libdl Issues running Kestrel on Linux: System.DllNotFoundException: libdl Nov 9, 2015
@CamiloBernal
Copy link

Greetings, I solved it by creating symbolic links to the correct location (Expected by Kestrel):

ln -s /usr/lib64/libdl.so.2 /usr/lib64/libdl
ln -s /usr/local/lib/libuv.so /usr/lib64/libuv.so.1
In my case it works.

You can see this thread: http://stackoverflow.com/questions/31855031/how-do-i-install-netcore-on-centos-machine

@BeranekCZ
Copy link

I have same problem like user thekane. I tried CamiloBernal solution (with lib not lib64), but it didn´t help. Any ideas?

@halter73
Copy link
Member

@BeranekCZ did you run ldconfig?

@faviocb
Copy link

faviocb commented Nov 13, 2015

Hi @BeranekCZ

What operating system are you using?

@muratg
Copy link
Contributor

muratg commented Nov 13, 2015

@BeranekCZ did you try lib64 as well?

@BeranekCZ
Copy link

I have Debian 8.2 (jessie), ASP.NET beta 8, mono 4.0.4. So same things as thekane. I tried ldconfig now, but error is still here. It is 32bit OS, so lib64 is not here.
And my run command is "web": "Microsoft.AspNet.Server.Kestrel "

@faviocb
Copy link

faviocb commented Nov 14, 2015

Thanks for your reply @BeranekCZ

Seems to me this is an issue in mono not supporting debian 32-bit.

I think I will have to switch to ubuntu 32-bit immediately. Eventually I will need to get a 64-bit PC to use debian (because coreclr won't support 32-bit anyway).

@gu1ll0me
Copy link

gu1ll0me commented Dec 9, 2015

On CentOS 7 x86_64, got this problem too (System.DllNotFoundException: libuv).
Compiled libuv from https://github.com/libuv/libuv.git.
Solved by creating this simlink : ln -s /usr/local/lib/libuv.so.1.0.0 /usr/lib/libuv.so

@acastaner
Copy link

Not sure if anybody noticed (I did only yesterday) but the documentation now includes installation instruction for Red Hat and derivatives and does mention the creation of the simlink:

https://docs.asp.net/en/latest/getting-started/installing-on-linux.html#installing-on-centos-fedora-and-derivatives

I hope someday we'll get a RPM for that, but in the meantime the steps in the documentation work nicely (on CentOS 6 anyway, I haven't tried CentOS 7).

@muratg
Copy link
Contributor

muratg commented Dec 9, 2015

Yeah, the documentation now has the right instructions. Acquisition on Linux is a little tricky because of the various differences between distros and even version differences between the same distro.

@gu1ll0me
Copy link

gu1ll0me commented Dec 9, 2015

"ln -s /usr/local/lib/libuv.so /usr/lib64/libuv.so.1" is not working for me, as described in the documentation. I still have the "System.DllNotFoundException: libuv" error when I try to start Kestrel.
To make it works, the simlink name must be "libuv.so".

@muratg
Copy link
Contributor

muratg commented Dec 9, 2015

@michaudg That's on CentOS 7?

@danroth27 A good data point to add to the doc.

@gu1ll0me
Copy link

gu1ll0me commented Dec 9, 2015

@muratg CentOS 7.1

@muratg muratg assigned cesarblum and unassigned troydai Dec 15, 2015
@cesarblum
Copy link
Contributor

Closing this since it's distro specific, there are workarounds, and the library search path is not Kestrel's concern.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests