AggregateException (Unable to load DLL 'Microsoft.Azure.Documents.ServiceInterop.dll') in ASP.NET Web API project #202
Comments
|
What is the exact exception? Was it looks like: If that is the case, you can try workaround in http://stackoverflow.com/questions/344608/unmanaged-dlls-fail-to-load-on-asp-net-server/4598747. |
|
@mingaliu Thanks for the quick response. You are correct, the exact exception is: InnerException {"Unable to load DLL 'Microsoft.Azure.Documents.ServiceInterop.dll': The specified module could not be found. (Exception from HRESULT: 0x8007007E)"} System.Exception {System.DllNotFoundException} I will try the workaround you mentioned and report back. Since our project is deployed to Azure from Visual Studio on my local machine, I wonder if the deployed version will have any problems? |
|
The workaround suggested works on my local machine. I wonder if it will work once deployed in production? |
|
@andresjacobo Does your ASP.NET Web project directly references DocumentDB Nuget package or is it through a second level dependency? I mean, is it like: ASP.NET Web project referencing a ASP.NET class library which references DocumentDB Nuget package. If it's second level, try to directly reference the DocumentDB Nuget package and ensure that Let me know if you still see the issue. |
|
@rnagpal To use DocumentDB on our ASP.NET Web project, I simply used Nuget inside Visual Studio to download the DocumentDB SDK. This, I believe, automatically adds a direct reference in the project's 'references' to So yes, I would say that even with a direct reference, the problem occurs. What can it be? |
|
@andresjacobo Apart from the assembly reference to Microsoft.Azure.Documents.Client.dll ensure that you also get the following import: Let me know if that helps. |
|
@rnagpal I do indeed have the first import. I then linked the other two libraries but when I ran the project I got the same exception (DLLNotFound). Is there a particular place I should paste the content you mentioned? I don't think it should make a difference, but I can't get it to work. |
|
@rnagpal I am refering this class library in unit test project.
Below is stack trace: Result StackTrace: |
|
@PratikChavda The issue in the first portion of your Stack trace has been fixed in 1.11.4 release. Please upgrade and let me know if you get DllNotFound exception. I'm investigating the issue reported by @andresjacobo for the ASP.Net project. |
|
@andresjacobo As long as it's in an ItemGroup tag under top level Project tag, you should be good. Can you confirm that you have both the native dlls: Microsoft.Azure.Documents.ServiceInterop.dll and DocumentDB.Spatial.Sql.dll in the bin dir? I've seen reports wherein if the Spatial dll is not there at the same place as ServiceInterop, you get this exception as ServiceInterop depends upon Spatial dll and both need to be present. Unfortunately, I haven't been able to repro this on my end using an ASP.NET Web app hosted on azure but given some folks have reported it to us, I believe there is some configuration missing that causes this behavior. Can you also try the workarounds mentioned in https://blogs.msdn.microsoft.com/jorman/2007/08/31/loading-c-assemblies-in-asp-net/ specifically: `In web.config, add the following…use a path where the Native C++ DLL is located: In global.asax, add the following: protected void Application_Start(object sender, EventArgs e){ |
|
Just to have this post contain all possible cases when you run into this, if you are seeing this exception from a VS Unit test project trying to test an ASP.NET class library project that has a reference to DocumentDB Nuget package, you may have to add the following as an attribute to the test class: |
|
@rnagpal Thanks for the info. I can confirm both DLLs you mention are in the bin dir of the ASP.NET project. I've tried every combination of all the advice you've given me, and this is what worked in the end: I added the following code to the global.asax file: With this in place, the DLLs seem to load fine and queries don't throw an AggregateException. This seems like a fairly good solution given that a) I don't pollute the System32 directory b) There's no version specific path that would break when the DocumentDB SDK is updated. My only remaining question is why would we need to do this? Surely there is something specific in my project that causes this to happen. Given how often DocumentDB is used inside ASP.NET, I wouldn't think this is a widespread issue. |
|
@andresjacobo Glad to hear that. I already reported this on our ASP.NET Discussions alias but haven't got any response yet. I'll update you when I hear from them. This is not specific to DocumentDB as other packages with native dlls have the same issue. Also since I'm unable to repro this on my end, I don't believe this is a widespread issue as we would have heard this loud enough. Do you mind creating a small repro app for me and attaching that to my email(listed in my profile) and I'll see if I can repro it? Just run this query on click of a button to repro this. Keep it as simple as possible. I can follow up with you offline in case I've any queries on the sample app. |
|
@rnagpal Sorry it took me some time. I've created a super simple ASP.NET project in VS and (un) fortunately I was able to reproduce the problem. To recreate the problem, all I did was:
I've omitted the DB key from the project. If you need to connect to our specific instance, I'd be happy to give it to you in a secure manner. Let me know if you are able to reproduce the issue. |
|
I have faced a weird sitution. I was working for a traditional ASP.NET 4.5.2 Framework (not Core). When I use the DocumentDB in IIS express in Visual Studio it works. After that I publish the whole project to Azure and it still works. However when I use that in local IIS it does not work and have the same exception as the start of this issue. Afterwards I have referred to the workaround solition suggested by @mingaliu and I put the |
|
@rnagpal : I think we're still stuck in that. I referenced the new 1.12.0 version of the Document DB NuGet and still having the error. Could not load file or assembly 'file:///D:\home\site\wwwroot\bin\DocumentDB.Spatial.Sql.dll' or one of its dependencies. The module was expected to contain an assembly manifest. Description: An unhandled exception occurred during the execution of the current web request. Please review the stack trace for more information about the error and where it originated in the code. Exception Details: System.BadImageFormatException: Could not load file or assembly 'file:///D:\home\site\wwwroot\bin\DocumentDB.Spatial.Sql.dll' or one of its dependencies. The module was expected to contain an assembly manifest. Source Error: An unhandled exception was generated during the execution of the current web request. Information regarding the origin and location of the exception can be identified using the exception stack trace below. Assembly Load Trace: The following information can be helpful to determine why the assembly 'file:///D:\home\site\wwwroot\bin\DocumentDB.Spatial.Sql.dll' could not be loaded. WRN: Assembly binding logging is turned OFF. Stack Trace: [BadImageFormatException: Could not load file or assembly 'file:///D:\home\site\wwwroot\bin\DocumentDB.Spatial.Sql.dll' or one of its dependencies. The module was expected to contain an assembly manifest.] [InvalidOperationException: The pre-application start initialization method Start on type WebActivator.PreApplicationStartCode threw an exception with the following error message: Could not load file or assembly 'file:///D:\home\site\wwwroot\bin\DocumentDB.Spatial.Sql.dll' or one of its dependencies. The module was expected to contain an assembly manifest..] [HttpException (0x80004005): The pre-application start initialization method Start on type WebActivator.PreApplicationStartCode threw an exception with the following error message: Could not load file or assembly 'file:///D:\home\site\wwwroot\bin\DocumentDB.Spatial.Sql.dll' or one of its dependencies. The module was expected to contain an assembly manifest..] |
|
Hi @thuru with regard to your case I think you should try adding the file to D:\home\site\wwwroot\bin\ or perhaps as of my case, adding it to C:\Windows\System32. In general I would say the problem is somehow related to .NET Framework itself not being able to the load the file in Nuget. |
|
@thuru I'll send you an invite for a quick call so that we can close this issue asap as it seems that you have been having this issue from long. @clifflo Instead of that hack, can you try the following code to the global.asax file: and see if that resolves the issue? |
|
@clifflo : The issue you are running into is different from Thuru. You might be seeing DllNotFoundException but Thuru is seeing BadImageFormatException which have different causes. @thuru I see that this exception is coming from an external package WebActivator that seems to load our native dlls. Since these are native dlls, Assembly.LoadFrom() call is bound to fail with BadImageFormatException. Please refer to this for more details: http://stackoverflow.com/questions/18325799/could-not-load-file-or-assembly-hunspellx64-dll-while-using-nhunspell-nuget-pack Can you try uninstalling the current version of WebActivator and install the latest one(looks like upgrade is not supported)? Please keep me updated on this issue. |
|
I got the same exception as the title of this issue indicates.
Here's my setup
I have another Service Fabric application which does not use Web API with a similar setup that does not throw the exception. The issue does seem to specific to mixing with Web API. Directly referencing the same library within the Web API project resolved that issue for me. 👍 |
|
@wonix If your target application is project.json based and you are referring a library project that internally references DocumentDB nuget package, those native dependencies will not be copied as they are based on .targets file which is not supported in project.json projects, so yes you would need to explicitly reference DocumentDB nuget package in your target application which is project.json. In your case, is Web API 2.0 a project.json based project? |
|
@rnagpal, my application is not project.json based. it was created using the Service Fabric Stateless Service template in Visual Studio 2015. The library project is also not project.json based. |
|
I'm having the same issue as well. I have the latest NuGet update (1.13.1). DocumentDB queries work fine an a console application, but when trying to query from a WebAPI controller with the exact same code I get the exception: System.DllNotFoundException: Unable to load DLL 'Microsoft.Azure.Documents.ServiceInterop.dll': The specified module could not be found. (Exception from HRESULT: 0x8007007E) Note this is only on my local machine - deploying the web service to Azure web apps works fine. |
|
I was able to fix this error when running locally by adding the following to my web.config under the system.web section:
|
|
thanks @lehn0058 for the fix, i got the same problem and hours wasted trying to fix!. |
|
I almost fight for a day and still unsolved. I already have two dll files (Interop and Spatial) in bin folder correctly. Setup almost all combinations of x64/AnyCPU whatever configurations. |
|
Today, I tested with fresh ASP.NET WebApi project.
In below table; Both have same DocDB library and code to read one DocDB record.
In summary, DocumentDB library should be used with In addition, I noticed that ALL configurations raise For me, (1) adding
|
|
@YoungjaeKim This is most likely an issue with your application trying to load the Spatial and Interop dlls that are native. If your process is trying to load these assemblies using Assembly.LoadFrom API it will fail with BadImageFormatException as expected: https://msdn.microsoft.com/en-us/library/system.badimageformatexception. Can you search for Assembly.LoadFrom in your application and ensure that this doesn't gets called for native assemblies? Please refer to https://stackoverflow.com/questions/18325799/could-not-load-file-or-assembly-hunspellx64-dll-while-using-nhunspell-nuget-pack in case you are running into this issue. Also, I see that you are using WebActivator package which might be the culprit. The above link says that the latest version of WebActivator seems to have fixed this issue. Can you double check that as well? Note that you will run into this issue even if you don't have a single line of DocDB code in your application. This issue happens at startup of your application which tries to load all the assemblies using Assembly.LoadFrom which is the culprit. Please let me know if that helps and if not we can investigate further. |
|
I just want to add to the discussion. We received
Our website is hosted in Azure App Service as Azure Website. When I went to the bin folder I noticed that both of the DLLs mentioned here: Microsoft.Azure.Documents.ServiceInterop.dll and DocumentDB.Spatial.Sql.dll were there twice, but the second one has the temp extension I am pretty sure that extension is there because of another problem we were having with deployment and locked files. This thread suggested to add the key Furthermore, we had redeployed the application and it started working again only to stop working after about an hour or so. Do you think these renamed files could be causing the problem? |
|
I get this same error message in the latest sdk (tools: 2.0.1-beta.23, runtime: 2.0.11587.0)... ...with only the following references... <ItemGroup>
<PackageReference Include="Microsoft.Azure.WebJobs.Extensions.CosmosDb" Version="3.0.0-beta6" />
<PackageReference Include="Microsoft.NET.Sdk.Functions" Version="1.0.9" />
</ItemGroup>...as well as a CosmosDBTrigger reference... [FunctionName(nameof(Function1))]
public static void Run([CosmosDBTrigger(
databaseName: "databaseName",
collectionName: "collectionName",
ConnectionStringSetting = "connectionString",
LeaseCollectionName = "leases")]IReadOnlyList<Document> input, TraceWriter log)
{
if (input != null && input.Count > 0)
{
log.Verbose("Documents modified " + input.Count);
log.Verbose("First document Id " + input[0].Id);
}
} |
|
@creyke The fix was in the single nuget which was rolled back. Here is a .net standard package which has the fix. Microsoft.Azure.DocumentDB.Core.1.9.1-dev-private.nupkg. |
|
@ausfeldt Thanks for uploading the dev package, however when I try and install it I get the following error, and it also does not seem to be a valid zip file... [Local] 'C:\Working\NuGet\Microsoft.Azure.DocumentDB.Core.1.9.1-dev-private.nupkg' is not a valid nupkg file. Error: 'End of Central Directory record could not be found.'.
|
|
@creyke I just tried and was not able to reproduce. https://github.com/Azure/azure-documentdb-dotnet/raw/DevTestPackages/DevTestPackages/Microsoft.Azure.DocumentDB.Core.1.9.1-dev-private.nupkg Install-Package I would try one more time. Maybe the download was corrupt. |
|
@ausfeldt Thanks again, I'm sorry for the confusion, it was corrupt. After referencing it... <PackageReference Include="Microsoft.Azure.DocumentDB.Core" Version="1.9.1-dev-private" />
<PackageReference Include="Microsoft.Azure.WebJobs.Extensions.CosmosDb" Version="3.0.0-beta6" />
<PackageReference Include="Microsoft.NET.Sdk.Functions" Version="1.0.9" />
<PackageReference Include="System.Data.SqlClient" Version="4.4.0" />...I still unfortunately get the same error... |
|
Sorry to chase, any options here? I am unable to deploy my code to Azure Functions so am completely blocked. |
|
@creyke Is the azure function 64-bit running on windows? |
|
I'm seeing the same issue in with F# scripts against 32-bit FSI. I can't load Interestingly I only see this issue if I try to do an IQueryable query. Simple getting all rows works: let myQuery = client.CreateDocumentQuery<Row>(pricesCollection.SelfLink)
myQuery |> Seq.toArray // works
query {
for doc in myQuery do
take 10
select (doc.Price, doc.Town) }
|> Seq.toArray // popAny ideas? |
|
@isaacabraham The nuget package is not 64-bit only. Microsoft.Azure.Documents.ServiceInterop.dll is 64-bit and native. The SDK will only use it if it's running on a Windows 64-bit OS. |
|
@ausfeldt I'm curious, given the agony that ServiceInterop.dll has created, is the performance boost on 64 bit windows worth it? Also, if you can run without it, can you guys simply catch the Exception and fall back gracefully? |
|
@jackbond This option is an option that is being considered. Long term we are planing to get rid of this dependency all together. Thanks. |
|
@ausfeldt Apologies for no response to previous question. I ran out of time and had to remove my dependencies on cosmos unfortunately. Will monitor the thread, but I no longer have a reproduceable case as I had to ship something (have gone back to http for now instead of cosmos db triggers). |
|
@ausfeldt I'm running on 64 bit Windows 10. I can't reference those two interop libraries - it says Bad CLI Header. Is there no way to "disable" this 64 bit optimisation? |
|
@isaacabraham There is currently no way to disable. The Nuget Package adds build props to make sure that the native DLL is copied to your bin directory. Native assemblies can't be referenced using the #r and #I. The working directory of the F# Interactive process is the path where it is installed, which is somewhere in C:\Program Files, where it will not find the DLL. You options are: Set the working directory where the DLL is: System.Environment.CurrentDirectory <- __SOURCE_DIRECTORY__Windows also considers the Path environment variables to look for the DLL: Environment.SetEnvironmentVariable("Path", Environment.GetEnvironmentVariable("Path") + ";" + __SOURCE_DIRECTORY__) |
|
@ausfeldt thanks for this. I'd say there are a few issues here:
The actual working directory for FSI isn't in C:\Program Files etc. - it varies depending on which IDE you use - but in VS Code, it's already set to the let x64 = IO.Path.GetFullPath @"packages\cosmos\Microsoft.Azure.DocumentDB\runtimes\win7-x64\native\"
Environment.SetEnvironmentVariable("Path", Environment.GetEnvironmentVariable "Path" + ";" + x64)Thanks for your help here. |
|
@ausfeldt I'm a little confused by these two comments:
Is the .Core package deprecated? Seems like they were both updated few weeks ago. Just want to make sure I'm not using a package that is now dead. I have read thru the entire chain and tried all the suggestions I could apply to an asp.net core 2.0 site. Every package version fails with the same error "Unable to load DLL 'Microsoft.Azure.Documents.ServiceInterop.dll': The specified module could not be found. (Exception from HRESULT: 0x8007007E)". I tried to also include the fx version which seems to have helped someone else, but no luck on my end. I also tried updating permissions on the directory and gave Everyone Full Control, but no luck. Works fine when run via unit test, just doesn't work at all behind a webapi. My only option at this point is to change my .netstandard lib to fx and reference the other DocumentDB package; adjusting any differences in code. Not sure if I can host this in an asp.net core 2.0 site or if needs to be asp.net fx. Hopefully that will at least let me ship this project. Kinda sucks that all the development was done against the unit test project until tests were passing only to find out the host has problems with this package T__T |
|
Unfortunately, no luck with the new package. I tried updating just the asp.net core 2.0 references assuming it would use the newer version, but same error. I also tried updating both the .netstandard 1.6 service library and the asp.net core 2.0 site references, but same error. :( Will be trying to convert back to full framework lib and full framework asp.net site. |
|
I was able to convert to using the fx package with no code changes required; just copied all the files into new full framework project so that's cool. The unit tests all still passed after making that change. I was able to do a basic test to see that the service does work when hosted in a asp.net full framework webapi project. However, I'm having other problems now due to System.Runtime not being found, but that's a different issue. |
|
@jackbond Here are the packages that will gracefully fallback if the native assemblies are not found. |
|
@ausfeldt Thanks Garrett. I managed to resolve this issue a while back by simply adding the DocDb nuget to all my executable projects, but still receive notifications on this issue. I'll be happy when the native stuff is gone entirely, as they seem to make little sense for this sort of project. |
|
I have started getting this issue now.. I had a working project on my master branch (which was published about a month ago) and today I needed to update my web.config (unrelated) and I started getting this error: "Unable to find an entry point named 'CreateServiceProvider' in DLL 'Microsoft.Azure.Documents.ServiceInterop.dll'." I have tried everything in this post and everything in the referenced post and I am still getting this error. I have checked the bin folder and both DLLs are present. This is annoying, because it just started happening ..... |
|
@r3plica Did you restart the App? That error can also appear if the ServiceInterop.dll is locked by some process. |
|
@ealsur What could be locking that .dll? Can you provide some examples or things to look for in code? |
|
@mcupito Lock in DLLs in Azure App Service sometimes occur if there is an ongoing process using it, sometimes due to an underlying runtime upgrade (sometimes App Service sends updates to the framework) and I've seen it also happen on deployments. I'm not 100% sure of the reason though, just that an App Service restart unlocks it. |
Absolutely FANTASTIC. That was 2 days of absolute hell trying to get the new DocumentDB libraries working on the Azure app service. Fixed just like that!!! Fantastic! |
and others
Similar to Issue #118.
I am running an ASP.NET project in .NET 4.6 using the latest 1.11.3 DocumentDB SDK and whenever I try to do any query using
CreateDocumentQuery, the query is created successfully but trying to iterate through the results throws an AggregateException, making DocumentDB useless since queries can't be performed.I've verified that
Microsoft.Azure.Documents.ServiceInterop.dllis indeed in the project's bin folder.What's interesting (and frustrating!) is that in a regular .NET console application using the same .NET SDK version and the same DocumentDB SDK version, the queries are executed without a problem, leading me to think the problem is specific to ASP.NET.
Please help.
Below is a stack trace of the exception thrown:
at Microsoft.Azure.Documents.ServiceInteropWrapper.CreateServiceProvider(String configJsonString, IntPtr& serviceProvider)\r\n at Microsoft.Azure.Documents.Query.QueryPartitionProvider.InitializeServiceProvider()\r\n at Microsoft.Azure.Documents.Query.QueryPartitionProvider.GetPartitionedQueryExecutionInfo(SqlQuerySpec querySpec, PartitionKeyDefinition partitionKeyDefinition)\r\n at Microsoft.Azure.Documents.Query.DocumentQueryExecutionContextBase.<GetPartitionedQueryExecutionInfoAsync>d__0.MoveNext()\r\n--- End of stack trace from previous location where exception was thrown ---\r\n at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)\r\n at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n at Microsoft.Azure.Documents.Query.DocumentQueryExecutionContextFactory.<CreateDocumentQueryExecutionContextAsync>d__3.MoveNext()\r\n--- End of stack trace from previous location where exception was thrown ---\r\n at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)\r\n at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n at Microsoft.Azure.Documents.Linq.DocumentQuery1.d__11.MoveNext()\r\n--- End of stack trace from previous location where exception was thrown ---\r\n at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)\r\n at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n at Microsoft.Azure.Documents.Linq.DocumentQuery1.<ExecuteAllAsync>d__1b.MoveNext()\r\n--- End of stack trace from previous location where exception was thrown ---\r\n at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)\r\n at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n at Microsoft.Azure.Documents.Linq.DocumentQuery1.d__d.MoveNext()`The text was updated successfully, but these errors were encountered: