New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Npgsql 3.2 installation from Nuget adds large number of dependent packages #1430

Closed
tcima7 opened this Issue Feb 5, 2017 · 21 comments

Comments

Projects
None yet
4 participants
@tcima7

tcima7 commented Feb 5, 2017

Steps to reproduce

Add or upgrade existing Npgsql Nuget package to 3.2.

The issue

37 dependent packages are added along with Npgsql 3.2. None of these were added with previous versions of Npgsql. This bloats the packages.config file and potentially the list of project references for any projects that have Npgsql referenced.

Is it possible to remove some or all of these dependencies from the Nuget package and package Npgsql 3.2 in a similar way as previous versions such that unnecessary dependency packages are not automatically added?

Full list of packages that are added with Npgsql 3.2 Nuget installation:

  • Microsoft.Extensions.DependencyInjection.Abstractions
  • Microsoft.Extensions.Logging
  • Microsoft.Extensions.Logging.Abstractions
  • Microsoft.NETCore.Platforms
  • NETStandard.Library
  • System.Collections
  • System.Collections.Concurrent
  • System.ComponentModel
  • System.Diagnostics.Debug
  • System.Diagnostics.Tools
  • System.Diagnostics.Tracing
  • System.Globalization
  • System.IO
  • System.IO.Compression
  • System.Linq
  • System.Linq.Expressions
  • System.Net.Http
  • System.Net.Primitives
  • System.ObjectModel
  • System.Reflection
  • System.Reflection.Extensions
  • System.Reflection.Primitives
  • System.Resources.ResourceManager
  • System.Runtime
  • System.Runtime.Extensions
  • System.Runtime.InteropServices
  • System.Runtime.InteropServices.RuntimeInformation
  • System.Runtime.Numerics
  • System.Text.Encoding
  • System.Text.Encoding.Extensions
  • System.Text.RegularExpressions
  • System.Threading
  • System.Threading.Tasks
  • System.Threading.Tasks.Extensions
  • System.Threading.Timer
  • System.Xml.ReaderWriter
  • System.Xml.XDocument
Exception message: N/A
Stack trace:

Further technical details

Npgsql version: 3.2
PostgreSQL version: 8.0.2
Operating system: MS Windows

Other details about my project setup:

We are using .NET Framework 4.5.1 for our projects (don't think this should matter much though)

@roji

This comment has been minimized.

Show comment
Hide comment
@roji

roji Feb 5, 2017

Member

Yes, this is expected - it's because Npgsql 3.2.0 takes a dependency on Microsoft.Extensions.Logging, which provides a .NET Standard 1.1 package. The dependency on NETStandard.Library is what's bringing everything in.

During development of 3.2.0 I had the same misgiving - all the extra bloat seemed out of place. However, after reading aspnet/HttpSysServer#268 (including referenced issues and some others), this seems to be the expected behavior. The extra DLLs you're seeing only contain type forwards to the actual .NET Framework implementation (in your GAC), and are very small. So while I sympathize (and felt the same way too!), this is how things are going to be with .NET moving forward...

You might be able to get away with removing some the dependencies which aren't really used in runtime, but I wouldn't go down that route. They shouldn't bother you in any way. Am going to close this issue, but am definitely open to more discussion and other views on this.

BTW are you actually still using PostgreSQL 8.0.2? Any specific reason (it hasn't received any patches for a long time now).

Member

roji commented Feb 5, 2017

Yes, this is expected - it's because Npgsql 3.2.0 takes a dependency on Microsoft.Extensions.Logging, which provides a .NET Standard 1.1 package. The dependency on NETStandard.Library is what's bringing everything in.

During development of 3.2.0 I had the same misgiving - all the extra bloat seemed out of place. However, after reading aspnet/HttpSysServer#268 (including referenced issues and some others), this seems to be the expected behavior. The extra DLLs you're seeing only contain type forwards to the actual .NET Framework implementation (in your GAC), and are very small. So while I sympathize (and felt the same way too!), this is how things are going to be with .NET moving forward...

You might be able to get away with removing some the dependencies which aren't really used in runtime, but I wouldn't go down that route. They shouldn't bother you in any way. Am going to close this issue, but am definitely open to more discussion and other views on this.

BTW are you actually still using PostgreSQL 8.0.2? Any specific reason (it hasn't received any patches for a long time now).

@roji roji closed this Feb 5, 2017

@tcima7

This comment has been minimized.

Show comment
Hide comment
@tcima7

tcima7 Feb 5, 2017

Ok, thanks for the explanation @roji. We use Npgsql to connect to AWS Redshift which implements PostgresSQL 8.0.2; as such, the PostgresSQL version used is out of our control.

tcima7 commented Feb 5, 2017

Ok, thanks for the explanation @roji. We use Npgsql to connect to AWS Redshift which implements PostgresSQL 8.0.2; as such, the PostgresSQL version used is out of our control.

@roji

This comment has been minimized.

Show comment
Hide comment
@roji

roji Feb 5, 2017

Member

Ah of course, Redshift was forked from 8.0.2, right.

Member

roji commented Feb 5, 2017

Ah of course, Redshift was forked from 8.0.2, right.

@chriswebb

This comment has been minimized.

Show comment
Hide comment
@chriswebb

chriswebb Feb 7, 2017

Contributor

So if we remove the use of that Library with say something like log4net, then all this unnecessary bloat goes away?

Contributor

chriswebb commented Feb 7, 2017

So if we remove the use of that Library with say something like log4net, then all this unnecessary bloat goes away?

@tcima7

This comment has been minimized.

Show comment
Hide comment
@tcima7

tcima7 Feb 7, 2017

+1 for moving to log4net.

tcima7 commented Feb 7, 2017

+1 for moving to log4net.

@roji

This comment has been minimized.

Show comment
Hide comment
@roji

roji Feb 7, 2017

Member

I feel your pain...

The problem with log4net (and NLog) is that it's a logging package (i.e. implementation) rather than a logging abstraction layer. Npgsql is a library, and may be used in applications which themselves use log4net, NLog, Serilog or any other logging implementation. The whole point of Microsoft.Extensions.Logging is to provide an abstraction, to which you can plug providers to use whatever specific implementation you want.

Again, these DLLs are supposed to be super-light (let me know if that's not the case), and apart from being there shouldn't bother anyone.,,

Member

roji commented Feb 7, 2017

I feel your pain...

The problem with log4net (and NLog) is that it's a logging package (i.e. implementation) rather than a logging abstraction layer. Npgsql is a library, and may be used in applications which themselves use log4net, NLog, Serilog or any other logging implementation. The whole point of Microsoft.Extensions.Logging is to provide an abstraction, to which you can plug providers to use whatever specific implementation you want.

Again, these DLLs are supposed to be super-light (let me know if that's not the case), and apart from being there shouldn't bother anyone.,,

@roji

This comment has been minimized.

Show comment
Hide comment
@roji

roji Feb 8, 2017

Member

@tcima7 @chriswebb, I opened aspnet/Logging#550 with Microsoft on this and there are some good news - .NET Standard 2.0 will resolve this issue. So the new package bloat is a temporary situation and will be gone when .NET Standard 2.0 is released (which isn't supposed to be too far away).

Member

roji commented Feb 8, 2017

@tcima7 @chriswebb, I opened aspnet/Logging#550 with Microsoft on this and there are some good news - .NET Standard 2.0 will resolve this issue. So the new package bloat is a temporary situation and will be gone when .NET Standard 2.0 is released (which isn't supposed to be too far away).

@chriswebb

This comment has been minimized.

Show comment
Hide comment
@chriswebb

chriswebb Feb 8, 2017

Contributor

@roji, that is great news! That definitely makes the pill a lot easier to swallow. Thanks for helping us understand the reasoning behind your decision to choose this approach.

Contributor

chriswebb commented Feb 8, 2017

@roji, that is great news! That definitely makes the pill a lot easier to swallow. Thanks for helping us understand the reasoning behind your decision to choose this approach.

@roji

This comment has been minimized.

Show comment
Hide comment
@roji

roji Feb 9, 2017

Member

@chriswebb sure thing, I also wasn't happy about the large number of dependencies, but didn't want that to be a blocker for adopting Microsoft.Extensions.Logging. Good to know that problem will go away.

Member

roji commented Feb 9, 2017

@chriswebb sure thing, I also wasn't happy about the large number of dependencies, but didn't want that to be a blocker for adopting Microsoft.Extensions.Logging. Good to know that problem will go away.

@phillip-haydon

This comment has been minimized.

Show comment
Hide comment
@phillip-haydon

phillip-haydon Feb 10, 2017

:( @roji I can't upgrade to 3.2 now.

All these dependencies causes conflicts with the GAC. I literally don't have time to sit and fix all the references in this project and our internal nuget packages. Sad times.

phillip-haydon commented Feb 10, 2017

:( @roji I can't upgrade to 3.2 now.

All these dependencies causes conflicts with the GAC. I literally don't have time to sit and fix all the references in this project and our internal nuget packages. Sad times.

@roji

This comment has been minimized.

Show comment
Hide comment
@roji

roji Feb 10, 2017

Member

@phillip-haydon it's very very odd for any dependencies to be in conflict with anything you have in the GAC... The packages Npgsql brings in are simply .NET Standard 1.1 (via the Microsoft.Logging.Extensions package); these packages are definitely not supposed to be in your GAC - that would be a very bad idea. Npgsql definitely wouldn't be the only package this would cause problems with (at least going forward).

I'll be happy to help understanding what's wrong, if you'd like to investigate this a little. Open a new issue with your errors and I'll take a look.

Member

roji commented Feb 10, 2017

@phillip-haydon it's very very odd for any dependencies to be in conflict with anything you have in the GAC... The packages Npgsql brings in are simply .NET Standard 1.1 (via the Microsoft.Logging.Extensions package); these packages are definitely not supposed to be in your GAC - that would be a very bad idea. Npgsql definitely wouldn't be the only package this would cause problems with (at least going forward).

I'll be happy to help understanding what's wrong, if you'd like to investigate this a little. Open a new issue with your errors and I'll take a look.

@phillip-haydon

This comment has been minimized.

Show comment
Hide comment
@phillip-haydon

phillip-haydon Feb 10, 2017

We have a bunch of old .NET 4.5 libraries compiled into our local Nuget.

The main app is 4.6.2.

After updating only npgsql, it gets ambiguous reference to all the additional references added from .NET Standard.

I can get you the exact errors on Monday.

phillip-haydon commented Feb 10, 2017

We have a bunch of old .NET 4.5 libraries compiled into our local Nuget.

The main app is 4.6.2.

After updating only npgsql, it gets ambiguous reference to all the additional references added from .NET Standard.

I can get you the exact errors on Monday.

@roji

This comment has been minimized.

Show comment
Hide comment
@roji

roji Feb 10, 2017

Member

We have a bunch of old .NET 4.5 libraries compiled into our local Nuget.

You mean into your local GAC, not nuget, right?

Send me the errors when you can, I'll take a look.

Member

roji commented Feb 10, 2017

We have a bunch of old .NET 4.5 libraries compiled into our local Nuget.

You mean into your local GAC, not nuget, right?

Send me the errors when you can, I'll take a look.

@phillip-haydon

This comment has been minimized.

Show comment
Hide comment
@phillip-haydon

phillip-haydon Feb 10, 2017

So the shared lib is built in team city and hosted in the team city Nuget server.

It has a wrapper around some things. For one, system.io.

After updating the main apps npgsql, it just has an ambiguous reference to system.io for the assembly in nuget (.net standard) and the version in the .net framework folder.

I'm on my way home so I'll get the exact error on Monday for you when I am at my work of again.

phillip-haydon commented Feb 10, 2017

So the shared lib is built in team city and hosted in the team city Nuget server.

It has a wrapper around some things. For one, system.io.

After updating the main apps npgsql, it just has an ambiguous reference to system.io for the assembly in nuget (.net standard) and the version in the .net framework folder.

I'm on my way home so I'll get the exact error on Monday for you when I am at my work of again.

@roji roji referenced this issue Feb 10, 2017

Closed

NETStandard ? #1436

@phillip-haydon

This comment has been minimized.

Show comment
Hide comment
@phillip-haydon

phillip-haydon Feb 13, 2017

Hey @roji,

I spent some time this morning investigating further and have fixed it.

Basically, 3 projects. Project's 1 / 2 contain controllers / codes and reference npgsql.

Project 3 references projects 1/2 and just configures web api.

Because npgsql isn't referenced into Project 3, it's copies npgsql into the bin, but ignores all the NETStandard references. So it would throw ambiguous reference when the app is run.

I referenced npgsql into Project 3 which pulls in NETStandard. This corrects the issues with the references by adding them into the web.config assembly binding redirects.

Hope that makes sense. It's all working great now. :)

phillip-haydon commented Feb 13, 2017

Hey @roji,

I spent some time this morning investigating further and have fixed it.

Basically, 3 projects. Project's 1 / 2 contain controllers / codes and reference npgsql.

Project 3 references projects 1/2 and just configures web api.

Because npgsql isn't referenced into Project 3, it's copies npgsql into the bin, but ignores all the NETStandard references. So it would throw ambiguous reference when the app is run.

I referenced npgsql into Project 3 which pulls in NETStandard. This corrects the issues with the references by adding them into the web.config assembly binding redirects.

Hope that makes sense. It's all working great now. :)

@phillip-haydon

This comment has been minimized.

Show comment
Hide comment
@phillip-haydon

phillip-haydon Feb 13, 2017

CSC error CS1703: Multiple assemblies with equivalent identity have been imported: 'T:\TeamCity\BuildAgent3\work\76ce86364964ffeb\packages\System.Xml.ReaderWriter.4.3.0\lib\net46\System.Xml.ReaderWriter.dll' and 'C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\.NETFramework\v4.6.2\Facades\System.Xml.ReaderWriter.dll'. Remove one of the duplicate references.
CSC error CS1703: Multiple assemblies with equivalent identity have been imported: 'T:\TeamCity\BuildAgent3\work\76ce86364964ffeb\packages\System.Runtime.InteropServices.4.3.0\lib\net462\System.Runtime.InteropServices.dll' and 'C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\.NETFramework\v4.6.2\Facades\System.Runtime.InteropServices.dll'. Remove one of the duplicate references.
CSC error CS1703: Multiple assemblies with equivalent identity have been imported: 'T:\TeamCity\BuildAgent3\work\76ce86364964ffeb\packages\System.Runtime.Extensions.4.3.0\lib\net462\System.Runtime.Extensions.dll' and 'C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\.NETFramework\v4.6.2\Facades\System.Runtime.Extensions.dll'. Remove one of the duplicate references.
CSC error CS1703: Multiple assemblies with equivalent identity have been imported: 'T:\TeamCity\BuildAgent3\work\76ce86364964ffeb\packages\System.Runtime.4.3.0\lib\net462\System.Runtime.dll' and 'C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\.NETFramework\v4.6.2\Facades\System.Runtime.dll'. Remove one of the duplicate references.
CSC error CS1703: Multiple assemblies with equivalent identity have been imported: 'T:\TeamCity\BuildAgent3\work\76ce86364964ffeb\packages\System.Reflection.4.3.0\lib\net462\System.Reflection.dll' and 'C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\.NETFramework\v4.6.2\Facades\System.Reflection.dll'. Remove one of the duplicate references.
CSC error CS1703: Multiple assemblies with equivalent identity have been imported: 'T:\TeamCity\BuildAgent3\work\76ce86364964ffeb\packages\System.IO.4.3.0\lib\net462\System.IO.dll' and 'C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\.NETFramework\v4.6.2\Facades\System.IO.dll'. Remove one of the duplicate references.
CSC error CS1703: Multiple assemblies with equivalent identity have been imported: 'T:\TeamCity\BuildAgent3\work\76ce86364964ffeb\packages\System.Diagnostics.Tracing.4.3.0\lib\net462\System.Diagnostics.Tracing.dll' and 'C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\.NETFramework\v4.6.2\Facades\System.Diagnostics.Tracing.dll'. Remove one of the duplicate references.

This is what happens.

(I get this on the build server, trying to figure out why it runs locally now but not in Team City)

phillip-haydon commented Feb 13, 2017

CSC error CS1703: Multiple assemblies with equivalent identity have been imported: 'T:\TeamCity\BuildAgent3\work\76ce86364964ffeb\packages\System.Xml.ReaderWriter.4.3.0\lib\net46\System.Xml.ReaderWriter.dll' and 'C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\.NETFramework\v4.6.2\Facades\System.Xml.ReaderWriter.dll'. Remove one of the duplicate references.
CSC error CS1703: Multiple assemblies with equivalent identity have been imported: 'T:\TeamCity\BuildAgent3\work\76ce86364964ffeb\packages\System.Runtime.InteropServices.4.3.0\lib\net462\System.Runtime.InteropServices.dll' and 'C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\.NETFramework\v4.6.2\Facades\System.Runtime.InteropServices.dll'. Remove one of the duplicate references.
CSC error CS1703: Multiple assemblies with equivalent identity have been imported: 'T:\TeamCity\BuildAgent3\work\76ce86364964ffeb\packages\System.Runtime.Extensions.4.3.0\lib\net462\System.Runtime.Extensions.dll' and 'C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\.NETFramework\v4.6.2\Facades\System.Runtime.Extensions.dll'. Remove one of the duplicate references.
CSC error CS1703: Multiple assemblies with equivalent identity have been imported: 'T:\TeamCity\BuildAgent3\work\76ce86364964ffeb\packages\System.Runtime.4.3.0\lib\net462\System.Runtime.dll' and 'C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\.NETFramework\v4.6.2\Facades\System.Runtime.dll'. Remove one of the duplicate references.
CSC error CS1703: Multiple assemblies with equivalent identity have been imported: 'T:\TeamCity\BuildAgent3\work\76ce86364964ffeb\packages\System.Reflection.4.3.0\lib\net462\System.Reflection.dll' and 'C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\.NETFramework\v4.6.2\Facades\System.Reflection.dll'. Remove one of the duplicate references.
CSC error CS1703: Multiple assemblies with equivalent identity have been imported: 'T:\TeamCity\BuildAgent3\work\76ce86364964ffeb\packages\System.IO.4.3.0\lib\net462\System.IO.dll' and 'C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\.NETFramework\v4.6.2\Facades\System.IO.dll'. Remove one of the duplicate references.
CSC error CS1703: Multiple assemblies with equivalent identity have been imported: 'T:\TeamCity\BuildAgent3\work\76ce86364964ffeb\packages\System.Diagnostics.Tracing.4.3.0\lib\net462\System.Diagnostics.Tracing.dll' and 'C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\.NETFramework\v4.6.2\Facades\System.Diagnostics.Tracing.dll'. Remove one of the duplicate references.

This is what happens.

(I get this on the build server, trying to figure out why it runs locally now but not in Team City)

@roji

This comment has been minimized.

Show comment
Hide comment
@roji

roji Feb 13, 2017

Member

@phillip-haydon, thanks for putting time into this.

Can you please post the project files (csproj or project.json) for your projects? If you can narrow it down by producing minimal projects files that would be even better. This is so that I can attempt to reproduce the problem here and then try to fix it.

Member

roji commented Feb 13, 2017

@phillip-haydon, thanks for putting time into this.

Can you please post the project files (csproj or project.json) for your projects? If you can narrow it down by producing minimal projects files that would be even better. This is so that I can attempt to reproduce the problem here and then try to fix it.

@phillip-haydon

This comment has been minimized.

Show comment
Hide comment
@phillip-haydon

phillip-haydon Feb 14, 2017

Hmmmm I can't work out how to reproduce it because it works locally here on multiple machines and only fails on the build server.

phillip-haydon commented Feb 14, 2017

Hmmmm I can't work out how to reproduce it because it works locally here on multiple machines and only fails on the build server.

@roji

This comment has been minimized.

Show comment
Hide comment
@roji

roji Feb 14, 2017

Member

That's definitely annoying... Maybe these assemblies somehow ended up in your build server's GAC without them supposed to be there? You may consider removing them if there's no real reason for them being there.

In any case I'll be here to help if you manage to narrow it down on your end (better to open a new issue).

Member

roji commented Feb 14, 2017

That's definitely annoying... Maybe these assemblies somehow ended up in your build server's GAC without them supposed to be there? You may consider removing them if there's no real reason for them being there.

In any case I'll be here to help if you manage to narrow it down on your end (better to open a new issue).

@roji

This comment has been minimized.

Show comment
Hide comment
@roji

roji Mar 21, 2017

Member

The Microsoft.Extensions.Logging dependency has been removed in 3.2., so the new dependencies should no longer appear (see #1504).

Member

roji commented Mar 21, 2017

The Microsoft.Extensions.Logging dependency has been removed in 3.2., so the new dependencies should no longer appear (see #1504).

@phillip-haydon

This comment has been minimized.

Show comment
Hide comment
@phillip-haydon

phillip-haydon Mar 21, 2017

I will give it a test tomorrow!

phillip-haydon commented Mar 21, 2017

I will give it a test tomorrow!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment