# Resolving .net standard depedencies #1883

Closed
opened this Issue Aug 24, 2016 · 60 comments

Projects
None yet
3 participants
Contributor

### Description

I'm experiencing the same problem as @baronfel from the gitter chat:

Chester Husk III @baronfel Jul 21 20:21
I've got a question about how we resolve nuget framework dependencies for a package with the
profile/framework/standard of a project. As a specific example, I have an F# project that is on
NetFramework 4.6.1. This constraint is set in my paket.dependencies to pin to that framework
specifically. This project has a reference to Marten at version 0.9.9.543, which has only a single
nuget dependency on NpgSql. Then I update Marten to version 0.9.12.563, which has a framework
dependency group for netstandard1.3. When the install occurs, because Marten has no specific
netframework4.6.1 or untethered dependencies anymore, my project breaks due to missing
dependencies. Is this something that can be addressed in Paket? Per the .netstandard spec we
should be able to infer that a netstandard1.3 runtime is usable by a .netframework4.6 client, unless
my understanding is wrong.



### Repro steps

Please provide the steps required to reproduce the problem

1. Use paket to install Marten
2. Try to compile getting started page

Here is a repo demonstrating the issue: https://github.com/TheAngryByrd/paketnetstandard

paket folder uses paket (VSCode Ionide console template) while nuget folder uses nuget (VS console template).

### Expected behavior

Should be able to compile and put correct dlls in bin

### Actual behavior

Does not compile.

Use nuget 😿

### Related information

• Operating system : Windows 10/OSX
• Branch : master
• .NET Runtime, CoreCLR or Mono Version: .NET/Mono

Build error from FAKE:

Build Time Report
---------------------------------------------------------------------
Target     Duration
------     --------
Clean      00:00:00.1086663
Total:     00:00:29.4679452
Status:    Failure
---------------------------------------------------------------------
1) Building z:\Documents\GitHub\paketnetstandard\vscode\src\vscode\vscode.fsproj failed with exitcode 1.
2) FS1108: z:\Documents\GitHub\paketnetstandard\vscode\src\vscode\vscode.fs(14,19): The type 'IsolationLevel' is required here and is unavailable. You must add a reference to assembly 'System.Data.Common, Version=4.0.1.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'.
3) FS0072: z:\Documents\GitHub\paketnetstandard\vscode\src\vscode\vscode.fs(16,5): Lookup on object of indeterminate type based on information prior to this program point. A type annotation may be needed prior to this program point to constrain the type of the object. This may allow the lookup to be resolved.---------------------------------------------------------------------

4) FS0072: z:\Documents\GitHub\paketnetstandard\vscode\src\vscode\vscode.fs(17,5): Lookup on object of indeterminate type based on information prior to this program point. A type annotation may be needed prior to this program point to constrain the type of the object. This may allow the lookup to be resolved.
5) FS0072: z:\Documents\GitHub\paketnetstandard\vscode\src\vscode\vscode.fs(18,20): Lookup on object of indeterminate type based on information prior to this program point. A type annotation may be needed prior to this program point to constrain the type of the object. This may allow the lookup to be resolved.


Member

### matthid commented Aug 25, 2016

 Yep that is a bug: The condition for the dependency itself are correct:  ..\..\packages\Marten\lib\netstandard1.3\Marten.dll True True  However for its transitive dependencies they are not (note that the full framework is missing):  ..\..\packages\System.Data.Common\ref\netstandard1.2\System.Data.Common.dll True True  It seems like netstandard dependencies are filtered out somewhere for other frameworks... Probably a leftover from the early days when we filtered everything containing netstandard.

### forki added a commit that referenced this issue Aug 25, 2016

 Resolving .net standard depedencies - references #1883 
 b54efc3 
Member

### forki commented Aug 25, 2016

 OK I'm not sure if this is going to work properly but just for testing purposes I released a alpha version with a potential fix. Could you please test it?
Member

### matthid commented Aug 25, 2016

 Doesn't seem to make any difference: .paket/paket.bootstrapper.exe prerelease .paket/paket.exe install --force grep -A 6 -B 3 "System.Data" src/vscode/vscode.fsproj  ..\..\packages\System.Data.Common\ref\netstandard1.2\System.Data.Common.dll True True 
Member

Member

### matthid commented Aug 25, 2016

 @TheAngryByrd The project file looks broken. Is Fake by any chance restoring the latest stable again? I mean it shouldn't actually matter, because Fake is only restoring (and shouldn't touch project files). Just compare the "System.Data.Common" section of @forki s version with the version you posted... @forki Can you tell me how Paket itself is broken For me the diff in the lockfile looks like this and no fsproj file is touched/changed (needless to say that everything is working for me)... Am I doing something wrong (testing with 3.17.0-alpha003)? diff --git a/paket.lock b/paket.lock index 1984dd6..c04c28b 100644 --- a/paket.lock +++ b/paket.lock @@ -4,44 +4,44 @@ NUGET Argu (3.2) Chessie (0.6) FSharp.Core (4.0.0.1) - redirects: force - Mono.Cecil (0.9.6.3) + Mono.Cecil (0.9.6.4) Newtonsoft.Json (9.0.1) - redirects: force GITHUB remote: fsharp/FAKE - src/app/FakeLib/Globbing/Globbing.fs (7f9fe8f546f6adec66febeec31e50821b60814d6) + src/app/FakeLib/Globbing/Globbing.fs (e5870cb85f9b500246e02c6daa94e5131dad7a6e) remote: fsprojects/FSharp.TypeProviders.StarterPack - src/AssemblyReader.fs (dfbca9b83fb70067e85abddcb8b89332aae4c28d) + src/AssemblyReader.fs (8f647a4cde71bdeda10e7f5b534ec3c879601570) GROUP Build NUGET remote: https://www.nuget.org/api/v2 - FAKE (4.31.1) + FAKE (4.39) FSharp.Compiler.Service (2.0.0.6) FSharp.Formatting (2.14.4) FSharp.Compiler.Service (2.0.0.6) FSharpVSPowerTools.Core (>= 2.3 < 2.4) FSharpVSPowerTools.Core (2.3) FSharp.Compiler.Service (>= 2.0.0.3) - ILRepack (2.0.10) + ILRepack (2.0.11) Microsoft.Bcl (1.1.10) - framework: net10, net11, net20, net30, net35, net40, net40-full Microsoft.Bcl.Build (>= 1.0.14) Microsoft.Bcl.Build (1.0.21) - import_targets: false, framework: net10, net11, net20, net30, net35, net40, net40-full Microsoft.Net.Http (2.2.29) - framework: net10, net11, net20, net30, net35, net40, net40-full Microsoft.Bcl (>= 1.1.10) Microsoft.Bcl.Build (>= 1.0.14) - Octokit (0.20) + Octokit (0.21.1) Microsoft.Net.Http - framework: net10, net11, net20, net30, net35, net40, net40-full GITHUB remote: fsharp/FAKE - modules/Octokit/Octokit.fsx (7f9fe8f546f6adec66febeec31e50821b60814d6) - Octokit + modules/Octokit/Octokit.fsx (e5870cb85f9b500246e02c6daa94e5131dad7a6e) + Octokit (>= 0.20) GROUP Test NUGET remote: https://www.nuget.org/api/v2 Castle.Core (3.3.3) - framework: >= net45 - FsCheck (2.5) + FsCheck (2.6) FSharp.Core (>= 3.1.2.5) FSharp.Core (4.0.0.1) - Moq (4.5.16) + Moq (4.5.21) Castle.Core (>= 3.3.3) - framework: >= net45 NUnit (3.4.1) NUnit.Console (3.4.1) \$ git status On branch master Your branch is ahead of 'matthid/master' by 1229 commits. (use "git push" to publish your local commits) Changes not staged for commit: (use "git add/rm ..." to update what will be committed) (use "git checkout -- ..." to discard changes in working directory) modified: docs/content/paket-push.md deleted: nupkgs/FAKE.4.31.1.nupkg deleted: nupkgs/FsCheck.2.5.0.nupkg deleted: nupkgs/ILRepack.2.0.10.nupkg deleted: nupkgs/Mono.Cecil.0.9.6.3.nupkg deleted: nupkgs/Moq.4.5.16.nupkg deleted: nupkgs/Octokit.0.20.0.nupkg modified: paket.lock modified: src/Paket.Bootstrapper/Properties/AssemblyInfo.cs modified: src/Paket.Core/AssemblyInfo.fs modified: src/Paket.PowerShell/AssemblyInfo.fs modified: src/Paket/AssemblyInfo.fs Untracked files: (use "git add ..." to include in what will be committed) nupkgs/FAKE.4.39.0.nupkg nupkgs/FsCheck.2.6.0.nupkg nupkgs/ILRepack.2.0.11.nupkg nupkgs/Mono.Cecil.0.9.6.4.nupkg nupkgs/Moq.4.5.21.nupkg nupkgs/Octokit.0.21.1.nupkg no changes added to commit (use "git add" and/or "git commit -a") 
Member

### forki commented Aug 26, 2016

 did you update -f? you will then see: Chessie (0.6) FSharp.Core - framework: net45, net451, net452, net453, net46, net461, net462 FSharp.Core (>= 4.0.1.7-alpha) - framework: >= net463 NETStandard.Library (>= 1.6) - framework: >= net463  which leads to paket installing FSharp.Core (4.0.1.7-alpha)
Member

### forki commented Aug 26, 2016

 ok that Chessie thing is fixed.
Member

### matthid commented Aug 26, 2016

 Since when do we need -f for everything? I always thought install is rewriting project files if needed and update includes install, but no I'm not so sure anymore ...
Member

### forki commented Aug 26, 2016

 you usually don't need that, but if you don't use it then paket uses cached package details from before we changed that logic and that doesn't run restriction optimizer again
Member

### forki commented Aug 26, 2016

 In other words: I should invalidate the cache by increasing the internal cache version number.
Member

### matthid commented Aug 26, 2016

 At first it's surprising that paket chooses an alpha version for a stable dependency (doesn't nuget even prevent uploading a stable with deps to unstable?). But its the only package which fullfills all requirements of Chessi. Now I'm wondering what would happen if we specify  All Frameworks FSharp.Core (< 4.0.1.7-alpha) .NETStandard 1.6 NETStandard.Library (>= 1.6.0) FSharp.Core (>= 4.0.1.7-alpha)  I think paket would stop with an error and we would need to specify a framework restriction. I don't see any particular issue here. Its just surprising and how paket works it tries to find the best package for ALL frameworks. One thing we could do here is tell paket to rather drop frameworks than to introduce alpha packages (ie keep things stable)?
Member

### forki commented Aug 26, 2016

 the problem would be the same if 4.0.1.7 would be stable...
Member

### matthid commented Aug 26, 2016

 Because you are arguing that the problem is now we basically reference most of that new .netstandard crap even in older projects And I think the problem here is that an alpha package was pulled. Paket always pulled unrelated frameworks, because that's how it works. Now another framework was added and paket adds support for it immediatly... If users don't want those deps they need to use framework restrictions. One thing we could do is make the restrictions more powerful. IE add blacklisting support like !netcoreapp !netstandard Long term we will be unable to prevent the pulling of netstandard packages, because more and more libraries will only provide a netstandard binary...
Member

### forki commented Aug 26, 2016

 @matthid the problem was: paket had that restriction on >= net45 and this intermediate alpha version still pulled in the .net standard stuff
Member

### matthid commented Aug 26, 2016 • edited Edited 1 time matthid edited Aug 26, 2016 (most recent)

 Hm, maybe a packaging bug of the alpha package. Shouldn't it contain an empty group for net40 because there actually is a "net40" binary included? Edit: not package i meant binary
Member

 maybe.
Member

### matthid commented Aug 26, 2016 • edited Edited 1 time matthid edited Aug 26, 2016 (most recent)

 The nuspec specs for this scenario are not really helpful at all. First I thought the package would only contain the netstandard1.6 binary and the behavior of paket seemed correct. But after recognizing that there actually is a net40 folder in the package it seems wrong that paket assumes all dependencies need to be resolved (as we are restricting to net45).... The spec says "All dependencies inside a group are installed together if the target framework is compatible with the project's framework profile." But I don't think even nuget does that. I think we should test if paket behaves differently with an empty group. If yes we might want to tell paket to always assume an empty group? (I still think this is a packaging bug though, question is can we reliable "fix" it without breaking things like using net40 dependencies for net45 (if no group is there))
Member

### matthid commented Aug 26, 2016

 Ok I found an example in FAKE: https://www.nuget.org/packages/Machine.Specifications Here we have the empty groups and after adding a framework restriction to >= net45 all netstandard stuff is still pulled... So an empty group doesn't help. Question is: Should it? I'm pretty sure nuget interprets dependency groups differently. - Machine.Specifications (0.9.3) + Machine.Specifications (0.11) + NETStandard.Library (>= 1.6) - framework: >= net46 + System.Diagnostics.TextWriterTraceListener (>= 4.0) - framework: >= net46 + System.Diagnostics.TraceSource (>= 4.0) - framework: >= net46 + System.Reflection (>= 4.1) - framework: >= net46 + System.Reflection.Extensions (>= 4.0.1) - framework: >= net46 + System.Reflection.TypeExtensions (>= 4.1) - framework: >= net46 + System.Xml.XPath (>= 4.0.1) - framework: >= net46 + System.Xml.XPath.XDocument (>= 4.0.1) - framework: >= net46 Btw: Is there a flag to just force install into fsproj instead of re-downloading everything again?
Member

### forki commented Aug 26, 2016

 nope there is no such flag. but run paket clear-cache and after you don't need to -f any more
Member

### forki commented Aug 26, 2016

 yeah we don't want that to happen. Let me investigate that case
Member

### matthid commented Aug 26, 2016

 I "think" what nuget does is use the best matching dependency group... Which imho is wrong in regards to the spec. But it definitely makes more sense than the spec...
Member

### matthid commented Aug 26, 2016

 Another question is where is that > relation between frameworks defined? Is this some paket internal thing?
Member

### matthid commented Aug 26, 2016

 Hm restricting to exactly net461 still pulls netstandard packages. Would it be correct if paket filters the groups with the current framework restrictions and keep only groups that fulfill the current restrictions (if > 0, because otherwise you need to take a group that doesn't fit with the current restrictions)
Member

### matthid commented Aug 26, 2016

 Scratch that it doesn't work for framework net461 only for framework >= net45.
Member

### forki commented Aug 26, 2016

 ok released new version with fix for mspec. please give it a try
Member

### matthid commented Aug 26, 2016

 I think there is still something wrong: I specified "framework: netstandard1.6" (in FAKE), then changed to framework: netstandard1.6, net461 and looked at the generated diff of paket update -f It seems some netstandard dependencies are added to net461, which I wouldn't expect: + NETStandard.Library (1.6) - framework: netstandard16 + System.Globalization.Calendars (>= 4.0.1) - framework: net461, netstandard16 It is added to the project file (Samples/ContinuousDeploymentWebsite/src/test/Fake_WebSite.Tests/Fake_WebSite.Tests.csproj) + + + + ..\..\..\..\..\packages\System.Globalization.Calendars\ref\net46\System.Globalization.Calendars.dll + True + True + + + True + + +  When I change to framework: net461 (ie remove netstandard) those references are removed. - - - - ..\..\..\..\..\packages\System.Globalization.Calendars\ref\net46\System.Globalization.Calendars.dll - True - True - - - True - - -  I would expect paket to generate always the same references for the same framework....
Member

### forki commented Aug 26, 2016

 https://www.nuget.org/packages/RabbitMQ.Client/ <- also fucked up
Member

### matthid commented Aug 26, 2016

 The package itself looks fine to me. I think paket is not delegating the framework restrictions properly. If a package is already restricted to netstandard1.6 (as NETStandard.Library above). It shouldn't loosen the restriction for its dependencies. System.Globalization.Calendars (>= 4.0.1) - framework: net461, netstandard16 <- Why did paket add net461 here? I think I found the culprit: https://www.nuget.org/packages/System.Net.Http It has dependencies for net461 (net46 rather) and only those seem to be pulled. But it's still wrong because System.Net.Http itself is only pulled for netstandard1.6...  System.Net.Http (4.1) - framework: netstandard16 Microsoft.NETCore.Platforms (>= 1.0.1) - framework: netstandard16 runtime.native.System (>= 4.0) - framework: netstandard16 runtime.native.System.Net.Http (>= 4.0.1) - framework: netstandard16 runtime.native.System.Security.Cryptography (>= 4.0) - framework: netstandard16 System.Collections (>= 4.0.11) - framework: netstandard16 System.Diagnostics.Debug (>= 4.0.11) - framework: netstandard16 System.Diagnostics.DiagnosticSource (>= 4.0) - framework: net461, netstandard16 System.Diagnostics.Tracing (>= 4.1) - framework: netstandard16 System.Globalization (>= 4.0.11) - framework: netstandard16 System.Globalization.Extensions (>= 4.0.1) - framework: netstandard16 System.IO (>= 4.1) - framework: netstandard16 System.IO.FileSystem (>= 4.0.1) - framework: netstandard16 System.Net.Primitives (>= 4.0.11) - framework: netstandard16 System.Resources.ResourceManager (>= 4.0.1) - framework: netstandard16 System.Runtime (>= 4.1) - framework: netstandard16 System.Runtime.Extensions (>= 4.1) - framework: netstandard16 System.Runtime.Handles (>= 4.0.1) - framework: netstandard16 System.Runtime.InteropServices (>= 4.1) - framework: netstandard16 System.Security.Cryptography.Algorithms (>= 4.2) - framework: netstandard16 System.Security.Cryptography.Encoding (>= 4.0) - framework: netstandard16 System.Security.Cryptography.OpenSsl (>= 4.0) - framework: netstandard16 System.Security.Cryptography.Primitives (>= 4.0) - framework: netstandard16 System.Security.Cryptography.X509Certificates (>= 4.1) - framework: net461, netstandard16 
Contributor

### TheAngryByrd commented Aug 26, 2016 • edited Edited 1 time TheAngryByrd edited Aug 26, 2016 (most recent)

 @matthid I started over again with an Ionide console template. Changed the TargetFrameworkVersion in the fsproj to 4.6.1. Made sure the build.(cmd/sh) used prelease for the paket.bootstrapper. Installed Marten, still same problem. This is as of 3.17.0-alpha008. If I go back to 3.17.0-alpha003 it seems to work. EDIT: Did some bisecting, looks like I can go back to 3.17.0-alpha006 and it works. Maybe I'm doing something stupidly and wrong?
Member

### matthid commented Aug 26, 2016

 @TheAngryByrd Yeah the test @forki added is failing again. I would say fixing this is complicated, therefore it could take a while to get this right...
Contributor

### TheAngryByrd commented Aug 26, 2016

 Yeah, it looks complex from the outside. Can't imagine what kind of hell it is to code it.

Member

Contributor

### TheAngryByrd commented Aug 26, 2016

 Working for me on Window and OSX! You rock!
Member

### matthid commented Aug 26, 2016

 @forki Paket is still adding additional/too many references to net461 for FAKE (for the Machine.Specifications package). But at least references seem to be consistent again. I might be able to take a look at this at the weekend...
Member

### forki commented Sep 2, 2016

 ok @TheAngryByrd please retry with latest. only to make sure I didn't break you again.
Contributor

### TheAngryByrd commented Sep 5, 2016 • edited Edited 1 time TheAngryByrd edited Sep 5, 2016 (most recent)

 Sorry, I was just able to get to this. Yeah it looks like it's broken again for me =\ Back to this silliness:  1) Building Z:\Documents\GitHub\paketnetstandard\paket\src\vscode\vscode.fsproj failed with exitcode 1. 2) FS1108: Z:\Documents\GitHub\paketnetstandard\paket\src\vscode\vscode.fs(14,19): The type 'IsolationLevel' is requir ed here and is unavailable. You must add a reference to assembly 'System.Data.Common, Version=4.0.1.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'. 3) FS0072: Z:\Documents\GitHub\paketnetstandard\paket\src\vscode\vscode.fs(16,5): Lookup on object of indeterminate ty pe based on information prior to this program point. A type annotation may be needed prior to this program point to cons train the type of the object. This may allow the lookup to be resolved. 4) FS0072: Z:\Documents\GitHub\paketnetstandard\paket\src\vscode\vscode.fs(17,5): Lookup on object of indeterminate ty pe based on information prior to this program point. A type annotation may be needed prior to this program point to cons train the type of the object. This may allow the lookup to be resolved. 5) FS0072: Z:\Documents\GitHub\paketnetstandard\paket\src\vscode\vscode.fs(18,20): Lookup on object of indeterminate t ype based on information prior to this program point. A type annotation may be needed prior to this program point to con strain the type of the object. This may allow the lookup to be resolved. --------------------------------------------------------------------- 
Member

### forki commented Sep 5, 2016

 on that demo project? with update -f?
Contributor

### TheAngryByrd commented Sep 5, 2016

 Yes. However on mono I get slightly different results:  1) Building /Users/jimmybyrd/Documents/GitHub/paketnetstandard/paket/src/vscode/vscode.fsproj failed with exitcode 1. 2) FS1108: /Users/jimmybyrd/Documents/GitHub/paketnetstandard/paket/src/vscode/vscode.fs(1,12): The type 'IQueryable1' is required here and is unavailable. You must add a reference to assembly 'System.Linq.Expressions, Version=4.0.10.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'. 3) FS0039: /Users/jimmybyrd/Documents/GitHub/paketnetstandard/paket/src/vscode/vscode.fs(13,17): The namespace or module 'DocumentStore' is not defined 4) FS0072: /Users/jimmybyrd/Documents/GitHub/paketnetstandard/paket/src/vscode/vscode.fs(14,19): Lookup on object of indeterminate type based on information prior to this program point. A type annotation may be needed prior to this program point to constrain the type of the object. This may allow the lookup to be resolved. 5) FS0072: /Users/jimmybyrd/Documents/GitHub/paketnetstandard/paket/src/vscode/vscode.fs(16,5): Lookup on object of indeterminate type based on information prior to this program point. A type annotation may be needed prior to this program point to constrain the type of the object. This may allow the lookup to be resolved. 6) FS0072: /Users/jimmybyrd/Documents/GitHub/paketnetstandard/paket/src/vscode/vscode.fs(17,5): Lookup on object of indeterminate type based on information prior to this program point. A type annotation may be needed prior to this program point to constrain the type of the object. This may allow the lookup to be resolved. 7) FS0072: /Users/jimmybyrd/Documents/GitHub/paketnetstandard/paket/src/vscode/vscode.fs(18,20): Lookup on object of indeterminate type based on information prior to this program point. A type annotation may be needed prior to this program point to constrain the type of the object. This may allow the lookup to be resolved. 
Member

### forki commented Sep 5, 2016

 ok will retry tomorrow. weird stuff
Member

### matthid commented Sep 5, 2016

 Note that it should work when you use the latest alpha of Marten (add "pre" to paket.dependencies). With nuget you are using that version as well... @forki I'm trying to find the reason and create another test for it. I think paket still gets a bit confused with when to apply which framework restrictions...
Member

### matthid commented Sep 5, 2016

 Might have been the failing test :)
Member

### matthid commented Sep 5, 2016

 This case is sent directly by the devil. What paket does seems to be perfectly valid. I think the root cause is that marten doesn't specify its dependencies correctly. Its binary directly depends on "System.Data.Common". Introducing such dependencies might be very very subtle (ie. if you call a method returning a class from a transitive dependency). Of course Paket is using the net46 version of Npgsql which doesn't have the "System.Data.Common" dependency. But Marten still has it (at least the compiler needs the reference assembly, it might not even be a runtime dependency). I think one way to solve this would be for paket to always use the lowest framework version a dependency introduces... I'll try to write a test case for this :)
Member

### matthid commented Sep 5, 2016

 I think one way to solve this would be for paket to always use the lowest framework version a dependency introduces... To clarify: This is at install time not at resolution time.
Member

### matthid commented Sep 5, 2016

 matthid@28efb12 But we can only do one way or the other. What should we do in a case where two packages (A, and B) depend on Y. A in netstandard group and B in net45 dependency group. Which binary should we reference? B might reference the net45 binary with more features -> potentially more dependencies. But A might need additional dependencies at compile time (reference assemblies)... I think there is no right or wrong here. What paket does right now is correct IMHO.
Member

### forki commented Sep 6, 2016

 @matthid can you try to send PR to marten then?

Member

### matthid commented Sep 6, 2016

 Why? The latest alpha should be working already...
Contributor

### TheAngryByrd commented Sep 6, 2016

 Yep. The alphas of Marten seem to work.

Merged