Global resolver strategy option #1160

mrinaldi · 2015-10-22T18:49:50Z

Currently, Paket uses the Max strategy and you can override it per package using the ! modifier

This PR adds a new setting to specify the default resolver strategy for the group.

Now, you can set the default strategy either by doing strategy min or strategy max in paket.dependencies. If no strategy is set, it'll continue using the Max by default.

In addition, a new strategy modifier has been introduced, similar to the ! modifier, the new modifier is the @, like in the example below:

strategy min
source https://nuget.org/api/v2

nuget FAKE @~> 4.0

This makes all transitive packages of FAKE resolve using the Max strategy, albeit the global strategy is Min.

Although improbable, to resolve conflicts between strategy modifiers (! and @), it'll compare depth of the requirements in conflict and if both are equal, it'll select the default strategy. In case no default strategy is set, it'll use Max by default.

In case this PR is merged, I'll begin working on the docs.

forki · 2015-10-22T18:53:52Z

on one hand I like the symmetry, but do we really need this extra mode?
Are you using it a lot?

mrinaldi · 2015-10-22T18:57:41Z

Actually I am.

As a matter of fact, I can't achieve what I want without this.

Some of my dependencies are both direct and transitive and unless I have a global strategy, those specific dependencies always get the latest version, regardless the ! modifier on its parent.

forki · 2015-10-22T19:02:58Z

ok that makes sense.
let's release this as alpha. and when docs are ready I release.

Global resolver strategy option

forki · 2015-10-23T08:35:56Z

Something broke.

2.16.0 finds resolution for https://github.com/ctaggart/coreclr-demo

deps file:

source https://www.myget.org/F/dotnet-core/
source https://nuget.org/api/v2

# CoreCLR Runtime
nuget Microsoft.NETCore.Runtime.CoreCLR-x64 prerelease

# has CoreConsole.exe
nuget Microsoft.NETCore.Runtime.CoreCLR.ConsoleHost-x64 prerelease

# code against these reference assemblies
nuget System.Runtime prerelease
nuget System.Console prerelease

# packages\System.Console\runtime.json says we need this for our platform
nuget runtime.win7.System.Console prerelease

# runtime dependencies
nuget System.IO prerelease
nuget System.IO.FileSystem.Primitives prerelease
nuget System.Runtime.InteropServices prerelease
nuget System.Text.Encoding prerelease
nuget System.Threading prerelease
nuget System.Threading.Tasks prerelease

lock file:

NUGET
  remote: https://www.myget.org/F/dotnet-core
  specs:
    Microsoft.NETCore.Runtime.CoreCLR.ConsoleHost-x64 (1.0.0-beta-22914)
    runtime.win7.System.Console (4.0.0-beta-23420)
      System.IO (>= 4.0.10)
      System.IO.FileSystem.Primitives (>= 4.0.0)
      System.Resources.ResourceManager (>= 4.0.0)
      System.Runtime (>= 4.0.20)
      System.Runtime.InteropServices (>= 4.0.20)
      System.Text.Encoding (>= 4.0.10)
      System.Text.Encoding.Extensions (>= 4.0.10)
      System.Threading (>= 4.0.10)
      System.Threading.Tasks (>= 4.0.10)
    System.Console (4.0.0-beta-23420)
      System.IO (>= 4.0.0)
      System.Runtime (>= 4.0.0)
    System.IO.FileSystem.Primitives (4.0.1-beta-23420)
      System.Runtime (>= 4.0.0)
      System.Runtime (>= 4.0.20)
    System.Linq (4.0.1-beta-23420) - framework: dnxcore50
      System.Collections (>= 4.0.0)
      System.Collections (>= 4.0.10)
      System.Diagnostics.Debug (>= 4.0.10)
      System.Resources.ResourceManager (>= 4.0.0)
      System.Runtime (>= 4.0.0)
      System.Runtime (>= 4.0.20)
      System.Runtime.Extensions (>= 4.0.10)
    System.Reflection.TypeExtensions (4.0.1-beta-23420) - framework: dnxcore50
      System.Diagnostics.Contracts (>= 4.0.0) - framework: dnxcore50
      System.Diagnostics.Debug (>= 4.0.10) - framework: dnxcore50
      System.Linq (>= 4.0.0) - framework: dnxcore50
      System.Reflection (>= 4.0.0)
      System.Reflection (>= 4.0.10) - framework: dnxcore50
      System.Reflection.Primitives (>= 4.0.0) - framework: dnxcore50
      System.Resources.ResourceManager (>= 4.0.0) - framework: dnxcore50
      System.Runtime (>= 4.0.0)
      System.Runtime (>= 4.0.20) - framework: dnxcore50
      System.Runtime.Extensions (>= 4.0.10) - framework: dnxcore50
  remote: https://nuget.org/api/v2
  specs:
    Microsoft.NETCore.Runtime.CoreCLR-x64 (1.0.1-beta-23225)
      System.Collections (4.0.11-beta-23225)
      System.Diagnostics.Contracts (4.0.1-beta-23225)
      System.Diagnostics.Debug (4.0.11-beta-23225)
      System.Diagnostics.StackTrace (4.0.1-beta-23225)
      System.Diagnostics.Tools (4.0.1-beta-23225)
      System.Diagnostics.Tracing (4.0.21-beta-23225)
      System.Globalization (4.0.11-beta-23225)
      System.Globalization.Calendars (4.0.1-beta-23225)
      System.IO (4.0.11-beta-23225)
      System.ObjectModel (4.0.11-beta-23225)
      System.Private.Uri (4.0.1-beta-23225)
      System.Reflection (4.1.0-beta-23225)
      System.Reflection.Extensions (4.0.1-beta-23225)
      System.Reflection.Primitives (4.0.1-beta-23225)
      System.Resources.ResourceManager (4.0.1-beta-23225)
      System.Runtime (4.0.21-beta-23225)
      System.Runtime.Extensions (4.0.11-beta-23225)
      System.Runtime.Handles (4.0.1-beta-23225)
      System.Runtime.InteropServices (4.0.21-beta-23225)
      System.Text.Encoding (4.0.11-beta-23225)
      System.Text.Encoding.Extensions (4.0.11-beta-23225)
      System.Threading (4.0.11-beta-23225)
      System.Threading.Tasks (4.0.11-beta-23225)
      System.Threading.Timer (4.0.1-beta-23225)
    System.Collections (4.0.11-beta-23225)
      System.Diagnostics.Debug (>= 4.0.0) - framework: dnxcore50
      System.Resources.ResourceManager (>= 4.0.0) - framework: dnxcore50
      System.Runtime (>= 4.0.0) - framework: dnxcore50
      System.Runtime (>= 4.0.21-beta-23225) - framework: dnxcore50
      System.Runtime.Extensions (>= 4.0.0) - framework: dnxcore50
      System.Threading (>= 4.0.0) - framework: dnxcore50
    System.Diagnostics.Contracts (4.0.1-beta-23225)
      System.Runtime (>= 4.0.0) - framework: dnxcore50
    System.Diagnostics.Debug (4.0.11-beta-23225)
      System.Runtime (>= 4.0.0) - framework: dnxcore50
    System.Diagnostics.StackTrace (4.0.1-beta-23225)
      System.Reflection (>= 4.0.0) - framework: dnxcore50
      System.Runtime (>= 4.0.0) - framework: dnxcore50
      System.Runtime (>= 4.0.21-beta-23225) - framework: dnxcore50
    System.Diagnostics.Tools (4.0.1-beta-23225)
      System.Runtime (>= 4.0.0) - framework: dnxcore50
    System.Diagnostics.Tracing (4.0.21-beta-23225)
      System.Collections (>= 4.0.10) - framework: dnxcore50
      System.Diagnostics.Contracts (>= 4.0.0) - framework: dnxcore50
      System.Globalization (>= 4.0.0) - framework: dnxcore50
      System.Reflection (>= 4.0.10) - framework: dnxcore50
      System.Reflection.Extensions (>= 4.0.0) - framework: dnxcore50
      System.Resources.ResourceManager (>= 4.0.0) - framework: dnxcore50
      System.Runtime (>= 4.0.0) - framework: dnxcore50
      System.Runtime (>= 4.0.20) - framework: dnxcore50
      System.Runtime.Extensions (>= 4.0.10) - framework: dnxcore50
      System.Runtime.InteropServices (>= 4.0.0) - framework: dnxcore50
      System.Text.Encoding (>= 4.0.0) - framework: dnxcore50
      System.Threading (>= 4.0.10) - framework: dnxcore50
    System.Globalization (4.0.11-beta-23225)
      System.Runtime (>= 4.0.0) - framework: dnxcore50
    System.Globalization.Calendars (4.0.1-beta-23225)
      System.Globalization (>= 4.0.0) - framework: dnxcore50
      System.Runtime (>= 4.0.0) - framework: dnxcore50
    System.IO (4.0.11-beta-23225)
      System.Globalization (>= 4.0.0) - framework: dnxcore50
      System.Runtime (>= 4.0.20) - framework: dnxcore50
      System.Text.Encoding (>= 4.0.0) - framework: dnxcore50
      System.Text.Encoding (>= 4.0.10) - framework: dnxcore50
      System.Text.Encoding.Extensions (>= 4.0.0) - framework: dnxcore50
      System.Threading (>= 4.0.0) - framework: dnxcore50
      System.Threading.Tasks (>= 4.0.0) - framework: dnxcore50
    System.ObjectModel (4.0.11-beta-23225)
      System.Collections (>= 4.0.10) - framework: dnxcore50
      System.Diagnostics.Debug (>= 4.0.10) - framework: dnxcore50
      System.Resources.ResourceManager (>= 4.0.0) - framework: dnxcore50
      System.Runtime (>= 4.0.20) - framework: dnxcore50
      System.Threading (>= 4.0.10) - framework: dnxcore50
    System.Private.Uri (4.0.1-beta-23225)
    System.Reflection (4.1.0-beta-23225)
      System.IO (>= 4.0.0) - framework: dnxcore50
      System.Reflection.Primitives (>= 4.0.0) - framework: dnxcore50
      System.Runtime (>= 4.0.20) - framework: dnxcore50
    System.Reflection.Extensions (4.0.1-beta-23225)
      System.Diagnostics.Debug (>= 4.0.10) - framework: dnxcore50
      System.Reflection (>= 4.0.0) - framework: dnxcore50
      System.Reflection (>= 4.0.10) - framework: dnxcore50
      System.Reflection.Primitives (>= 4.0.0) - framework: dnxcore50
      System.Reflection.TypeExtensions (>= 4.0.1-beta-23225) - framework: dnxcore50
      System.Resources.ResourceManager (>= 4.0.0) - framework: dnxcore50
      System.Runtime (>= 4.0.0) - framework: dnxcore50
      System.Runtime (>= 4.0.20) - framework: dnxcore50
      System.Runtime.Extensions (>= 4.0.10) - framework: dnxcore50
    System.Reflection.Primitives (4.0.1-beta-23225)
      System.Runtime (>= 4.0.0) - framework: dnxcore50
      System.Threading (>= 4.0.0) - framework: dnxcore50
    System.Resources.ResourceManager (4.0.1-beta-23225)
      System.Globalization (>= 4.0.0) - framework: dnxcore50
      System.Reflection (>= 4.0.0) - framework: dnxcore50
      System.Reflection (>= 4.0.10) - framework: dnxcore50
      System.Runtime (>= 4.0.0) - framework: dnxcore50
      System.Runtime (>= 4.0.20) - framework: dnxcore50
    System.Runtime (4.0.21-beta-23225)
      System.Private.Uri (>= 4.0.1-beta-23225) - framework: dnxcore50
    System.Runtime.Extensions (4.0.11-beta-23225)
      System.Runtime (>= 4.0.20) - framework: dnxcore50
    System.Runtime.Handles (4.0.1-beta-23225)
      System.Runtime (>= 4.0.0) - framework: dnxcore50
    System.Runtime.InteropServices (4.0.21-beta-23225)
      System.Reflection (>= 4.0.0) - framework: dnxcore50
      System.Reflection.Primitives (>= 4.0.0) - framework: dnxcore50
      System.Runtime (>= 4.0.0) - framework: dnxcore50
      System.Runtime.Handles (>= 4.0.0) - framework: dnxcore50
    System.Text.Encoding (4.0.11-beta-23225)
      System.Runtime (>= 4.0.0) - framework: dnxcore50
    System.Text.Encoding.Extensions (4.0.11-beta-23225)
      System.Runtime (>= 4.0.0) - framework: dnxcore50
      System.Text.Encoding (>= 4.0.10) - framework: dnxcore50
    System.Threading (4.0.11-beta-23225)
      System.Runtime (>= 4.0.0) - framework: dnxcore50
      System.Threading.Tasks (>= 4.0.0) - framework: dnxcore50
    System.Threading.Tasks (4.0.11-beta-23225)
      System.Runtime (>= 4.0.0) - framework: dnxcore50
    System.Threading.Timer (4.0.1-beta-23225)
      System.Runtime (>= 4.0.0) - framework: dnxcore50

but paket 2.17.0-alphaxxx fails to resolve.

mrinaldi · 2015-10-23T11:34:44Z

I can't repro. Is there an error message?

What exactly are you trying to do?
paket update? paket install with paket.dependencies and no paket.lock?

forki · 2015-10-23T11:37:26Z

no. the issue is the resolver doesn't find a resolution. (maybe it will eventually but I stopped after 10min)

I'm trying to do paket update on the coreclr sample repo. As I said 2.16 finds a resolution (in under a minute)

mrinaldi · 2015-10-23T11:54:31Z

This is weird.

Here's my output using 2.17.0-alpha002:

D:\Projects\paket-test>paket update
Paket version 2.17.0.0
Resolving packages for group Main:
 - Microsoft.NETCore.Runtime.CoreCLR.ConsoleHost-x64 1.0.0-beta-22914
 - Microsoft.NETCore.Runtime.CoreCLR-x64 1.0.1-beta-23225
 - runtime.win7.System.Console 4.0.0-beta-23409
 - System.Console 4.0.0-beta-23409
 - System.IO 4.0.11-beta-23225
 - System.IO.FileSystem.Primitives 4.0.0
 - System.Runtime 4.0.21-beta-23225
 - System.Runtime.InteropServices 4.0.21-beta-23225
 - System.Text.Encoding 4.0.11-beta-23225
 - System.Threading 4.0.11-beta-23225
 - System.Threading.Tasks 4.0.11-beta-23225
 - System.Reflection 4.1.0-beta-23225
 - System.Diagnostics.Tracing 4.0.21-beta-23225
 - System.Collections 4.0.11-beta-23225
 - System.Diagnostics.Debug 4.0.11-beta-23225
 - System.Globalization 4.0.11-beta-23225
 - System.ObjectModel 4.0.11-beta-23225
 - System.Runtime.Extensions 4.0.11-beta-23225
 - System.Text.Encoding.Extensions 4.0.11-beta-23225
 - System.Diagnostics.Contracts 4.0.1-beta-23225
 - System.Diagnostics.StackTrace 4.0.1-beta-23225
 - System.Diagnostics.Tools 4.0.1-beta-23225
 - System.Globalization.Calendars 4.0.1-beta-23225
 - System.Private.Uri 4.0.1-beta-23225
 - System.Reflection.Extensions 4.0.1-beta-23225
 - System.Reflection.Primitives 4.0.1-beta-23225
 - System.Resources.ResourceManager 4.0.1-beta-23225
 - System.Runtime.Handles 4.0.1-beta-23225
 - System.Threading.Timer 4.0.1-beta-23225
 - System.Reflection.TypeExtensions 4.0.1-beta-23409
 - System.Linq 4.0.0
Locked version resolution written to D:\Projects\paket-test\paket.lock
5 seconds - ready.

forki · 2015-10-23T11:59:23Z

mhm. ok. then I guees I need to check what's wrong here. thanks for testing

2015-10-23 13:54 GMT+02:00 Marcio Rinaldi notifications@github.com:

This is weird.

Here's my output using 2.17.0-alpha002:

D:\Projects\paket-test>paket update
Paket version 2.17.0.0
Resolving packages for group Main:

Microsoft.NETCore.Runtime.CoreCLR.ConsoleHost-x64 1.0.0-beta-22914

Microsoft.NETCore.Runtime.CoreCLR-x64 1.0.1-beta-23225

runtime.win7.System.Console 4.0.0-beta-23409

System.Console 4.0.0-beta-23409

System.IO 4.0.11-beta-23225

System.IO.FileSystem.Primitives 4.0.0

System.Runtime 4.0.21-beta-23225

System.Runtime.InteropServices 4.0.21-beta-23225

System.Text.Encoding 4.0.11-beta-23225

System.Threading 4.0.11-beta-23225

System.Threading.Tasks 4.0.11-beta-23225

System.Reflection 4.1.0-beta-23225

System.Diagnostics.Tracing 4.0.21-beta-23225

System.Collections 4.0.11-beta-23225

System.Diagnostics.Debug 4.0.11-beta-23225

System.Globalization 4.0.11-beta-23225

System.ObjectModel 4.0.11-beta-23225

System.Runtime.Extensions 4.0.11-beta-23225

System.Text.Encoding.Extensions 4.0.11-beta-23225

System.Diagnostics.Contracts 4.0.1-beta-23225

System.Diagnostics.StackTrace 4.0.1-beta-23225

System.Diagnostics.Tools 4.0.1-beta-23225

System.Globalization.Calendars 4.0.1-beta-23225

System.Private.Uri 4.0.1-beta-23225

System.Reflection.Extensions 4.0.1-beta-23225

System.Reflection.Primitives 4.0.1-beta-23225

System.Resources.ResourceManager 4.0.1-beta-23225

System.Runtime.Handles 4.0.1-beta-23225

System.Threading.Timer 4.0.1-beta-23225

System.Reflection.TypeExtensions 4.0.1-beta-23409

System.Linq 4.0.0
Locked version resolution written to D:\Projects\paket-test\paket.lock
5 seconds - ready.

—
Reply to this email directly or view it on GitHub
#1160 (comment).

mrinaldi · 2015-10-23T12:09:38Z

No problem.

Did you happen to run it in verbose mode? Does it happen every time you run it?
Let me know if you need any help with that.

forki · 2015-10-23T18:16:04Z

Just to verify my findings could you please try it again. Maybe like 5 times?
I guess the following is happening:

we have 2 sources and ask both in parallel about "give me all versions numbers of the following package"
whatever comes back first (with valid results) wins

And it seems we have results for both sources and whatever returns first decides if we find a resolution.

…es to #1160

forki · 2015-10-23T18:33:19Z

with the changes in #1163 we lose a bit of perf, but it seems to always resolve correctly.

mrinaldi added 15 commits October 22, 2015 15:59

Changes ResolverStrategy to option

5676421

Adds paket's strategy operator as a valid operator

3eac914

Adds strategy option for groups

78a68a0

Parses strategy option

ab5759f

Uses group resolver strategy when there is no strategy for the package

b9babef

Always use strategy max when the package is a root requirement

ca63e7b

Updates package to lastest but not its conflicting transitive packages

4cbc11b

Removing unnecessary condition

1543d1b

Update single package respects strategy overrides

5a5b6fd

Cleanup

5b504be

Resolves updating package first

6e58f4b

Cleanup

e41d70f

Resolves conflicts in ResolverStrategy

a2f107a

Clenaup

5bdff3b

Resolves conflicts in ResolverStrategy with Max

f1abb93

forki added a commit that referenced this pull request Oct 22, 2015

Merge pull request #1160 from mrinaldi/resolver_strategy

15762b4

Global resolver strategy option

forki merged commit 15762b4 into fsprojects:master Oct 22, 2015

forki mentioned this pull request Oct 23, 2015

Can't find resolution ctaggart/coreclr-demo#2

Closed

forki added a commit that referenced this pull request Oct 23, 2015

Queries every NuGet feed in parallel and combines the results - relat…

9d37f3e

…es to #1160

forki added a commit that referenced this pull request Oct 23, 2015

Queries every NuGet feed in parallel and combines the results - relat…

893a5a3

…es to #1160

forki mentioned this pull request Oct 23, 2015

Queries every NuGet feed in parallel and combines the results #1163

Merged

mrinaldi deleted the resolver_strategy branch October 24, 2015 00:16

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Global resolver strategy option #1160

Global resolver strategy option #1160

mrinaldi commented Oct 22, 2015

forki commented Oct 22, 2015

mrinaldi commented Oct 22, 2015

forki commented Oct 22, 2015

forki commented Oct 23, 2015

mrinaldi commented Oct 23, 2015

forki commented Oct 23, 2015

mrinaldi commented Oct 23, 2015

forki commented Oct 23, 2015

mrinaldi commented Oct 23, 2015

forki commented Oct 23, 2015

forki commented Oct 23, 2015

Global resolver strategy option #1160

Global resolver strategy option #1160

Conversation

mrinaldi commented Oct 22, 2015

forki commented Oct 22, 2015

mrinaldi commented Oct 22, 2015

forki commented Oct 22, 2015

forki commented Oct 23, 2015

mrinaldi commented Oct 23, 2015

forki commented Oct 23, 2015

mrinaldi commented Oct 23, 2015

forki commented Oct 23, 2015

mrinaldi commented Oct 23, 2015

forki commented Oct 23, 2015

forki commented Oct 23, 2015