Dependency manager

[MecuSorin]

Description

Attempted to play with the new features in FSharpInteractive using the FSharp.DependencyManager.Paket, but encountered some issues. Was thinking that we should provide an easier experience for newcomers.

Repro steps

In a new folder:

dotnet new tool-manifest
dotnet tool install paket
dotnet paket init
dotnet paket add FSharp.DependencyManager.Paket

Changed the settings.json for VSCode to include
{
"FSharp.fsiExtraParameters": [
"--compilertool:c:/Users/Sorin/.nuget/packages/fsharp.dependencymanager.paket/6.0.0-alpha053/lib/netstandard2.0"
]
}
Then created something.fsx with the content:

#r "paket: nuget FParsec"
open FParsec

Expected behavior

The script to run in FSI without issues

Actual behavior

exception while resolving dependencies:

System.Exception: Paket was not found in 'd:\dev\fsharp\sandbox\mlnet' or a parent directory, or 'C:\Users\Sorin'. Please download the tool and place it in one of the locations.

Known workarounds

To manually copy paket.exe in a reachable location for the FSI. The problem with that is that on a tool version change most probably the tooling around FSI will not work properly anymore.

[MecuSorin]

For reference I copy paste here the discussion from the gitter channel:
mecusorin @mecusorin_twitter Nov 12 12:46
Wanted to play with the new fsi features in a new folder:

dotnet new tool-manifest
dotnet tool install paket
dotnet paket init
dotnet paket add FSharp.DependencyManager.Paket
Change the settings.json for VSCode to include

{
"FSharp.fsiExtraParameters": [
"--compilertool:c:/Users/Sorin/.nuget/packages/fsharp.dependencymanager.paket/6.0.0-alpha053"
]
}
Then create something.fsx with the content:

#r "paket: FParsec"
open FParsec
And send to the FSI the content of the file
I get the error

somethig.fsx(2,1): error FS3216: Package manager key 'paket'
was not registered in [C:\Program Files\dotnet\sdk\5.0.100\FSharp; C:\Program Files\dotnet\sdk\5.0.100\FSharp], [c:/Users/Sorin/.nuget/packages/fsharp.dependencymanager.paket/6.0.0-alpha053]. Currently registered: nuget
What I am doing wrong?

mecusorin @mecusorin_twitter Nov 12 12:56
Also attempted with settings.json containing:
{
"FSharp.fsiExtraParameters": [
"--compilertool:c:/Users/Sorin/.nuget/packages/fsharp.dependencymanager.paket/6.0.0-alpha053/lib/netstandard2.0"
]
}

mecusorin @mecusorin_twitter Nov 12 13:04
the error was:

exception while resolving dependencies: System.Exception: Paket was not found in 'd:\dev\fsharp\sandbox\mlnet' or a parent directory, or 'C:\Users\Sorin'. Please download the tool and place it in one of the locations.
We need to make the tool search the default tools folder

I iterate: What is the idiomatic way to make this work? What I am doing wrong in my approach?

Gauthier Segay @smoothdeveloper 11:38
@mecusorin_twitter Hello, you also need to have paket.exe in either location up the directory structure, or in your user directory, it expects a .paket folder with paket.exe
if that solves the issue, could you help improve the current documentation: https://github.com/fsprojects/Paket/blob/master/docs/content/fsi-integration.md ?
I personally go with the first route, I copy the assembly next to fsi.exe
the other thing, is how you reference with paket in the script, it keeps the same syntax as paket.dependencies file, so you'd need:
#r "paket: nuget FParsec"
open FParsec

mecusorin @mecusorin_twitter 12:05
@smoothdeveloper Thanks for reply. Don't want to copy paket.exe near fsi.exe, because is the short path to a lot of funny issues as soon as paket changes major (or even minor) version. I was thinking that the best bet is to add to the paket tool a command to setup itself (or a list of steps for the user to follow) for fsi integration. Something similar with the generate-load-scripts. About the actual script good catch!

Gauthier Segay @smoothdeveloper 12:05
@mecusorin_twitter it is just the extension dll you need to copy there
I've made a PR with changes to the doc: https://github.com/fsprojects/Paket/blob/6cf49cb4fa5806b1a83db32d544037d7acef11ee/docs/content/fsi-integration.md
ok, yes we should consider better installation scenario, for now I'm trying to keep the documentation as straight forward so it works for usual scenarios, for me it is working well with dotnet fsi and Visual Studio 2019 copying the dll and having a version of paket under ~/.paket/paket.exe

mecusorin @mecusorin_twitter 12:08
@smoothdeveloper Why is not having another default to search for ~.nuget\packages\paket\ ... ?

Gauthier Segay @smoothdeveloper 12:08
@mecusorin_twitter I don't think this is default? is it?

mecusorin @mecusorin_twitter 12:09
I was saying in addition to the current default

Gauthier Segay @smoothdeveloper 12:09
because it would need to check the latest version, which is more logic in the extension
I think we'd just want to rely on dotnet paket in the future, by the time we made the prototype of the extension, I don't know if it was already used as a dotnet tool.
Would you mind creating a discussion issue on paket repository to list some of the improvements you'd like to see and we can try to bring some of the maintainers to see what makes most sense.

mecusorin @mecusorin_twitter 12:11
And again the same issue, I need to manually copy something somewhere I will forgot about and will byte me back without knowing what is wrong

Gauthier Segay @smoothdeveloper 12:11
historically, paket users are expected to have a .paket somewhere in the folder hierarchy where they do their scripts, and leverage generate-load-scripts.

mecusorin @mecusorin_twitter 12:12

For reference this are my home folders we talked about:

Gauthier Segay @smoothdeveloper 12:13
for now this is only deployment scenario, copying dll or passing argument to fsi / tooling, to make it more robust will require some engineering, I don't think it is simple, and you can see that Microsoft just copies the nuget version of the extension next to FSI, for now most natural is to do the same.

mecusorin @mecusorin_twitter 12:13
I have paket installed as global tool also

Gauthier Segay @smoothdeveloper 12:13
You may contribute to the extension, it is not lots of code.
https://github.com/fsprojects/Paket/tree/master/src/FSharp.DependencyManager.Paket

mecusorin @mecusorin_twitter 12:16
Will take a look

Gauthier Segay @smoothdeveloper 12:16

Paket/src/FSharp.DependencyManager.Paket/ReferenceLoading.PaketHandler.fs

Lines 60 to 80 in a489335

	let findPaketExe (prioritizedSearchPaths: string seq) (baseDir: DirectoryInfo) =
	let prioritizedSearchPaths = prioritizedSearchPaths |> Seq.map (fun d -> DirectoryInfo d)
	// for each given directory, we look for paket.exe and .paket/paket.exe
	let getPaketAndExe (directory: DirectoryInfo) =
	match directory.GetFiles(PM_EXE) with
	| [| exe |] -> Some exe.FullName
	| _ ->
	match directory.GetDirectories(PM_DIR) with
	| [| dir |] ->
	match dir.GetFiles(PM_EXE) with
	| [| exe |] -> Some exe.FullName
	| _ -> None
	| _ -> None
	let allDirs =
	Seq.concat [prioritizedSearchPaths ; getDirectoryAndAllParentDirectories baseDir]
	allDirs
	|> Seq.choose getPaketAndExe
	|> Seq.tryHead

this is the specific part, I think it is ok to add location you mention to try.
if you are able to get it working with the updated instructions, could you leave a comment to my PR just to confirm: #3949

mecusorin @mecusorin_twitter 12:49
@smoothdeveloper Looking at the code: I observed that is invoking the tool, why not doing dotnet paket instead directly, and if that fails to fallback to searching the tool to invoke it the way is doing it now?

Gauthier Segay @smoothdeveloper 12:52
yes that sounds reasonable, I'm not sure it is doable to test if dotnet paket exists because then it would incur the cost of running the command and checking the return value or parsing the output, not sure if that amounts for significant time before checking the folders.

There is also the question of the priority order in which those checks would go.

I guess when most of this code was put in first place (was in fsharp repo) the dotnet tooling was not yet available as it is since dotnet core 3

mecusorin @mecusorin_twitter 12:56
We need to talk with @forki about this, I want to invest some time in this, but only when everything is clear about the best course of action

mecusorin @mecusorin_twitter 13:12
I suppose that we can check the dotnet tool manifest file for paket

Gauthier Segay @smoothdeveloper 13:26
@mecusorin_twitter you may either create an issue or append comment to my PR on the doc to keep track of the idea to run dotnet paket and additional folders in the lookup, I think that will help Steffen to pick up and put his answer in the issue / PR

Gauthier Segay @smoothdeveloper 13:32
and yes, no problem you don't have to look at implementing in the project, you may play around in script to try to call dotnet paket or check the manifest.

It seems dotnet tool list would give easy output to parse, looking if the implementation of that command can be referenced in the code.

~/.dotnet/tools/paket.exe would be for the global install.

[MecuSorin]

My ideas from that discussion:

• the tool to attempt to use in the following order

  • local dotnet paket
  • local .paket/paket.exe
  • global dotnet paket tool
  • C:\Users\Sorin\.paket\paket.exe
  • most recent paket.exe compatible with the current framework used from c:\Users\Sorin\.nuget\packages\paket\

• either use a symlink folder for paket tooling and just update that link on new paket versions

A helper subcommand for paket (similar to generate-load-scripts) would be nice: to setup the environment for FSI (what can be automated, like the first few steps from my repo) and to guide the user on how can activate the extension through clear instructions.

[smoothdeveloper]

@MecuSorin this very sensible outline of the adjustments to better pick up paket.exe in the environment.

Also, if the resolution fails, we should list what has been attempted so the user has better time actionning towards a solution:

:paket> couldn't find paket.exe after searching for it:
:paket>    - "dotnet paket" local tool
:paket>    - /long/path/to/there/.paket/paket.exe
:paket>    - /long/path/to/.paket/paket.exe
:paket>    - /long/path/.paket/paket.exe
:paket>    - /long/.paket/paket.exe
:paket>    - /.paket/paket.exe
:paket>    - paket.exe from PATH environment variable
:paket>    - C:\Users\Sorin\.paket\paket.exe
:paket>    - C:\Users\Sorin\.nuget\packages\paket\

It may also print

:paket> Found paket: C:\Users\Sorin\.paket\paket.exe

[smoothdeveloper]

@MecuSorin the latest release of the extension should have some of the extra search paths and error message which improves the situation.

The only thing not implemented is picking up local dotnet paket.