Paket/Fake tooling

[WalternativE]

Hi!

I saw that there are already paket and fake fragments in the codebase which appear do not have the full functionality at the moment. Trying to build using the scripts fails.

Building in VS2017 works as expected (normal nuget restore and MSBuild execution by the IDE).

Is it the goal to transition to a more paket/fake focused tooling approach? If so I'd be glad to help. I'd just need to know what your "vision" for the project would be. If lay out work items in the issue tracker and mark them as up-for-grabs I could get going - or any other mode you prefer (just thought it's already here and doesn't cost anything extra).

Regards
Gregor

[HLWeil]

Hey!

Thanks for pointing out the build problem. I made some adjustments and it should be buildable with the scripts now.
This project was initiated using the ProjectScaffold and should therefore come with functional paket and FAKE tools.

My vision for this project is to create a plentiful toolbox for functional graph related tasks. You are kindly invited to participate by adding functions or ideas. I will add some tasks for easier tracking.

Greetings
Lukas

[WalternativE]

Hi!

Thank you for the update. Building with the scripts works now on Windows10. I could not build on Linux (Ubuntu on Windows Subsystem for Linux - Mono >= 5). Do you currently have Travis and appveyor running (saw the configs in the repo)?

Also: would you be interested to transition to .NET Standard/.NET Core and the new SDK based fsproj format? The project isn't very big and the move should be "rather" painless. Would have a couple of hours to kill today if you'd like a PR in this direction (I'd also fix the paket bootstrapper version in the same go - the version in the repo seems to be still affected by the GitHub TLS problem https://twitter.com/sforkmann/status/966952214819418112?ref_src=twsrc%5Etfw&ref_url=https%3A%2F%2Fsergeytihon.com%2F2018%2F02%2F24%2Ff-weekly-8-2018-paket-github-tls-disaster%2F).

Regards
Gregor

[HLWeil]

Hey

I neither use Travis nor appveyor. The fragments in the library also came from ProjectScaffold.

Yes those changes have to be done sooner or later, so I'd be thankful for you implementing them. The transition to .NET-Standard should also increase the compatibility with Mono if I'm not mistaken?

Greetings
Lukas

[WalternativE]

Hi,

I'll test both travis as well as appveyor config while working on the infrastructure transition. Yes - compability to all .NET frameworks that are standard compliant should be better afterwards (full .NET framework, Mono, Xamarin, netcore). Switching to dotnetcli tooling should also increase the reproducability on other systems (Linux, Mac, Windows, etc).

My current WIP branch is here https://github.com/WalternativE/FSharp.FGL/tree/project-infra-transition - fsproj files are almost all ready - still have to refactor the build script and test around a bit.

Do you want to keep NUnit? I'm pretty indifferent about testing frameworks so I'd just get it up to date and maybe get some simple tests running. If you want to get all the F# goodness Expecto (https://github.com/haf/expecto) would be worth a look. As far as I can see there are no tests yet anyway.

Also: are you using Octokit - I'd might get rid of it to reduce complexity (the same for sourcelink - except if you see pressing reasons to keep it - can be introduced again to the project at a later point in time anyway).

Do you already have plans for a NuGet release?

Sorry for asking so many questions by the way - really don't want to bother you.

Greetings
Gregor

[HLWeil]

What would be the benefits of using Expecto over NUnit?

And regarding dotnetcli: This switch should have some graver impacts on the Modus Operandi of developers, shouldn't it? In principle keeping the workflow similar between the CSBiology projects is important for our workgroup. How would this switch affect this?

Thanks a lot for the effort! The changes you've made so far in your repository look good. I'll read a bit more into your further suggestions and get back at you afterwards.

Greetings
Lukas

[WalternativE]

Both are testing frameworks - Expecto has a stronger focus on functional concepts but in the end it doesn't realy matter. As you mentioned in you answer the most sane thing would be to use what the rest of the group is using - if you tell me what your colleagues like best I'll just stick with that.

What is the current workflow of your team? If you use a current version of Visual Studio there shouldn't be any changes for you. Depending on all the other steps you normally have when releasing docs, bumping versions, etc I would need to know what those steps are :)

Additionaly: do you use paket (eg via the extension) in Visual studio as well? Just asking because of the current redundancies regarding the packages.config files.

Regarding testing: I've been browsing through a lot of major F# projects to get a general feeling what people are using for testing - apparently there is no consensus. xUnit seems to very popular though. The Visual Studio support is very good, Ionide test explorer support is coming and both fake as well as the dotnet cli seem to work well with it. Apart from that it has a lot of documentation and even FSCheck integration. I'd almost suggest to go for it.

Regarding sourcelink: I played around with sourcelink2 and the tooling seems to work well - unfortunately I can't verify that it works - the dotnet sourcelink test tools fails me. That could be because I'm working on a fork of the original github repo or because I missed to switch to the portable pdb format somewhere. I used the Argu build script as a reference to see how it was used (https://github.com/fsprojects/Argu/blob/10b0b1ba251eb340335633e5ff2a965ec6e50319/build.fsx#L131-L138)

--> I added a couple of changes to my WIP branch - you can check it out if the general direction is suitable for the project.

Regards
Gregor

[HLWeil]

Hey Gregor,

sry for the long delay, I did not want to answer mindlessly. Again many thanks for helping out. I've looked into the topics you mentioned and here's a more thorough answer.

Testing: As you've probably already seen. There's not really any testing done in our projects. Therefore there's no need to keep NUnit if some other Testing framework might be better suited. If you've got some good experience with another Framework feel free to exchange them.
Octokit: We've not used Octokit and I don't see any need to keep it atm.
CI: That's probably one of the more important things. I have not been using those despite their practicality. Were you able to build the library using travis or/and appveyor?
.paket: I don't use the extension I just edit the .dependancies and the .reference directly
Sourcelink: Similar to Octokit, this isn't really on my "Important"-checklist.
Nuget: The library is far from being refined enough to be released as a nuget package. Therefore I have no current plans for releasing it shortly.
Your branch: I've received an error when trying to build your branch, which was caused by sourcelink.

C:\Users\HLWei.nuget\packages\sourcelink.create.commandline\2.8.0\build\SourceLink.Create.CommandLine.targets(21,5): error MSB3073: The command "git rev-parse --show-toplevel" exited with code 128. [C:\Users\HLWei\Downloads\FSharp.FGL-project-infra-transition\src\FSharp.FGL\FSharp.FGL.fsproj]

Build FAILED.

Workflow: Well actually there's not much to say ^^. I use the integrated, FAKE-and-FSharpFormatting-based ProjectScaffold build tools. For documentation I altered them a bit. What exactly would the dotnetcli add in your opinion?

I hope this clears some things up.

Best regards
Lukas

[WalternativE]

Hmmmmm....sourcelink acting up again. Good - I'll simplify it as far as I can for now - and work on the CI tooling (will come around to it this weekend most likely - maybe Friday).

Dotnet Cli is just a nice XPlat way to work with .NET Standard and Core projects. The SDK based project format enables better tooling support (VS2017 goodies) and better standard nuget handling (doesn't matter this much as paket is already working really well). There are also other benefits to it - in the end VS still does its thing and shouldn't be bothered by the changes. .NET standard is the future anyway - embracing it doesn't hurt :)

[WalternativE]

Sorry for not getting back to this earlier. I had less time than I anticipated and was overly confident with this issue. I think I solved the cross platform and cross framework issues for now. It bugs me that sourcelink is acting up - but it can be reintroduced to the project at a later point in time (should not be the main concern right now).

If you take a look at my WIP branch you'll see that I made a couple of changes to the build file as well as the CI/CD configurations. I got travis and appveyor to work and set up the test project that it runs on both Linux, Windows, Mono, Full Framework and .NET core (core on both operating systems). The project also runs in Ionide and VS2017 (in VS the Test explorer works as well out of the box without the need for another plugin). I tested the VS setup without any plugins (VS2017 insider) as well as with the paket plugin (VS2017 Community). I only edited the project on Windows 10 so I don't have any data how it performs on VS Code in Linux or VS for Mac or the like.

The CI/CD pipes are currently watching my repo (as I can't configure them for CSBiology organisation - but it's super easy - you won't have any problems with that). The links to the builds are here:
https://travis-ci.org/WalternativE/FSharp.FGL
https://ci.appveyor.com/project/WalternativE/fsharp-fgl

Hope these changes are to your liking. If you like the state of the project I would open a PR for you.

[HLWeil]

Building your current WIP branch works like a charm and it's great that you got to run the build on the build servers too! Also nice that you set up the new testing environment.
Weirdly though I can't run any of the functions in fsharp script files because the "Aether.dll" does not get copied into the binaries folder.
Do you know what might be the reason for this? If that's fixed I'd say the present commits are good to go for a pull request

[WalternativE]

Do you have a gist of the full fsx file you are currently using? I'll investigate the problem asap. You are experiencing the problem in VS2017, aren't you?

[HLWeil]

Just this tiny snippet

#r @"src\FSharp.FGL\bin\Release\netstandard2.0\FSharp.FGL.dll"

open FSharp.FGL
open FSharp.FGL.Directed
Models.gilbert (fun x -> (x,"hey")) 10 0.5

With the following error:

File name: 'Aether, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null'
at FSharp.FGL.Directed.Edges.add[Vertex,Edge,Label](Vertex _arg1_0, Vertex _arg1_1, Edge _arg1_2, FSharpMap2 g) at FSharp.FGL.Algorithm.Models.innerAddEdges@15[Graph,Vertex](FSharpFunc2 fVertexKey, FSharpFunc2 fAddEdge, Int32 vertexCount, Double p, Random rnd, Int32 i, Int32 j, Graph g) at FSharp.FGL.Algorithm.Models.addEdges@24[Graph,Vertex](FSharpFunc2 fVertexKey, FSharpFunc2 fAddEdge, Int32 vertexCount, Double p, Random rnd, Int32 i, Graph g) at FSharp.FGL.Algorithm.Models.addEdges@24-1.Invoke(Int32 i, Graph g) at FSharp.FGL.Directed.Models.gilbert[Vertex,Label](FSharpFunc2 nodeInitializer, Int32 n, Double p)
at <StartupCode$FSI_0003>.$FSI_0003.main@()

WRN: Assembly binding logging is turned OFF.
To enable assembly bind failure logging, set the registry value [HKLM\Software\Microsoft\Fusion!EnableLog] (DWORD) to 1.
Note: There is some performance penalty associated with assembly bind failure logging.
To turn this feature off, remove the registry value [HKLM\Software\Microsoft\Fusion!EnableLog].

I built the project using the build.cmd. The build tools were installed together with VS2017 on Windows 10.

EDIT: If I add the Aether.dll manually from the packages folder it works fine.

[WalternativE]

I looked into it. As far as I know you still need to reference (as you did) all dependencies of the dll in your script for it to work (they would have to be merged into the main dll in order to make it work with a single reference - and I'd strongly advise against that). In the netstandard directory you have a *deps.json file which documents all the dependencies on the different runtimes.

For scripting purposes I usually either use this referencing style for library development

#r @"C:\Users\Gregor\.nuget\packages\aether\8.2.0\lib\netstandard1.6\Aether.dll"
#r @"bin\Release\netstandard2.0\FSharp.FGL.dll"

or this one (which I prefer)

#r @"C:\Users\Gregor\.nuget\packages\aether\8.2.0\lib\netstandard1.6\Aether.dll"
#load @"c:\dev\code\FSharp.FGL\src\FSharp.FGL\Graph.fs"
#load @"c:\dev\code\FSharp.FGL\src\FSharp.FGL\Models.fs"
#load @"c:\dev\code\FSharp.FGL\src\FSharp.FGL\Undirected.fs"
#load @"c:\dev\code\FSharp.FGL\src\FSharp.FGL\Directed.fs"

Naturally the relative paths vary depending on the location of your script file.

Are there other problems open to discuss?

[HLWeil]

No I didn't reference the Aether.dll in my fsx but just copied it to the folder containing the FSharp.FGL.dll. Sry for the ambiguous wording.
No other problems ^^. It's very tidy as far as I can tell.
Just copying the Aether.dll to the bin folder after the build would top things off. It's just that, given the dependancy, I don't see a case where anyone could make use of just the FSharp.FGL withouth Aether. So you can add the copy command and create the PR or create the PR and I'll add it afterwards.

[WalternativE]

Ok, I get it now. Copying the dependency directly in the bin directory isn't the prefered way to do it with netstandard assemblied - I'd rather not do this. Clients will in the long run consume the library via NuGet/Paket anyway - so the dependency will be rather explicit.

We already have a client in the project which uses FSharp.FGL with the dedicated runtimes - the test project. As the tests have to actually run on the machine the build directory will have the 'classic' content.

You could reference it like this

#r @"C:\dev\code\FSharp.FGL\tests\FSharp.FGL.Tests\bin\Release\net461\FSharp.FGL.dll"

The path would naturally look different on your machine (project location, configuration and framework version could differ).

We could also have multiple targets in the project build so it would create the 'proper' directories for it (and even pack them explicitly). I can research how other people approach this.

[HLWeil]

Maybe not the most elegant but definitely a very comfortable strategy was used in many CSBiology projects. Just storing a copy of every essential dll in the general "bin" directory of the solution via postbuild events. This way one who wants to run the code in a script file can easily grip the dll's he wants without the need to explicitly load any other libraries they depend on.
I'm not quite sure if I just can't see the elefant in the room right now, but to me this seems at least more intuitive.
Regarding the different targets: I don't think the repositoy is really big enough to make this worthwile. What kind of targets do you have in mind?

[WalternativE]

It's comfortable in the usage but is rather uncomfortable to maintain (the usual tradeoff). Postbuild events are pretty much 'the devil' and have to die - doing this with fake is an option, adding msbuild tasks is another (even though I'm not the biggest fan of writing msbuild tasks).

I added net461 as another build target for the library. The builds appear to go through on all platforms.
https://travis-ci.org/WalternativE/FSharp.FGL/builds/359017953
https://ci.appveyor.com/project/WalternativE/fsharp-fgl/build/0.0.1.11

You can now just build the library and reference the dll in the net461 build directory like this

#r @"C:\dev\code\FSharp.FGL\src\FSharp.FGL\bin\Release\net461\FSharp.FGL.dll"

This makes the final nuget package a bit bulkier, but it should be a tradeoff one can live with. What do you think?

[HLWeil]

Yes that's a good idea.
How about building both (the way you implemented in your current branch) but just pack the .netstandard version as a nuget package?
That to me seems to be the best of both worlds. I honestly have no experience with building nuget packages, but that should be doable, shouldn't it?

[WalternativE]

Yes, totally doable :) Do we want to open a thread for the nuget topic and close this one with a PR to merge to current state?

[HLWeil]

Yes. That seems appropriate