New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fable support #2310

Closed
alfonsogarciacaro opened this Issue May 2, 2017 · 14 comments

Comments

Projects
None yet
4 participants
@alfonsogarciacaro

Related: fable-compiler/Fable#856

I've been thinking about how we could use Paket for Fable dependencies. In 1.0 cycle, Fable libraries directly distribute their source code and it's the final user who takes care of compiling all the files (and bundle them) in order to get a better control of the whole compilation. The final user references libraries as project references.

There are two main things to consider here:

Distributing the packages and version management

After considering this, I think the easiest thing to do would be to just move Fable packages to Nuget (they're currently in npm), so downloading, version management, etc. is automatically handled by Paket. Fable lib authors would just have to make sure that the sources of their projects are distributed as content in their Nuget packages.

Reference Fable libs from user project

This is currently done by following the convention that Fable libs must have a single .fsproj in the root folder which users reference as <ProjectReference Include="path/to/node_modules/package-name/Project.Name.fsproj" />. This more or less works but the ugliest part is how to resolve the references among Fable dependencies, as the paths differ from lib development and when being consumed by the final user. At the moment this is solved by using a hard-coded condition that detects if the .fsproj file is in node_modules folder or not, if so it assumes other dependencies are on the same level.

An alternative way would be to make Paket create an artificial .fsproj file in the obj folder containing all the files from the dependency projects and reference it from the user project. I think @enricosada suggested something like this. The main trick is to create the whole dependency graph to order the files properly. Fable itself follows a similar strategy as it collects all files from referenced projects before sending them to FCS.

Thoughts?

@cloudRoutine

This comment has been minimized.

Show comment
Hide comment
@cloudRoutine

cloudRoutine May 2, 2017

Member
  • Could you go into some more detail about what this process involving paket is supposed to look like?
  • If the development and consumption paths are different can't this be handled by some sort of fable pack to create the desired nupkg structure?
  • When you say sources for the project I'm assuming you mean the .js files, do you mean they should be found in content or contentFiles?
  • Could this be handled by a .targets file so that all paket would need to do is add the targets to the fsproj?

I worry about putting additional stuff into the /obj/ dir as detritus in it has led to weird tooling issues for me in the past, but maybe it won't matter in this case. If you're talking about creating the same resource, props, and targets files that msbuild uses as manifest resources for the new msbuild15 fsproj, we're still relying on nuget to do that and I'm not sure if we could use the same mechanism for fable or if this would have to wait until we implement our own way to do this that doesn't involve nuget at all.

But overall I'm still really unclear on what specifically you want us to do.

Member

cloudRoutine commented May 2, 2017

  • Could you go into some more detail about what this process involving paket is supposed to look like?
  • If the development and consumption paths are different can't this be handled by some sort of fable pack to create the desired nupkg structure?
  • When you say sources for the project I'm assuming you mean the .js files, do you mean they should be found in content or contentFiles?
  • Could this be handled by a .targets file so that all paket would need to do is add the targets to the fsproj?

I worry about putting additional stuff into the /obj/ dir as detritus in it has led to weird tooling issues for me in the past, but maybe it won't matter in this case. If you're talking about creating the same resource, props, and targets files that msbuild uses as manifest resources for the new msbuild15 fsproj, we're still relying on nuget to do that and I'm not sure if we could use the same mechanism for fable or if this would have to wait until we implement our own way to do this that doesn't involve nuget at all.

But overall I'm still really unclear on what specifically you want us to do.

@forki

This comment has been minimized.

Show comment
Hide comment
@forki

forki May 3, 2017

Member

If I understand correctly we would scan for fsproj files in all referenced packages and build a targets file that imports all of these fsproj. We also would need to ensure some kind of order. Is that correct? Anything else to do?

Member

forki commented May 3, 2017

If I understand correctly we would scan for fsproj files in all referenced packages and build a targets file that imports all of these fsproj. We also would need to ensure some kind of order. Is that correct? Anything else to do?

@alfonsogarciacaro

This comment has been minimized.

Show comment
Hide comment
@alfonsogarciacaro

alfonsogarciacaro May 5, 2017

Hi again! Sorry, after considering this further and checking with other stakeholders (I always wanted to say that), we're going to explore distribution of .dll + F# source files. This makes it very easy for Paket and IDEs because then Fable packages are indistinguishable from others, and then Fable can pick the F# sources for the compilation.

The only thing that Fable would need is the full list of Fable dependencies in compilation order. For example, if the user had Fable.Elmisht.React in paket.dependencies, it'd be very nice if Paket could somehow provide the following:

Fable.PowerPack
Fable.React
Fable.Elmish
Fable.Elmish.React

Fable already contains an algorithm to resolve dependency graphs, so it's also ok to have something like this:

Fable.Elmish.React
    Fable.React
    Fable.Elmish
Fable.React
Fable.Elmish
    Fable.PowerPack
Fable.PowerPack

For now it should be fine to just assume packages prefixed with Fable. are Fable packages, we can think of something else later.

Hi again! Sorry, after considering this further and checking with other stakeholders (I always wanted to say that), we're going to explore distribution of .dll + F# source files. This makes it very easy for Paket and IDEs because then Fable packages are indistinguishable from others, and then Fable can pick the F# sources for the compilation.

The only thing that Fable would need is the full list of Fable dependencies in compilation order. For example, if the user had Fable.Elmisht.React in paket.dependencies, it'd be very nice if Paket could somehow provide the following:

Fable.PowerPack
Fable.React
Fable.Elmish
Fable.Elmish.React

Fable already contains an algorithm to resolve dependency graphs, so it's also ok to have something like this:

Fable.Elmish.React
    Fable.React
    Fable.Elmish
Fable.React
Fable.Elmish
    Fable.PowerPack
Fable.PowerPack

For now it should be fine to just assume packages prefixed with Fable. are Fable packages, we can think of something else later.

@matthid

This comment has been minimized.

Show comment
Hide comment
@matthid

matthid May 5, 2017

Member

Crazy thinking:

What about adding a frontend to npm into paket to stay in npm ecosystem as @forki is suggesting time and time again?

So having something like fablegroup where you can add npm packages. Resolving and installing via npm and adding some fable specific logic on top?

Member

matthid commented May 5, 2017

Crazy thinking:

What about adding a frontend to npm into paket to stay in npm ecosystem as @forki is suggesting time and time again?

So having something like fablegroup where you can add npm packages. Resolving and installing via npm and adding some fable specific logic on top?

@matthid

This comment has been minimized.

Show comment
Hide comment
@matthid

matthid May 5, 2017

Member

Disclaimer again: I have not used fable just jet. So I have no idea how possible this is technically...

Member

matthid commented May 5, 2017

Disclaimer again: I have not used fable just jet. So I have no idea how possible this is technically...

@alfonsogarciacaro

This comment has been minimized.

Show comment
Hide comment
@alfonsogarciacaro

alfonsogarciacaro May 5, 2017

npm is going to be needed anyways for tooling like Webpack, Babel, and other JS libraries, so even if we push Fable libs to Nuget, it could still be a good idea for Paket to interact with npm for people who prefer to use only one package manager.

npm is going to be needed anyways for tooling like Webpack, Babel, and other JS libraries, so even if we push Fable libs to Nuget, it could still be a good idea for Paket to interact with npm for people who prefer to use only one package manager.

@forki

This comment has been minimized.

Show comment
Hide comment
@forki

forki May 7, 2017

Member

@alfonsogarciacaro for MSBuild 15 projects we already put a *.fsproj.references file into the obj folder (on restore). This file contains a list of all deps and has following structure:

PackageName,Version,[Transitive|Direct]

would this be enough?

Member

forki commented May 7, 2017

@alfonsogarciacaro for MSBuild 15 projects we already put a *.fsproj.references file into the obj folder (on restore). This file contains a list of all deps and has following structure:

PackageName,Version,[Transitive|Direct]

would this be enough?

@alfonsogarciacaro

This comment has been minimized.

Show comment
Hide comment
@alfonsogarciacaro

alfonsogarciacaro May 7, 2017

True, I had seen that file before but it didn't cross my mind that it could be used for this purpose.

I just gave it a quick try: I created a project with a paket.references file including Fable.Compiler. However, after running paket install and dotnet restore, I cannot find in the obj/myproj.fsproj.references file the dependencies of Fable.Compiler (like Fable.Core or Newtonsoft.Json), which means it's not useful to solve the full dependency graph. Or am I missing something?

True, I had seen that file before but it didn't cross my mind that it could be used for this purpose.

I just gave it a quick try: I created a project with a paket.references file including Fable.Compiler. However, after running paket install and dotnet restore, I cannot find in the obj/myproj.fsproj.references file the dependencies of Fable.Compiler (like Fable.Core or Newtonsoft.Json), which means it's not useful to solve the full dependency graph. Or am I missing something?

@forki

This comment has been minimized.

Show comment
Hide comment
@forki

forki May 7, 2017

Member

can you upload your test project for me to repro?

Member

forki commented May 7, 2017

can you upload your test project for me to repro?

@alfonsogarciacaro

This comment has been minimized.

Show comment
Hide comment
@alfonsogarciacaro

alfonsogarciacaro May 7, 2017

Ups, it seems I did mess up things: when building the repro repo (you see what I did here, right?) I did see all the dependencies, so that should be fine. Now the only issue would be the order of the dependencies. If I only include dotnet-fable in paket.references I correctly get the dependencies in (inverse) compilation order: dotnet-fable < Fable.Compiler < Fable.Core. However, if I list all the dependencies in paket.references, they will appear exactly in the same order in the *.fsproj.references, so this may be a problem if the user lists them in arbitrary order:

FSharp.Core
dotnet-fable
Fable.Core
Fable.Compiler

Repro repo: https://github.com/alfonsogarciacaro/TestProject

Ups, it seems I did mess up things: when building the repro repo (you see what I did here, right?) I did see all the dependencies, so that should be fine. Now the only issue would be the order of the dependencies. If I only include dotnet-fable in paket.references I correctly get the dependencies in (inverse) compilation order: dotnet-fable < Fable.Compiler < Fable.Core. However, if I list all the dependencies in paket.references, they will appear exactly in the same order in the *.fsproj.references, so this may be a problem if the user lists them in arbitrary order:

FSharp.Core
dotnet-fable
Fable.Core
Fable.Compiler

Repro repo: https://github.com/alfonsogarciacaro/TestProject

@forki

This comment has been minimized.

Show comment
Hide comment
@forki

forki May 7, 2017

Member

ok fixed in latest paket 5 alpha.

Member

forki commented May 7, 2017

ok fixed in latest paket 5 alpha.

@forki

This comment has been minimized.

Show comment
Hide comment
@forki

forki May 7, 2017

Member

they are now ordered so that a package down in the list only references packages above itself (similar to what we do with fs files in a fsproj).

Member

forki commented May 7, 2017

they are now ordered so that a package down in the list only references packages above itself (similar to what we do with fs files in a fsproj).

@alfonsogarciacaro

This comment has been minimized.

Show comment
Hide comment
@alfonsogarciacaro

alfonsogarciacaro May 7, 2017

Super-fast, thank you! I'll try to have a prototype asap 👍 I will also put Fable.Core JS files in the Nuget package to prevent the Fable.Core / fable-core split.

BTW, not directly related but what would it take for Paket to manage also DotNetCliToolReferences? That would be the last piece to manage all Fable libs and tools (at least on .NET side).

I'm ccing @enricosada just because I like to bother him constantly ;)

Super-fast, thank you! I'll try to have a prototype asap 👍 I will also put Fable.Core JS files in the Nuget package to prevent the Fable.Core / fable-core split.

BTW, not directly related but what would it take for Paket to manage also DotNetCliToolReferences? That would be the last piece to manage all Fable libs and tools (at least on .NET side).

I'm ccing @enricosada just because I like to bother him constantly ;)

@forki

This comment has been minimized.

Show comment
Hide comment
@forki

forki May 7, 2017

Member
Member

forki commented May 7, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment