Proposal: two-stage NuGet-aware build scripts (Fake.Boot) #116

Closed
t0yv0 opened this Issue Mar 29, 2013 · 6 comments

Comments

Projects
None yet
3 participants
@t0yv0
Contributor

t0yv0 commented Mar 29, 2013

Hi,

I have a proposal - draft currently implemented here:

https://github.com/intellifactory/FAKE

The gist of it is to allow two-stage execution of FAKE scripts - first stage is BOOT, where F# code runs to resolve dependencies with NuGet, and second stage is as usual - and you can use the resolved dependencies there.

The downside - I pulled in a dependency on NuGet.Core into FakeLib - is this acceptable?

Also, how do I test this precisely? Will try on Mono myself, have not yet.

To elaborate, why this is useful: you can have a simple worflow:

emacs build.fsx
fake

With a script like this (https://gist.github.com/toyvo/8f1b541a051830b36a18), this pulls in the deps and you get a running website. You also do not mess with any explicit references inside build.fsx.

It also eases the pain of using NuGet repository for build automation. Extending FAKE is now easy - now people can just put their "tasks" on NuGet and then just use them.

How it works: FAKE first invokes the script with BOOT constant set - this stage uses NuGet API to resolve packages (in-process), this creates packages folder, then it computes the reference set using NuGet rules again, and generates a little F# script with "#r" directives. Then FAKE invokes the script again without the BOOT constant, and now the script can include the auto directive file and use the references resolved from NuGet.

Thanks,

Anton

@panesofglass

This comment has been minimized.

Show comment
Hide comment
@panesofglass

panesofglass Mar 30, 2013

Contributor

👍

Contributor

panesofglass commented Mar 30, 2013

👍

@t0yv0

This comment has been minimized.

Show comment
Hide comment
@t0yv0

t0yv0 Mar 30, 2013

Contributor

Question from Steffen: can we do this without a compiler flag.

Good question. There are two options that do not involve a compiler flag:

  1. Having two FSX files (boot.fsx and build.fsx) - this should be easy. I would keep the compiler flag as an option though - it is a matter of taste.
  2. Make changes to FSI to support staged execution. This would be ideal is it currently a realistic option? I guess if there was a way to do that so that I would get the modified FSI in my VS environment and on the build server, this could be OK.

The reason you cannot have a single script file and no compiler flags is hopefully very obvious - F# would complain that it cannot find the nuget-fetched references in the boot stage. I do not think there is currently (on stock FSI) a way to execute code in stages (feeding it to FSI in chunks), is there?

Contributor

t0yv0 commented Mar 30, 2013

Question from Steffen: can we do this without a compiler flag.

Good question. There are two options that do not involve a compiler flag:

  1. Having two FSX files (boot.fsx and build.fsx) - this should be easy. I would keep the compiler flag as an option though - it is a matter of taste.
  2. Make changes to FSI to support staged execution. This would be ideal is it currently a realistic option? I guess if there was a way to do that so that I would get the modified FSI in my VS environment and on the build server, this could be OK.

The reason you cannot have a single script file and no compiler flags is hopefully very obvious - F# would complain that it cannot find the nuget-fetched references in the boot stage. I do not think there is currently (on stock FSI) a way to execute code in stages (feeding it to FSI in chunks), is there?

@t0yv0

This comment has been minimized.

Show comment
Hide comment
@t0yv0

t0yv0 Apr 1, 2013

Contributor

I am doing some further testing and fixes. Apparently, assembly references should be specified in the correct order to work at all. In addition, on Mono 3.0.6 and latest fsharpi, the references do not get loaded in correct order because of this bug:

fsharp/fsharp#119

In addition, current release of NuGet.Core does not work on Mono - it fails to find resolved assemblies in the local repository. However, using the alpha version of the latest NuGet.Core from NuGet resolves this.

I think I can get ordering fixed this week and also add an option to do this with two files instead of compiler flags in one file. Overall, my changes should not affect the existing FAKE users (only specially marked FAKE scripts will run in a two-stage bootstrapping way), except the change would put a dependency from FakeLib.dll to NuGet.Core.dll.

Does that sound good? Will this have a chance of being merged into FAKE then? If NuGet.Core.dll dependency is considered a serious problem, could introduce dynamic loading to try to feature-detect FakeLib.NuGet.dll depending on NuGet.Core.dll, but I would prefer not to go there unless there is a real benefit.

Contributor

t0yv0 commented Apr 1, 2013

I am doing some further testing and fixes. Apparently, assembly references should be specified in the correct order to work at all. In addition, on Mono 3.0.6 and latest fsharpi, the references do not get loaded in correct order because of this bug:

fsharp/fsharp#119

In addition, current release of NuGet.Core does not work on Mono - it fails to find resolved assemblies in the local repository. However, using the alpha version of the latest NuGet.Core from NuGet resolves this.

I think I can get ordering fixed this week and also add an option to do this with two files instead of compiler flags in one file. Overall, my changes should not affect the existing FAKE users (only specially marked FAKE scripts will run in a two-stage bootstrapping way), except the change would put a dependency from FakeLib.dll to NuGet.Core.dll.

Does that sound good? Will this have a chance of being merged into FAKE then? If NuGet.Core.dll dependency is considered a serious problem, could introduce dynamic loading to try to feature-detect FakeLib.NuGet.dll depending on NuGet.Core.dll, but I would prefer not to go there unless there is a real benefit.

@forki

This comment has been minimized.

Show comment
Hide comment
@forki

forki Apr 2, 2013

Member

This sounds great. NuGet.Core.dll dependency is not a problem.
Cool stuff.

Member

forki commented Apr 2, 2013

This sounds great. NuGet.Core.dll dependency is not a problem.
Cool stuff.

@t0yv0

This comment has been minimized.

Show comment
Hide comment
@t0yv0

t0yv0 Apr 3, 2013

Contributor

I now also added a dependency on Mono.Cecil to resolve in which order to reference assemblies. This is unfortunately needed - see fsharp/fsharp#119 for details. With the correct order, and my suggested fix to F# for Mono, references work now.

I added the separate file workflow as requested - conf.fsx and build.fsx instead of build.fsx with compiler flags. The previous form is also available.

I also added some command line support, run fake boot --help.

Example workflow:

mkdir mydir
cd mydir
fake boot init # or `fake boot init one` for a single-script version
emacs conf.fsx # add a reference to say FParsec package
emacs build.fsx # you can `open FParsec` here and do something
fake boot auto # downloads packages, resolves references and runs `build.fsx` script
fake boot # only runs the `build.fsx` script
Contributor

t0yv0 commented Apr 3, 2013

I now also added a dependency on Mono.Cecil to resolve in which order to reference assemblies. This is unfortunately needed - see fsharp/fsharp#119 for details. With the correct order, and my suggested fix to F# for Mono, references work now.

I added the separate file workflow as requested - conf.fsx and build.fsx instead of build.fsx with compiler flags. The previous form is also available.

I also added some command line support, run fake boot --help.

Example workflow:

mkdir mydir
cd mydir
fake boot init # or `fake boot init one` for a single-script version
emacs conf.fsx # add a reference to say FParsec package
emacs build.fsx # you can `open FParsec` here and do something
fake boot auto # downloads packages, resolves references and runs `build.fsx` script
fake boot # only runs the `build.fsx` script
@t0yv0

This comment has been minimized.

Show comment
Hide comment
@t0yv0

t0yv0 Apr 4, 2013

Contributor

Available in FAKE now. Thanks.

Contributor

t0yv0 commented Apr 4, 2013

Available in FAKE now. Thanks.

@t0yv0 t0yv0 closed this Apr 4, 2013

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment