Allow easy build/format/build cycle of external projects (#233) (#254)

* Initial implementation of testing external projects (git clone, build, format, build again). For now tests with the Argu project which fails ATM * Improve building of external projects for regression testing. Add configuration function that allows platform specific configuration of external projects build exectuable and arguments TestExternalProjects is now dependent on Build only for faster testing Platform specific selection of Fanotmas executable from previous build step Shallow cloning of external repos for faster clones; switched to always cloning fresh repo instead of resetting * Fix 452 path * Attempt to add build stages to enable long running external tests on certain conditions in travisCI * Set sudo to true which will AFAIK disable container based builds * Switch back to container based builds, try Argu as an example external project on CI * Reverting sudo to true again since container builds take much more time in startup * Improve build script process name/args handling, fix paths for nix dotnet core testing * Cleanup of accidental changes vs master * More cleanup * Revert accidental whitespace changes in test file * Add testing of external projects only when on a branch named build-external-projects * Final cleanup and comments * Dummy commit to trigger rebuilds after github issues
fsprojects · Oct 23, 2018 · 693baa3 · 693baa3
1 parent d4c766f
commit 693baa3
Show file tree

Hide file tree

Showing 8 changed files with 144 additions and 48 deletions.
diff --git a/.gitignore b/.gitignore
@@ -177,4 +177,7 @@ paket-files/
 
 # JetBrains Rider
 .idea/
-*.sln.iml
+*.sln.iml
+
+# external projects directory for extensive regression tests
+external-project-tests/
diff --git a/.travis.yml b/.travis.yml
@@ -2,11 +2,12 @@ language: csharp
 dotnet: 2.1.300
 mono: latest
 
-sudo: true  # use the new container-based Travis infrastructure 
+sudo: true  # as of October 2018 setting this to false is much slower as mono and dotnet are installed into the containers
 
-branches:
-  only:
-    - master
-
-script: 
-  - ./build.sh All
+jobs:
+  include:
+    - script:
+      - ./build.sh All
+    - script:
+      - ./build.sh TestExternalProjects
+      if: branch = build-external-projects
diff --git a/README.md b/README.md
@@ -10,7 +10,7 @@ F# source code formatter, inspired by [scalariform](https://github.com/mdr/scala
 
 ## How to use
 
-### Command line tool / API 
+### Command line tool / API
 Use this command to install Fantomas as a dotnet SDK global tool:
 
 ```
@@ -76,7 +76,7 @@ Fantomas follows the formatting guideline being described in [A comprehensive gu
 ## Use cases
 The project is developed with the following use cases in mind:
 
- - Reformatting an unfamiliar code base. It gives readability when you are not the one originally writing the code. 
+ - Reformatting an unfamiliar code base. It gives readability when you are not the one originally writing the code.
 To illustrate, the following example
 
 	```fsharp
@@ -93,24 +93,24 @@ To illustrate, the following example
 	will be rewritten to
 
 	```fsharp
-	type Type = 
+	type Type =
 	    | TyLam of Type * Type
 	    | TyVar of string
 	    | TyCon of string * Type list
-	    override this.ToString() = 
+	    override this.ToString() =
 	        match this with
 	        | TyLam(t1, t2) -> sprintf "(%s -> %s)" (t1.ToString()) (t2.ToString())
 	        | TyVar a -> a
 	        | TyCon(s, ts) -> s
 	```
 
- - Converting from verbose syntax to light syntax. 
+ - Converting from verbose syntax to light syntax.
 Feeding a source file in verbose mode, Fantomas will format it appropriately in light mode.
 This might be helpful for code generation since generating verbose source files is much easier.
 For example, this code fragment
 
 	```fsharp
-	let Multiple9x9 () = 
+	let Multiple9x9 () =
 	    for i in 1 .. 9 do
 	        printf "\n";
 	        for j in 1 .. 9 do
@@ -119,17 +119,17 @@ For example, this code fragment
 	        done;
 	    done;;
 	Multiple9x9 ();;
-	```	
-	is reformulated to 
-	
+	```
+	is reformulated to
+
 	```fsharp
-	let Multiple9x9() = 
+	let Multiple9x9() =
 	    for i in 1..9 do
 	        printf "\n"
 	        for j in 1..9 do
 	            let k = i * j
 	            printf "%d x %d = %2d " i j k
-	
+
 	Multiple9x9()
 	```
 
@@ -138,22 +138,22 @@ For example, this code fragment
 For more complex examples, please take a look at F# outputs of [20 language shootout programs](tests/languageshootout_output) and [10 CodeReview.SE source files](tests/stackexchange_output).
 
 ## Installation
-The code base is written in F# 4.X /.NET standard 2.0. 
+The code base is written in F# 4.X /.NET standard 2.0.
 The solution file can be opened in Visual Studio 2017, VS Code (with the [ionide plugin](http://ionide.io/)) & [Jetbrains Rider](http://jetbrains.com/rider/).
 Paket is used to manage external packages.
 The [test project](src/Fantomas.Tests) depends on FsUnit and NUnit.
 However, the [library project](src/Fantomas) and [command line interface](src/Fantomas.Cmd) have no dependency on external packages.
 
 ## Testing and validation
 We have tried to be careful in testing the project.
-There are 329 unit tests and 30 validated test examples, 
+There are 329 unit tests and 30 validated test examples,
 but it seems some corner cases of the language haven't been covered.
 Feel free to suggests tests if they haven't been handled correctly.
 
 ## Why the name "Fantomas"?
-There are a few reasons to choose the name as such. 
-First, it starts with an "F" just like many other F# projects. 
-Second, Fantomas is my favourite character in the literature. 
+There are a few reasons to choose the name as such.
+First, it starts with an "F" just like many other F# projects.
+Second, Fantomas is my favourite character in the literature.
 Finally, Fantomas has the same Greek root as "[phantom](https://en.wiktionary.org/wiki/phantom)"; coincidentally F# ASTs and formatting rules are so *mysterious* to be handled correctly.
 
 ## How to contribute
@@ -168,7 +168,7 @@ Fantomas' features are basically two commands: *format a document* or *format a
 
 They both consist in the same two stages:
 
-- parse the code and generate the F# AST (Abstract Syntax Tree). This is provided by the 
+- parse the code and generate the F# AST (Abstract Syntax Tree). This is provided by the
 by the FSharp.Compiler.Services library (see the `parse` function in
 [CodeFormatterImpl.fs](src/Fantomas/CodeFormatterImpl.fs)).
 - rewrite the code based on the AST (previous step) and formatting settings.
@@ -193,7 +193,7 @@ let ``discriminated unions declaration``() =
     formatSourceString false "type X = private | A of AParameters | B" config
     |> prepend newline
     |> should equal """
-type X = 
+type X =
     private
     | A of AParameters
     | B
@@ -222,7 +222,7 @@ that formats the string in input and prints it.
 The time it took to contribute is sometimes mentioned, as a side note.
 
 #### Fixing code generation
- - Record type formatting generated invalid F# code in some cases ([#197](https://github.com/dungpa/fantomas/pull/197)) - (2h fix)  
+ - Record type formatting generated invalid F# code in some cases ([#197](https://github.com/dungpa/fantomas/pull/197)) - (2h fix)
 
 ## Credits
 We would like to gratefully thank the following persons for their contributions.
@@ -237,5 +237,5 @@ We would like to gratefully thank the following persons for their contributions.
  - [Enrico Sada](https://github.com/enricosada)
 
 ## License
-The library and tool are available under Apache 2.0 license. 
+The library and tool are available under Apache 2.0 license.
 For more information see the [License file](LICENSE.md).
diff --git a/build.fsx b/build.fsx
@@ -1,5 +1,5 @@
 // --------------------------------------------------------------------------------------
-// FAKE build script 
+// FAKE build script
 // --------------------------------------------------------------------------------------
 
 #r @"packages/build/FAKE/tools/FakeLib.dll"
@@ -10,12 +10,12 @@ open System
 
 
 // Git configuration (used for publishing documentation in gh-pages branch)
-// The profile where the project is posted 
+// The profile where the project is posted
 let gitHome = "https://github.com/dungpa"
 // The name of the project on GitHub
 let gitName = "fantomas"
 
-// The name of the project 
+// The name of the project
 // (used by attributes in AssemblyInfo, name of a NuGet package and directory in 'src')
 let project = "Fantomas"
 
@@ -32,11 +32,11 @@ let configuration = "Release"
 
 // Longer description of the project
 // (used as a description for NuGet package; line breaks are automatically cleaned up)
-let description = """This library aims at formatting F# source files based on a given configuration. 
-Fantomas will ensure correct indentation and consistent spacing between elements in the source files. 
-Some common use cases include 
-(1) Reformatting a code base to conform a universal page width 
-(2) Converting legacy code from verbose syntax to light syntax 
+let description = """This library aims at formatting F# source files based on a given configuration.
+Fantomas will ensure correct indentation and consistent spacing between elements in the source files.
+Some common use cases include
+(1) Reformatting a code base to conform a universal page width
+(2) Converting legacy code from verbose syntax to light syntax
 (3) Formatting auto-generated F# signatures."""
 
 // List of author names (for NuGet package)
@@ -50,12 +50,47 @@ let solutionFile  = "fantomas"
 // Environment.CurrentDirectory <- __SOURCE_DIRECTORY__
 let release = parseReleaseNotes (IO.File.ReadAllLines "RELEASE_NOTES.md")
 
+// Types and helper functions for building external projects (see the TestExternalProjects target below)
+type ProcessStartInfo =
+    { ProcessName : string
+      Arguments : string list }
+
+type ExternalProjectInfo =
+    { GitUrl : string
+      DirectoryName : string
+      Tag : string
+      SourceSubDirectory : string
+      BuildConfigurationFn : (string -> ProcessStartInfo) }
+
+// Construct the commands/arguments for running an external project build script for both windows and linux
+// For linux we run this by invoking sh explicitly and passing the build.sh script as an argument as some
+// projects generated on windows don't have the executable permission set for .sh scripts. On windows we
+// treat .cmd files as executable
+let configureBuildCommandFromDefaultFakeBuildScripts pathToProject =
+    if Fake.EnvironmentHelper.isWindows
+    then { ProcessName = pathToProject </> "build.cmd"; Arguments = [ "Build" ] }
+    else { ProcessName = "sh"; Arguments = [ sprintf "%s/build.sh Build" pathToProject ] }
+
+// Construct the path of the fantomas executable to use for external project tests
+let fantomasExecutableForExternalTests projectdir =
+    if Fake.EnvironmentHelper.isWindows
+    then { ProcessName = sprintf "%s/src/Fantomas.Cmd/bin/%s/net452/dotnet-fantomas.exe" projectdir configuration; Arguments = [] }
+    else { ProcessName = "dotnet"; Arguments = [ sprintf "%s/src/Fantomas.CoreGlobalTool/bin/%s/netcoreapp2.1/fantomas-tool.dll" projectdir configuration ] }
+
+let externalProjectsToTest = [
+    { GitUrl = @"https://github.com/fsprojects/Argu"
+      DirectoryName = "Argu"
+      Tag = "5.1.0"
+      SourceSubDirectory = "src"
+      BuildConfigurationFn = configureBuildCommandFromDefaultFakeBuildScripts }
+    ]
+
 // --------------------------------------------------------------------------------------
 // Clean build results & restore NuGet packages
 
 Target "Clean" (fun _ ->
     CleanDirs [
-        "bin" 
+        "bin"
         "nuget"
         "src/Fantomas/bin"
         "src/Fantomas/obj"
@@ -71,7 +106,7 @@ Target "AssemblyInfo" (fun _ ->
       [ Attribute.Product project
         Attribute.Description summary
         Attribute.Version release.AssemblyVersion
-        Attribute.FileVersion release.AssemblyVersion ] 
+        Attribute.FileVersion release.AssemblyVersion ]
 
   CreateFSharpAssemblyInfo "src/Fantomas/AssemblyInfo.fs"
       ( Attribute.InternalsVisibleTo "Fantomas.Tests" :: Attribute.Title "FantomasLib" :: shared)
@@ -94,21 +129,21 @@ Target "ProjectVersion" (fun _ ->
 // Build library & test project
 Target "Build" (fun _ ->
     DotNetCli.Build (fun p ->
-        { p with 
-            Project = (sprintf "src/%s.sln" solutionFile) 
+        { p with
+            Project = (sprintf "src/%s.sln" solutionFile)
             Configuration = configuration
         }
     )
 )
 
 Target "UnitTests" (fun _ ->
     DotNetCli.Test (fun p ->
-        { p with 
+        { p with
             Project = "src/Fantomas.Tests/Fantomas.Tests.fsproj"
             Configuration = configuration
             AdditionalArgs = ["--no-build --no-restore --test-adapter-path:. --logger:nunit;LogFilePath=../../TestResults.xml"]
         }
-    ) 
+    )
 )
 
 // --------------------------------------------------------------------------------------
@@ -136,12 +171,66 @@ Target "Pack" (fun _ ->
             ] |> String.concat " "
         "pack src/"+project+"/"+project+".fsproj -c "+ configuration + " -o ../../bin " + packParameters
         |> DotNetCli.RunCommand id
-    
+
     pack "Fantomas"
     pack "Fantomas.Cmd"
     pack "Fantomas.CoreGlobalTool"
 )
 
+
+// This takes the list of external projects defined above, does a git checkout of the specified repo and tag,
+// tries to build the project, then reformats with fantomas and tries to build the project again. If this fails
+// then there was a regression in fantomas that mangles the source code
+
+Target "TestExternalProjects" (fun _ ->
+    let externalBuildErrors =
+        externalProjectsToTest
+        |> List.map (fun project ->
+            let relativeProjectDir = sprintf "external-project-tests/%s" project.DirectoryName
+
+            Fake.FileHelper.CleanDir relativeProjectDir
+            // Use "shallow" clone by setting depth to 1 to only check out the one commit we want to build
+            Fake.Git.CommandHelper.gitCommand "." (sprintf "clone --branch %s --depth 1 %s %s" project.Tag project.GitUrl relativeProjectDir)
+
+            let fullProjectPath = sprintf "%s/%s" __SOURCE_DIRECTORY__ relativeProjectDir
+            let buildStartInfo = project.BuildConfigurationFn fullProjectPath
+
+            let buildExternalProject() = ExecProcess
+                                            (fun info -> info.FileName <- buildStartInfo.ProcessName
+                                                         info.WorkingDirectory <- relativeProjectDir
+                                                         info.Arguments <- String.Join(" ", buildStartInfo.Arguments))
+                                            (TimeSpan.FromMinutes 5.0)
+
+            let cleanResult = buildExternalProject()
+            if cleanResult <> 0 then failwithf "Initial build of external project %s returned with a non-zero exit code" project.DirectoryName
+
+            let fantomasStartInfo =
+                fantomasExecutableForExternalTests __SOURCE_DIRECTORY__
+            let arguments =
+                fantomasStartInfo.Arguments @ [ sprintf "--recurse %s" project.SourceSubDirectory ]
+                |> fun args -> String.Join(" ", args)
+            let invokeFantomas() = ExecProcess
+                                    (fun info -> info.FileName <- fantomasStartInfo.ProcessName
+                                                 info.WorkingDirectory <- sprintf "%s/%s" __SOURCE_DIRECTORY__ relativeProjectDir
+                                                 info.Arguments <- arguments)
+                                    (TimeSpan.FromMinutes 5.0)
+            let fantomasResult = invokeFantomas()
+
+            if fantomasResult <> 0
+            then Some <| sprintf "Fantomas invokation for %s returned with a non-zero exit code" project.DirectoryName
+            else
+                let formattedResult = buildExternalProject()
+                if formattedResult <> 0
+                then Some <| sprintf "Build of external project after fantomas formatting failed for project %s" project.DirectoryName
+                else
+                    printfn "Successfully built %s after reformatting" project.DirectoryName
+                    None
+        )
+        |> List.choose id
+    if not (List.isEmpty externalBuildErrors)
+    then failwith (String.Join("\n", externalBuildErrors) )
+)
+
 Target "Push" (fun _ -> Paket.Push (fun p -> { p with WorkingDir = "bin" }))
 
 // --------------------------------------------------------------------------------------
@@ -158,4 +247,7 @@ Target "All" DoNothing
   ==> "All"
   ==> "Push"
 
-RunTargetOrDefault "All"
+"Build"
+  ==> "TestExternalProjects"
+
+RunTargetOrDefault "All"
diff --git a/src/Fantomas.Cmd/Fantomas.Cmd.fsproj b/src/Fantomas.Cmd/Fantomas.Cmd.fsproj
@@ -17,4 +17,4 @@
     <ProjectReference Include="..\Fantomas\Fantomas.fsproj" />
   </ItemGroup>
   <Import Project="..\..\.paket\Paket.Restore.targets" />
-</Project>
+</Project>
diff --git a/src/Fantomas.CoreGlobalTool/Fantomas.CoreGlobalTool.fsproj b/src/Fantomas.CoreGlobalTool/Fantomas.CoreGlobalTool.fsproj
@@ -18,4 +18,4 @@
     <ProjectReference Include="..\Fantomas\Fantomas.fsproj" />
   </ItemGroup>
   <Import Project="..\..\.paket\Paket.Restore.targets" />
-</Project>
+</Project>
diff --git a/src/Fantomas.Tests/Fantomas.Tests.fsproj b/src/Fantomas.Tests/Fantomas.Tests.fsproj
@@ -46,4 +46,4 @@
     <ProjectReference Include="..\Fantomas\Fantomas.fsproj" />
   </ItemGroup>
   <Import Project="..\..\.paket\Paket.Restore.targets" />
-</Project>
+</Project>
diff --git a/src/Fantomas/Fantomas.fsproj b/src/Fantomas/Fantomas.fsproj
@@ -21,4 +21,4 @@
     <None Include="CodeFormatter.fsx" />
   </ItemGroup>
   <Import Project="..\..\.paket\Paket.Restore.targets" />
-</Project>
+</Project>