[PSCore Align] Adopt build.ps1 build concept

[TylerLeonhardt]

PowerShell Core has this concept of a build.psm1 file that contains a bunch of functions for building and testing PSCore: https://github.com/PowerShell/PowerShell/blob/master/build.psm1

We should align with those processes so that it's consistent across PowerShell repos.

We can still continue to use InvokeBuild, but we can wrap the Invoke-Build * with the same entrypoints that PSCore has.

[TravisEz13]

We should discuss what is actually required here.

[MartinSGill]

I've just been trying to build using vscode-powershell.build.ps1 and got a load of errors ; because I actually just ran the script. So I'm very interested in improvements to the build.

Does this change mean removing the dependency on InvokeBuild?

build.psm1 (powershell/powershell) exports the following methods.

• Clear-PSRepo
• Compress-TestContent
• Convert-TxtResourceToXml
• Find-Dotnet
• Get-EnvironmentInformation
• Get-PackageVersionAsMajorMinorBuildRevision
• Get-PesterTag
• Get-PSCommitId
• Get-PSLatestTag
• Get-PSOptions
• Get-PSOutput
• Get-PSVersion
• Get-RedHatPackageManager
• Install-Dotnet
• log
• logerror
• New-MSIPackage
• New-PSOptions
• precheck
• Publish-NuGetFeed
• Publish-PSTestTools
• Restore-GitModule
• Restore-PSModule
• Restore-PSModuleToBuild
• Restore-PSPester
• Set-PSOptions
• Show-PSPesterError
• Start-BuildNativeUnixBinaries
• Start-BuildNativeWindowsBinaries
• Start-CrossGen
• Start-DevPowerShell
• Start-NativeExecution
• Start-PSBootstrap
• Start-PSBuild
• Start-PSPester
• Start-PSxUnit
• Start-ResGen
• Start-TypeGen
• Start-UnelevatedProcess
• Sync-PSTags
• Test-PSPesterResults
• Test-Win10SDK
• Test-XUnitTestResults
• Use-MSBuild

Which of these constitutes an "entrypoint" that should be implemented? None of these look like standard/obvious commands to call; as I discovered when I wasn't able to figure out how to build PSCore without reading the docs/appveyor.yml.

I feel the following points ought to be considered:

1. build.psm1 - Had to read docs/appveyor.yml to figure out how to build it.
2. The existing build.ps1 is also not intuitive to use. You need to run it with Invoke-Build, instead of just calling it as you would a normal script.
3. One advantage of a framework like PSake, MSBuild, Maven, et. al. is they make it easy to figure out how to build; especially as there are defaults and they enforce an interface. See a .proj, call msbuild, see a .pom call maven etc.

[MartinSGill]

Since msbuild is already being used in most projects, why not standardise on that?

vscode-powershell.proj
powershell.proj
powershell-editor-services.proj
etc.

This defines the targets and doesn't need to be more than a wrapper around powershell scripts.

[rkeithhill]

@tylerl0706 While I agree that consistency is nice PowerShell Core might not be the best "baseline" for every other PowerShell project. It is quite unique and quite a large build.

It seems to me we have this "grouping" of PowerShell projects:

• PowerShell modules - mostly script modules e.g. PowerShellGet, Plaster. InvokeBuild/psake work fine here
• PowerShell modules - binary (or binary & script) e.g. PSReadLine, PSScriptAnalyzer. This requires invoking msbuild to build C# code.
• VSCode extension - this "build" is largely driven by NPM but can be orchestrated by InvokeBuild or psake.
• PS Core - large, fairly complex custom build system.

So I don't think it makes sense to try to apply the same "custom" build system to other, much simpler to build projects. OTOH it might be nice to perhaps have a common way to start the build no matter what tool is used to build the project. Maybe that could be build.ps1 or <project-name>.build.ps1?

BTW the latest version of psake now automatically recognizes a file called psakefile.ps1. So in a directory with a psakefile.ps1 file in it, you can simply run invoke-psake and it will run the default task in that file - which is typically build.

@MartinSGill vscode-powershell.build.ps1 is an InvokeBuild recipe. You have to pass that into Invoke-Build. That said, I never invoke that script directly. I invoke it from within VSCode via the task runner. If you open the extension's root folder in VSCode (after you have git cloned PSES into a peer folder), run task Install and then task Build or just press F5 to start debugging which will execute the BuildAll task. BuildAll builds PSES as well.

[MartinSGill]

@rkeithhill While using VSCode is nice and hardly an outlandish suggestion given that it's a vscode extension, I've always taken the view that a build system that cannot give you the most common build scenario with a single command is incomplete; be that command "maven", "msbuild", "gradle", Invoke-Build or whatever else.

I also feel that easy discovery is important. Powershell made this deliberate choice with the cmdlet naming convention and standardised verbs. The commands might get a bit long winded at times, but this type of design decision is what attracted me to powershell in the first place.

In many ways first impressions count, and the first impression anyone who thinks of contributing code to a project has is how easily they can get it to build.

Powershell doesn't really have a recognisable build system yet, so simply seeing a *.csproj or *.pom file like you would in a C# or Java project to identify build system isn't an option.

• vscode-powershell.build.ps1 looks as though it should be run to build the project. Yet it throws an error if you try; and not a very helpful one.
• build.psm1 doesn't look like it should be run, it looks like something else uses it.
• a psakefile.ps1, while it still looks like you should run it, at least provides a good clue, just as a Rakefile does for ruby, excepting of course that rakefiles don't have an extension, despite being ruby source.
• Personal experience tends me towards Invoke-Build over psake, but invoke-build doesn't have such an obvious build file.

I think the jury is still out on the preferred build system for powershell related projects, nothing seems to have established itself as the defacto standard yet. Whatever the various powershell projects align on will have a big part in establishing such a defacto standard.

[rkeithhill]

@MartinSGill I agree with just about everything you're saying. Where I work - doing cross-plat C++ development - we have settled upon using build.ps1 at the root of the repo. Any dev can walk up to a system and run build.ps1 and it will generate the cmake cache and the build the project. Many times devs only want the cmake cache gen'd and will load the generated SLN into Visual Studio. In that case, they can run build.ps1 -CMakeCacheGenOnly. If they want to do any of this with a "clean" they just add -Clean.

We have quite a bit of stuff that needs to be installed & configured for dev/build machines: app installs, PS module installs, Conan, NPM & Artifactory config. I've started to implement that in build.ps1 via the switch -BootstrapBuildEnv. This is similar to the PS Core's Start-PSBootstrap command but our build's are still simple enough to handle this with a parameter, rather than a separate command.

I could see VSCode having a build.ps1 file that simply runs:

#requires -Modules InvokeBuild
Invoke-Build Restore
Invoke-Build Build # or maybe BuildAll

For an example of this see: https://github.com/PowerShell/Plaster/blob/master/build.ps1 While you're there, check out:
https://github.com/PowerShell/Plaster/blob/master/build.settings.ps1 and
https://github.com/PowerShell/Plaster/blob/master/build.psake.ps1

At one point, I had this idea that PowerShell needs a project system like the C# project system - where NOBODY worries about writing custom MSBuild targets files to embed resources or strong name or these days - emit a NuGet pkg. You simply provide the correct properties for the MSBuild targets file for C#. Personally I still like this idea for PowerShell. I can see an equivalent "targets" file for PowerShell modules that can handle various phases (some optional) such as: Clean, CompileSources (C#), GenerateHelp (platyPS), CopyArtifactsToOutDir (signing here doesn't generate Git changes), Analyze (PSSA), RunTests (Pester), SignFiles, GenerateModuleCatalog. And as a manual step - PublishModule.

It is interesting to see various build tools trying to replicate MSBuild but with syntax specific for that language e.g. rake, psake, cake, InvokeBuild, etc. It really makes me wish MSBuild had a way to allow for DSLs to use it. Short of that, I wonder if you could create a PowerShell friendly syntax that you could then render to a temp MSBuild file (kind of like MSBuild does for SLN files) and then use MSBuild to build it? Hmm....

[nightroman]

You can make Invoke-Build scripts directly invokable easily

Project.build.ps1

param(
    [Parameter(Position=0)]
    $Tasks,
    #... other script parameters
)

if ($MyInvocation.ScriptName -notlike '*Invoke-Build.ps1') {
    Invoke-Build $Tasks $MyInvocation.MyCommand.Path @PSBoundParameters
    return
}

# The usual build script stuff
task ...

See the full topic Tasks/Direct.

And you can extend this idea in order to add automatic bootstrapping to such a script.
See the topic Tasks/Paket.
This sample uses paket but you can use other package managers in the same way.

As a result, a user may just clone a project and run Project.build.ps1 directly.
This triggers bootstrapping and then invokes Build by just downloaded Invoke-Build.

[nightroman]

As far as this project now uses Invoke-Build, let me suggest a tweak. The project appveyor.yml fixes IB version to 3.2.1, quite old. Please use the latest 4.2.0 (nothing breaking for the project script). You will get colored Invoke-Build output in AppVeyor and some improved error logging, also done for AppVeyor. See PSReadLine AppVeyor log and its log with an error.