Proposal - module build & publish via the task runner and psake #107

Closed
rkeithhill opened this Issue Mar 3, 2016 · 42 comments

Projects

None yet

7 participants

@rkeithhill
Collaborator

There is interest in being able to publish a module to the PowerShell Gallery from VSCode using this extension. I have been doing a fair amount of work with ASP.NET Core projects. Those projects use gulp as a task runner to perform during the "publish" of a web site. VSCode has a task runner that can be configured to run various tools including gulp. However for PowerShell module publish, I think we should use something more "PowerShell oriented". So I've been playing with PSake to build and publish modules within VSCode.

Here is the psake.ps1 file (equivalent to gulpfile.js) that would be in the workspace folder:

Properties {    
    $NugetApiKeyEnc = @'XXXX'@
    $ReleaseNotesPath = "$PSScriptRoot\ReleaseNotes.md"

    $PublishDir = "$PSScriptRoot\.publish"
    $Exclude = @(
        'psake.ps1',
        '*Tests.ps1',
        '.git*',
        '.publish',
        '.vscode'
    )
}

Task default -depends Build

Task Publish -depends Test {
    $secureString = $NugetApiKeyEnc | ConvertTo-SecureString
    $nugetApiKeyBstr = [Runtime.InteropServices.Marshal]::SecureStringToBSTR($secureString)
    $nugetApiKey = [Runtime.InteropServices.Marshal]::PtrToStringAuto($nugetApiKeyBstr)
    [Runtime.InteropServices.Marshal]::ZeroFreeBSTR($nugetApiKeyBstr)

    $ReleaseNotes = Get-Content $ReleaseNotesPath -Raw
    Publish-Module -Path $PublishDir -NuGetApiKey $nugetApiKey `
                   -ReleaseNotes $ReleaseNotes -WhatIf
}

Task Test -depends Build {
    Import-Module Pester
    Invoke-Pester $PSScriptRoot
}

Task Build -depends Clean {
    Copy-Item $PSScriptRoot\* -Destination $PublishDir -Exclude $Exclude -Recurse -Container 
}

Task Clean -depends Init {
    Remove-Item $PublishDir\* -Recurse -Force
}

Task Init {
   if (!(Test-Path $PublishDir)) {
       New-Item -ItemType Directory $PublishDir
   }
}

Note that most of this file is boilerplate except for the Properties section. That is what the user will spend the most time configuring for their module. The NuGetApi key is something that should be protected and not checked in. What you see above is the output from:

$ss = Read-Host 'Enter NuGet Api Key' -AsSecureString
$ss | ConvertFrom-SecureString | Out-Clipboard

Now to get this to work from VSCode, I've modified the tasks.json file to:

// A task runner that invokes Pester to run all Pester tests under the
// current workspace folder.

// NOTE: This Pester task runner requires an updated version of Pester (>=3.4.0)
// in order for the problemMatcher to find failed test information (message, line, file).
// If you don't have that version, you can update Pester from the PSGallery like so:
//
// PS C:\> Update-Module Pester
//
// If that gives an error like:
// "Module 'Pester' was not installed by using Install-Module, so it cannot be updated."
// then execute:
//
// PS C:\> Install-Module Pester -Scope CurrentUser -Force
//

// Available variables which can be used inside of strings.
// ${workspaceRoot}: the root folder of the team
// ${file}: the current opened file
// ${fileBasename}: the current opened file's basename
// ${fileDirname}: the current opened file's dirname
// ${fileExtname}: the current opened file's extension
// ${cwd}: the current working directory of the spawned process
{
    "version": "0.1.0",

    // Start PowerShell
    "command": "${env.windir}\\sysnative\\windowspowershell\\v1.0\\PowerShell.exe",

    // The command is a shell script
    "isShellCommand": true,

    // Show the output window always
    "showOutput": "always",

    "args": [
        "-NoProfile", "-ExecutionPolicy", "Bypass"        
    ],

    // Associate with test task runner
    "tasks": [
        {
            "taskName": "Build",
            "suppressTaskName": true,
            "isBuildCommand": true,
            "args": [
                "Write-Host 'Invoking PSake...'; Invoke-PSake psake.ps1;",
                "Invoke-Command { Write-Host \"Completed Build task in task runner\" }"                
            ]
        },
        {
            "taskName": "Publish",
            "suppressTaskName": true,
            "args": [
                "Write-Host 'Invoking PSake...'; Invoke-PSake psake.ps1 Publish;",
                "Invoke-Command { Write-Host \"Completed Publish task in task runner\" }"                
            ]
        },
        {
            "taskName": "Pester",
            "suppressTaskName": true,
            "isTestCommand": true,
            "args": [
                "Write-Host 'Invoking Pester...'; Invoke-Pester -PesterOption @{IncludeVSCodeMarker=$true};",
                "Invoke-Command { Write-Host \"Completed all tests tasks in task runner\" }"
            ],
            "problemMatcher": [
                {
                    "owner": "powershell",
                    "fileLocation": ["absolute"],
                    "severity": "error",
                    "pattern": [
                        {
                            "regexp": "^\\s*(\\[-\\]\\s*.*?)(\\d+)ms\\s*$",
                            "message": 1
                        },
                        {
                            "regexp": "^\\s+at\\s+[^,]+,\\s*(.*?):\\s+line\\s+(\\d+)$",
                            "file": 1,
                            "line": 2
                        }
                    ]
                }
            ]
        }
    ]
}

The result looks like this:

vscodepublish

Still a ways to go. The Build's copy task isn't copying the en-US dir correctly. And I have no problemMatcher for the psake output yet. Still this shows some promise.

What we still need though is a command for scaffolding a new module manifest file from the VSCode UI. And there should be a .gitignore file with .publish in it.

@daviwil
Member
daviwil commented Mar 3, 2016

This is really cool! I'd really prefer it if PowerShellEditorServices had built-in module publishing support but the reality is that we'd end up needing something like PSake anyway because each module has its own unique build steps. I think we could require that PSake be installed to use that command, though. Doing the initial work using VS Code tasks is helpful to learn how this should work.

I really like the idea of a .publish file though, or at least some standard naming convention for module build files. If we have well-established task names then we'll easily be able to run anyone's clean/build/publish tasks.

Now I'm curious to use PSake for Editor Services and ISE Preview ;)

Related: https://github.com/psake/psake-vscode

@daviwil daviwil added the discussion label Mar 3, 2016
@KirkMunro
Collaborator

I don't agree that each module has its own unique build steps. Most modules don't have build steps at all. Most of them are just unsigned script modules. Some modules have unique build steps, but the vast majority of them don't, other than perhaps signing the release.

I like the idea of simple publishing support (by simple I mean sign and publish) in PowerShellEditorServices, along with extension points where that process can do other required work if necessary to cover the unique build steps. For example, similar to $ExecutionContext.InvokeCommand.PreCommandLookupAction and $ExecutionContext.InvokeCommand.PostCommandLookupAction, it would be great if module authors could specify pre-/post- publish steps that are just PowerShell scripts that do whatever work is required to publish the module.

@daviwil
Member
daviwil commented Mar 3, 2016

Good point! I suppose if we had simple hooks around a general publish action then the user could use whatever they wanted (including PSake) to inject the build behavior they need for their module project.

@rkeithhill
Collaborator

@KirkMunro I started out with the opinion that folks would only need to tweak these properties for their module:

Properties {    
    $NugetApiKeyEnc = @'XXXX'@
    $ProjectUri = 'http://pscx.codeplex.com/'
    $LicenseUri = 'http://pscx.codeplex.com/license'
    $IconUri = 'http://download-codeplex.sec.s-msft.com/Download?ProjectName=pscx&DownloadId=8046&Build=20941'
    $Tags = 'Utilities','Xml','Zip','Clipboard','Base64'    
    $ReleaseNotesPath = "$PSScriptRoot\ReleaseNotes.md"

    $PublishDir = "$PSScriptRoot\.publish"
    $Exclude = @(
        'psake.ps1',
        '*Tests.ps1',
        '.git*',
        '.publish',
        '.vscode'
    )
}

But then I realized that sometimes you want to publish to an internal repo. No biggie. We can use the -Repository parameter and add another property that defaults to PSGallery (if I understand how that param works - never used it before). But then what if someone wants to publish to an internal file share? Then you no longer want to use Publish-Module, you just need to use Copy-Item and most of the properties don't apply. In addition, I've seen some cases where folks extract module version numbers from other files or assemblies. So while it is mostly a standard build process it can vary and I think whatever we do it should anticipate that.

@daviwil Note that .publish is a folder in which copy the artifacts to be published. You don't want to publish directly from the workspace folder because A) it likely contains more artifacts than should be published (ie .vscode, your test files, .gitignore, etc). B) I could be wrong here but I believe the name of the folder you publish needs to be your module name and your workspace folder may or may not have that name and C) ideally you want to run your tests in your publish folder (to ensure you didn't forget a file) and then remove those tests before publish. So whatever approach we take I believe it requires copying parts of the workspace's content to a temp dir of some sort from which the publish will happen. BTW I'm not stuck on .publish perhaps the dir should be called out?

WRT how the PowerShell supports module publishing, I believe "this" is the way. I encourage folks to check out the Tasks docs here. The first two paragraphs read:

Lots of tools exist to automate tasks like building, packaging, testing or deploying software systems. Examples include Make, Ant, Gulp, Jake, Rake and MSBuild.

These tools are mostly run from the command line and automate jobs outside the inner software development loop (edit, compile, test and debug). Given their importance in the development life-cycle, it is very helpful to be able run them and analyze their results from within VS Code. Please note that task support is only available when working on a workspace folder. It is not available when editing single files.

As I mentioned web sites are using gulpfile.js to do this sort of stuff (preparing, copying, minifying, etc). I think the VSCode way is to use tasks. I just object to using gulp. :-) I think something like psake would make more sense for a PowerShell user. Unfortunately PSake is currently not on the PSGallery but @dlwyatt is on it.

BTW I'm not sure if the file should be called psake.ps1. Perhaps it should be called build.ps1 or publish.ps1?

I believe I'm going to write this up in a blog post as folks can use the existing infrastructure to do this today with the current VSCode and PowerShell extension. I'm still hoping we get better support for pre-configuring the tasks.json file. Until then, we could look into contributing a task snippet tsc - build & publish PowerShell module that folks could use. And for the build/publish ps1 file, that could be a PowerShell snippet. BTW we could also do a snippet that would be the equivalent of New-ModuleManifest. Although I would love it if VSCode supported adding template files defined by an extension i.e. right click in navigator and select Add File template -> PowerShell -> Module manifest file.

@rkeithhill
Collaborator

Submitted a feature request to VSCode related to this proposal.

@rkeithhill
Collaborator

Also, what do folks recommend in terms of protecting the NugetApiKey? The example above uses secure strings encrypted with DPAPI. This means that the user who created the encrypted key can put it in a file. Check it in, push to the public repo safely. However, on a multi-user project, only that user can execute the publish and only on the same machine the user created the encrypted api key on.

Is there a GitHub feature for storing secrets that can be safely accessed only by project team members and not others?

@dlwyatt
dlwyatt commented Mar 3, 2016

For something built into an editor, I'd prefer to see it prompt me to enter the API key, with a checkbox to save it. How the saved value gets encrypted is entirely up to you; could just put it in $env:APPDATA\VSCode\Secrets or something, and in that case, your current DPAPI code would be fine (since $env:APPDATA is in the user's profile anyway.) The only difference is the bit where if the encrypted / saved copy isn't found, it prompts the user.

@daviwil
Member
daviwil commented Mar 3, 2016

I could be wrong here but I believe the name of the folder you publish needs to be your module name and your workspace folder may or may not have that name

Yep, that is true.

Although I would love it if VSCode supported adding template files defined by an extension

VS Code uses Yeoman for creating things from templates but I'm not sure if there's an option for creating a single file (as opposed to a whole "project"). I had started a generator-powershell project on the PowerShell GitHub org to get a basic PowerShell module project generator going but haven't made progress or worked on it in a while.

I like Dave's idea for storing the API key after first request. I don't think there's a GitHub feature for shared secrets but it'd be handy if they added something like that.

@rkeithhill
Collaborator

I'd prefer to see it prompt me to enter the API key

@daviwil Can a VSCode extension pop a dialog box for text entry? Or are we limited to the Debug Console and Read-Host prompting at runtime?

@daviwil
Member
daviwil commented Mar 3, 2016

Yep, check out showInputBox on the API page. I've actually wired it up to Read-Host via a protocol message in the language service as you can see here.

@KirkMunro
Collaborator

But then I realized that sometimes you want to publish to an internal repo. No biggie. We can use the -Repository parameter and add another property that defaults to PSGallery (if I understand how that param works - never used it before). But then what if someone wants to publish to an internal file share? Then you no longer want to use Publish-Module, you just need to use Copy-Item and most of the properties don't apply. In addition, I've seen some cases where folks extract module version numbers from other files or assemblies. So while it is mostly a standard build process it can vary and I whatever we do should anticipate that.

@rkeithhill I agree that the build/publish process can vary. For publication to a file share though, Publish-Module still works just fine (PowerShellGet supports using a file share as the repository -- it's incredibly easy to set up and works very well for this scenario). So the publication portion (pushing the collection of files to the repository) should remain largely the same. The being/end work may change (updating version numbers before publishing, automated tweeting after publishing, etc.), and that's where PrePublish and PostPublish actions would come in.

@KirkMunro
Collaborator

+1 for David's idea for prompting for the key and storing it for each user who needs to publish.

@rkeithhill
Collaborator

Yep, check out showInputBox on the API page. I've actually wired it up to Read-Host via a protocol message in the language service as you can see here.

That's not in 0.4.1 though, right? Read-Host prompts via the Debug Console (and also doesn't support -AsSecureString). :-) It's also not as obvious where you are supposed to type in the response as a dialog box would be.

So a build script could look for the api key in a set location, and if it is not found, use Read-Host to allow the user to enter the key. We could then encrypt that for storage on the user's file system so that the next time, we wouldn't need to prompt them. Of course, if they ever wanted to use a new key they would have to know how to either A) delete the encrypted key file or B) force the script to prompt for a new key.

FWIW I'd like for there to be not too much magic in how this works. That is why I like using a PSake file. While I love me some VSCode w/PowerShell there are times I do stuff from the PowerShell prompt. I'd love to be able to wander into my module workspace directory and type Invoke-PSake build.ps1 to allow me to manually test my module before publishing and Invoke-PSake build.ps1 Publish to publish it.

@rkeithhill
Collaborator

WRT to prompting for the NuGet API key, what about the rest of this information:

    $ProjectUri = 'http://pscx.codeplex.com/'
    $LicenseUri = 'http://pscx.codeplex.com/license'
    $IconUri = 'http://download-codeplex.sec.s-msft.com/Download?ProjectName=pscx&DownloadId=8046&Build=20941'
    $Tags = 'Utilities','Xml','Zip','Clipboard','Base64'    
    $ReleaseNotesPath = "$PSScriptRoot\ReleaseNotes.md"

I don't think VSCode supports anything as fancy as a form, does it? If not, then would you be willing to be peppered with input dialogs for each of the above properties?

@daviwil
Member
daviwil commented Mar 3, 2016

That's not in 0.4.1 though, right? Read-Host prompts via the Debug Console

Yeah, in the Debug Console we just use the console. If you put Read-Host -Prompt "Enter Text" in a file, select it in the editor and hit F8 you'll see a UI prompt show up.

Nah, no forms in VS Code yet (if ever). I agree, being prompted multiple times would be pretty annoying.

@dfinke
Contributor
dfinke commented Mar 3, 2016

-1 for prompting. I set my nuget key to a variable in my profile and reference it in my PublishToGallery.ps1 script (which could be moved to a psake task).

@dlwyatt
dlwyatt commented Mar 3, 2016

WRT to prompting for the NuGet API key, what about the rest of this information:

That stuff should probably be in the module manifest anyway. No need to duplicate it in a publish settings file.

FWIW I'd like for there to be not too much magic in how this works. That is why I like using a PSake file. While I love me some VSCode w/PowerShell there are times I do stuff from the PowerShell prompt. I'd love to be able to wander into my module workspace directory and type Invoke-PSake build.ps1 to allow me to manually test my module before publishing and Invoke-PSake build.ps1 Publish to publish it.

No reason it has to be an "either/or" thing. The psake file can have the nuget key as a property which can be passed in via the command line, and VSCode can offer the prompt-and-remember functionality which just passed a value to that psake property.

@KirkMunro
Collaborator

I agree with @dlwyatt, and was just coming to ask why you want to get details that are in the manifest, and why prompting for them would be a question when you're in a code editor where those values can be updated.

As for @dfinke's comments, I also have a variable in my profile that I reference in my own publish script. I look at this like a mandatory parameter. If I provide my value (the variable I have in my profile), then it just works. If I don't, then I get prompted, with an option to save it encrypted so that it doesn't need to prompt me every day. Then I can just use $PSDefaultParameterValues to set the default to the value that is read as part of my profile, and I won't ever get prompted, allowing me to continue to manage that myself in my profile. I don't know exactly how that ties into VSCode features/functionality, but that is essentially what I would like the UX to be like.

@rkeithhill
Collaborator

@KirkMunro

why you want to get details that are in the manifest

Hehe, good point. I guess I've never scrolled down far enough to see the PrivateData.PSData section. :-) That said, I would personally never embed my releaseNotes inside my psd1 file. Presumably though, if I used the Publish-Module -ReleaseNotes parameter, it override the empty string inside the psd1 file.

I envision that the build/publish.ps1 PSake script would:

  • Provide a parameter to allow the user to pass in the unencrypted api key
  • If the parameter is not supplied, look in a location where this script would have stored the encrypted key during a previous execution. If the parameter is supplied, then encrypt it and store in a well known location.
  • If the encrypted key file was not found, prompt the user (Read-Host) to enter the api key, encrypt it and store it in a well known location (probably something like what @dlwyatt suggests).

@dlwyatt

VSCode can offer the prompt-and-remember functionality which just passed a value to that psake property.

The way I'm envisioning this is invoking PSake via VSCode task runner. So the PowerShell extension doesn't really get "involved" during a Publish. Where PowerShell extension got involved was in the creation/modification of the tasks.json file to add the Publish task:

        {
            "taskName": "Publish",
            "suppressTaskName": true,
            "args": [
                "Write-Host 'Invoking PSake...'; Invoke-PSake psake.ps1 Publish;",
                "Invoke-Command { Write-Host \"Completed Publish task in task runner\" }"                
            ]
        },

We "could" embed script in the args to look for & prompt for the key if not found (blech) but I think it is better for this to be in the psake script. Then you could run it from PowerShell console or even the ISE console pane.

Now perhaps I'm thinking about this all wrong but I really do believe using the VSCode task support is the way to go.

@rkeithhill
Collaborator

I wonder if we should consider either using this VSCode Yo extension or do something like it. I kind of like not re-inventing the wheel. Maybe we contribute a PowerShell sub-generator to Yeoman for scaffolding various project types: PowerShell Module Project, PowerShell Gallery Module Project, PowerShell Module Manifest File, etc?

@daviwil
Member
daviwil commented Mar 4, 2016

My impression is that this is just an extension for running Yeoman generators that are available online, not that it includes its own. I could be wrong about that, though. That sounds like a great set of initial scaffolds!

@daviwil
Member
daviwil commented Mar 4, 2016

I should clarify, that extension is meant to be the official front-end for Yeoman generators in VS Code, so I think all we need to do is write our PowerShell generator package and it should show up there.

@rkeithhill
Collaborator

FWIW I proposed an "Add File Template..." UI feature for VSCode. Erich Gamma's response was to (for now) look into the Yo extension for scaffolding.

@daviwil
Member
daviwil commented Mar 4, 2016

Ok, I just asked the author of the Yeoman extension how generators get resolved:

SamVerschueren/vscode-yo#37

Apparently the user has to globally install a Yeoman generator via npm before it shows up in the list. This is definitely an adoption blocker for PowerShell folks.

I think we may need to have our own scaffolding implementation in Editor Services (probably using New-ModuleManifest plus some extra goodies). The benefit here is that the same scaffolding functionality could be used in the ISE, Sublime Text, and anywhere else Editor Services is used. Could also have some hooks to add custom scaffolding types.

Thoughts?

@rkeithhill
Collaborator

Yeah it would be different if this feature were just built into VSCode (as I think it should be). Anyway, this feature would be mostly implemented in the vscode-powershell side, right? At the end of the day, we ask the user for a "project" or "file" name and maybe an API key but after that it's just spitting out text into a file or a set of files into a workspace. I guess for some of this we could use the PowerShell service to say generate an empty Module Manifest file?

BTW when we create a PowerShell Module project, we should create a Tests\<ModuleName>.Tests.ps1 file with an initial test of Test-ModuleManifest <ModuleName>.psd1; $? | Should Be $true.

@daviwil
Member
daviwil commented Mar 4, 2016

It'd be nice if their desired templating system didn't require knowledge or usage of npm but I guess that's the current state of things :/

For now we could implement most of the logic in the VS Code extension and then figure out how to generalize it later in Editor Services. If we register commands in the extension's package.json that don't include the usual editorTextFocus && editorLangId == 'powershell' condition then those commands should be globally visible. To make sure that the extension code is loaded when the command is run we can add a new activation event for the templating command(s) like how vscode-yo does:

https://github.com/SamVerschueren/vscode-yo/blob/master/package.json#L23

Currently the PowerShell extension is only activated on the onLanguage:powershell event, but we can add an arbitrary number of onCommand:commandName entries to the activationEvents list. Since activating the extension would cause the language service to be loaded, we could eventually invoke some command there to perform the scaffolding work, but I'm fine with doing it solely in TypeScript for the moment.

@rkeithhill
Collaborator

After reading about folks having issues getting the "right" version of a module to publish based on module name, I propose the solution we provide generates the module dir & contents to a subdirectory of the VSCode workspace and publish based on path instead. We also probably should generate a .gitignore file that prevents that directory from being committed. I had proposed the directory name .publish but bin or out could work as well.

@daviwil
Member
daviwil commented Mar 6, 2016

Yep, definitely. This is the process I use for publishing the ISE Preview package. I have a release folder with a subfolder that matches the module's name then I copy all relevant files there to be published with Publish-Module -Path.

@rkeithhill
Collaborator

OK, I think I have a PSake script that works pretty well. Here is the top part that folks would customize:

# This is a PSake script that supports the following tasks: 
# clean, build, test and publish.  The default task is build.  
#
# The publish task uses the Publish-Module command to publish
# to either the PowerShell Gallery (the default) or you can change
# the $Repository property to the name of an alternate repository.
#
# The test task invokes Pester to run any Pester tests in your 
# workspace folder. Name your test scripts <TestName>.Tests.ps1
# and Pester will find and run the tests contained in the files.
#
# You can run this build script directly using the invoke-psake
# command which will execute the build task.  This task "builds"
# a temporary folder from which the module can be published.
#
# PS C:\> invoke-psake build.ps1
#
# You can run your Pester tests (if any) by running the following command.
#
# PS C:\> invoke-psake build.ps1 -taskList test
#
# You can execute the publish task with the following command. Note that
# the publish task will run the test task first. The Pester tests must pass
# before the publish task will run.  The first time you run the publish
# command, you will be prompted to enter your PowerShell Gallery NuGetApiKey.
# After entering the key, it is encrypted and stored so you will not have to
# enter it again.
# 
# PS C:\> invoke-psake build.ps1 -taskList publish
#
# You can verify the stored and encrypted NuGetApiKey by running the following 
# command. This will display your NuGetApiKey in plain text!
#
# PS C:\> invoke-psake build.ps1 -taskList showKey
#
# You can store a new NuGetApiKey with this command. You can leave off 
# the -properties parameter and you'll be prompted for the key.
# 
# PS C:\> invoke-psake build.ps1 -taskList storeKey -properties @{NuGetApiKey='test123'}
#

###############################################################################
# Customize these properties for your module.
###############################################################################
Properties {
    # The name of your module should match the basename of the PSD1 file.
    $ModuleName = (Get-Item $PSScriptRoot\*.psd1)[0].BaseName

    # Path to the release notes file.  Set to $null if the release notes reside in the manifest file.
    $ReleaseNotesPath = "$PSScriptRoot\ReleaseNotes.md"    

    # The directory used to publish the module from.  If you are using Git, the
    # $PublishDir should be excluded if it is under the workspace directory.
    $PublishDir = "$PSScriptRoot\.publish\$ModuleName"

    # The following items will not be copied to the $PublishDir. 
    # Add items that should not be published with the module.
    $Exclude = @(
        '*.Tests.ps1',
        '.git*',
        '.publish',
        '.vscode',
        (Split-Path $PSCommandPath -Leaf)
    )

    # Name of the repository you wish to publish to. Default repo is the PSGallery.
    $PublishRepository = $null

    # Your NuGet API key for the PSGallery.  Leave it as $null and the first time
    # you publish you will be prompted to enter your API key.  The build will 
    # store the key encrypted in a file, so that on subsequent publishes you
    # will no longer be prompted for the API key.
    $NuGetApiKey = $null           
    $EncryptedApiKeyPath = "$env:LOCALAPPDATA\vscode-powershell\NuGetApiKey.clixml"
}

###############################################################################
# Customize these tasks for performing operations before and/or after publish.
###############################################################################
Task PrePublish {
}

Task PostPublish { 
}

The entire psake build.ps1 script is in this gist. When you run the publish task from VSCode, this is the output psake generates in the VSCode output console:

Invoking PSake...
psake version 4.5.0
Copyright (c) 2010-2014 James Kovacs & Contributors
Executing Init
Executing Clean
Executing Build
Executing Test
Describing Module Manifest Tests
 [+] Passes Test-ModuleManifest 451ms
Describing Test Get-Foo
 [+] Should return a foo of size 1024 228ms
Tests completed in 679ms
Passed: 2 Failed: 0 Skipped: 0 Pending: 0 Inconclusive: 0
Executing PrePublish
Executing PublishImpl
What if: Performing the operation "Publish-Module" on target "Version '1.0' of module 'MyModule'".
Executing PostPublish
Executing Publish
Build Succeeded!
----------------------------------------------------------------------
Build Time Report
----------------------------------------------------------------------
Name        Duration        
----        --------        
Init        00:00:00.0418430
Clean       00:00:00.0224899
Build       00:00:01.0279609
Test        00:00:01.0478605
PrePublish  00:00:00.0022610
PublishImpl 00:00:06.5223184
PostPublish 00:00:00.0183579
Publish     00:00:00.0035106
Total:      00:00:08.9377985
Completed build task in task runner

The resulting workspace in VSCode looks like this:
vscodepublish2

This is looking even more promising to me. This reminds me a bit of the MSBuild C# "canned" targets plus the csproj file where I can hook into various targets. I like that it is "open" to my customization needs. Now we just need to get the psake folks to release 4.6.0 and publish to the gallery. Well, and there's that small issue of getting various files scaffolded into the project. At the very least I think we need to scaffold the following based on the user supplied "module-name":

  • .vscode/tasks.json (define tasks for clean, build, test, publish)
  • Tests/{module-name}.Tests.ps1
  • .build.ps1
  • .gitignore (ignore .publish dir)
  • {module-name}.psd1
  • {module-name}.psm1
  • ReleaseNotes.md
@daviwil
Member
daviwil commented Mar 7, 2016

This is great! Will make it a lot easier to get a new module project kicked off. Since we'd be including a whole source file in their project code, I wonder if there's any license info that needs to be put in it there to say where it came from and that they can freely use it. I'll ask a legal person about it.

The only comment I have on the script is on this bit:

$EncryptedApiKeyPath = "$env:LOCALAPPDATA\vscode-powershell\NuGetApiKey.clixml"

Maybe we should store that in a path relative to the module name (or even in the .publish folder?) This would avoid problems for those who may need to use multiple Gallery accounts. Someone may publish different modules to the PSGallery and their own internal NuGet repository.

@dlwyatt
dlwyatt commented Mar 7, 2016

That's just the default in the template, right? If people have more advanced scenarios, they can tweak those values as needed.

@daviwil
Member
daviwil commented Mar 7, 2016

Ahhh, yes, good point :)

@rkeithhill
Collaborator

@dlwyatt

That's just the default in the template, right? If people have more advanced scenarios, they can tweak those values as needed.

👍

@rkeithhill rkeithhill referenced this issue in psake/psake Mar 7, 2016
Closed

Publish Psake on PowerShellGallery #143

@rkeithhill
Collaborator

@daviwil

Since we'd be including a whole source file in their project code

BTW we "could" put the bulk of the "canned" tasks in the extension's install dir and then dot source it into the build.ps1 file. If I strip out the huge comment at the top of build.ps1, you wind up with these contents:

###############################################################################
# Customize these properties for your module.
###############################################################################
Properties {
    # The name of your module should match the basename of the PSD1 file.
    $ModuleName = (Get-Item $PSScriptRoot\*.psd1)[0].BaseName

    # Path to the release notes file.  Set to $null if the release notes reside in the manifest file.
    $ReleaseNotesPath = "$PSScriptRoot\ReleaseNotes.md"

    # The directory used to publish the module from.  If you are using Git, the
    # $PublishDir should be ignored if it is under the workspace directory.
    $PublishDir = "$PSScriptRoot\.publish\$ModuleName"

    # The following items will not be copied to the $PublishDir.
    # Add items that should not be published with the module.
    $Exclude = @(
        'Tests',
        '.git*',
        '.publish',
        '.vscode',
        (Split-Path $PSCommandPath -Leaf)
    )

    # Name of the repository you wish to publish to. Default repo is the PSGallery.
    $PublishRepository = $null

    # Your NuGet API key for the PSGallery.  Leave it as $null and the first time
    # you publish you will be prompted to enter your API key.  The build will
    # store the key encrypted in a file, so that on subsequent publishes you
    # will no longer be prompted for the API key.
    $NuGetApiKey = $null
    $EncryptedApiKeyPath = "$env:LOCALAPPDATA\vscode-powershell\NuGetApiKey.clixml"

    # This gets used by the dot sourced BuildTasks.ps1 file as it cannot use $PSScriptRoot
    $WorkspaceRoot = $PSScriptRoot
}

###############################################################################
# Customize these tasks for performing operations before and/or after publish.
###############################################################################
Task PrePublish {
}

Task PostPublish {
}

# Dot source the PSake build tasks for publishing a PowerShell module.
. $home\.vscode-alpha\extensions\ms-vscode.PowerShell-0.4.1\BuildTasks.ps1

The only trick there is to scaffold out the build.ps1 file with the correct path to the extension's install dir for the BuildTasks.ps1 file.

I could go either way on this. Separating the files is probably a bit less intimidating than having everything in one file. But if they need to really customize the process, that's makes it harder. OTOH, we can add more hooks like the PrePublish and PostPublish task to anticipate that. And at the very least, it is still a text file that they can pull directly into their build.ps1 file (replacing the dot source line).

Then again, is it that big of a deal to put this all in a single file - placing the customization stuff at the top?

@daviwil
Member
daviwil commented Mar 7, 2016

Dot-sourcing a file from the extension install folder seems pretty risky since there are a few ways it could end up breaking for the user. I'd prefer it if the solution was self-contained either through everything being in a single build script or split between two scripts that are included in the generated project. For the sake of simplicity I'd say just a single script is better.

@rkeithhill
Collaborator

Yeah, that's my preference as well. Just wanted to throw the option out there.

@doesitscript

I'm of the mindset that dot sourcing different components makes the code easier to keep organized. If I'm troubleshooting my build tasks, its nice from both a logical and visible standpoint, to have one file open that contains build tasks. It is easy to break but not hard to fix, just start a new project from source and import your code worst cast scenario... Or just troubleshoot the missing file.

@randomchance
Contributor

I really like using dot sourcing to organize code, however I find it to be too fragile for importing resources from other locations. I think it would be ideal if you scaffold a .buildtasks\ directory in the project root and copy the canned tasks into it.

This has the benefit of:

  • Still accomplishes goal of distributing common tasks
  • Gives new users an example to examine
  • Allows easier debugging of builds.
  • Promotes a nice organizational structure
  • Still allows the build tasks to function without editor services being directly installed.

The last one is huge deal, using my environment as an example the build/deployment tasks could be called on:

  • any of 20 developer machines (that might have the repository stored in different locations)
  • build servers
  • demo servers
  • release / deployment servers

A constant headache with msbuild was attempting to get it working correctly without installing visual studio on the build servers, I'd like to avoid that if at all possible.

@daviwil
Member
daviwil commented May 18, 2016

Hey Chance, you should check out our new project Plaster: https://github.com/PowerShell/Plaster. We're building a project scaffolder which will stamp out standalone projects in a structure much like the one Keith is proposing in this issue. Those projects will definitely not require Editor Services or the PowerShell extension to be installed.

@randomchance
Contributor

That's pretty nice, I'm gonna test it out after we wrap up the next deployments. Do you see that being leveraged in editor services? Or would ES have independently maintained tooling?

@daviwil
Member
daviwil commented May 19, 2016

Yep, part of the reason for making it is to use it in all editors that leverage Editor Services :)

@daviwil
Member
daviwil commented Dec 15, 2016

Plaster integration is coming to 0.8.0, closing this now :)

@daviwil daviwil closed this Dec 15, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment