New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add container to Linux CI #17233
Add container to Linux CI #17233
Changes from all commits
859c411
b775b25
f0a8ccf
1ee59cc
7f1564d
215e05f
8e189d5
f712d18
38d6e8d
191faba
fc7059a
10c0ab1
a6ed369
7ad1005
b017cc6
16b1369
8323bde
c4b8e34
287aaf6
3cc5e29
7fcd510
cc28651
55380bd
64724bf
12923a1
a67f0c3
10d8698
b3a8ae7
f57dfa3
ef57181
b4563f1
9f376ad
6b67596
82d5034
0d05851
35226cc
8b385ef
06a6001
27e103a
ac65b38
969ad82
e2ca78e
3425bff
2ca09ba
c4e09b4
a08d688
3c703c8
a0614b7
6fdaa77
35c8e49
ce3ca85
41270f7
fc5a15d
95a5897
8f615c2
0a8dfba
5ba5be6
42b7aaf
c876b4e
0b1e737
ec88d98
165118e
9bbaa4d
1ae439b
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change | ||
---|---|---|---|---|
|
@@ -47,11 +47,15 @@ variables: | |||
__SuppressAnsiEscapeSequences: 1 | ||||
|
||||
resources: | ||||
- repo: self | ||||
clean: true | ||||
repositories: | ||||
- repository: Docker | ||||
type: github | ||||
endpoint: PowerShell | ||||
name: PowerShell/PowerShell-Docker | ||||
ref: master | ||||
|
||||
stages: | ||||
- stage: BuildLinux | ||||
- stage: BuildLinuxStage | ||||
displayName: Build for Linux | ||||
jobs: | ||||
- template: templates/ci-build.yml | ||||
|
@@ -60,33 +64,34 @@ stages: | |||
jobName: linux_build | ||||
displayName: linux Build | ||||
|
||||
- stage: TestLinux | ||||
displayName: Test for Linux | ||||
- stage: TestUbuntu | ||||
displayName: Test for Ubuntu | ||||
dependsOn: [BuildLinuxStage] | ||||
jobs: | ||||
- template: templates/nix-test.yml | ||||
parameters: | ||||
name: Linux | ||||
name: Ubuntu | ||||
pool: ubuntu-20.04 | ||||
purpose: UnelevatedPesterTests | ||||
tagSet: CI | ||||
|
||||
- template: templates/nix-test.yml | ||||
parameters: | ||||
name: Linux | ||||
name: Ubuntu | ||||
pool: ubuntu-20.04 | ||||
purpose: ElevatedPesterTests | ||||
tagSet: CI | ||||
|
||||
- template: templates/nix-test.yml | ||||
parameters: | ||||
name: Linux | ||||
name: Ubuntu | ||||
pool: ubuntu-20.04 | ||||
purpose: UnelevatedPesterTests | ||||
tagSet: Others | ||||
|
||||
- template: templates/nix-test.yml | ||||
parameters: | ||||
name: Linux | ||||
name: Ubuntu | ||||
pool: ubuntu-20.04 | ||||
purpose: ElevatedPesterTests | ||||
tagSet: Others | ||||
|
@@ -95,24 +100,72 @@ stages: | |||
parameters: | ||||
pool: ubuntu-20.04 | ||||
|
||||
- stage: TestContainer | ||||
displayName: Test in a container | ||||
dependsOn: [BuildLinuxStage] | ||||
jobs: | ||||
- job: getContainerJob | ||||
displayName: Choose a container | ||||
pool: | ||||
vmImage: ubuntu-20.04 | ||||
steps: | ||||
- checkout: self | ||||
clean: true | ||||
|
||||
- checkout: Docker | ||||
clean: true | ||||
|
||||
- pwsh: | | ||||
$matrix = ./PowerShell-Docker/build.ps1 -GenerateMatrixJson -FullJson -Channel preview | ConvertFrom-Json | ||||
# Filter out containers that won't run on AzDevOps Linux host as an agent | ||||
# Windows, nano server, alpine, and any ARM image | ||||
$linuxImages = $matrix.preview | | ||||
TravisEz13 marked this conversation as resolved.
Show resolved
Hide resolved
|
||||
Where-Object {$_.osversion -notmatch 'windows|nano|alpine|arm'} | | ||||
Select-Object JobName, Taglist | | ||||
Sort-Object -property JobName | ||||
# Force mariner for now, later we will add automated rotation by the month. | ||||
$marinerImage = $linuxImages | where-object {$_.JobName -like 'mariner*'} | ||||
TravisEz13 marked this conversation as resolved.
Show resolved
Hide resolved
|
||||
Import-Module ./PowerShell/tools/ci.psm1 | ||||
$tag = $marinerImage.Taglist -split ';' | select-object -first 1 | ||||
$containerName = "mcr.microsoft.com/powershell/test-deps:$tag" | ||||
Set-BuildVariable -Name containerName -Value $containerName -IsOutput | ||||
Set-BuildVariable -Name containerBuildName -Value $marinerImage.JobName -IsOutput | ||||
name: getContainerTask | ||||
displayName: Get Container | ||||
continueOnError: true | ||||
|
||||
- template: templates/test/nix-container-test.yml | ||||
parameters: | ||||
name: container | ||||
pool: ubuntu-20.04 | ||||
purpose: UnelevatedPesterTests | ||||
tagSet: CI | ||||
|
||||
- template: templates/test/nix-container-test.yml | ||||
parameters: | ||||
name: container | ||||
pool: ubuntu-20.04 | ||||
purpose: ElevatedPesterTests | ||||
tagSet: CI | ||||
|
||||
- template: templates/test/nix-container-test.yml | ||||
parameters: | ||||
name: container | ||||
pool: ubuntu-20.04 | ||||
purpose: UnelevatedPesterTests | ||||
tagSet: Others | ||||
|
||||
- template: templates/test/nix-container-test.yml | ||||
parameters: | ||||
name: container | ||||
pool: ubuntu-20.04 | ||||
purpose: ElevatedPesterTests | ||||
tagSet: Others | ||||
|
||||
- stage: PackageLinux | ||||
displayName: Package Linux | ||||
dependsOn: ["BuildLinux"] | ||||
dependsOn: ["BuildLinuxStage"] | ||||
jobs: | ||||
- template: linux/templates/packaging.yml | ||||
parameters: | ||||
pool: ubuntu-20.04 | ||||
|
||||
- stage: CodeCovTestPackage | ||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. On checking a part of this is required, where we create test packages. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Line 373 in 36fcec3
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. spoke offline and we use this from another pipeline. |
||||
displayName: CodeCoverage and Test Packages | ||||
dependsOn: [] # by specifying an empty array, this stage doesn't depend on the stage before it | ||||
jobs: | ||||
- job: CodeCovTestPackage | ||||
displayName: CodeCoverage and Test Packages | ||||
pool: | ||||
vmImage: ubuntu-20.04 | ||||
steps: | ||||
- pwsh: | | ||||
Import-Module .\tools\ci.psm1 | ||||
New-CodeCoverageAndTestPackage | ||||
displayName: CodeCoverage and Test Package |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,71 +1,19 @@ | ||
parameters: | ||
pool: 'macOS-latest' | ||
parentJobs: [] | ||
purpose: '' | ||
tagSet: 'CI' | ||
name: 'mac' | ||
|
||
jobs: | ||
- job: ${{ parameters.name }}_test_${{ parameters.purpose }}_${{ parameters.tagSet }} | ||
dependsOn: | ||
${{ parameters.parentJobs }} | ||
|
||
pool: | ||
vmImage: ${{ parameters.pool }} | ||
|
||
displayName: ${{ parameters.name }} Test - ${{ parameters.purpose }} - ${{ parameters.tagSet }} | ||
|
||
steps: | ||
- pwsh: | | ||
Get-ChildItem -Path env: | ||
displayName: Capture Environment | ||
condition: succeededOrFailed() | ||
|
||
- task: DownloadBuildArtifacts@0 | ||
displayName: 'Download build artifacts' | ||
inputs: | ||
downloadType: specific | ||
itemPattern: | | ||
build/**/* | ||
downloadPath: '$(System.ArtifactsDirectory)' | ||
|
||
- pwsh: | | ||
Get-ChildItem "$(System.ArtifactsDirectory)\*" -Recurse | ||
displayName: 'Capture Artifacts Directory' | ||
continueOnError: true | ||
|
||
- pwsh: | | ||
Import-Module .\tools\ci.psm1 | ||
Invoke-CIInstall -SkipUser | ||
displayName: Bootstrap | ||
|
||
- task: ExtractFiles@1 | ||
displayName: 'Extract Build ZIP' | ||
inputs: | ||
archiveFilePatterns: '$(System.ArtifactsDirectory)/build/build.zip' | ||
destinationFolder: '$(System.ArtifactsDirectory)/bins' | ||
|
||
- bash: | | ||
find "$(System.ArtifactsDirectory)/bins" -type d -exec chmod +rwx {} \; | ||
find "$(System.ArtifactsDirectory)/bins" -type f -exec chmod +rw {} \; | ||
displayName: 'Fix permissions' | ||
continueOnError: true | ||
|
||
- pwsh: | | ||
Get-ChildItem "$(System.ArtifactsDirectory)\bins\*" -Recurse -ErrorAction SilentlyContinue | ||
displayName: 'Capture Extracted Build ZIP' | ||
continueOnError: true | ||
|
||
- pwsh: | | ||
Import-Module .\tools\ci.psm1 | ||
Restore-PSOptions -PSOptionsPath '$(System.ArtifactsDirectory)\build\psoptions.json' | ||
$options = (Get-PSOptions) | ||
$rootPath = '$(System.ArtifactsDirectory)\bins' | ||
$originalRootPath = Split-Path -path $options.Output | ||
$path = Join-Path -path $rootPath -ChildPath (split-path -leaf -path $originalRootPath) | ||
$pwshPath = Join-Path -path $path -ChildPath 'pwsh' | ||
chmod a+x $pwshPath | ||
$options.Output = $pwshPath | ||
Set-PSOptions $options | ||
Invoke-CITest -Purpose '${{ parameters.purpose }}' -TagSet '${{ parameters.tagSet }}' | ||
displayName: Test | ||
condition: succeeded() | ||
- template: ./test/nix-test-steps.yml | ||
parameters: | ||
purpose: ${{ parameters.purpose }} | ||
tagSet: ${{ parameters.tagSet }} |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,28 @@ | ||
parameters: | ||
pool: 'macOS-latest' | ||
purpose: '' | ||
tagSet: 'CI' | ||
name: 'mac' | ||
|
||
jobs: | ||
- job: ${{ parameters.name }}_test_${{ parameters.purpose }}_${{ parameters.tagSet }} | ||
|
||
dependsOn: | ||
- getContainerJob | ||
|
||
variables: | ||
getContainerJob: $[ dependencies.getContainerJob.outputs['getContainerTask.containerName'] ] | ||
__INCONTAINER: 1 | ||
|
||
container: $[ variables.getContainerJob ] | ||
|
||
pool: | ||
vmImage: ${{ parameters.pool }} | ||
|
||
displayName: ${{ parameters.name }} Test - ${{ parameters.purpose }} - ${{ parameters.tagSet }} | ||
|
||
steps: | ||
- template: ./nix-test-steps.yml | ||
parameters: | ||
purpose: ${{ parameters.purpose }} | ||
tagSet: ${{ parameters.tagSet }} |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,59 @@ | ||
parameters: | ||
purpose: '' | ||
tagSet: 'CI' | ||
|
||
steps: | ||
- pwsh: | | ||
Get-ChildItem -Path env: | ||
displayName: Capture Environment | ||
condition: succeededOrFailed() | ||
|
||
- task: DownloadBuildArtifacts@0 | ||
displayName: 'Download build artifacts' | ||
inputs: | ||
downloadType: specific | ||
itemPattern: | | ||
build/**/* | ||
downloadPath: '$(System.ArtifactsDirectory)' | ||
|
||
- pwsh: | | ||
Get-ChildItem "$(System.ArtifactsDirectory)\*" -Recurse | ||
displayName: 'Capture Artifacts Directory' | ||
continueOnError: true | ||
|
||
- pwsh: | | ||
Import-Module .\tools\ci.psm1 | ||
Invoke-CIInstall -SkipUser | ||
displayName: Bootstrap | ||
|
||
- task: ExtractFiles@1 | ||
displayName: 'Extract Build ZIP' | ||
inputs: | ||
archiveFilePatterns: '$(System.ArtifactsDirectory)/build/build.zip' | ||
destinationFolder: '$(System.ArtifactsDirectory)/bins' | ||
|
||
- bash: | | ||
find "$(System.ArtifactsDirectory)/bins" -type d -exec chmod +rwx {} \; | ||
find "$(System.ArtifactsDirectory)/bins" -type f -exec chmod +rw {} \; | ||
TravisEz13 marked this conversation as resolved.
Show resolved
Hide resolved
|
||
displayName: 'Fix permissions' | ||
continueOnError: true | ||
|
||
- pwsh: | | ||
Get-ChildItem "$(System.ArtifactsDirectory)\bins\*" -Recurse -ErrorAction SilentlyContinue | ||
displayName: 'Capture Extracted Build ZIP' | ||
continueOnError: true | ||
|
||
- pwsh: | | ||
Import-Module .\tools\ci.psm1 | ||
Restore-PSOptions -PSOptionsPath '$(System.ArtifactsDirectory)\build\psoptions.json' | ||
$options = (Get-PSOptions) | ||
$rootPath = '$(System.ArtifactsDirectory)\bins' | ||
$originalRootPath = Split-Path -path $options.Output | ||
$path = Join-Path -path $rootPath -ChildPath (split-path -leaf -path $originalRootPath) | ||
$pwshPath = Join-Path -path $path -ChildPath 'pwsh' | ||
chmod a+x $pwshPath | ||
$options.Output = $pwshPath | ||
Set-PSOptions $options | ||
Invoke-CITest -Purpose '${{ parameters.purpose }}' -TagSet '${{ parameters.tagSet }}' | ||
displayName: Test | ||
condition: succeeded() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
i suppose eventually we should be using
ubuntu-latest
and provide a little more future proofingThere was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let's open a separate PR for that. I don't particularly like that because the people on build duty usually just decide to roll back if there is an actual version upgrade.