Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CI/CD conceptualisation: Migrate Jenkins build job "cyberduck-release-windows" to GitHub Actions #15555

Draft
wants to merge 86 commits into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
86 commits
Select commit Hold shift + click to select a range
490a3c2
set distributionManagement repos to local
RaveenaRamany Dec 10, 2023
ccabc97
Merge pull request #1 from RaveenaRamany/features/migration-cyberduck…
RaveenaRamany Dec 10, 2023
7c6015a
Add workflow cyberduck-release-windows
RaveenaRamany Dec 10, 2023
8795297
Fix missing transforms
RaveenaRamany Dec 10, 2023
5bdd666
update Repository secrets
RaveenaRamany Dec 10, 2023
70e22c7
Fix secret names
RaveenaRamany Dec 10, 2023
b05f3ef
Change runner to github hosted windows runner
RaveenaRamany Dec 10, 2023
c7ad044
Fix typo in secret name
RaveenaRamany Dec 10, 2023
e9297de
Merge pull request #2 from RaveenaRamany/convert-cyberduck-release-wi…
RaveenaRamany Dec 10, 2023
f67f6ef
Add option for git tag
RaveenaRamany Dec 10, 2023
3404b02
Merge pull request #3 from RaveenaRamany/features/migration-cyberduck…
RaveenaRamany Dec 10, 2023
547c0b8
Fix WORKSPACE variable
RaveenaRamany Dec 10, 2023
9f7cd51
Merge pull request #4 from RaveenaRamany/features/migration-cyberduck…
RaveenaRamany Dec 10, 2023
0689757
Fix sparkle updater certificate path
RaveenaRamany Dec 10, 2023
67679c0
Add quotes
RaveenaRamany Dec 10, 2023
60255dc
env test
RaveenaRamany Dec 10, 2023
5cbf6d6
env test
RaveenaRamany Dec 10, 2023
07fed6c
env test
RaveenaRamany Dec 10, 2023
7a6faaa
Fix environment variables
RaveenaRamany Dec 10, 2023
9a129dd
Fix maven-settings-action
RaveenaRamany Dec 10, 2023
e175cf7
Fix maven settings.xml
RaveenaRamany Dec 10, 2023
52633d0
Add workflow cyberduck-release-windows-chocolatey
RaveenaRamany Dec 11, 2023
b415011
Merge pull request #5 from RaveenaRamany/convert-cyberduck-release-wi…
RaveenaRamany Dec 11, 2023
dc9cbb6
comment env-key
RaveenaRamany Dec 11, 2023
4750959
Update cyberduck-release-windows-chocolatey.yml
RaveenaRamany Dec 12, 2023
06ca975
fix Variable GITHUB_REPOSITROY
RaveenaRamany Dec 14, 2023
40c20d8
update runs-on
RaveenaRamany Dec 14, 2023
6132d45
Merge branch 'iterate-ch:master' into master
RaveenaRamany Dec 14, 2023
1e30dc4
Merge pull request #6 from RaveenaRamany/features/migration-cyberduck…
RaveenaRamany Dec 14, 2023
66e1d6a
customize upload
RaveenaRamany Dec 16, 2023
fb97d74
change credential values
RaveenaRamany Dec 16, 2023
89ae06d
update aws bucket location
RaveenaRamany Dec 16, 2023
14c4335
update server.xml credentials
RaveenaRamany Dec 16, 2023
19b212f
remove interactivemode setting
RaveenaRamany Dec 16, 2023
0402597
remove interactive mode & add debugging
RaveenaRamany Dec 16, 2023
42aff91
show server.xml
RaveenaRamany Dec 16, 2023
a88efc1
test server.xml
RaveenaRamany Dec 16, 2023
0143980
comment out server.xml generation
RaveenaRamany Dec 16, 2023
d4cd2c1
comment out servers.xml
RaveenaRamany Dec 16, 2023
a965c78
settings.xml test
RaveenaRamany Dec 16, 2023
015c434
fix setup-java overwriting settings.xml
RaveenaRamany Dec 16, 2023
66010a2
change description of setup-java action
RaveenaRamany Dec 16, 2023
ef7cfa9
output settings.xml
RaveenaRamany Dec 16, 2023
cf60354
change settings.xml path to default
RaveenaRamany Dec 16, 2023
d2dec59
remove output of settings.xml
RaveenaRamany Dec 16, 2023
4133db3
change settings.xml path to github.workspace
RaveenaRamany Dec 16, 2023
c04c677
output content of settings.xml
RaveenaRamany Dec 16, 2023
5117f28
remove output of settings.xml
RaveenaRamany Dec 16, 2023
281b117
delete debug modus
RaveenaRamany Dec 16, 2023
9a33b95
remove space
RaveenaRamany Dec 16, 2023
97b2974
maven enable debugging
RaveenaRamany Dec 17, 2023
8c9d9d9
enable fetching all history
RaveenaRamany Dec 18, 2023
54e8022
Add MSBuild.exe to PATH
RaveenaRamany Dec 18, 2023
4d8e6c9
Install build requirement Bonjour Print Services
RaveenaRamany Dec 18, 2023
dc9ac7c
Move build requirements installation before build execution
RaveenaRamany Dec 18, 2023
bf31d55
Skip package signing
RaveenaRamany Dec 18, 2023
0b49d4f
Skip package signing
RaveenaRamany Dec 18, 2023
05f7642
Install cygwin to provide sh and openssl
RaveenaRamany Dec 18, 2023
9529548
Only build cyberduck for windows
RaveenaRamany Dec 18, 2023
fcf4033
Only build Cyberduck.Native
RaveenaRamany Dec 18, 2023
e64eb1f
Build all projects
RaveenaRamany Dec 18, 2023
92ff5b4
choco yes to all
RaveenaRamany Dec 19, 2023
f0a745c
install openssl
RaveenaRamany Dec 19, 2023
f30c2ae
Only build core/native
RaveenaRamany Dec 19, 2023
d1cd6e0
only build windows module
RaveenaRamany Dec 19, 2023
9ab78ab
add install Cmake
RaveenaRamany Dec 19, 2023
fb755bd
Add cmake to PATH
RaveenaRamany Dec 19, 2023
3a53a0f
add cpack using chocolatey
RaveenaRamany Dec 20, 2023
b9ec705
Remove unnecessary CPack (CMake) build requirment and make yaml more …
RaveenaRamany Dec 20, 2023
5b1089a
Build all modules
RaveenaRamany Dec 20, 2023
9109341
output env Path
RaveenaRamany Dec 20, 2023
b67150b
Refresh env for troubleshooting unknown command duck
RaveenaRamany Dec 20, 2023
9ad1c06
Skip building for AWS upload troubleshooting
RaveenaRamany Dec 20, 2023
8eb3621
Display duck install path
RaveenaRamany Dec 20, 2023
cd03e5e
Remove bonjour & openssl install for testing behaviour of windows pat…
RaveenaRamany Dec 20, 2023
1fe7fee
Add duck cli to GitHub runner persistent path
RaveenaRamany Dec 20, 2023
9d0b815
Switch powershell version to 6+
RaveenaRamany Dec 20, 2023
5928617
Refactor adding variable to persistent GitHub PATH
RaveenaRamany Dec 20, 2023
bf459a0
Switch to powershell core version 6+
RaveenaRamany Dec 20, 2023
7a4e22b
Import revreshenv helper
RaveenaRamany Dec 20, 2023
f93dbf3
output GitHub env
RaveenaRamany Dec 20, 2023
2218f53
Show directories under GITHUB_WORKSPACE
RaveenaRamany Dec 20, 2023
f6091a1
Output GITHUB_PATH
RaveenaRamany Dec 20, 2023
ccdab04
Only test duck install
RaveenaRamany Dec 20, 2023
daf2993
Writeback current PATH containing cyberduck
RaveenaRamany Dec 20, 2023
ec9f070
Writeback current PATH after installing packages
RaveenaRamany Dec 21, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 19 additions & 0 deletions .github/workflows/cyberduck-release-windows-chocolatey.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
name: cyberduck-release-windows-chocolatey
on:
workflow_dispatch:
env:
CHOCOLATEY_API: "${{ secrets.CHOCOLATEY_API_KEY }}"
# # TimestamperBuildWrapper was not converted because the behavior is available by default in GitHub Actions and/or it is not configurable
jobs:
build:
runs-on:
- windows-latest
steps:
- name: checkout
uses: actions/checkout@v4.1.0
- name: run batch command
shell: cmd
run: FOR %%c in ("%GITHUB_REPOSITORY%\..\cyberduck-release-windows\windows\target\release\cyberduck.*.nupkg") DO C:\ProgramData\chocolatey\bin\cpush --verbose --api-key %CHOCOLATEY_API% %%c
# # Mailer plugin was not converted because GitHub Actions will email the actor after failed build and does not support emailing a list of recipients


208 changes: 208 additions & 0 deletions .github/workflows/cyberduck-release-windows.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,208 @@
name: cyberduck-release-windows
on:
workflow_dispatch:
inputs:
GIT_TAG:
required: false

env:
AWS_ACCESS_KEY_DEPLOYMENT: "${{ secrets.AWS_ACCESS_KEY_DEPLOYMENT }}"
AWS_SECRET_KEY_DEPLOYMENT: "${{ secrets.AWS_SECRET_KEY_DEPLOYMENT }}"
AWS_CUSTOM_IP5_ACCESS_KEY_DEPLOYMENT: "${{ secrets.AWS_CUSTOM_IP5_ACCESS_KEY_DEPLOYMENT }}"
AWS_CUSTOM_IP5_SECRET_KEY_DEPLOYMENT: "${{ secrets.AWS_CUSTOM_IP5_SECRET_KEY_DEPLOYMENT }}"
RACKSPACE_USER: "${{ secrets.RACKSPACE_USER }}"
RACKSPACE_PASSWORD: "${{ secrets.RACKSPACE_PASSWORD }}"

jobs:
build:
runs-on:
- windows-latest
steps:
- name: clean workspace
shell: ruby {0}
run: |-
require "fileutils"
Dir.chdir(ENV["GITHUB_WORKSPACE"]) do
paths = Dir.glob(["**/*"])
paths -= Dir.glob([".git/**", ".repository/**"])
paths.each do |path|
File.delete(path) if File.file?(path)
FileUtils.rm_rf(path) if File.directory?(path)
end
end

- name: checkout
uses: actions/checkout@v4.1.0
with:
fetch-depth: '0'
ref: ${{ inputs.GIT_TAG }}

- name: Copy Sparkle Updater Private Key (DSA)
shell: pwsh
env:
SPARKLE_UPDATER_PK: ${{ secrets.WIN_SPARKLE_UPDATER_PRIVATE_KEY }}
run: ${env:SPARKLE_UPDATER_PK} > "${env:GITHUB_WORKSPACE}\www\update\private.pem"

- name: Generate maven settings.xml
uses: s4u/maven-settings-action@v2.8.0
with:
path: "${{ github.workspace }}/settings.xml"
servers: |
[{
"id": "custom-ip5-maven.cyberduck.io-release",
"username": "${{ secrets.AWS_CUSTOM_IP5_ACCESS_KEY_DEPLOYMENT }}",
"password": "${{ secrets.AWS_CUSTOM_IP5_SECRET_KEY_DEPLOYMENT }}"
},
{
"id": "custom-ip5-maven.cyberduck.io-snapshot",
"username": "${{ secrets.AWS_CUSTOM_IP5_ACCESS_KEY_DEPLOYMENT }}",
"password": "${{ secrets.AWS_CUSTOM_IP5_SECRET_KEY_DEPLOYMENT }}"
},
{
"id": "maven.iterate.ch-release",
"username": "${{ secrets.AWS_ACCESS_KEY_DEPLOYMENT }}",
"password": "${{ secrets.AWS_SECRET_KEY_DEPLOYMENT }}"
},
{
"id": "maven.iterate.ch-snapshot",
"username": "${{ secrets.AWS_ACCESS_KEY_DEPLOYMENT }}",
"password": "${{ secrets.AWS_SECRET_KEY_DEPLOYMENT }}"
},
{
"id": "maven.cyberduck.io-release",
"username": "${{ secrets.AWS_ACCESS_KEY_DEPLOYMENT }}",
"password": "${{ secrets.AWS_SECRET_KEY_DEPLOYMENT }}"
},
{
"id": "maven.cyberduck.io-snapshot",
"username": "${{ secrets.AWS_ACCESS_KEY_DEPLOYMENT }}",
"password": "${{ secrets.AWS_SECRET_KEY_DEPLOYMENT }}"
}]

- name: Set up JDK 17 and skip overwriting of settings.xml
uses: actions/setup-java@v4.0.0
with:
distribution: 'temurin'
java-version: '17'
settings-path: "${{ github.workspace }}"
overwrite-settings: false

- name: Install upload requirement Cyberduck CLI using chocolatey
shell: pwsh
run: Import-Module "${env:ChocolateyInstall}\helpers\chocolateyProfile.psm1" && choco install duck -y && refreshenv && "${env:Path}" | Out-File -FilePath ${env:GITHUB_PATH} -Append
# run: Import-Module "${env:ChocolateyInstall}\helpers\chocolateyProfile.psm1" && choco install duck -y && refreshenv && Get-Command duck | Split-Path -Parent | Out-File -FilePath ${env:GITHUB_PATH} -Append

- name: Install build requirement Bonjour Print Services for Windows (https://community.chocolatey.org/packages/bonjour)
shell: pwsh
run: Import-Module "${env:ChocolateyInstall}\helpers\chocolateyProfile.psm1" && choco install bonjour -y && refreshenv && "${env:Path}" | Out-File -FilePath ${env:GITHUB_PATH} -Append

- name: Install build requirement openssl using chocolatey
shell: pwsh
run: Import-Module "${env:ChocolateyInstall}\helpers\chocolateyProfile.psm1" && choco install openssl -y && refreshenv && "${env:Path}" | Out-File -FilePath ${env:GITHUB_PATH} -Append

- name: Add MSBuild.exe to PATH
uses: microsoft/setup-msbuild@v1.1

- name: Run maven
# Todo: Remove "-e" and "-DskipSign=true" Options for production
run: mvn clean deploy -DskipSign=true -e --settings ${env:GITHUB_WORKSPACE}/settings.xml -DskipTests -D"sparkle.feed="

- name: Upload files to Rackspace
shell: pwsh
run: echo "Dummy rackspace upload"
# run: Get-ChildItem "${Env:GITHUB_WORKSPACE}\windows\target\release\" -Filter Cyberduck-* | Foreach-Object -process { duck --upload rackspace:/cdn.cyberduck.ch/ $_.FullName --username ${Env:RACKSPACE_USER} --password ${Env:RACKSPACE_PASSWORD} --retry --quiet --existing overwrite --region DFW }
# run: FOR %%c in ("%GITHUB_WORKSPACE%\windows\target\release\Cyberduck-*") DO "C:\Program Files\Cyberduck CLI\duck.exe" --upload rackspace:/cdn.cyberduck.ch/ "%%c" --username %RACKSPACE_USER% --password %RACKSPACE_PASSWORD% --retry --quiet --existing overwrite --region DFW

- name: Upload files to AWS S3 Bucket
shell: pwsh
# Todo: Remove echos
run: dir "${Env:GITHUB_WORKSPACE}"; echo ${Env:GITHUB_WORKSPACE}\windows\target\release\; Get-ChildItem "${Env:GITHUB_WORKSPACE}\windows\target\release\" -Filter Cyberduck-* | Foreach-Object -process { duck --upload s3:/ip5-cicd-cyberduck/ $_.FullName --username ${Env:AWS_CUSTOM_IP5_ACCESS_KEY_DEPLOYMENT} --password ${Env:AWS_CUSTOM_IP5_SECRET_KEY_DEPLOYMENT} --retry --quiet --existing overwrite --region eu-west-1}
# run: Get-ChildItem "${Env:GITHUB_WORKSPACE}\windows\target\release\" -Filter Cyberduck-* | Foreach-Object -process { duck --upload s3:/release.cyberduck.io/ $_.FullName --username ${Env:AWS_ACCESS_KEY_DEPLOYMENT} --password ${Env:AWS_SECRET_KEY_DEPLOYMENT} --retry --quiet --existing overwrite --region us-east-1 }
# run: FOR %%c in ("%GITHUB_WORKSPACE%\windows\target\release\Cyberduck-*") DO "C:\Program Files\Cyberduck CLI\duck.exe" --upload s3:/release.cyberduck.io/ "%%c" --username %AWS_ACCESS_KEY_DEPLOYMENT% --password %AWS_SECRET_KEY_DEPLOYMENT% --retry --quiet --existing overwrite --region us-east-1

- name: Upload Artifacts
uses: actions/upload-artifact@v3.1.3
if: always()
with:
path: |-
windows\target\release\*.exe
windows\target\release\*.msi
windows\target\release\*.appx
!**/*~
!**/#*#
!**/.#*
!**/%*%
!**/._*
!**/CVS
!**/CVS/**
!**/.cvsignore
!**/SCCS
!**/SCCS/**
!**/vssver.scc
!**/.svn
!**/.svn/**
!**/.DS_Store
!**/.git
!**/.git/**
!**/.gitattributes
!**/.gitignore
!**/.gitmodules
!**/.hg
!**/.hg/**
!**/.hgignore
!**/.hgsub
!**/.hgsubstate
!**/.hgtags
!**/.bzr
!**/.bzr/**
!**/.bzrignore

# # Fingerprinter was not converted because the behavior is available by default in GitHub Actions and/or it is not configurable
# # Mailer plugin was not converted because GitHub Actions will email the actor after failed build and does not support emailing a list of recipients
# Ensure parameter if_key_exists is set correctly
- name: Install SSH key
uses: shimataro/ssh-key-action@v2.6.1
with:
key: "${{ secrets.VERSION_CYBERDUCK_IO_SSH_KEY }}"
name: id_rsa-version_cyberduck_io
known_hosts: "${{ secrets.VERSION_CYBERDUCK_IO_KNOWN_HOSTS }}"
if_key_exists: replace # replace will allow us to update the ssh-key on an existing build
config: |
Host VERSION_CYBERDUCK_IO
HostName ${{ secrets.VERSION_CYBERDUCK_IO_HOST_NAME }}
User ${{ secrets.VERSION_CYBERDUCK_IO_USER }}
IdentityFile ~/.ssh/id_rsa-version_cyberduck_io
if: always()

- name: setup file transfer file
uses: actions/github-script@v7.0.0
with:
script: |-
const fs = require('fs').promises
const path = require('path')
const patterns = "windows/target/update/changelog.*,!**/*~,!**/#*#,!**/.#*,!**/%*%,!**/._*,!**/CVS,!**/CVS/**,!**/.cvsignore,!**/SCCS,!**/SCCS/**,!**/vssver.scc,!**/.svn,!**/.svn/**,!**/.DS_Store,!**/.git,!**/.git/**,!**/.gitattributes,!**/.gitignore,!**/.gitmodules,!**/.hg,!**/.hg/**,!**/.hgignore,!**/.hgsub,!**/.hgsubstate,!**/.hgtags,!**/.bzr,!**/.bzr/**,!**/.bzrignore"
const globber = await glob.create(patterns.replace(/,/g, "\n"))
const files = []
for await (const file of globber.globGenerator()) {
if ((await fs.lstat(file)).isDirectory()) continue
files.push(path.relative(process.cwd(), file))
}
fs.writeFile("version_cyberduck_io-transfer.txt", files.join("\n"), (err) => {})
if: always()

- name: run file transfers
run: |-
ssh VERSION_CYBERDUCK_IO 'mkdir -p windows'
tar -cvf version_cyberduck_io-transfer.tar --files-from version_cyberduck_io-transfer.txt
scp version_cyberduck_io-transfer.tar VERSION_CYBERDUCK_IO:
ssh VERSION_CYBERDUCK_IO 'tar -xvf version_cyberduck_io-transfer.tar -C windows && rm version_cyberduck_io-transfer.tar'
if: always()
# # This item has no matching transformer
# - uses: hudson.tasks.BuildTrigger
# with:
# childProjects: cyberduck-release-windows-chocolatey
# threshold:
# name: SUCCESS
# ordinal: '0'
# color: BLUE
# completeBuild: 'true'
6 changes: 4 additions & 2 deletions pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -859,12 +859,14 @@
<distributionManagement>
<repository>
<id>maven.cyberduck.io-release</id>
<url>s3://repo.maven.cyberduck.io/releases/</url>
<!-- <url>s3://repo.maven.cyberduck.io/releases/</url>
-->
<url>file://${user.home}</url>
<layout>default</layout>
</repository>
<snapshotRepository>
<id>maven.cyberduck.io-snapshot</id>
<url>s3://repo.maven.cyberduck.io/snapshots/</url>
<url>file://${user.home}</url>
</snapshotRepository>
<site>
<id>github-pages-site</id>
Expand Down