WIndowsPb: gpg signature verification for ANT binary #3018

Haroon-Khel · 2023-04-05T11:46:31Z

commit message has one of the standard prefixes
faq.md updated if appropriate
other documentation is changed or added (if applicable)
playbook changes run through VPC or QPC (if you have access)
VPC/QPC not applicable for this PR
for inventory.yml changes, bastillion/nagios/jenkins updated accordingly

It seems only the Ant download supplies a .asc or .sig file. For the other downloads, we supply a fixed checksum string which, since we do not plan on changing, provide adequate verification of a download.

Theres still the java downloads in the playbook which are capable of having gpg verification. Getting the signature file is a bit tricky, on unix we do something like this

curl -s 'https://api.adoptium.net/v3/assets/feature_releases/{{ jdk_version }}/ga?architecture={{ api_architecture }}&heap_size=normal&image_type=jdk&jvm_impl={{ bootjdk }}&os={{ platformLinux }}&page=0&page_size=1&project=jdk&vendor=eclipse' | grep signature_link | awk '{split($0,a,"\""); print a[4]}'

which does not seem to be working well in a windows environment. So until I can figure this out, I thought id merge my work so far

Haroon-Khel · 2023-04-05T11:55:46Z

windows vpc https://ci.adoptium.net/job/VagrantPlaybookCheck/1661/console

Haroon-Khel · 2023-04-05T12:11:15Z

Latest vpc https://ci.adoptium.net/job/VagrantPlaybookCheck/1662/console

Haroon-Khel · 2023-04-05T13:53:32Z

TASK [ANT : Import GPG Key] ****************************************************
fatal: [127.0.0.1]: FAILED! => {"changed": true, "cmd": "gpg --keyserver keyserver.ubuntu.com --recv-keys \"CE8075A251547BEE249BC151A2115AE15F6B8B72\"", "delta": "0:00:00.072935", "end": "2023-04-05 12:25:35.885799", "msg": "non-zero return code", "rc": 2, "start": "2023-04-05 12:25:35.812864", "stderr": "gpg: fatal: can't create directory `/home/runneradmin/.gnupg': No such file or directory\nsecmem usage: 0/0 bytes in 0/0 blocks of pool 0/65536\n", "stderr_lines": ["gpg: fatal: can't create directory `/home/runneradmin/.gnupg': No such file or directory", "secmem usage: 0/0 bytes in 0/0 blocks of pool 0/65536"], "stdout": "", "stdout_lines": []}

I wouldnt expect there to be a problem with making directories in /home/$user once cygwin is installed

Haroon-Khel · 2023-04-05T13:54:56Z

Latest vpc hit the same error

14:47:20 TASK [ANT : Import GPG Key] ****************************************************
14:47:23 fatal: [127.0.0.1]: FAILED! => {"changed": true, "cmd": "gpg --keyserver keyserver.ubuntu.com --recv-keys \"CE8075A251547BEE249BC151A2115AE15F6B8B72\"", "delta": "0:00:00.297306", "end": "2023-04-05 01:30:39.610441", "msg": "non-zero return code", "rc": 2, "start": "2023-04-05 01:30:39.313134", "stderr": "gpg: fatal: can't create directory `/home/vagrant/.gnupg': No such file or directory\nsecmem usage: 0/0 bytes in 0/0 blocks of pool 0/65536\n", "stderr_lines": ["gpg: fatal: can't create directory `/home/vagrant/.gnupg': No such file or directory", "secmem usage: 0/0 bytes in 0/0 blocks of pool 0/65536"], "stdout": "", "stdout_lines": []}

Did not get this error when testing on test-azure-win2012r2-x64-1

sxa · 2023-04-05T15:52:42Z

Latest vpc hit the same error

14:47:20 TASK [ANT : Import GPG Key] ****************************************************
14:47:23 fatal: [127.0.0.1]: FAILED! => {"changed": true, "cmd": "gpg --keyserver keyserver.ubuntu.com --recv-keys \"CE8075A251547BEE249BC151A2115AE15F6B8B72\"", "delta": "0:00:00.297306", "end": "2023-04-05 01:30:39.610441", "msg": "non-zero return code", "rc": 2, "start": "2023-04-05 01:30:39.313134", "stderr": "gpg: fatal: can't create directory `/home/vagrant/.gnupg': No such file or directory\nsecmem usage: 0/0 bytes in 0/0 blocks of pool 0/65536\n", "stderr_lines": ["gpg: fatal: can't create directory `/home/vagrant/.gnupg': No such file or directory", "secmem usage: 0/0 bytes in 0/0 blocks of pool 0/65536"], "stdout": "", "stdout_lines": []}

Did not get this error when testing on test-azure-win2012r2-x64-1

I would guess that's because it's running under cmd.exe (The windows shell) and not cygwin's bash and therefore doesn't understand /home/vagrant. When you tested outside ansible was it from a cygwin shell?

sxa · 2023-04-05T15:56:58Z

You can set an environment variable GNUPGHOME=$WORKSPACE/.gnupg to a specific directory to use, so that may give you a way forward to use an alternate location (I use this for the Temurin GPG signing to store it in the jenkins workspace)

(There's probably a command line option to set it too, but I've not used that)

Haroon-Khel · 2023-04-06T11:25:09Z

I would guess that's because it's running under cmd.exe (The windows shell) and not cygwin's bash and therefore doesn't understand /home/vagrant. When you tested outside ansible was it from a cygwin shell?

Works in a cmd.exe environment too. I think with cygwin installed, any shell can access /home/ directories?

Haroon-Khel · 2023-04-06T11:31:46Z

(There's probably a command line option to set it too, but I've not used that)

--homedir looks like it should work, just need to figure out how to use it properly

(both forward and backslash path locations)

sxa · 2023-04-06T14:04:42Z

Interesting ... the other thing to check would be whether there is another gpg tool on the machine that's being picked up instead of the cygwin one. Could run a test where you fully qualify c:\cygwin\bin\gpg to run it and see if that makes a difference.

sxa

LGTM but my only comment would be that it may not be necessary to import the key each time (e.g. it could be skipped if it already exists on the machine)

karianna · 2023-04-12T23:59:27Z

Playbook checkers are failing, related?

sxa · 2023-04-13T08:47:17Z

Playbook checkers are failing, related?

Looks like it. (My review was based on reading the code, and so the check failures need to be understood and addressed before this is merged)

Create the home/.gnupg directory within windows/cygwin environment.

karianna · 2023-05-18T10:25:49Z

@Haroon-Khel needs a linter fix

karianna · 2023-05-18T19:54:46Z

ansible/playbooks/AdoptOpenJDK_Windows_Playbook/roles/ANT/tasks/main.yml

+    file: "{{playbook_dir}}/roles/GPG_signature_verification/tasks/main.yml"
+  vars:
+    file_path: c:\temp\ant.zip
+    signature_link: "https://archive.apache.org/dist/ant/binaries/apache-ant-1.10.5-bin.zip.asc"


Ant is on 1.10.13 now - should we upgrade?

karianna

Ant version

sxa · 2023-05-22T09:52:12Z

Ant version

I would suggest that we put the security checks in place and then discuss that as a separate issue. I'd want there to be additional testing on a new ant version in order to ensure we don't have compatibility issues before just bumping it up (especially sine this PR is specific to Windows and we use the same fixed version across all platforms IIRC) - perhaps create a new issue proposing an upgrade? It's probably about time we did look at that.

ant is critical to the test case executions and also now for the SBOM generation.

Haroon-Khel · 2023-05-23T10:08:58Z

Ant contrib is failing to download on the 2022 windows workflow

ASK [ANT : Download ant-contrib] **********************************************
fatal: [127.0.0.1]: FAILED! => {"changed": false, "dest": "c:\\temp\\ant-contrib.zip", "elapsed": 0.8295483, "msg": "Error downloading 'https://sourceforge.net/projects/ant-contrib/files/ant-contrib/ant-contrib-1.0b2/ant-contrib-1.0b2-bin.zip' to 'c:\\temp\\ant-contrib.zip': The request was aborted: Could not create SSL/TLS secure channel.", "status_code": null, "url": "https://sourceforge.net/projects/ant-contrib/files/ant-contrib/ant-contrib-1.0b2/ant-contrib-1.0b2-bin.zip"}

however that error is unrelated to this pr, the gpg check on the ant binary just before it passes without error

A bit confused by this linter error


Run # Ansible code static analysis
WARNING  Listing 1 violation(s) that are fatal
syntax-check[missing-file]: Unable to retrieve file contents
ansible/playbooks/AdoptOpenJDK_Windows_Playbook/roles/ANT:1:1 Could not find or access '/tmp/roles/GPG_signature_verification/tasks/main.yml' on the Ansible Controller.

Error: Unable to retrieve file contents

                    Rule Violation Summary                     
 count tag                        profile rule associated tags 
     1 syntax-check[missing-file] min     core, unskippable    

Failed after : 1 failure(s), 0 warning(s) on 505 files.
Error: Process completed with exit code 2.

Haroon-Khel · 2023-05-23T17:02:30Z

Linter issue is solved. The pr checks are running now because I had to rebase (branch was out of date) but this pr is ready to be merged

Haroon-Khel · 2023-05-23T17:03:23Z

The ant contrib error mentioned in #3018 (comment) is intermittent

steelhead31

LGTM

gpg signature verification for ANT binary

7c6d880

Haroon-Khel requested a review from sxa April 5, 2023 11:46

Haroon-Khel mentioned this pull request Apr 5, 2023

Add GPG signature verification to adoptium binary downloads #2908

Closed

3 tasks

github-actions bot added the ansible label Apr 5, 2023

fix conditional

19023aa

sxa approved these changes Apr 11, 2023

View reviewed changes

Haroon-Khel mentioned this pull request May 17, 2023

Windows Playbook Checker PR check failing signature verification steps in playbook #3067

Closed

steelhead31 and others added 7 commits May 18, 2023 08:34

Create gnupg dir when doesnt exist

7e95341

Create the home/.gnupg directory within windows/cygwin environment.

Linter fix

b822ae8

Linter fix.

1b6710c

Linter fix

46a69a2

Tweak path to GPG tasks

fe9afac

Linter Fix

c2587df

Merge branch 'master' into windows.gpg

69cf056

steelhead31 added 2 commits May 18, 2023 15:21

Test Regsvr without silent for vs2017

26825ac

Undo regsvr test

8bf6be9

karianna reviewed May 18, 2023

View reviewed changes

Merge branch 'master' into windows.gpg

80c1784

Haroon-Khel and others added 3 commits May 23, 2023 13:53

remove playbooks_dir variable

e3025bf

change import to include

7d7bcb5

Merge branch 'master' into windows.gpg

64ae227

Haroon-Khel requested review from steelhead31 and karianna May 23, 2023 17:02

steelhead31 approved these changes May 24, 2023

View reviewed changes

Haroon-Khel merged commit d9b705d into adoptium:master May 24, 2023
6 of 7 checks passed

karianna mentioned this pull request May 25, 2023

> Update Ant version to 1.10.13 #3074

Open

Haroon-Khel deleted the windows.gpg branch June 5, 2023 15:43

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

WIndowsPb: gpg signature verification for ANT binary #3018

WIndowsPb: gpg signature verification for ANT binary #3018

Haroon-Khel commented Apr 5, 2023 •

edited

Loading

Haroon-Khel commented Apr 5, 2023

Haroon-Khel commented Apr 5, 2023

Haroon-Khel commented Apr 5, 2023 •

edited

Loading

Haroon-Khel commented Apr 5, 2023

sxa commented Apr 5, 2023 •

edited

Loading

sxa commented Apr 5, 2023 •

edited

Loading

Haroon-Khel commented Apr 6, 2023 •

edited

Loading

Haroon-Khel commented Apr 6, 2023

sxa commented Apr 6, 2023

sxa left a comment

karianna commented Apr 12, 2023

sxa commented Apr 13, 2023

karianna commented May 18, 2023

karianna May 18, 2023

karianna left a comment

sxa commented May 22, 2023

Haroon-Khel commented May 23, 2023

Haroon-Khel commented May 23, 2023

Haroon-Khel commented May 23, 2023

steelhead31 left a comment

WIndowsPb: gpg signature verification for ANT binary #3018

WIndowsPb: gpg signature verification for ANT binary #3018

Conversation

Haroon-Khel commented Apr 5, 2023 • edited Loading

Haroon-Khel commented Apr 5, 2023

Haroon-Khel commented Apr 5, 2023

Haroon-Khel commented Apr 5, 2023 • edited Loading

Haroon-Khel commented Apr 5, 2023

sxa commented Apr 5, 2023 • edited Loading

sxa commented Apr 5, 2023 • edited Loading

Haroon-Khel commented Apr 6, 2023 • edited Loading

Haroon-Khel commented Apr 6, 2023

sxa commented Apr 6, 2023

sxa left a comment

Choose a reason for hiding this comment

karianna commented Apr 12, 2023

sxa commented Apr 13, 2023

karianna commented May 18, 2023

karianna May 18, 2023

Choose a reason for hiding this comment

karianna left a comment

Choose a reason for hiding this comment

sxa commented May 22, 2023

Haroon-Khel commented May 23, 2023

Haroon-Khel commented May 23, 2023

Haroon-Khel commented May 23, 2023

steelhead31 left a comment

Choose a reason for hiding this comment

Haroon-Khel commented Apr 5, 2023 •

edited

Loading

Haroon-Khel commented Apr 5, 2023 •

edited

Loading

sxa commented Apr 5, 2023 •

edited

Loading

sxa commented Apr 5, 2023 •

edited

Loading

Haroon-Khel commented Apr 6, 2023 •

edited

Loading