Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP: DO NOT MERGE: RHEL 9 smoke test #538

Closed
wants to merge 1 commit into from

Conversation

frobware
Copy link
Contributor

No description provided.

@openshift-ci openshift-ci bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Nov 17, 2023
Copy link
Contributor

openshift-ci bot commented Nov 17, 2023

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please ask for approval from frobware. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@frobware
Copy link
Contributor Author

/test all

@frobware
Copy link
Contributor Author

/test perfscale-aws-ingress-perf

@frobware
Copy link
Contributor Author

An additional sample:

/test perfscale-aws-ingress-perf

@frobware
Copy link
Contributor Author

/test e2e-upgrade

@frobware
Copy link
Contributor Author

/test perfscale-aws-ingress-perf

1 similar comment
@frobware
Copy link
Contributor Author

/test perfscale-aws-ingress-perf

@frobware
Copy link
Contributor Author

/test e2e-upgrade

@frobware
Copy link
Contributor Author

/retest

@frobware
Copy link
Contributor Author

The origin test "when FIPS is disabled the HAProxy router should serve routes when configured with a 1024-bit RSA key" fails because HAProxy 2.6.13 with OpenSSL 3 (on RHEL 9) fails to start.

Pulling out the cert data from: https://github.com/openshift/origin/blob/master/test/extended/router/certs.go#L79 we see the following:

% ocp-haproxy-2.6.13 -c -f ./testrun/haproxy/
[NOTICE]   (46579) : haproxy version is 2.6.13-234aa6d
[NOTICE]   (46579) : path to executable is /etc/profiles/per-user/aim/bin/ocp-haproxy-2.6.13
[ALERT]    (46579) : config : parsing [./testrun/haproxy//haproxy.cfg:133] : 'bind unix@/tmp/haproxy-sni.sock' in section 'frontend' : unable to load SSL certificate into SSL Context 'testrun/certs/domain.pem'.
[ALERT]    (46579) : config : parsing [./testrun/haproxy//haproxy.cfg:175] : 'bind unix@/tmp/haproxy-no-sni.sock' in section 'frontend' : unable to load SSL certificate into SSL Context 'testrun/certs/domain.pem'.
[ALERT]    (46579) : config : parsing [./testrun/haproxy//haproxy.cfg:219] : 'bind :9443' in section 'frontend' : unable to load SSL certificate into SSL Context 'testrun/certs/domain.pem'.
[ALERT]    (46579) : config : Error(s) found in configuration file : ./testrun/haproxy//haproxy.cfg
[ALERT]    (46579) : config : Fatal errors found in configuration.

which actually doesn't tell us very much. If you run with haproxy-2.8.3 then you see a hint at what's actually at fault:

% ocp-haproxy-2.8.3 -c -f ./testrun/haproxy/
[NOTICE]   (46609) : haproxy version is 2.8.3-86e043a
[NOTICE]   (46609) : path to executable is /etc/profiles/per-user/aim/bin/ocp-haproxy-2.8.3
[ALERT]    (46609) : config : parsing [./testrun/haproxy//haproxy.cfg:133] : 'bind unix@/tmp/haproxy-sni.sock' in section 'frontend' : unable to load SSL certificate into SSL Context 'testrun/certs/domain.pem': ca md too weak.
[ALERT]    (46609) : config : parsing [./testrun/haproxy//haproxy.cfg:175] : 'bind unix@/tmp/haproxy-no-sni.sock' in section 'frontend' : unable to load SSL certificate into SSL Context 'testrun/certs/domain.pem': ca md too weak.
[ALERT]    (46609) : config : parsing [./testrun/haproxy//haproxy.cfg:219] : 'bind :9443' in section 'frontend' : unable to load SSL certificate into SSL Context 'testrun/certs/domain.pem': ca md too weak.
[ALERT]    (46609) : config : Error(s) found in configuration file : ./testrun/haproxy//haproxy.cfg
[ALERT]    (46609) : config : Fatal errors found in configuration.

Notably "ca md too weak".

Peeking at the certificate I see that the signature is based on SHA-1:

% openssl x509 -in ./testrun/certs/domain.pem -text -noout

Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number: 6 (0x6)
        Signature Algorithm: sha1WithRSAEncryption
        Issuer: C = US, ST = SC, L = Default City, O = Default Company Ltd, OU = Test CA, CN = www.exampleca.com, emailAddress = example@example.com
        Validity
            Not Before: Jan 13 19:40:57 2016 GMT
            Not After : Jan 10 19:40:57 2026 GMT
        Subject: CN = www.example.com, ST = SC, C = US, emailAddress = example@example.com, O = Example, OU = Example
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                Public-Key: (1024 bit)
                Modulus:
                    00:cd:01:bb:ef:a8:1d:5d:70:72:98:56:45:b3:0b:
                    51:fb:7c:7c:3e:e7:3c:6f:79:c6:ce:d4:78:b3:e9:
                    84:56:6e:4a:12:21:85:d0:10:32:9b:dc:b0:57:a0:
                    f8:18:3d:3d:60:ae:bf:78:d9:96:39:a4:d3:a3:0b:
                    be:e5:8b:7f:08:d8:7e:8f:6a:fd:5f:aa:39:95:5f:
                    4d:51:c7:6d:a2:0e:e5:14:d7:24:c0:b6:36:7f:80:
                    08:5d:24:1c:19:8d:71:bb:61:38:e8:05:f0:bf:39:
                    e1:af:cf:f2:d8:e6:d6:23:d1:fb:eb:c7:9e:47:b1:
                    d0:d1:fb:97:2a:2c:55:cf:ef
                Exponent: 65537 (0x10001)
        X509v3 extensions:
            X509v3 Basic Constraints:
                CA:FALSE
    Signature Algorithm: sha1WithRSAEncryption
    Signature Value:
        38:c5:9b:90:25:34:7d:c3:35:3d:54:cf:1d:e6:eb:a7:c5:8e:
        aa:ab:1c:4b:36:e6:07:93:c8:0d:eb:bc:1b:b2:9d:f9:44:1d:
        03:bd:e4:cf:fc:c0:2c:58:57:2b:80:c7:86:75:8b:86:03:0b:
        bb:71:d2:f9:8e:9f:fd:86:5d:91:d8:f4:c0:23:d4:8f:0a:0a:
        7c:2d:26:29:3b:13:6a:f5:ff:26:34:32:b9:1a:c2:58:ce:03:
        18:ac:5e:cf:36:d6:42:7e:67:fe:75:51:dd:ad:e6:e8:46:c5:
        2e:a1:27:2a:5b:1a:ca:4c:4b:b3:ad:05:bc:3b:f2:84:e1:2b:
        b7:16:b4:25:ce:95:d3:45:ea:bb:aa:63:56:a0:5a:78:51:74:
        0a:b9:a9:23:27:b8:15:ee:89:c3:8f:d0:a6:ed:c7:9d:5a:d6:
        e8:04:dd:29:ef:41:e9:c9:9d:9c:5d:b5:85:17:bc:4b:07:64:
        2f:fa:6c:0b:53:7c:d8:7e:fd:2c:20:c5:02:6e:2f:66:4c:f9:
        51:b7:5f:62:48:29:8f:31:4c:3b:a2:fa:29:7b:b9:38:78:db:
        69:6a:c0:40:fc:bb:02:a5:f9:b3:a3:eb:11:43:cb:0e:20:5a:
        80:71:46:86:ec:6c:56:8c:13:ca:64:ca:43:f5:f8:69:ca:0d:
        b7:ab:c8:03

SHA-1 is now considered a weak hashing algorithm due to its susceptibility to collision attacks. This is likely the reason HAProxy is reporting "ca md too weak".

Running:

% openssl genrsa -out testkey.pem 1024

% openssl req -new -x509 -sha256 -key testkey.pem -out testcert.pem -days 3650

You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [AU]:
State or Province Name (full name) [Some-State]:
Locality Name (eg, city) []:
Organization Name (eg, company) [Internet Widgits Pty Ltd]:
Organizational Unit Name (eg, section) []:
Common Name (e.g. server FQDN or YOUR name) []:www.example.com
Email Address []:

% cat testcert.pem testkey.pem > testrun/certs/domain.pem

and checking again with haproxy's -c flag we see that this is now OK.

% ocp-haproxy-2.8.3 -c -f ./testrun/haproxy/
Configuration file is valid

% ocp-haproxy-2.6.13 -c -f ./testrun/haproxy/
Configuration file is valid

Looking at the signature of the new certs we now have SHA-256:

% openssl x509 -in ./testrun/certs/domain.pem -text -noout

Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
            49:7b:4b:c3:bc:f5:28:6d:18:23:72:8e:96:b6:ea:5a:f5:82:ac:79
        Signature Algorithm: sha256WithRSAEncryption
        Issuer: C = AU, ST = Some-State, O = Internet Widgits Pty Ltd, CN = www.example.com
        Validity
            Not Before: Nov 29 13:39:39 2023 GMT
            Not After : Nov 26 13:39:39 2033 GMT
        Subject: C = AU, ST = Some-State, O = Internet Widgits Pty Ltd, CN = www.example.com
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                Public-Key: (1024 bit)
                Modulus:
                    00:de:1f:f4:52:55:60:a0:60:5f:07:b6:ba:45:60:
                    23:c9:26:7d:1c:a4:4f:0e:59:f8:5e:90:38:0f:53:
                    8d:34:89:dd:3b:46:a6:b3:bf:5c:16:0b:4e:9b:13:
                    49:18:9f:00:e6:6f:7a:c1:2a:17:9f:2a:7d:78:34:
                    a7:4c:cb:b4:f6:1c:17:7a:57:1d:23:ae:09:62:13:
                    c8:83:7d:17:d2:1c:7f:27:c2:90:9d:06:33:53:53:
                    29:1e:96:f8:49:a8:e9:c0:98:7a:d2:d2:28:83:38:
                    76:9b:0d:d2:56:62:3f:f6:05:85:da:a4:0e:7d:ee:
                    be:1c:8a:42:34:f1:d2:d0:2f
                Exponent: 65537 (0x10001)
        X509v3 extensions:
            X509v3 Subject Key Identifier:
                AE:C0:7E:B1:2C:00:73:A5:A2:B6:A1:A9:EE:61:ED:36:48:4F:3F:69
            X509v3 Authority Key Identifier:
                AE:C0:7E:B1:2C:00:73:A5:A2:B6:A1:A9:EE:61:ED:36:48:4F:3F:69
            X509v3 Basic Constraints: critical
                CA:TRUE
    Signature Algorithm: sha256WithRSAEncryption
    Signature Value:
        25:ab:a7:b8:1f:84:bd:1e:5b:eb:19:fd:f1:d9:ec:05:b9:49:
        3d:80:22:90:2d:1b:6f:ec:fa:99:78:52:20:ba:36:7c:48:47:
        5e:5d:43:d1:4f:0e:57:4b:b4:aa:af:79:00:e9:2f:36:b1:a4:
        9a:e5:f4:97:34:5a:26:1e:0b:ce:2e:ad:c8:1c:5d:7c:27:33:
        04:0e:6e:bd:1c:3b:dc:4c:40:35:8a:15:f3:2b:c1:cd:a1:83:
        f7:94:7a:5c:bb:75:40:79:7e:64:39:97:5b:96:d0:19:a0:df:
        07:d7:cf:35:4d:6b:05:a7:61:c7:be:56:57:47:14:60:d7:56:
        f0:2a

@frobware
Copy link
Contributor Author

The RPM associated with this PR has been untagged. Any further test iterations on this PR will fail because the RPM is no longer available.

Slack thread: https://redhat-internal.slack.com/archives/CB95J6R4N/p1701271843429409?thread_ts=1674550176.823119&cid=CB95J6R4N.

@gcs278
Copy link
Contributor

gcs278 commented Dec 8, 2023

Created rhaos-4.16-rhel-9 branch and built haproxy-2.6.13-4.rhaos4.16.el9. We are just waiting for the RPM to propagate

/test images

@gcs278
Copy link
Contributor

gcs278 commented Dec 11, 2023

/test images

1 similar comment
@gcs278
Copy link
Contributor

gcs278 commented Dec 11, 2023

/test images

Copy link
Contributor

openshift-ci bot commented Dec 11, 2023

@frobware: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/e2e-agnostic 2dbec98 link true /test e2e-agnostic
ci/prow/images 2dbec98 link true /test images

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

WORKDIR /go/src/github.com/openshift/router
COPY . .
RUN make

FROM registry.ci.openshift.org/ocp/4.15:base
FROM registry.ci.openshift.org/ocp/4.15:base-rhel9
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Needs to be /4.16:...

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah thanks

@gcs278
Copy link
Contributor

gcs278 commented Dec 11, 2023

Created #548 to continue RHEL9 testing in 4.16 (I didn't have permission to change @frobware repo).

@gcs278
Copy link
Contributor

gcs278 commented Dec 13, 2023

/close
closing in favor of #548

@openshift-merge-robot openshift-merge-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Dec 13, 2023
@openshift-merge-robot
Copy link
Contributor

PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@openshift-ci openshift-ci bot closed this Dec 13, 2023
Copy link
Contributor

openshift-ci bot commented Dec 13, 2023

@gcs278: Closed this PR.

In response to this:

/close
closing in favor of #548

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@frobware frobware deleted the rhel9-smoke-test branch May 1, 2024 12:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants