Exception in ValidationContext.validate_usage() during OCSP response parsing #246

bbczeuz · 2022-11-17T18:00:25Z

While checking a certificate using OCSP, I get the following exception:
(Hashed lines added using print(...) just before the exception happens)

# ocsp url=http://ocsp.quovadisglobal.com
# ocsp request=<urllib.request.Request object at 0x7efe1df7ada0>
# ocsp dump=b'0|0z0Q0O0M0\t\x06\x05+\x0e\x03\x02\x1a\x05\x00\x04\x14\xf2\x85\xc2\x91\xd4\x0e\x17\x85\x02\xc5e\x1by\xbb\xe4\xfcL;\x18u\x04\x14\x1a\x84b\xbcHL3%\x04\xd4\xee\xd0\xf6\x03\xc4\x19F\xd1\x94k\x02\x14\x1an\xe8\x93\xc3t\x978\xe1*\xcc\xc7z\x8c\n\xcb\x16~\xaf\x14\xa2%0#0!\x06\t+\x06\x01\x05\x05\x070\x01\x02\x04\x14\x04\x12\x04\x10k\xaf1n\xfd\x0e=\x9f\xacwQ\xd5#\x9d\xc7x'
# urlopen status=200, reason=OK, headers=Date: Thu, 17 Nov 2022 17:50:55 GMT
Server: Apache
Content-Type: application/ocsp-response
Content-Length: 5
Connection: close


# ocsp response=<asn1crypto.ocsp.OCSPResponse 139629889564456 b'0\x03\n\x01\x06'>
# request nonce=<asn1crypto.core.OctetString 139629889697216 b'\x04\x12\x04\x10k\xaf1n\xfd\x0e=\x9f\xacwQ\xd5#\x9d\xc7x'>
UNKNOWN: Exception caught during EAPOL check: 'Void' object is not subscriptable
Traceback (most recent call last):
  File "./check_ocsp.py", line 102, in main
    file_name, verbose=verbose
  File "./check_ocsp.py", line 54, in validate_certificate_chain
    validator.validate_usage(set(usage))
  File "/home/zeuz/git/wpa_eapol_checker/venv/lib64/python3.6/site-packages/certvalidator/__init__.py", line 193, in validate_usage
    self._validate_path()
  File "/home/zeuz/git/wpa_eapol_checker/venv/lib64/python3.6/site-packages/certvalidator/__init__.py", line 121, in _validate_path
    validate_path(self._context, candidate_path)
  File "/home/zeuz/git/wpa_eapol_checker/venv/lib64/python3.6/site-packages/certvalidator/validate.py", line 50, in validate_path
    return _validate_path(validation_context, path)
  File "/home/zeuz/git/wpa_eapol_checker/venv/lib64/python3.6/site-packages/certvalidator/validate.py", line 386, in _validate_path
    end_entity_name_override=end_entity_name_override
  File "/home/zeuz/git/wpa_eapol_checker/venv/lib64/python3.6/site-packages/certvalidator/validate.py", line 891, in verify_ocsp_response
    ocsp_responses = validation_context.retrieve_ocsps(cert, issuer)
  File "/home/zeuz/git/wpa_eapol_checker/venv/lib64/python3.6/site-packages/certvalidator/context.py", line 497, in retrieve_ocsps
    **self._ocsp_fetch_params
  File "/home/zeuz/git/wpa_eapol_checker/venv/lib64/python3.6/site-packages/certvalidator/ocsp_client.py", line 114, in fetch
    response_nonce = ocsp_response.nonce_value
  File "/home/zeuz/git/wpa_eapol_checker/venv/lib64/python3.6/site-packages/asn1crypto/ocsp.py", line 671, in nonce_value
    self._set_extensions()
  File "/home/zeuz/git/wpa_eapol_checker/venv/lib64/python3.6/site-packages/asn1crypto/ocsp.py", line 631, in _set_extensions
    for extension in self['response_bytes']['response'].parsed['tbs_response_data']['response_extensions']:
TypeError: 'Void' object is not subscriptable

Minimum working implementation:

# High-level API for cert validation
from asn1crypto import pem
from certvalidator import CertificateValidator, ValidationContext, errors

# Lower-level API for cert info
from cryptography import x509


def load_certs(cert_file):
    end_entity_cert = None
    intermediates = []
    with open(cert_file, "rb") as f:
        data = f.read()
        if not pem.detect(data):
            raise RuntimeError("Invalid cert contents")
        # Usually, the cert is the first part and then the intermediates follow; Seemingly Radiator doesn't follow this convention.
        for _type_name, _headers, der_bytes in pem.unarmor(data, multiple=True):
            cert = x509.load_der_x509_certificate(der_bytes)
            key_usages = cert.extensions.get_extension_for_class(x509.KeyUsage)
            if key_usages.value.key_cert_sign:
                intermediates.append(der_bytes)
            else:
                end_entity_cert = der_bytes
    return intermediates, end_entity_cert


def validate_certificate_chain( cert_file, usage=["key_encipherment"]):
    # Checks certificate chain for: usage
    rc = 1
    msg = "Unknown validation status"
    intermediates, end_entity_cert = load_certs(cert_file)
    try:
        validation_context = ValidationContext( allow_fetching=True, revocation_mode="soft-fail")
        validator = CertificateValidator( end_entity_cert, intermediates, validation_context)
        validator.validate_usage(set(usage))
        rc = 0
        msg = "Cert valid"
    except (errors.PathValidationError) as e:
        msg = getattr(e, "message", e)
        rc = 1
    except (errors.PathBuildingError) as e:
        msg = getattr(e, "message", e)
        rc = 1
    return rc, msg

validate_certificate_chain('ocsp-fail.pem')

--> If I set allow_fetching=False when creating ValidationContext, I don't get the exception, but OCSP is not validated.

The text was updated successfully, but these errors were encountered:

bbczeuz · 2022-11-17T18:02:40Z

Github doesn't want me to attach the pem formatted cert, so I paste it here:

Intermediate
-----BEGIN CERTIFICATE-----
MIIFpDCCA4ygAwIBAgIUGm7ok8N0lzjhKszHeowKyxZ+rxQwDQYJKoZIhvcNAQELBQAwRTEL
MAkGA1UEBhMCQk0xGTAXBgNVBAoTEFF1b1ZhZGlzIExpbWl0ZWQxGzAZBgNVBAMTElF1b1Zh
ZGlzIFJvb3QgQ0EgMjAeFw0yMDA5MjIxOTE1NTlaFw0yMzA2MDExMzM1MDVaME0xCzAJBgNV
BAYTAkJNMRkwFwYDVQQKExBRdW9WYWRpcyBMaW1pdGVkMSMwIQYDVQQDExpRdW9WYWRpcyBH
bG9iYWwgU1NMIElDQSBHMjCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAOHhhWmU
wI9X+jT+wbho5JmQqYh6zle30OS1VMIYfdDDGeipY4D3t9zSGaNasGDZdrQdMlY18WyjnEKh
i4ojNZdBewVphCiOzh5Ni2Ak8bSI/sBQ9sKPrpd0+UCqbvaGs6Tpx190ZRT0Pdy+TqOYZF/j
BmzBj7YfXJmWxlfCy62UiQ6tvv+4C6W2OPu1R4HUD8oJ8Qo7Eg0cD+GFsBM2w8soffyl+Dc6
pKtARmOClUC7EqyWP0V9953lA34kuJZlYxxdgghBTn9rWoaQw/Lr5Fn0Xgd7fYS3/zGhmXYv
VsuAxIn8Gk+YaeoLZ8H9tUvnDD3lEHzvIsMPxqtd7IgcVaMCAwEAAaOCAYIwggF+MBIGA1Ud
EwEB/wQIMAYBAf8CAQAwHwYDVR0jBBgwFoAUGoRivEhMMyUE1O7Q9gPEGUbRlGswcgYIKwYB
BQUHAQEEZjBkMDYGCCsGAQUFBzAChipodHRwOi8vdHJ1c3QucXVvdmFkaXNnbG9iYWwuY29t
L3F2cmNhMi5jcnQwKgYIKwYBBQUHMAGGHmh0dHA6Ly9vY3NwLnF1b3ZhZGlzZ2xvYmFsLmNv
bTBKBgNVHSAEQzBBMD8GBFUdIAAwNzA1BggrBgEFBQcCARYpaHR0cHM6Ly93d3cucXVvdmFk
aXNnbG9iYWwuY29tL3JlcG9zaXRvcnkwHQYDVR0lBBYwFAYIKwYBBQUHAwIGCCsGAQUFBwMB
MDkGA1UdHwQyMDAwLqAsoCqGKGh0dHA6Ly9jcmwucXVvdmFkaXNnbG9iYWwuY29tL3F2cmNh
Mi5jcmwwHQYDVR0OBBYEFJEZYq1bF6cw+/DeOSWxvYy5uFEnMA4GA1UdDwEB/wQEAwIBBjAN
BgkqhkiG9w0BAQsFAAOCAgEAWNELUhzNSHcK+M1HkEDA4ty/O9VC3idOyrAEm72NKE+iLJ6c
jN3ofG2+xFDBR+yExpg+WT/fn/H1mCTQdCkrsDIFe/rqTv9PB1PSRoH/MnsWdN9uutlgfBXL
3EdrdjyJH0s8Fpbbm5JXSnsfl35NO+0Hppe6gBIZ8njcVIMg7j84f7b8iYm0YuBTW7gPvKfs
0wRVRguyH++9g+UDQI6e55aqIF6bKBqB3WhWnvx2F6hSZhhmJLhHJNtvKjbyjUqO4EI+TJCn
M1nnNffVO4PI1BuIc4yPSSSAHZ9nvKJ4rTfWB8edjON1OLSFM5MBEnZD7Gni79McDlBP/Sln
FwOfq2xhmlinaLXe0QLipUkq5EH4QnUdz6ShtQfSd8QauTtLZdUNRwsuu6z5sQGmJdSjTzF5
Wn1Y4/XpCwf63gWQDqj7kXC0VI46TjcrdzQXp3IscYAmBF7mALa0wLuBKy8ZB4wMlRMAY7j8
KSsyg1Sz1rWbq+eap/IATpQoiymHKgwvP8ERSJX6XWFwWvAQjIv+aZu4yh+h+h1OjVY/VXH/
M/lvwf7crVy4n0G+dGROcHQO8sN+MF8/JLXZDQGR7spYYG19Mw849kcbYcaWEd57LH1jGhuY
+IdemBUALfw6V8VIvJfLWGBG+35DjkBiin9kKcTT9ySYu9PzGK/VnF31OSI=
-----END CERTIFICATE-----

Final
-----BEGIN CERTIFICATE-----
MIIG7jCCBdagAwIBAgIUI5fwP+KV2AcFYXhrkzNzREyyboMwDQYJKoZIhvcNAQELBQAwTTEL
MAkGA1UEBhMCQk0xGTAXBgNVBAoTEFF1b1ZhZGlzIExpbWl0ZWQxIzAhBgNVBAMTGlF1b1Zh
ZGlzIEdsb2JhbCBTU0wgSUNBIEcyMB4XDTIyMDEwNDA5MjcxN1oXDTIzMDEwNDA5MzcwMFow
ZjELMAkGA1UEBhMCQ0gxDDAKBgNVBAgMA1p1ZzEMMAoGA1UEBwwDWnVnMRowGAYDVQQKDBFL
YW50b25zc2NodWxlIFp1ZzEfMB0GA1UEAwwWZ2liei5lZHVyb2FtLmVkdS16Zy5jaDCCASIw
DQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBALOmIazzjasur9pHw5/fSu3JqQMB1rOW8/pk
GoI7NRZXjI+IPo0hFuc8dsXlMdFk4R1hBS8ND7oUjGeKEgIKAIQG8Sd6/NtJ/4an/NuRkeB+
rEaIb4guLqbjcALCwZzHfLVN44cEVvP2DQkCHXmAtlmxc1Z2SDGo0mGfERDZ6tIBZn9SIvWS
0C8T7qRyas5ZYHCUszdUaBsckgLNF6Gmxn9b1UJMCYqnmWyTH5wJzlgy+gzge4UkeMtv03Xr
gzFDgRmhfYFibFb9lc08bavYJj210zg0xF40FmnSueXKFtezUcEvlETrfVqD6fbJ6cXhuFwq
WobVE/5tW3uVDsfg+RMCAwEAAaOCA6swggOnMAkGA1UdEwQCMAAwHwYDVR0jBBgwFoAUkRli
rVsXpzD78N45JbG9jLm4UScwcwYIKwYBBQUHAQEEZzBlMDcGCCsGAQUFBzAChitodHRwOi8v
dHJ1c3QucXVvdmFkaXNnbG9iYWwuY29tL3F2c3NsZzIuY3J0MCoGCCsGAQUFBzABhh5odHRw
Oi8vb2NzcC5xdW92YWRpc2dsb2JhbC5jb20wIQYDVR0RBBowGIIWZ2liei5lZHVyb2FtLmVk
dS16Zy5jaDBbBgNVHSAEVDBSMEYGDCsGAQQBvlgAAmQBATA2MDQGCCsGAQUFBwIBFihodHRw
Oi8vd3d3LnF1b3ZhZGlzZ2xvYmFsLmNvbS9yZXBvc2l0b3J5MAgGBmeBDAECAjAdBgNVHSUE
FjAUBggrBgEFBQcDAgYIKwYBBQUHAwEwOgYDVR0fBDMwMTAvoC2gK4YpaHR0cDovL2NybC5x
dW92YWRpc2dsb2JhbC5jb20vcXZzc2xnMi5jcmwwHQYDVR0OBBYEFLpXNemCwmZ8UIwCgo1O
iFvreAPfMA4GA1UdDwEB/wQEAwIFoDCCAfgGCisGAQQB1nkCBAIEggHoBIIB5AHiAHcAejKM
VNi3LbYg6jjgUh7phBZwMhOFTTvSK8E6V6NS61IAAAF+JHM3jQAABAMASDBGAiEA4qNU0TPr
JLt0lFSdI2fpYVWE3LZLi4PVPaDx/hHKCokCIQDxh4NOHz9oubw97DySqR5SvERe9nsDFAhv
Ltpw43tT2wB2AFYUBpov18Ls0/XhvUSyPsdGdrm8mRFcwO+UmFXWidDdAAABfiRzN8MAAAQD
AEcwRQIgKM6VJ9OUgZWnFFtcxVm+GA/xqQKYioR/0Adm6CSdEj4CIQDa6GiVxjWucYGevGGf
NJQm4f7wwk5Q5IS75AzjjWe5qwB3AK33vvp8/xDIi509nB4+GGq0Zyldz7EMJMqFhjTr3IKK
AAABfiRzN4QAAAQDAEgwRgIhAJI1F6BgwoicNlZGl74gG1hLiEwwVOITEMOqzWGHAbnfAiEA
19KJFY5Rx5Y6WifiZcRr67MQcTBg+g/jWqVClTPDDMUAdgBVgdTCFpA2AUrqC5tXPFPwwOQ4
eHAlCBcvo6odBxPTDAAAAX4kczeQAAAEAwBHMEUCIQCVLYhH5Da0ssDqAhyv6t40Jn4nfxmR
zqiy7OCoTC3+mQIgNLvTC5Yava2jl3GOccTWz6KPHsloVI2/eSXcd6GjgEMwDQYJKoZIhvcN
AQELBQADggEBAHT5wTLJojfavUwMgZH8fqtI78Ycty4AECynVad8LUL9AO423GADxFHIZiCB
wYmhXfaPxCAfgEo4txsJmOxHmnS8GRIIyyY+FequXYKXv0poEngsPY9gS8Cjf+1Lvdg50Qzy
rRY0F7ISXn9/RVHGxcie6M+/Jc7uYgaZJsLoQ9xXHjfz5fgNr2vJqnLgUuyDNIzF/LsCkebC
LM5OpwvV2Wn2+LxDbDwIjbkdH/p1BnjKS+X7xJMAnF2RhdRDbzKxMj9bPP6H+9nxtkPaLRqd
AQIjUU+UMG47hBdCoTlEJZy9LABoSmgr7p+6/5JhJmSef9kknqzzEzhuhm/5MT7G98Q=
-----END CERTIFICATE-----

bbczeuz · 2022-11-17T18:03:22Z

I'm not really good at debugging the ocsp.py code, any pointers are welcome.

MatthiasValvekens · 2022-11-17T19:33:05Z

I tried to reproduce the bug you're seeing, but on my end everything seems to work fine. I also manually fetched an OCSP response from that QuoVadis responder and was able to decode it (including the nonce extension) just fine using asn1crypto 1.5.1.

That said, that stack trace has all the hallmarks of "garbage in, garbage out". Parsing in asn1crypto is lazy, and that nonce_value read just happens to be the very first piece of information that certvalidator tries to access after obtaining the response... That leads me to suspect that certvalidator is passing on garbage data (perhaps a server error response?), and asn1crypto isn't actually doing anything wrong.

Also, I couldn't help but notice that your line numbers in ocsp_client.py look off... The stack trace mentions line 114, but that file has only 109 lines, and hasn't changed since 2017... Which version of certvalidator are you running? Maybe this is a very old bug that has been fixed already.

bbczeuz · 2022-11-25T13:21:26Z

Hi Matthias,
Thanks for your comments and sorry for my late response. My line numbers in ocsp_client.py are a little bit off because I added some debugging code; the code itself is from tag 1.5.1. I was using the most recent tagged release of certvalidator (0.11.1 from 2016); Unfortunately using the git version certvalidator didn't help either.
The packet capture shows a regular request, but Quovadis returning OCSP responseStatus: unauthorized (6). According to https://www.rfc-editor.org/rfc/rfc6960.txt, section 4.2.1, if responseStatus is not successful (0), the field ResponseBytes may not exist. I now catch this error shown as below:
(ocsp.py#L622)

    def _set_extensions(self):
        """
        Sets common named extensions to private attributes and creates a list
        of critical extensions
        """

        self._critical_extensions = set()

        if self['response_status'] != 0:
            raise NotImplementedError(unwrap(
                '''
                OCSP response error status handling. Status = %s
                ''',
                repr(self['response_status'])
                ))

        for extension in self['response_bytes']['response'].parsed['tbs_response_data']['response_extensions']:
            name = extension['extn_id'].native
            attribute_name = '_%s_value' % name
            if hasattr(self, attribute_name):
                setattr(self, attribute_name, extension['extn_value'].parsed)
            if extension['critical'].native:
                self._critical_extensions.add(name)
        self._processed_extensions = True

bbczeuz · 2022-11-25T19:35:47Z

Issue seems not to be specific to asn1crypto nor certvalidator:

# ocsp x509 -noout -text -in ocsp-fail.pem
[...]
            Authority Information Access: 
                CA Issuers - URI:http://trust.quovadisglobal.com/qvsslg2.crt
                OCSP - URI:http://ocsp.quovadisglobal.com
[...]

# openssl ocsp -issuer ocsp-fail-interm.pem -cert ocsp-fail.pem -text -url http://ocsp.quovadisglobal.com
OCSP Request Data:
    Version: 1 (0x0)
    Requestor List:
        Certificate ID:
          Hash Algorithm: sha1
          Issuer Name Hash: F285C291D40E178502C5651B79BBE4FC4C3B1875
          Issuer Key Hash: 911962AD5B17A730FBF0DE3925B1BD8CB9B85127
          Serial Number: 1A6EE893C3749738E12ACCC77A8C0ACB167EAF14
    Request Extensions:
        OCSP Nonce: 
            0410B34D25D827DEDD0A261187293332A1C8
Responder Error: unauthorized (6)

I think the PR helps debugging similar cases, but it's not a final solution yet. Please feel free to comment.

bbczeuz mentioned this issue Nov 25, 2022

Raise exception on OCSP status fail #247

Closed

wbond mentioned this issue Aug 23, 2023

OCSP response extension parsing fails #262

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Exception in ValidationContext.validate_usage() during OCSP response parsing #246

Exception in ValidationContext.validate_usage() during OCSP response parsing #246

bbczeuz commented Nov 17, 2022

bbczeuz commented Nov 17, 2022

bbczeuz commented Nov 17, 2022

MatthiasValvekens commented Nov 17, 2022

bbczeuz commented Nov 25, 2022

bbczeuz commented Nov 25, 2022

Exception in ValidationContext.validate_usage() during OCSP response parsing #246

Exception in ValidationContext.validate_usage() during OCSP response parsing #246

Comments

bbczeuz commented Nov 17, 2022

bbczeuz commented Nov 17, 2022

bbczeuz commented Nov 17, 2022

MatthiasValvekens commented Nov 17, 2022

bbczeuz commented Nov 25, 2022

bbczeuz commented Nov 25, 2022