Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

oscrypto segfaults on catalina #35

Closed
worldwise001 opened this issue Nov 21, 2019 · 15 comments
Closed

oscrypto segfaults on catalina #35

worldwise001 opened this issue Nov 21, 2019 · 15 comments

Comments

@worldwise001
Copy link
Contributor

@worldwise001 worldwise001 commented Nov 21, 2019

I've come here after an all-day debugging session. tl;dr I think oscrypto is picking up the wrong SSL backend for catalina (OS X 10.15.1) which causes a crash.

There's some workarounds as documented here: https://forums.developer.apple.com/thread/119429
Though that seems pretty gnarly.

Specifically I'm finding it failing in _libcrypto_cffi:

libcrypto_path = _backend_config().get('libcrypto_path')
if libcrypto_path is None:
    libcrypto_path = find_library('crypto')
if not libcrypto_path:
    raise LibraryNotFoundError('The library libcrypto could not be found')

try:
    vffi = FFI()
    vffi.cdef("const char *SSLeay_version(int type);")
    version_string = vffi.string(vffi.dlopen(libcrypto_path).SSLeay_version(0)).decode('utf-8') #### fails here
except (AttributeError):
    vffi = FFI()
    vffi.cdef("const char *OpenSSL_version(int type);")
    version_string = vffi.string(vffi.dlopen(libcrypto_path).OpenSSL_version(0)).decode('utf-8')

I suspect we are just picking a version that makes Catalina extremely upset, and so if we can pick a pinned version that would be better.

I posted something similar here: snowflakedb/snowflake-connector-python#235

@wbond

This comment has been minimized.

Copy link
Owner

@wbond wbond commented Nov 21, 2019

Can you provide a way to reproduce this segfault? Running the test suite doesn't seem to cause theses issues, so maybe we need to add something.

@worldwise001

This comment has been minimized.

Copy link
Contributor Author

@worldwise001 worldwise001 commented Nov 21, 2019

So from oscrypto import asymmetric was triggering it in the snowflake library. The code snippet from snowflake-connector-python that's relevant was in oscp_asn1crypto in verify_signature:

    def verify_signature(self, signature_algorithm, signature, cert, data):
        pubkey = asymmetric.load_public_key(cert.public_key).unwrap().dump() ### failing line
        rsakey = RSA.importKey(pubkey)
        signer = PKCS1_v1_5.new(rsakey)
        if signature_algorithm in SnowflakeOCSPAsn1Crypto.SIGNATURE_ALGORITHM_TO_DIGEST_CLASS:
            digest = \
                SnowflakeOCSPAsn1Crypto.SIGNATURE_ALGORITHM_TO_DIGEST_CLASS[
                    signature_algorithm].new()
@worldwise001

This comment has been minimized.

Copy link
Contributor Author

@worldwise001 worldwise001 commented Nov 21, 2019

I created a PR with a workaround for them too: snowflakedb/snowflake-connector-python#241

@worldwise001

This comment has been minimized.

Copy link
Contributor Author

@worldwise001 worldwise001 commented Nov 21, 2019

I have been doing some more digging (to see if I can make a fix PR) and found the following w.r.t. libcrypto libraries on catalina:

Wed 20-11 (shh@shh-at-square) : /usr/lib ls -la libcrypto*
-rwxr-xr-x  1 root  wheel  1095424 Oct 23 17:40 libcrypto.0.9.7.dylib
-rwxr-xr-x  1 root  wheel  1390720 Oct 23 17:40 libcrypto.0.9.8.dylib
-rwxr-xr-x  1 root  wheel  1488080 Oct 23 17:40 libcrypto.35.dylib
-rwxr-xr-x  1 root  wheel  1500560 Oct 23 17:40 libcrypto.41.dylib
-rwxr-xr-x  1 root  wheel  1500592 Oct 23 17:40 libcrypto.42.dylib
-rwxr-xr-x  1 root  wheel  1519664 Oct 23 17:40 libcrypto.44.dylib
-rwxr-xr-x  1 root  wheel    32992 Oct 23 17:40 libcrypto.dylib

Based on the filesize, I suspect libcrypto.dylib unversioned is super small and its only purpose is to raise SIGABRT with a message telling people to use versioned libraries instead.

@worldwise001

This comment has been minimized.

Copy link
Contributor Author

@worldwise001 worldwise001 commented Nov 21, 2019

I also just filed https://bugs.python.org/issue38873 with python ctypes. I'm not sure if it's something they should fix, but I wonder if they can do us a favor and provide a means to specify version or something.

@wbond

This comment has been minimized.

Copy link
Owner

@wbond wbond commented Nov 21, 2019

I wonder what is different about my machine that I don't get the abort.

Either way, we can just update it to use the versioned libcrypto as far as I am concerned. You don't happen to have a 10.14 install around do you? I can ensure we use a versioned lib on 10.15, but it wouldn't be terrible to include older releases if they have stable versions also.

worldwise001 added a commit to worldwise001/oscrypto that referenced this issue Nov 21, 2019
@worldwise001

This comment has been minimized.

Copy link
Contributor Author

@worldwise001 worldwise001 commented Nov 21, 2019

I asked a coworker who is still on 10.14 to list his /usr/lib/libcrypto:

/usr/lib/libcrypto.0.9.7.dylib
/usr/lib/libcrypto.0.9.8.dylib
/usr/lib/libcrypto.35.dylib
/usr/lib/libcrypto.41.dylib
/usr/lib/libcrypto.42.dylib
/usr/lib/libcrypto.dylib

So 44 appears to only be on catalina at least/

@wbond

This comment has been minimized.

Copy link
Owner

@wbond wbond commented Nov 21, 2019

Based on https://bugs.python.org/issue38814, I tend to doubt python core will deal with the issue, which I don't mind.

@worldwise001

This comment has been minimized.

Copy link
Contributor Author

@worldwise001 worldwise001 commented Nov 21, 2019

Opened this for you: #36

@wbond

This comment has been minimized.

Copy link
Owner

@wbond wbond commented Nov 21, 2019

The only other useful thing would be to know what version of libressl the various versioned libcryptos are. Not that I expect any of them to break the APIs being used, since they are fairly pedestrian.

@worldwise001

This comment has been minimized.

Copy link
Contributor Author

@worldwise001 worldwise001 commented Nov 21, 2019

Wed 20-11 (shh@shh-at-square) : /usr/lib for i in `ls libcrypto.*.dylib`; do echo -n $i:; strings $i | grep libressl | head -n1 | cut -d'/' -f9; echo; done
libcrypto.0.9.7.dylib:
libcrypto.0.9.8.dylib:
libcrypto.35.dylib:libressl-2.2

libcrypto.41.dylib:libressl-2.5

libcrypto.42.dylib:libressl-2.6

libcrypto.44.dylib:libressl-2.8
@wbond

This comment has been minimized.

Copy link
Owner

@wbond wbond commented Nov 21, 2019

Awesome - thanks for digging that up!

worldwise001 added a commit to worldwise001/oscrypto that referenced this issue Nov 21, 2019
worldwise001 added a commit to worldwise001/oscrypto that referenced this issue Nov 21, 2019
@wbond wbond closed this in 9e05537 Nov 21, 2019
wbond added a commit that referenced this issue Nov 21, 2019
Fixes #35: Pin libcrypto and libssl dylibs only on Catalina
@worldwise001

This comment has been minimized.

Copy link
Contributor Author

@worldwise001 worldwise001 commented Nov 21, 2019

Any chance we can cut a release with this fix? It’s blocking my development right now. Keep me posted and thank you for your extremely prompt response!

@wbond

This comment has been minimized.

Copy link
Owner

@wbond wbond commented Nov 21, 2019

I just need the CI on master to pass. I updated the CI config to pull in some revised matrix assignments across different CI providers that I made for asn1crypto. The major benefit being running the tests on 10.15 via GitHub Actions, however something is broken in regards to Circle and the macOS tests there. I'll need to make sure those look good before the next release.

@wbond

This comment has been minimized.

Copy link
Owner

@wbond wbond commented Jan 4, 2020

This fix was included with the 1.1.1 release

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants
You can’t perform that action at this time.