Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

SNOW-886899: Cannot create Sessions if env uses OpenSSL 3.0.10 or later #992

Closed
sfc-gh-jfreeberg opened this issue Aug 4, 2023 · 6 comments
Closed

SNOW-886899: Cannot create Sessions if env uses OpenSSL 3.0.10 or later #992

sfc-gh-jfreeberg opened this issue Aug 4, 2023 · 6 comments
Labels
bug Something isn't working

Comments

@sfc-gh-jfreeberg
Copy link
Collaborator

sfc-gh-jfreeberg commented Aug 4, 2023 •
edited
Loading

Please answer these questions before submitting your issue. Thanks!

  1. What version of Python are you using?

3.8.17

  1. What operating system and processor architecture are you using?

    On macOS-10.16-x86_64-i386-64bit I get a warning:

     ../../../anaconda3/envs/snowpark/lib/python3.8/site-packages/snowflake/snowpark/session.py:22
   /Users/jfreeberg/anaconda3/envs/snowpark/lib/python3.8/site-packages/snowflake/snowpark/session.py:22: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
     import pkg_resources
 
 ../../../anaconda3/envs/snowpark/lib/python3.8/site-packages/pkg_resources/__init__.py:2871
   /Users/jfreeberg/anaconda3/envs/snowpark/lib/python3.8/site-packages/pkg_resources/__init__.py:2871: DeprecationWarning: Deprecated call to `pkg_resources.declare_namespace('snowflake')`.
   Implementing implicit namespace packages (as specified in PEP 420) is preferred to `pkg_resources.declare_namespace`. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
     declare_namespace(pkg)

    On Github Actions Linux runners I get an error:

    platform linux -- Python 3.8.17, pytest-7.4.0, pluggy-1.2.0
rootdir: /home/runner/work/snowpark-python-template/snowpark-python-template
configfile: pytest.ini
plugins: snowflake-vcrpy-0.1.0
collected 2 items

test/test_app.py E                                                       [ 50%]
test/test_functions.py .                                                 [100%]

==================================== ERRORS ====================================
________________________ ERROR at setup of test_app_dim ________________________

scope = 'module'

    @pytest.fixture
    def session(scope='module'):
        # pylint: disable=unused-argument
        """
        Creates a Session object for tests
        """
    
>       return Session.builder.configs(get_env_var_config()).create()

test/conftest.py:16: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/usr/share/miniconda/envs/snowpark/lib/python3.8/site-packages/snowflake/snowpark/session.py:3[15](https://github.com/Snowflake-Labs/snowpark-python-template/actions/runs/5766183664/job/15633628936#step:4:16): in create
    session = self._create_internal(self._options.get("connection"))
/usr/share/miniconda/envs/snowpark/lib/python3.8/site-packages/snowflake/snowpark/session.py:335: in _create_internal
    ServerConnection({}, conn) if conn else ServerConnection(self._options),
/usr/share/miniconda/envs/snowpark/lib/python3.8/site-packages/snowflake/snowpark/_internal/server_connection.py:136: in __init__
    self._conn = conn if conn else connect(**self._lower_case_parameters)
/usr/share/miniconda/envs/snowpark/lib/python3.8/site-packages/snowflake/connector/__init__.py:51: in Connect
    return SnowflakeConnection(**kwargs)
/usr/share/miniconda/envs/snowpark/lib/python3.8/site-packages/snowflake/connector/connection.py:351: in __init__
    self.connect(**kwargs)
/usr/share/miniconda/envs/snowpark/lib/python3.8/site-packages/snowflake/connector/connection.py:623: in connect
    self.__open_connection()
/usr/share/miniconda/envs/snowpark/lib/python3.8/site-packages/snowflake/connector/connection.py:896: in __open_connection
    self.authenticate_with_retry(self.auth_class)
/usr/share/miniconda/envs/snowpark/lib/python3.8/site-packages/snowflake/connector/connection.py:1[16](https://github.com/Snowflake-Labs/snowpark-python-template/actions/runs/5766183664/job/15633628936#step:4:17)5: in authenticate_with_retry
    self._authenticate(auth_instance)
/usr/share/miniconda/envs/snowpark/lib/python3.8/site-packages/snowflake/connector/connection.py:1191: in _authenticate
    auth.authenticate(
/usr/share/miniconda/envs/snowpark/lib/python3.8/site-packages/snowflake/connector/auth/_auth.py:250: in authenticate
    ret = self._rest._post_request(
/usr/share/miniconda/envs/snowpark/lib/python3.8/site-packages/snowflake/connector/network.py:723: in _post_request
    ret = self.fetch(
/usr/share/miniconda/envs/snowpark/lib/python3.8/site-packages/snowflake/connector/network.py:842: in fetch
    ret = self._request_exec_wrapper(
/usr/share/miniconda/envs/snowpark/lib/python3.8/site-packages/snowflake/connector/network.py:980: in _request_exec_wrapper
    raise e
/usr/share/miniconda/envs/snowpark/lib/python3.8/site-packages/snowflake/connector/network.py:884: in _request_exec_wrapper
    return_object = self._request_exec(
/usr/share/miniconda/envs/snowpark/lib/python3.8/site-packages/snowflake/connector/network.py:1[18](https://github.com/Snowflake-Labs/snowpark-python-template/actions/runs/5766183664/job/15633628936#step:4:19)2: in _request_exec
    raise err
/usr/share/miniconda/envs/snowpark/lib/python3.8/site-packages/snowflake/connector/network.py:1077: in _request_exec
    raw_ret = session.request(
/usr/share/miniconda/envs/snowpark/lib/python3.8/site-packages/snowflake/connector/vendored/requests/sessions.py:587: in request
    resp = self.send(prep, **send_kwargs)
/usr/share/miniconda/envs/snowpark/lib/python3.8/site-packages/snowflake/connector/vendored/requests/sessions.py:701: in send
    r = adapter.send(request, **kwargs)
/usr/share/miniconda/envs/snowpark/lib/python3.8/site-packages/snowflake/connector/vendored/requests/adapters.py:487: in send
    resp = conn.urlopen(
/usr/share/miniconda/envs/snowpark/lib/python3.8/site-packages/snowflake/connector/vendored/urllib3/connectionpool.py:703: in urlopen
    httplib_response = self._make_request(
/usr/share/miniconda/envs/snowpark/lib/python3.8/site-packages/snowflake/connector/vendored/urllib3/connectionpool.py:386: in _make_request
    self._validate_conn(conn)
/usr/share/miniconda/envs/snowpark/lib/python3.8/site-packages/snowflake/connector/vendored/urllib3/connectionpool.py:1042: in _validate_conn
    conn.connect()
/usr/share/miniconda/envs/snowpark/lib/python3.8/site-packages/snowflake/connector/vendored/urllib3/connection.py:4[19](https://github.com/Snowflake-Labs/snowpark-python-template/actions/runs/5766183664/job/15633628936#step:4:20): in connect
    self.sock = ssl_wrap_socket(
/usr/share/miniconda/envs/snowpark/lib/python3.8/site-packages/snowflake/connector/ssl_wrap_socket.py:85: in ssl_wrap_socket_with_ocsp
    from .ocsp_asn1crypto import SnowflakeOCSPAsn1Crypto as SFOCSP
/usr/share/miniconda/envs/snowpark/lib/python3.8/site-packages/snowflake/connector/ocsp_asn1crypto.py:62: in <module>
    from oscrypto import asymmetric
/usr/share/miniconda/envs/snowpark/lib/python3.8/site-packages/oscrypto/asymmetric.py:19: in <module>
    from ._asymmetric import _unwrap_private_key_info
/usr/share/miniconda/envs/snowpark/lib/python3.8/site-packages/oscrypto/_asymmetric.py:27: in <module>
    from .kdf import pbkdf1, pbkdf2, pkcs12_kdf
/usr/share/miniconda/envs/snowpark/lib/python3.8/site-packages/oscrypto/kdf.py:9: in <module>
    from .util import rand_bytes
/usr/share/miniconda/envs/snowpark/lib/python3.8/site-packages/oscrypto/util.py:14: in <module>
    from ._openssl.util import rand_bytes
/usr/share/miniconda/envs/snowpark/lib/python3.8/site-packages/oscrypto/_openssl/util.py:6: in <module>
    from ._libcrypto import libcrypto, libcrypto_version_info, handle_openssl_error
/usr/share/miniconda/envs/snowpark/lib/python3.8/site-packages/oscrypto/_openssl/_libcrypto.py:9: in <module>
    from ._libcrypto_cffi import (
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

    from __future__ import unicode_literals, division, absolute_import, print_function
    
    import re
    
    from .. import _backend_config
    from .._errors import pretty_message
    from .._ffi import get_library, register_ffi
    from ..errors import LibraryNotFoundError
    
    from cffi import FFI
    
    
    __all__ = [
        'is_libressl',
        'libcrypto',
        'libressl_version',
        'libressl_version_info',
        'version',
        'version_info',
    ]
    
    libcrypto_path = _backend_config().get('libcrypto_path')
    if libcrypto_path is None:
        libcrypto_path = get_library('crypto', 'libcrypto.dylib', '42')
    if not libcrypto_path:
        raise LibraryNotFoundError('The library libcrypto could not be found')
    
    try:
        vffi = FFI()
        vffi.cdef("const char *SSLeay_version(int type);")
        version_string = vffi.string(vffi.dlopen(libcrypto_path).SSLeay_version(0)).decode('utf-8')
    except (AttributeError):
        vffi = FFI()
        vffi.cdef("const char *OpenSSL_version(int type);")
        version_string = vffi.string(vffi.dlopen(libcrypto_path).OpenSSL_version(0)).decode('utf-8')
    
    is_libressl = 'LibreSSL' in version_string
    
    version_match = re.search('\\b(\\d\\.\\d\\.\\d[a-z]*)\\b', version_string)
    if not version_match:
        version_match = re.search('(?<=LibreSSL )(\\d\\.\\d(\\.\\d)?)\\b', version_string)
    if not version_match:
>       raise LibraryNotFoundError('Error detecting the version of libcrypto')
E       oscrypto.errors.LibraryNotFoundError: Error detecting the version of libcrypto

/usr/share/miniconda/envs/snowpark/lib/python3.8/site-packages/oscrypto/_openssl/_libcrypto_cffi.py:44: LibraryNotFoundError
=============================== warnings summary ===============================
../../../../../usr/share/miniconda/envs/snowpark/lib/python3.8/site-packages/snowflake/snowpark/session.py:22
  /usr/share/miniconda/envs/snowpark/lib/python3.8/site-packages/snowflake/snowpark/session.py:22: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
    import pkg_resources

../../../../../usr/share/miniconda/envs/snowpark/lib/python3.8/site-packages/pkg_resources/__init__.py:2871
  /usr/share/miniconda/envs/snowpark/lib/python3.8/site-packages/pkg_resources/__init__.py:2871: DeprecationWarning: Deprecated call to `pkg_resources.declare_namespace('snowflake')`.
  Implementing implicit namespace packages (as specified in PEP 4[20](https://github.com/Snowflake-Labs/snowpark-python-template/actions/runs/5766183664/job/15633628936#step:4:21)) is preferred to `pkg_resources.declare_namespace`. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
    declare_namespace(pkg)

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=========================== short test summary info ============================
ERROR test/test_app.py::test_app_dim - oscrypto.errors.LibraryNotFoundError: Error detecting the version of libcrypto

  2. What are the component versions in the environment (pip freeze)?

    asn1crypto==1.5.1
certifi==2023.7.22
cffi==1.15.1
charset-normalizer==3.2.0
cloudpickle==2.0.0
cryptography==41.0.3
-e git+https://github.com/Snowflake-Labs/snowpark-python-template.git@64864264d8faada2467f10f45c02c876e13950ea#egg=Example_Snowpark_Python_project
exceptiongroup==1.1.2
filelock==3.12.2
idna==3.4
iniconfig==2.0.0
multidict==6.0.4
numpy==1.24.4
oscrypto==1.3.0
packaging==23.1
pandas==2.0.3
platformdirs==3.8.1
pluggy==1.2.0
pyarrow==10.0.1
pycparser==2.21
pycryptodomex==3.18.0
PyJWT==2.8.0
pyOpenSSL==23.2.0
pytest==7.4.0
python-dateutil==2.8.2
pytz==2023.3
PyYAML==6.0.1
requests==2.31.0
six==1.16.0
snowflake-connector-python==3.1.0
snowflake-snowpark-python==1.6.1
snowflake-vcrpy @ git+https://github.com/Snowflake-Labs/snowflake-vcrpy.git@c9f7380c3321b55a0bfe26a8e01f35ebd05b77eb
sortedcontainers==2.4.0
toml==0.10.2
tomli==2.0.1
tomlkit==0.12.1
typing_extensions==4.7.1
tzdata==2023.3
urllib3==1.26.16
wrapt==1.15.0
yarl==1.9.2

  3. What did you do?

    Source code is in this PR: v1.2.0 update Snowflake-Labs/snowpark-python-template#7

    I tried to run PyTest. This suite worked with version 1.4.0 of Snowpark Python

  4. What did you expect to see?

    What should have happened and what happened instead?

  5. Can you set logging to DEBUG and collect the logs?

    Logs are here: https://github.com/Snowflake-Labs/snowpark-python-template/actions/runs/5766183664/job/15633628936
@sfc-gh-jfreeberg sfc-gh-jfreeberg added bug Something isn't working needs triage Initial RCA is required labels Aug 4, 2023
@github-actions github-actions bot changed the title Creating Session fails with oscrypto.errors.LibraryNotFoundError: Error detecting the version of libcrypto SNOW-886899: Creating Session fails with oscrypto.errors.LibraryNotFoundError: Error detecting the version of libcrypto Aug 4, 2023
@sfc-gh-jfreeberg sfc-gh-jfreeberg mentioned this issue Aug 4, 2023
Merged
3 tasks
@pinnaclepwr2
Copy link

Hi, I am using Snowpark-Python for my application. Now I am not able to create the session. Would you happen to have any updates on this issue? Any quick fix from our end?

@sfc-gh-jfreeberg
Copy link
Collaborator Author

@pinnaclepwr2 -- The core issue is that OpenSSL upgraded from 3.0.9 to 3.0.10, and the oscrypto module does not correctly parse the double-digit patch number. More info: wbond/oscrypto#75

A plausible workaround would be to explicitly set the OpenSSL version to 3.0.9 on your machine.

@sfc-gh-jfreeberg
Copy link
Collaborator Author

@pinnaclepwr2 -- If you're using conda, you can set the OpenSSL version like this: https://github.com/Snowflake-Labs/snowpark-python-template/blob/main/environment.yml#L9

@sfc-gh-jfreeberg sfc-gh-jfreeberg changed the title SNOW-886899: Creating Session fails with oscrypto.errors.LibraryNotFoundError: Error detecting the version of libcrypto SNOW-886899: Cannot create Sessions if env uses OpenSSL 3.0.10 or later Aug 8, 2023
@BibMartin BibMartin mentioned this issue Aug 9, 2023
Merged
@sfc-gh-stan sfc-gh-stan removed the needs triage Initial RCA is required label Oct 2, 2023
@sfc-gh-stan
Copy link
Collaborator

The fix PR in oscrypto has been merged, closing this.

@aspexdaniel aspexdaniel mentioned this issue Oct 16, 2023
Open
@rd-andreas-lay
Copy link

FYI: As of 2023/10/16 the new oscrypto version is not yet released on PyPI. On a side note I don't think you should rely on third-party dependencies that much, especially when they're so small.

@Isaac-Flath
Copy link

I am disappointed to see snowflake relying on an small OS project and OS developer so heavily without sponsoring that person or project. I wish snowflake supported the open source projects they build on top of.

@ykeremy ykeremy mentioned this issue Oct 16, 2023
Merged
4 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@Isaac-Flath @rd-andreas-lay @sfc-gh-stan @pinnaclepwr2 @sfc-gh-jfreeberg