botocore.exceptions.SSLError: SSL validation failed on aws3 endpoint #2630

snirbenyosef · 2022-03-07T10:51:27Z

Describe the bug
i'm trying to download from s3 public bucket and got ssl error, while 2 weeks ago everything works and i didnt change anything on my workspace. i'm not sure why?

Steps to reproduce

    import boto3
    import botocore
    import urllib3

    from botocore.handlers import disable_signing

    s3_resource = boto3.resource(
                "s3",
                endpoint_url="https://s3.amazonaws.com",
                verify=False,
                region_name=None,
                aws_access_key_id=None,
                aws_secret_access_key=None,
            )
    s3_resource.meta.client.meta.events.register('choose-signer.s3.*', disable_signing)
    
    s3_resource.Bucket("mybucket").download_file(
                    "logo_detection_data/images/1.jpg", "/logo_detection_data/images/1.jpg", ExtraArgs=None)

Expected behavior
download from s3 amazon public bucket.

Debug logs
botocore.exceptions.SSLError: SSL validation failed for https://s3.amazonaws.com/libhub-readme/logo_detection_data/images/1.jpg [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:852)

The text was updated successfully, but these errors were encountered:

kdaily · 2022-03-07T17:31:18Z

Hi @snirbenyosef,

Sorry to hear you're having an issue. Can you please post debug logs using boto3.set_stream_logger(''), redacting any sensitive information?

Does the network you're on use a certificate proxy of any kind?

Also, it seems that you're disabling signing to download from the public bucket. The preferred way to do this is by configuring the session to use the UNSIGNED signature version:

import boto3
import botocore
from botocore import UNSIGNED
from botocore.config import Config

s3_resource = boto3.resource("s3", config=Config(signature_version=UNSIGNED))

snirbenyosef · 2022-03-08T07:23:04Z

@kdaily

Does the network you're on use a certificate proxy of any kind? i'm not sure but i didnt change anything in my workspace last 2 weeks.

2022-03-08 07:20:47,012 root [DEBUG] CMD line args: {'debug': False, 'endpoint': 'https://s3.amazonaws.com', 'region_name': None, 'command': 'download', 'bucket': 'mybucket', 'bucketname': True, 'localdir': '/cnvrg', 'overwrite': False, 'versionid': None, 'filename': None, 'prefix': 'logo_detection_data/', 'cnvrg_dataset': None, 'func': <function cmd_download at 0x7f68a9acbbf8>}
https://s3.amazonaws.com
None
None
None
2022-03-08 07:20:47,069 root [DEBUG] Checking if bucket exist: mybucket
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/urllib3/connectionpool.py", line 677, in urlopen
    chunked=chunked,
  File "/usr/local/lib/python3.6/dist-packages/urllib3/connectionpool.py", line 381, in _make_request
    self._validate_conn(conn)
  File "/usr/local/lib/python3.6/dist-packages/urllib3/connectionpool.py", line 978, in _validate_conn
    conn.connect()
  File "/usr/local/lib/python3.6/dist-packages/urllib3/connection.py", line 371, in connect
    ssl_context=context,
  File "/usr/local/lib/python3.6/dist-packages/urllib3/util/ssl_.py", line 386, in ssl_wrap_socket
    return context.wrap_socket(sock, server_hostname=server_hostname)
  File "/usr/lib/python3.6/ssl.py", line 407, in wrap_socket
    _context=self, _session=session)
  File "/usr/lib/python3.6/ssl.py", line 817, in __init__
    self.do_handshake()
  File "/usr/lib/python3.6/ssl.py", line 1077, in do_handshake
    self._sslobj.do_handshake()
  File "/usr/lib/python3.6/ssl.py", line 689, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:852)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/botocore/httpsession.py", line 394, in send
    chunked=self._chunked(request.headers),
  File "/usr/local/lib/python3.6/dist-packages/urllib3/connectionpool.py", line 727, in urlopen
    method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
  File "/usr/local/lib/python3.6/dist-packages/urllib3/util/retry.py", line 386, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/usr/local/lib/python3.6/dist-packages/urllib3/packages/six.py", line 734, in reraise
    raise value.with_traceback(tb)
  File "/usr/local/lib/python3.6/dist-packages/urllib3/connectionpool.py", line 677, in urlopen
    chunked=chunked,
  File "/usr/local/lib/python3.6/dist-packages/urllib3/connectionpool.py", line 381, in _make_request
    self._validate_conn(conn)
  File "/usr/local/lib/python3.6/dist-packages/urllib3/connectionpool.py", line 978, in _validate_conn
    conn.connect()
  File "/usr/local/lib/python3.6/dist-packages/urllib3/connection.py", line 371, in connect
    ssl_context=context,
  File "/usr/local/lib/python3.6/dist-packages/urllib3/util/ssl_.py", line 386, in ssl_wrap_socket
    return context.wrap_socket(sock, server_hostname=server_hostname)
  File "/usr/lib/python3.6/ssl.py", line 407, in wrap_socket
    _context=self, _session=session)
  File "/usr/lib/python3.6/ssl.py", line 817, in __init__
    self.do_handshake()
  File "/usr/lib/python3.6/ssl.py", line 1077, in do_handshake
    self._sslobj.do_handshake()
  File "/usr/lib/python3.6/ssl.py", line 689, in do_handshake
    self._sslobj.do_handshake()
urllib3.exceptions.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:852)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "s3-connector.py", line 922, in <module>
    main()
  File "s3-connector.py", line 915, in main
    args.func(s3, args)
  File "s3-connector.py", line 842, in cmd_download
    if not s3.check_bucket_exist(args.bucket):
  File "s3-connector.py", line 414, in check_bucket_exist
    self.s3_resource.meta.client.head_bucket(Bucket=bucket_name)
  File "/usr/local/lib/python3.6/dist-packages/botocore/client.py", line 391, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/usr/local/lib/python3.6/dist-packages/botocore/client.py", line 706, in _make_api_call
    operation_model, request_dict, request_context)
  File "/usr/local/lib/python3.6/dist-packages/botocore/client.py", line 725, in _make_request
    return self._endpoint.make_request(operation_model, request_dict)
  File "/usr/local/lib/python3.6/dist-packages/botocore/endpoint.py", line 102, in make_request
    return self._send_request(request_dict, operation_model)
  File "/usr/local/lib/python3.6/dist-packages/botocore/endpoint.py", line 137, in _send_request
    success_response, exception):
  File "/usr/local/lib/python3.6/dist-packages/botocore/endpoint.py", line 255, in _needs_retry
    caught_exception=caught_exception, request_dict=request_dict)
  File "/usr/local/lib/python3.6/dist-packages/botocore/hooks.py", line 357, in emit
    return self._emitter.emit(aliased_event_name, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/botocore/hooks.py", line 228, in emit
    return self._emit(event_name, kwargs)
  File "/usr/local/lib/python3.6/dist-packages/botocore/hooks.py", line 211, in _emit
    response = handler(**kwargs)
  File "/usr/local/lib/python3.6/dist-packages/botocore/retryhandler.py", line 183, in __call__
    if self._checker(attempts, response, caught_exception):
  File "/usr/local/lib/python3.6/dist-packages/botocore/retryhandler.py", line 251, in __call__
    caught_exception)
  File "/usr/local/lib/python3.6/dist-packages/botocore/retryhandler.py", line 277, in _should_retry
    return self._checker(attempt_number, response, caught_exception)
  File "/usr/local/lib/python3.6/dist-packages/botocore/retryhandler.py", line 317, in __call__
    caught_exception)
  File "/usr/local/lib/python3.6/dist-packages/botocore/retryhandler.py", line 223, in __call__
    attempt_number, caught_exception)
  File "/usr/local/lib/python3.6/dist-packages/botocore/retryhandler.py", line 359, in _check_caught_exception
    raise caught_exception
  File "/usr/local/lib/python3.6/dist-packages/botocore/endpoint.py", line 199, in _do_get_response
    http_response = self._send(request)
  File "/usr/local/lib/python3.6/dist-packages/botocore/endpoint.py", line 268, in _send
    return self.http_session.send(request)
  File "/usr/local/lib/python3.6/dist-packages/botocore/httpsession.py", line 412, in send
    raise SSLError(endpoint_url=request.url, error=e)
botocore.exceptions.SSLError: SSL validation failed for https://s3.amazonaws.com/mybucket [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:852)

kdaily · 2022-03-25T16:59:15Z

Hi @snirbenyosef,

Thanks for the logs. Looks like they're truncated though, there should be a lot more information about what boto3/botocore are doing. Are you sure you added boto3.set_stream_logger('') to your code?

Can you also provide details on the version of boto3/botocore you're using?

You would not have made any changes regarding certificate proxies or networking, but your system administrator might have. Can you replicate the error on another workspace in the same network environment?

Another thing you can check is to force which certificate bundle to use by setting the AWS_CA_BUNDLE environment variable or ca_bundle configuration file parameter. You would provide the path to your certificates. A certificate bundle is provided and used by default with botocore.

github-actions · 2022-03-30T17:17:09Z

Greetings! It looks like this issue hasn’t been active in longer than five days. We encourage you to check if this is still an issue in the latest release. In the absence of more information, we will be closing this issue soon. If you find that this is still a problem, please feel free to provide a comment or upvote with a reaction on the initial post to prevent automatic closure. If the issue is already closed, please feel free to open a new one.

dmitrytretyakov · 2022-04-27T14:27:22Z

@snirbenyosef , did you find the reason?
I faced the same issue using ansible community aws collection and investigating src code led me to botocore

dmitrytretyakov · 2022-04-27T17:15:18Z

the reason in my case and probably in @snirbenyosef is botocore of 1.23.0 is not compatible with python 3.6 (at least with verify=False option)
Is that expected @kdaily ?

nateprewitt · 2022-04-27T17:29:06Z

Hi @dmitrytretyakov, could you clarify what isn’t compatible with Python 3.6? We’ll need logs as @kdaily requested above to verify the issue.

dmitrytretyakov · 2022-04-27T20:11:19Z

In case of using invalid certificates and trying to skip validation:

import boto3

client = boto3.client('s3', 
    aws_access_key_id='ID',
    aws_secret_access_key='reallysecret',
    endpoint_url='https://myaws',
    verify=False
)
client.list_buckets()

Python 3.6.9
boto3 (1.20.1)
botocore (1.22.12)
IS OK
Python 3.6.9
boto3 (1.20.1)
botocore (1.23.1)
IS NOT OK:

Traceback. Click to expand

Traceback (most recent call last): File "/opt/k8s-module-venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 677, in urlopen chunked=chunked, File "/opt/k8s-module-venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 381, in _make_request self._validate_conn(conn) File "/opt/k8s-module-venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 978, in _validate_conn conn.connect() File "/opt/k8s-module-venv/lib/python3.6/site-packages/urllib3/connection.py", line 371, in connect ssl_context=context, File "/opt/k8s-module-venv/lib/python3.6/site-packages/urllib3/util/ssl_.py", line 386, in ssl_wrap_socket return context.wrap_socket(sock, server_hostname=server_hostname) File "/usr/lib/python3.6/ssl.py", line 407, in wrap_socket _context=self, _session=session) File "/usr/lib/python3.6/ssl.py", line 817, in __init__ self.do_handshake() File "/usr/lib/python3.6/ssl.py", line 1077, in do_handshake self._sslobj.do_handshake() File "/usr/lib/python3.6/ssl.py", line 689, in do_handshake self._sslobj.do_handshake() ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:852)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/opt/k8s-module-venv/lib/python3.6/site-packages/botocore/httpsession.py", line 394, in send
chunked=self._chunked(request.headers),
File "/opt/k8s-module-venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 727, in urlopen
method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
File "/opt/k8s-module-venv/lib/python3.6/site-packages/urllib3/util/retry.py", line 386, in increment
raise six.reraise(type(error), error, _stacktrace)
File "/opt/k8s-module-venv/lib/python3.6/site-packages/urllib3/packages/six.py", line 734, in reraise
raise value.with_traceback(tb)
File "/opt/k8s-module-venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 677, in urlopen
chunked=chunked,
File "/opt/k8s-module-venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 381, in _make_request
self._validate_conn(conn)
File "/opt/k8s-module-venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 978, in validate_conn
conn.connect()
File "/opt/k8s-module-venv/lib/python3.6/site-packages/urllib3/connection.py", line 371, in connect
ssl_context=context,
File "/opt/k8s-module-venv/lib/python3.6/site-packages/urllib3/util/ssl.py", line 386, in ssl_wrap_socket
return context.wrap_socket(sock, server_hostname=server_hostname)
File "/usr/lib/python3.6/ssl.py", line 407, in wrap_socket
_context=self, _session=session)
File "/usr/lib/python3.6/ssl.py", line 817, in init
self.do_handshake()
File "/usr/lib/python3.6/ssl.py", line 1077, in do_handshake
self._sslobj.do_handshake()
File "/usr/lib/python3.6/ssl.py", line 689, in do_handshake
self._sslobj.do_handshake()
urllib3.exceptions.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:852)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "boto3-bug.py", line 8, in
print(client.list_buckets())
File "/opt/k8s-module-venv/lib/python3.6/site-packages/botocore/client.py", line 391, in _api_call
return self._make_api_call(operation_name, kwargs)
File "/opt/k8s-module-venv/lib/python3.6/site-packages/botocore/client.py", line 706, in _make_api_call
operation_model, request_dict, request_context)
File "/opt/k8s-module-venv/lib/python3.6/site-packages/botocore/client.py", line 725, in _make_request
return self._endpoint.make_request(operation_model, request_dict)
File "/opt/k8s-module-venv/lib/python3.6/site-packages/botocore/endpoint.py", line 102, in make_request
return self._send_request(request_dict, operation_model)
File "/opt/k8s-module-venv/lib/python3.6/site-packages/botocore/endpoint.py", line 137, in _send_request
success_response, exception):
File "/opt/k8s-module-venv/lib/python3.6/site-packages/botocore/endpoint.py", line 255, in _needs_retry
caught_exception=caught_exception, request_dict=request_dict)
File "/opt/k8s-module-venv/lib/python3.6/site-packages/botocore/hooks.py", line 357, in emit
return self._emitter.emit(aliased_event_name, **kwargs)
File "/opt/k8s-module-venv/lib/python3.6/site-packages/botocore/hooks.py", line 228, in emit
return self._emit(event_name, kwargs)
File "/opt/k8s-module-venv/lib/python3.6/site-packages/botocore/hooks.py", line 211, in _emit
response = handler(**kwargs)
File "/opt/k8s-module-venv/lib/python3.6/site-packages/botocore/retryhandler.py", line 183, in call
if self._checker(attempts, response, caught_exception):
File "/opt/k8s-module-venv/lib/python3.6/site-packages/botocore/retryhandler.py", line 251, in call
caught_exception)
File "/opt/k8s-module-venv/lib/python3.6/site-packages/botocore/retryhandler.py", line 277, in _should_retry
return self._checker(attempt_number, response, caught_exception)
File "/opt/k8s-module-venv/lib/python3.6/site-packages/botocore/retryhandler.py", line 317, in call
caught_exception)
File "/opt/k8s-module-venv/lib/python3.6/site-packages/botocore/retryhandler.py", line 223, in call
attempt_number, caught_exception)
File "/opt/k8s-module-venv/lib/python3.6/site-packages/botocore/retryhandler.py", line 359, in _check_caught_exception
raise caught_exception
File "/opt/k8s-module-venv/lib/python3.6/site-packages/botocore/endpoint.py", line 199, in _do_get_response
http_response = self._send(request)
File "/opt/k8s-module-venv/lib/python3.6/site-packages/botocore/endpoint.py", line 268, in _send
return self.http_session.send(request)
File "/opt/k8s-module-venv/lib/python3.6/site-packages/botocore/httpsession.py", line 412, in send
raise SSLError(endpoint_url=request.url, error=e)
botocore.exceptions.SSLError: SSL validation failed for https://myaws/ [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:852)

tim-finnigan · 2022-11-23T20:42:56Z

Checking in - support for Python 3.6 has since been deprecated as of May 2022 per this announcement. If this is still an issue in Python 3.7+ please let us know.

diptripa · 2023-07-25T08:05:50Z

@tim-finnigan, not sure if its the same but getting this error woth below versions:
boto3~=1.28.10
botocore~=1.31.10
python~= 3.11
openssl~=3.1.1
`
urllib3.exceptions.SSLError: [SSL: UNSAFE_LEGACY_RENEGOTIATION_DISABLED] unsafe legacy renegotiation disabled (_ssl.c:1002)

During handling of the above exception, another exception occurred:
botocore.exceptions.SSLError: SSL validation failed for https://alln-cloud-storage-1.cisco.com/ [SSL: UNSAFE_LEGACY_RENEGOTIATION_DISABLED] unsafe legacy renegotiation disabled (_ssl.c:1002)
`

zackees · 2024-05-15T08:19:02Z

Hitting this constantly with Python 3.11.7.

Do I really have to use verify=False on the official boto library?

snirbenyosef added the needs-triage This issue or PR still needs to be triaged. label Mar 7, 2022

kdaily self-assigned this Mar 7, 2022

kdaily added investigating This issue is being investigated and/or work is in progress to resolve the issue. guidance Question that needs advice or information. and removed needs-triage This issue or PR still needs to be triaged. labels Mar 7, 2022

kdaily added certs and removed investigating This issue is being investigated and/or work is in progress to resolve the issue. labels Mar 7, 2022

kdaily added the response-requested Waiting on additional info and feedback. label Mar 25, 2022

github-actions bot added closing-soon closed-for-staleness and removed closing-soon labels Mar 30, 2022

github-actions bot closed this as completed Apr 1, 2022

kdaily reopened this Apr 27, 2022

kdaily added response-requested Waiting on additional info and feedback. and removed closed-for-staleness response-requested Waiting on additional info and feedback. labels Apr 27, 2022

github-actions bot removed the response-requested Waiting on additional info and feedback. label Apr 27, 2022

RyanFitzSimmonsAK added the p3 This is a minor priority issue label Nov 4, 2022

tim-finnigan added the closing-soon label Nov 23, 2022

github-actions bot added closed-for-staleness and removed closing-soon labels Nov 25, 2022

github-actions bot closed this as completed Nov 25, 2022

1049451037 mentioned this issue Aug 17, 2023

模型推理时遇到的问题 THUDM/VisualGLM-6B#236

Open

1049451037 mentioned this issue Aug 24, 2023

fintune下载模型问题 THUDM/VisualGLM-6B#250

Open

johanneskoester mentioned this issue Dec 12, 2023

local() file not uploaded to storage snakemake/snakemake-executor-plugin-googlebatch#17

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

botocore.exceptions.SSLError: SSL validation failed on aws3 endpoint #2630

botocore.exceptions.SSLError: SSL validation failed on aws3 endpoint #2630

snirbenyosef commented Mar 7, 2022

kdaily commented Mar 7, 2022

snirbenyosef commented Mar 8, 2022

kdaily commented Mar 25, 2022

github-actions bot commented Mar 30, 2022

dmitrytretyakov commented Apr 27, 2022

dmitrytretyakov commented Apr 27, 2022 •

edited

Loading

nateprewitt commented Apr 27, 2022

dmitrytretyakov commented Apr 27, 2022

tim-finnigan commented Nov 23, 2022

diptripa commented Jul 25, 2023

zackees commented May 15, 2024

botocore.exceptions.SSLError: SSL validation failed on aws3 endpoint #2630

botocore.exceptions.SSLError: SSL validation failed on aws3 endpoint #2630

Comments

snirbenyosef commented Mar 7, 2022

kdaily commented Mar 7, 2022

snirbenyosef commented Mar 8, 2022

kdaily commented Mar 25, 2022

github-actions bot commented Mar 30, 2022

dmitrytretyakov commented Apr 27, 2022

dmitrytretyakov commented Apr 27, 2022 • edited Loading

nateprewitt commented Apr 27, 2022

dmitrytretyakov commented Apr 27, 2022

tim-finnigan commented Nov 23, 2022

diptripa commented Jul 25, 2023

zackees commented May 15, 2024

dmitrytretyakov commented Apr 27, 2022 •

edited

Loading