Botocore max dependency range

[cpvandehey]

Im very curious why the maintainers insist on restricting the max version of botocore.

I would highly suggest making the max botocore version allowed to be <2.0.0 and potentially doing the same for the extras installed below it. If there is a reason for these restrictions, a comment should be added to explain why.

Thanks! We appreciate aiobotocore!

[thehesiod]

https://github.com/aio-libs/aiobotocore/blob/master/CONTRIBUTING.rst#background-and-implementation

[kloczek]

Looks like pytest fails with botocore version 1.34.44

+ PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-aiobotocore-2.11.2-2.fc35.x86_64/usr/lib64/python3.8/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-aiobotocore-2.11.2-2.fc35.x86_64/usr/lib/python3.8/site-packages
+ /usr/bin/pytest -ra -m 'not network' -m moto --deselect tests/test_lambda.py::test_run_lambda
============================= test session starts ==============================
platform linux -- Python 3.8.18, pytest-8.0.1, pluggy-1.3.0
cachedir: /tmp/pytest_aiobotocore_cache
rootdir: /home/tkloczko/rpmbuild/BUILD/aiobotocore-2.11.2
configfile: pyproject.toml
plugins: asyncio-0.23.5
asyncio: mode=auto
collected 206 items / 23 deselected / 183 selected

tests/boto_tests/test_credentials.py ..................................  [ 18%]
tests/boto_tests/test_signers.py .                                       [ 19%]
tests/boto_tests/test_utils.py ........                                  [ 23%]
tests/python3.8/boto_tests/test_credentials.py ...............           [ 31%]
tests/python3.8/boto_tests/test_signers.py ........                      [ 36%]
tests/python3.8/boto_tests/test_tokens.py ....................           [ 46%]
tests/python3.8/boto_tests/test_utils.py ........................        [ 60%]
tests/test_basic_s3.py .......................x                          [ 73%]
tests/test_batch.py .                                                    [ 73%]
tests/test_config.py .....                                               [ 76%]
tests/test_dynamodb.py ......                                            [ 79%]
tests/test_ec2.py .                                                      [ 80%]
tests/test_eventstreams.py ..                                            [ 81%]
tests/test_monitor.py .                                                  [ 81%]
tests/test_patches.py .F                                                 [ 83%]
tests/test_response.py ...............                                   [ 91%]
tests/test_session.py ...                                                [ 92%]
tests/test_sns.py .....                                                  [ 95%]
tests/test_sqs.py ....                                                   [ 97%]
tests/test_stubber.py ..                                                 [ 98%]
tests/test_version.py .                                                  [ 99%]
tests/test_waiter.py .                                                   [100%]

=================================== FAILURES ===================================
_________________________________ test_patches _________________________________

    @pytest.mark.moto
    def test_patches():
        print(f"Botocore version: {botocore.__version__}")

        success = True
        for obj, digests in _API_DIGESTS.items():
            try:
                source = getsource(obj)
            except TypeError:
                obj = obj.fget
                source = getsource(obj)

            digest = hashlib.sha1(source.encode('utf-8')).hexdigest()

            if digest not in digests:
                print(
                    "Digest of {}:{} not found in: {}".format(
                        obj.__qualname__, digest, digests
                    )
                )
                success = False

>       assert success
E       assert False

tests/test_patches.py:729: AssertionError
----------------------------- Captured stdout call -----------------------------
Botocore version: 1.34.44
Digest of ContainerProvider._retrieve_or_fail:c99153a4c68927810a3edde09ee98c5ba33d3697 not found in: {'7c14f1cdee07217f847a71068866bdd10c3fa0fa'}
Digest of ContainerProvider._create_fetcher:a921ee40b9b4779f238adcf369a3757b19857fc7 not found in: {'935ae28fdb1c76f419523d4030265f8c4d9d0b00'}
================================== XFAILURES ===================================
_____________________ test_put_object_sha256[False-https] ______________________

s3_client = <aiobotocore.client.S3 object at 0x7f2786f6c6a0>
bucket_name = 'vzdjnuiywbhkqfaemsocltpgrx'

    @pytest.mark.xfail(
        reason="moto does not yet support Checksum: https://github.com/spulec/moto/issues/5719"
    )
    @pytest.mark.parametrize('server_scheme', ['https'])
    @pytest.mark.parametrize('s3_verify', [False])
    @pytest.mark.moto
    @pytest.mark.asyncio
    async def test_put_object_sha256(s3_client, bucket_name):
        data = b'test1234'
        digest = hashlib.sha256(data).digest().hex()

        resp = await s3_client.put_object(
            Bucket=bucket_name,
            Key='foobarbaz',
            Body=data,
            ChecksumAlgorithm='SHA256',
        )
        sha256_trailer_checksum = base64.b64decode(resp['ChecksumSHA256'])

>       assert digest == sha256_trailer_checksum
E       assert '937e8d5fbb48bd4949536cd65b8d35c426b80d2f830c5c308e2cdec422ae2244' == b'\x93~\x8d_\xbbH\xbdIISl\xd6[\x8d5\xc4&\xb8\r/\x83\x0c\\0\x8e,\xde\xc4"\xae"D'

tests/test_basic_s3.py:660: AssertionError
------------------------------ Captured log setup ------------------------------
INFO     werkzeug:_internal.py:96 127.0.0.1 - - [19/Feb/2024 08:49:26] "GET /static HTTP/1.1" 404 -
INFO     werkzeug:_internal.py:96 127.0.0.1 - - [19/Feb/2024 08:49:26] "PUT /vzdjnuiywbhkqfaemsocltpgrx HTTP/1.1" 200 -
INFO     werkzeug:_internal.py:96 127.0.0.1 - - [19/Feb/2024 08:49:26] "PUT /vzdjnuiywbhkqfaemsocltpgrx?versioning HTTP/1.1" 200 -
------------------------------ Captured log call -------------------------------
INFO     werkzeug:_internal.py:96 127.0.0.1 - - [19/Feb/2024 08:49:26] "PUT /vzdjnuiywbhkqfaemsocltpgrx/foobarbaz HTTP/1.1" 200 -
---------------------------- Captured log teardown -----------------------------
INFO     werkzeug:_internal.py:96 127.0.0.1 - - [19/Feb/2024 08:49:26] "GET /vzdjnuiywbhkqfaemsocltpgrx?versions&prefix=&encoding-type=url HTTP/1.1" 200 -
INFO     werkzeug:_internal.py:96 127.0.0.1 - - [19/Feb/2024 08:49:26] "DELETE /vzdjnuiywbhkqfaemsocltpgrx/foobarbaz?versionId=79f0205b-6e2a-4175-b1f0-054e68dfc509 HTTP/1.1" 204 -
INFO     werkzeug:_internal.py:96 127.0.0.1 - - [19/Feb/2024 08:49:26] "DELETE /vzdjnuiywbhkqfaemsocltpgrx HTTP/1.1" 204 -
=============================== warnings summary ===============================
tests/test_dynamodb.py:123
  tests/test_dynamodb.py:123: PytestDeprecationWarning: test_waiter_table_exists[v4] is asynchronous and explicitly requests the "event_loop" fixture. Asynchronous fixtures and test functions should use "asyncio.get_running_loop()" instead.
    @pytest.mark.moto

tests/boto_tests/test_signers.py: 2 warnings
tests/test_basic_s3.py: 47 warnings
tests/test_batch.py: 2 warnings
tests/test_dynamodb.py: 12 warnings
tests/test_ec2.py: 2 warnings
tests/test_eventstreams.py: 4 warnings
tests/test_monitor.py: 2 warnings
tests/test_sns.py: 10 warnings
tests/test_sqs.py: 8 warnings
tests/test_waiter.py: 2 warnings
  /usr/lib64/python3.8/site-packages/aiohttp/client.py:1194: DeprecationWarning: verify_ssl is deprecated, use ssl=False instead
    self._resp = await self._coro

tests/python3.8/boto_tests/test_credentials.py::test_refreshablecredentials_get_credentials_set
  /usr/lib64/python3.8/selectors.py:72: RuntimeWarning: coroutine 'fake_aiohttp_session.<locals>.FakeAioHttpSession.FakeResponse._content' was never awaited
    return self._selector._fd_to_key[fd]
  Enable tracemalloc to get traceback where the object was allocated.
  See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.

tests/python3.8/boto_tests/test_credentials.py::test_refreshablecredentials_get_credentials_set
  /usr/lib64/python3.8/selectors.py:72: RuntimeWarning: coroutine 'fake_aiohttp_session.<locals>.FakeAioHttpSession.FakeResponse._text' was never awaited
    return self._selector._fd_to_key[fd]
  Enable tracemalloc to get traceback where the object was allocated.
  See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.

tests/python3.8/boto_tests/test_utils.py::test_containermetadatafetcher_retrieve_url_bad_status
  /usr/lib64/python3.8/unittest/mock.py:2030: RuntimeWarning: coroutine 'fake_aiohttp_session.<locals>.FakeAioHttpSession.FakeResponse._text' was never awaited
    setattr(_type, entry, MagicProxy(entry, self))
  Enable tracemalloc to get traceback where the object was allocated.
  See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.

tests/python3.8/boto_tests/test_utils.py::test_containermetadatafetcher_retrieve_url_not_json
  /usr/lib64/python3.8/_collections_abc.py:837: RuntimeWarning: coroutine 'fake_aiohttp_session.<locals>.FakeAioHttpSession.FakeResponse._text' was never awaited
    for key, value in other:
  Enable tracemalloc to get traceback where the object was allocated.
  See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.

tests/test_basic_s3.py::test_can_make_request
  /usr/lib64/python3.8/inspect.py:2821: RuntimeWarning: coroutine 'fake_aiohttp_session.<locals>.FakeAioHttpSession.FakeResponse._text' was never awaited
    params = OrderedDict(((param.name, param)
  Enable tracemalloc to get traceback where the object was allocated.
  See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.

tests/test_basic_s3.py::test_can_make_request_no_verify[False]
tests/test_basic_s3.py::test_put_object_sha256[False-https]
  /home/tkloczko/rpmbuild/BUILD/aiobotocore-2.11.2/aiobotocore/httpsession.py:111: DeprecationWarning: verify_ssl is deprecated, use ssl=False instead
    self._create_connector = lambda: aiohttp.TCPConnector(

tests/test_version.py::test_release_versions
  /home/tkloczko/rpmbuild/BUILD/aiobotocore-2.11.2/tests/test_version.py:35: DeprecationWarning: The frontend.OptionParser class will be replaced by a subclass of argparse.ArgumentParser in Docutils 0.21 or later.
    settings = docutils.frontend.OptionParser(

tests/test_version.py: 69 warnings
  /usr/lib64/python3.8/optparse.py:1000: DeprecationWarning: The frontend.Option class will be removed in Docutils 0.21 or later.
    option = self.option_class(*args, **kwargs)

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=========================== short test summary info ============================
XFAIL tests/test_basic_s3.py::test_put_object_sha256[False-https] - moto does not yet support Checksum: https://github.com/spulec/moto/issues/5719
FAILED tests/test_patches.py::test_patches - assert False
= 1 failed, 181 passed, 23 deselected, 1 xfailed, 169 warnings in 91.06s (0:01:31) =

[jakob-keller]

Looks like with botocore version 1.34.44 pytest fails

Thanks for reporting. botocore 1.34.44 is not (yet) supported by aiobotocore.