Project Limit Request: AWS CRT - 50 GB

[TingDaoK]

Project URL

https://pypi.org/project/awscrt

Does this project already exist?

• Yes

New limit

100 GB

Update issue title

• I have updated the title.

Which indexes

TestPyPI, PyPI

About the project

AWS CRT is a set of foundational components for communicating with AWS services. AWS Python SDK and CLI depend on it, and anyone could use to build custom sdks.

There was a previous ask to bump it up to 30GB #2190
After ~3 years, we hit the limits again.
We did support ABI3 now, but we still need to support the old python version from 3.8 to 3.10.

How large is each release?

~250mb

we provide wheels for the following configurations for several latest cpython versions (cp313 with abi3, cp311 with abi3, cp310, cp39, cp38 on latest release):

• manylinux aarch64, x86_64
• musllinux x64, aarch64
• macos universal2
• win 32 and amd64

How frequently do you make a release?

about twice a month.

Code of Conduct

• I agree to follow the PSF Code of Conduct

[Thespi-Brain]

Hi @TingDaoK,

Is it possible to reduce the wheel sizes per architecture? If you are able to do so, I can bump up the project limit to 50 GB. Thanks!

[waahm7]

@Thespi-Brain I am not sure how we can reduce the wheel size per architecture as everything we bundle is needed. Do you have any ideas? We have already moved towards abi3-compatible wheels and are expecting size to reduce once we drop older Python versions. We would appreciate a limit increase since this is blocking our release pipeline.

[TingDaoK]

@Thespi-Brain not really, we need libcrypto from aws-lc in our wheel, which contributes to most of our package size. And I don't see an alternative to it.

[TingDaoK]

@Thespi-Brain Can we at least get the 50GB bump first?
It's blocking our release now.

We can investigate on reducing the size a bit, but I don't think there is much we can reduce.

Is it possible for us to host the wheels somewhere like S3/Github for pypi?

[TingDaoK]

@Thespi-Brain We did find the debug symbol was included in the wheels are really larger than expected.
And we trimmed it down see awslabs/aws-crt-python#653

Again, can we please get more storage, it's blocking us, and will soon be a blocker for aws-cli and boto3 as well.

[TingDaoK]

Apologies for another ping, but we're approaching one month without being able to release. @di @miketheman @Thespi-Brain is there anything we're able to do to help expedite turn around as this is a production impacting issue. We'd like to increase our limit to 50GB. We've taken steps to reduce our wheel and package size going forward which should shrink our releases from 320MB to 168 MB

[Thespi-Brain]

Hi @TingDaoK

I've set the upload limit for AWS CRT to 50 GB on PyPI and Test.PyPI.