Clean up deprecated behaviors from the S3BotoStorage

[jdufresne]

The ACL deprecation warnings produced by S3BotoStorage can be a false positives and create a lot of noise. I suggest the project move forward with removing these features as scheduled. Dropping the
deprecated behavior now means:

• The minimum supported version of boto3 is now 1.4.4.

• The S3Boto3Storage backend no longer accepts the argument acl. Use the ACL key in AWS_S3_OBJECT_PARAMETERS instead.

• The S3Boto3Storage backend no longer accepts the argument bucket. Use bucket_name or the setting AWS_STORAGE_BUCKET_NAME instead.

• The S3Boto3Storage backend no longer automatically creates the bucket. Doing so had encouraged using overly broad credentials. As a result, the AWS_BUCKET_ACL setting has been removed.

• The S3Boto3Storage backend no longer not longer supports the settings AWS_DEFAULT_ACL, AWS_REDUCED_REDUNDANCY and AWS_S3_ENCRYPTION. They have been removed in favor of the AWS_S3_OBJECT_PARAMETERS setting. Using AWS_S3_OBJECT_PARAMETERS allows for full control over these S3 parameters.

• The S3Boto3Storage backend no longer supports the undocumented AWS_PRELOAD_METADATA setting.

[jdufresne]

@jschneier Any thoughts on this? If you're onboard, I would be really nice to clean up the AWS_DEFAULT_ACL warnings. Right now it floods the test output of my projects.

[jdufresne]

I have rebased and updated to handle all newly deprecated features. The new changelog entry is:

• The minimum supported version of boto3 is now 1.4.4.

• The S3Boto3Storage backend no longer accepts the argument acl. Use the ACL key in AWS_S3_OBJECT_PARAMETERS instead.

• The S3Boto3Storage backend no longer accepts the argument bucket. Use bucket_name or the setting AWS_STORAGE_BUCKET_NAME instead.

• The S3Boto3Storage backend no longer automatically creates the bucket. Doing so had encouraged using overly broad credentials. As a result, the AWS_BUCKET_ACL setting has been removed.

• The S3Boto3Storage backend no longer not longer supports the settings AWS_DEFAULT_ACL, AWS_REDUCED_REDUNDANCY and AWS_S3_ENCRYPTION. They have been removed in favor of the AWS_S3_OBJECT_PARAMETERS setting. Using AWS_S3_OBJECT_PARAMETERS allows for full control over these S3 parameters.

• The S3Boto3Storage backend no longer supports the undocumented AWS_PRELOAD_METADATA setting.

[jdufresne]

Rebased 🙂

[WhyNotHugo]

These were some very breaking changes that were release under a minor release, so have triggered breakage on lots of pipelines where I had pinned dependencies.

I'd appreciate if you could publish a major release (rather than minor) if there's going to be large breaking changes and drops of so many features.

[WhyNotHugo]

Now that default_acl="private", is dropped, how do I get an equivalent behaviour? The PR points to AWS_S3_OBJECT_PARAMETERS, but the docs for that just take **kwargs.

At this point, I've lost track of where to look or how to figure it out.

[jschneier]

Now that default_acl="private", is dropped, how do I get an equivalent behaviour? The PR points to AWS_S3_OBJECT_PARAMETERS, but the docs for that just take **kwargs.

At this point, I've lost track of where to look or how to figure it out.

I will add a link to the Boto3 docs. The method of uploading is to use .put(), its params are here. So in your case it would be ACL. If your bucket is configured I don't think you should need to do anything though which is a big advantage to the new changes.

These were some very breaking changes that were release under a minor release, so have triggered breakage on lots of pipelines where I had pinned dependencies.

I'd appreciate if you could publish a major release (rather than minor) if there's going to be large breaking changes and drops of so many features.

Yeah, I went back and forth on this. I'm not a huge fan of semantic versioning. If this library was just the one backend I definitely would have but its really difficult and unwieldy to work with. If you're not already doing this, I strongly recommend pinning to the exact version and using something like dependabot to help keep track of upgrade releases.

[jschneier]

Sorry for the churn, I would be very surprised if there is something similar in the future. I've wanted to drop some of this stuff since I first forked a very long time ago.

[WhyNotHugo]

I strongly recommend pinning to the exact version and using something like dependabot to help keep track of upgrade releases.

That's regrettably, not an options for me. dependabot does not support GitLab.

I already use poetry to pin and track outdated dependencies. The main issue is that pretty much all other dependencies follow a versioning scheme, and this particular library doesn't.

But yeah, I'll probably pin the exact version (or even 1.9.1, since I still have to work around dropped features which are critical to me).