Failed uploads do not get cleaned out of s3 storage, potentially leaving PII for longer than the data retention period.

[rahearn]

Seen in the wild for a demo upload that included phone numbers that weren't part of the team.

Instead, we should delete the upload from s3 as soon as the user is given the error message stating that the file could not be used.

[terrazoon]

Ideally we would do this with a lifecycle policy attached to the bucket, but CloudFoundry is giving me some trouble there. It needs to call GetMetadata but doesn't have the EC2 IMDS role or something.

I've tried doing this as a task also, and maybe I will go back to that approach. The issue I saw there was that the call to list_multipart_uploads was not returning an Uploads key in the response. I assume my bucket never had multipart uploads, but still the key should be there even if it has an empty array.

[ccostino]

🤔 regarding the lifecycle policy, that may still be an option, but we'd have to request support from the cloud.gov team. If it's helpful, I can check in with them when it makes sense and see what options might be available to us.