Question: When send a DELETE request to tusd with minio s3 storage backend, how do I remove associated multipart and tmp resources?

[dcs3spp]

Hi,

Hopefully, posting this in the correct place.

I am a newbie using tusd for serving uploaded files on minio S3 storage with uppy incorporated into the front end with the tus plugin. So far, finding all of these excellent, they have made development much easier

I am struggling with the process of implementing removal of an uploaded file and associated resources. From what I have read the tus protocol specifies the termination Tus-Extension to DELETE uploaded resources.

I have used the Rested plugin in Opera browser to send a DELETE request and can see that the file object in the minio bucket is removed in addition to the associated metadata info file.

Upon inspecting the minio /data folder there are two folders: multipart and tmp located within .minio.sys

The subfolders associated with the uploaded file do not appear to be removed from these folders when a DELETE request is issued. Each upload seems to create a subfolder within multipart and sometimes tmp. The subfolders appear to be empty.

Is there any way that I can remove these resources when a DELETE request is issued? I am guessing that they are controlled by minio? How can I trigger the removal of these empty subfolders?

Kind regards

dcs3spp

[Acconut]

Hopefully, posting this in the correct place.

Yes, absolutely. This is the best place for discussion those topics.

Is there any way that I can remove these resources when a DELETE request is issued? I am guessing that they are controlled by minio? How can I trigger the removal of these empty subfolders?

When you issue a DELETE request, tusd will abort the corresponding multipart upload and delete any associated S3 objects (see https://github.com/tus/tusd/blob/master/s3store/s3store.go#L401-L432). As far as I know this should free all the resources hold by minio, at least that's what minio/minio#5222 (comment) indicates.

What version of minio are you using? The issue I linked mentions that RELEASE.2017-10-27T18-59-02Z would already be too old for that.

I can very much image that this is a problem with minio and not directly with tusd.

[dcs3spp]

Thanks for responding @Acconut.

When you issue a DELETE request, tusd will abort the corresponding multipart upload and delete any associated S3 objects (see https://github.com/tus/tusd/blob/master/s3store/s3store.go#L401-L432). As far as I know this should free all the resources hold by minio, at least that's what minio/minio#5222 (comment) indicates.

Yep, upon inspecting the data directory S3 object and associated meta data .info are removed. Also, the mc client utility confirms that there are no incomplete uploads when I issue an
mc ls --incomplete.

What version of minio are you using? The issue I linked mentions that RELEASE.2017-10-27T18-59-02Z would already be too old for that.

I am using Minio version 2018-11-30T03:56:59Z. Thanks for the link, appreciated.

I can very much image that this is a problem with minio and not directly with tusd.

Tusd is definitely aborting the multipart upload since there are no incomplete uploads listed with
mc ls --incomplete and related object file and meta data .info file are removed.

The empty subfolders in .minio.sys/multipart and resources in .minio.sys/tmp do not appear to be cleaned by minio when a DELETE request is issued. The contents of .minio.sys/tmp do appear to be cleaned after minio is restarted. However, the empty subfolders in .minio.sys/multipart remain.

Will investigate further and then raise question / issue on minio.

Thanks again, @Acconut. Appreciated 👍

[Acconut]

Thank you for bringing this to the attention of the minio team. Since this looks very much like a topic for their project, I will close this issue and follow the minio discussion.

Feel free for adding another comment here if the situation changes.