Allow writers to overwrite existing data

[JAEarly]

Description of changes:

CloudUploader has an exist_ok kwarg that is set to False by default.
Writers (e.g. MDSWriter) cannot overwrite this.

This PR makes two changes:

1. exist_ok can now be set by a writer.
2. If exist_ok is True, existing files are now deleted. This allows easy debugging when developing streaming datasets as the files do not have to be manually deleted each time the code is run.

Tests and docs updated accordingly.

Merge Checklist:

Put an x without space in the boxes that apply. If you are unsure about any checklist, please don't hesitate to ask. We are here to help! This is simply a reminder of what we are going to look for before merging your pull request.

General

• I have read the contributor guidelines
• This is a documentation change or typo fix. If so, skip the rest of this checklist.
• I certify that the changes I am introducing will be backward compatible, and I have discussed concerns about this, if any, with the MosaicML team.
• I have updated any necessary documentation, including README and API docs (if appropriate).

Tests

• I ran pre-commit on my change. (check out the pre-commit section of prerequisites)
• I have added tests that prove my fix is effective or that my feature works (if appropriate).
• I ran the tests locally to make sure it pass. (check out testing)
• I have added unit and/or integration tests as appropriate to ensure backward compatibility of the changes.

[karan6181]

Thank you for creating the PR. The changes looks good to me. I have posted some minor comments.

@XiaohanZhangCMU Can you please also take a look?

[JAEarly]

Looking at the tests that are failing: I believe there's now a conflict when exists_ok=True and keep_local=True, or at least this seems to be what the GitHub test log says. I'm having issues running the failing tests locally so I'm unable to debug. @karan6181 do you have further insight on this?

[XiaohanZhangCMU]

@JAEarly Currently exists_ok=True in clouduploader.py is only used by merging index (streaming/base/utils/merge_index). Your change will make that fail.

I suggest you add the "file-removing behavior" to mdswriter instead of upload.py.

[XiaohanZhangCMU]

Lgtm. Wait for ci checks /

[JAEarly]

Should be good to merge, I've rebased just need the CI checks to be run again, thanks!

[XiaohanZhangCMU]

Sure. Just kicked off the CI tests.

[XiaohanZhangCMU]

LGTM! Thanks for the improvement!