Feature request: support fsspec for FileBackend

[mjkanji]

Feature description

Use case

Is there a specific goal that this would help you accomplish, or can you provide any other context about how you would like to use this feature?

I'd like to persist the Filesystem cache in object storage (S3) instead of my local machine for easier portability across machines/users.

Workarounds

Is there an existing workaround to accomplish this?

One could potentially just pass in a universal_pathlib's UPath as an input. Though, I wonder how the sqlite database that's mentioned here would work with that set up.

Plan to implement

Are you interested in submitting a PR to implement this?

I don't think I'd know enough to help implement this.

[JWCook]

I think that's a reasonable idea. I will look into it.

Meanwhile, have you looked into the other remote backends (MongoDB, Redis, and DynamoDB)? If you're already on AWS, DynamoDB would probably be a good choice. It does a better job than S3 for large quantities of tiny files/documents, which is what you'll typically have with an HTTP cache. On the other hand, if you have large responses (like images or other file downloads), S3 would be a better option.

[mjkanji]

I think that's a reasonable idea. I will look into it.

Thanks!

Meanwhile, have you looked into the other remote backends (MongoDB, Redis, and DynamoDB)?

My current plan is, indeed, to experiment with using the DynamoDB backend.