testing/plugins: migrate cloud fixtures to use fsspec filesystems #8917
Labels
p3-nice-to-have
It should be done this or next sprint
testing
Related to the tests and the testing infrastructure
Cloud
fixtures currently depend on each plugin to implement the cloud/path-like utility methods (read_text/bytes
,write_text/bytes
, etc) and we do this by using the underlying libraries ourselves directly in dvc plugins (i.e.dvc_s3.tests.cloud
usesboto
directly). It would be easier to add new functionality if the plugin specified an fsspec filesystem to use, and then the genericdvc.testing.Cloud
used the fsspec api calls instead. That way adding new functionality for things like retrieving specific version ID's can be added without having to use the underlying aws/gcs/adlfs libraries ourselves.Basically, since we already added these things in the underlying fsspec implementations, it doesn't make sense for us to be re-implementing them all again in each plugin's
Cloud
test fixtures.This would also let us implement a generic
Cloud.remove()
(currently we use DVCutils.fs.remove()
for removing local files, but have no equivalent for the cloud/remote fixtures)related: #8850
The text was updated successfully, but these errors were encountered: