Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is a proof of concept of an abstract test harness in
fsspecthat can be used to define, in a fairly generic way, the set of tests that every implementation should pass. The tests only need to be written once infsspecand each of the implementations can inherit and run these tests with minimal code.There are just 3 tests included so far, one each of
cp,getandput, to test remote to remote, remote to local and local to remote transfers. I haves3fsandgcsfscode to work with this, and all these run and pass on my local machine.The tests are defined in
fspec/test/abstract, one class perfsspecfunction to be tested and each class contains one (it will be more!) test. There is anAbstractFixturesclass that provides local filesystem functionality.There are two implementations,
localandmemory. The code to inherit and hence to run the tests is really simple, just a class that inherits from each of theAbstracttest classes and also from a mixin class that derives fromAbstractFixtures. The mixin class, e.g.MemoryFixtures, provides the filesystem to be used by the tests and is simple forlocalandmemorybut will be much more complicated fors3fsandgcsfsfor example. The test class inheritance and use of mixin for fixtures wasn't the first approach I tried but it works well with thepytestapproach of dynamically finding tests and fixtures.At this stage the plan is just to run it through CI to see what fails, I expect Windows tests will produce some problems.