add tests for (Dataset|Image)Folder

[pmeier]

The failing test is valid. We first _find_classes()

vision/torchvision/datasets/folder.py

Line 126 in 9846569

classes, class_to_idx = self._find_classes(self.root)

by scanning the root directory for subdirectories

vision/torchvision/datasets/folder.py

Line 164 in 9846569

classes = [d.name for d in os.scandir(dir) if d.is_dir()]

and afterwards collect all samples

vision/torchvision/datasets/folder.py

Line 127 in 9846569

samples = self.make_dataset(self.root, class_to_idx, extensions, is_valid_file)

In the case that a subdirectory contains no or no matching files, it is still listed in the classes attribute.

---

A brute force fix would be to re-iterate over the samples remove all classes that are not present. A more efficient way would be flag all classes that contain at least one sample during make_dataset and simply remove empty ones. But this would require to change the output which is BC breaking. Thoughts?

[fmassa]

@pmeier I'm not sure I get this. Is this a bug in the dataset implementation or in the test?

In principle, having an empty folder in the root dir shouldn't be a problem, so I would say I'd change the test to take this into account.

[pmeier]

Depends. Do we allow entries in the classes attribute that do not have a single example? If yes, my assumptions were wrong and I can simply adapt the test. If no, this is a bug.

[fmassa]

Following the discussion we had for make_dataset in the other issue, we can move the assert of empty classes to make_dataset, and fix the test so that no class should be empty

[pmeier]

Blocked by #3496.

[pmeier]

This was missed in #3496.

[pmeier]

This will close #963 since this adds tests for DatasetFolder which is currently our only untested dataset 🎉

[fmassa]

Thanks!

[fmassa]

There is a conflict, can you look into it?

[codecov]

Codecov Report

Merging #3477 (c39c240) into master (7cc941f) will decrease coverage by 0.01%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master    #3477      +/-   ##
==========================================
- Coverage   79.48%   79.47%   -0.02%     
==========================================
  Files         105      105              
  Lines        9822     9822              
  Branches     1582     1582              
==========================================
- Hits         7807     7806       -1     
  Misses       1527     1527              
- Partials      488      489       +1     

Impacted Files	Coverage Δ
torchvision/datasets/folder.py	77.55% <100.00%> (-1.03%)	⬇️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7cc941f...c39c240. Read the comment docs.