Some problem with tqdm(dataloader). Somehow it couldn't load the image properly

[abc123yuanrui]

Somehow it couldn't load the image properly
In terminal:
# python simple_extractor.py --dataset 'atr' --model-restore './exp-schp-201908301523-atr.pth' --input-dir './input' --output-dir './output'
Evaluating total class number 18 with ['Background', 'Hat', 'Hair', 'Sunglasses', 'Upper-clothes', 'Skirt', 'Pants', 'Dress', 'Belt', 'Left-shoe', 'Right-shoe', 'Face', 'Left-leg', 'Right-leg', 'Left-arm', 'Right-arm', 'Bag', 'Scarf']
0%| | 0/2 [00:00<?, ?it/s]
Traceback (most recent call last):
File "simple_extractor.py", line 155, in
main()
File "simple_extractor.py", line 128, in main
for idx, batch in enumerate(tqdm(dataloader)):
File "/usr/local/lib/python3.6/dist-packages/tqdm/std.py", line 1130, in iter
for obj in iterable:
File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/dataloader.py", line 346, in next
data = self.dataset_fetcher.fetch(index) # may raise StopIteration
File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/_utils/fetch.py", line 44, in fetch
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/_utils/fetch.py", line 44, in
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/notebooks/datasets/simple_extractor_dataset.py", line 54, in getitem
h, w, _ = img.shape
AttributeError: 'NoneType' object has no attribute 'shape'

[abc123yuanrui]

After some trouble shootings, the problem was caused by Jupyter checkpoint in the input directory. This issue can be fixed by add a simple line in SimpleFolderDataset function detecting whether the file is a legit image.

[jasmineytt]

How do you solve the problem? Can you share you code?

[abc123yuanrui]

Hi jasmineytt,

I don't have the code for the moment, but what you can do is to check the image folder and see if there is any mid-product generated by Jupyter Notebook. Those checkpoints would cause problem as they are not images. A simple function would be solely read the image file by checking the file names.