NotImplementedError while loading Distillation dataset

[YCAyca]

Hello, I try to use zscl method with the suggested datasets (ImageNet and Conceptual Captions). I prepared the distillation dataset using https://github.com/ml-jku/cloob repository so I have "Validation_GCC-1.1.0-Validation_output.csv" file already. But it throws me the following error:

0%| | 0/1301 [00:00<?, ?it/s]
Error executing job with overrides: []
Traceback (most recent call last):
File "main.py", line 51, in continual_clip
model.adaptation(task_id, cfg, train_dataset, train_classes_names)
File "/workspace/ZSCL/cil/continual_clip/models.py", line 48, in adaptation
self.train(task_id, cfg, train_dataset, train_classes_names)
File "/workspace/ZSCL/cil/continual_clip/models.py", line 230, in train
ref_images, ref_labels = next(ref_iter)
File "/opt/conda/lib/python3.7/site-packages/torch/utils/data/dataset.py", line 33, in getitem
raise NotImplementedError
NotImplementedError

I know that the problem is about Distillation Dataset (Conceptual Captions) but I cant fix it. Can anybody help??

[Thunderbeee]

Thanks for reaching out us! Below is the instruction provided by Doctor Zangwei Zheng:

First, download Validation_GCC-1.1.0-Validation.tsv from the Conceptual Captions dataset here.
Then, use gather_cc.py to download the images. After running the script, you should get both a folder of images and a Validation_GCC-1.1.0-Validation_output.csv file.
The process of Conceptual Captions is the same with this repo.

python gather_cc.py Validation_GCC-1.1.0-Validation.tsv

[YCAyca]

Thanks, but the question is not how to create Validation_GCC-1.1.0-Validation_output.csv, Its totally different. I have already done these steps and I have Validation_GCC-1.1.0-Validation_output.csv file, but the code gives me mentioned error, something about conceptual_captions(Dataset) class should be wrong, but I dont understand why. The error explains that there is no getitem function implemented for conceptual_captions class, but it is implemented. Below is the cc.py I have only changed the path to .csv file and added two print() blocks, apparently initialization is fine but getitem never called.

class CsvDataset(Dataset):
   def __init__(self, input_filename, transforms, img_key, caption_key, sep="\t"):
       df = pd.read_csv(input_filename, sep=sep)
       print("INIT")
       self.location = os.path.dirname(input_filename)
       self.images = df[img_key].tolist()
       self.captions = df[caption_key].tolist()
       self.transforms = transforms

   def __len__(self):
       return len(self.captions)

   def __getitem__(self, idx):
       print("GETITEM")
       image_path = os.path.join(self.location, str(self.images[idx]))
       images = self.transforms(Image.open(image_path))
       texts = clip.tokenize([str(self.captions[idx])])[0]
       return images, texts


class conceptual_captions(Dataset):
   def __init__(
       self, transforms, location, batch_size, *args, num_workers=16, **kwargs
   ):
       file_name = "/workspace/ZSCL/cil/data/Validation_GCC-1.1.0-Validation_output.csv"
       file_path = file_name
       self.template = lambda c: f"a photo of a {c}."
       self.train_dataset = CsvDataset(
           input_filename=file_path,
           transforms=transforms,
           img_key="filepath",
           caption_key="title",
       )
       # breakpoint()
       self.train_loader = torch.utils.data.DataLoader(
           self.train_dataset,
           batch_size=batch_size,
           shuffle=True,
           num_workers=0,
       )

[YCAyca]

Okey I found the problem and the solution seems to work now. In cc.py class conceptual_captions(Dataset) is defined already as a class inherited from Dataset, and it initialize an instance of CSVDataset by self.train_dataset = CsvDataset(...) but it never use it.... That's why print("INIT) line was working but not print("GETITEM") line, because when in models.py in line 232 ref_images, ref_labels = next(ref_iter) call made, it calls the getitem of conceptual_captions class, since ref_iter is an instance of this class. But there is no getitem function in this class so it doesnt work. I will make a pull request about that to push the fixed code soon!