How is it possible to test it on custom dataset?

[SamuelePino]

I'm looking for a model which is able to accomplish recipe retrieval on food images, and this model seems to be very promising. I have a custom dataset of images along with their corresponding titles and ingredient lists, but I'm unsure how to adapt it to this particular model.

I haven't yet understood how to set up my dataset in order to use the preprocessing scripts provided, nor do I know what folder structure is required and what files are needed to run the model on my dataset.

My confusion arises from the 'Download and Prepare Dataset' section of the README, where you mention downloading files like 'Layers' and 'data.h5', which are specifically related to the Recipe1M dataset. How can I obtain files like these if I'm using my custom dataset instead? Are they necessary only for testing the model? [There appears to be a typo in 'Download Processed Data': 'downoload']

What about the other files in folders such as 'data_dir/recip1m/text' or 'data_dir/recip1m/context_annotation'? How do they fit into the process?

Lastly, could you provide an example of an evaluation command using a generic custom dataset?"

[mshukor]

Hi,
you can take inspiration from how the dataset for the recipe1m_13m dataset. Another option is to implement your dataset class with your style and change how the model read your dataset.

To use the same model weights, you only need to keep the architecture intact, otherwise it is possible to customize the code to your use case.

[VladAndronik]

@SamuelePino have you been able to accomplish something with custom dataset? I am also curious to test it.

[SamuelePino]

@SamuelePino have you been able to accomplish something with custom dataset? I am also curious to test it.

Not yet, I am focused on data preprocessing right now. I'll notify you here in case

[VladAndronik]

@mshukor seems like it is a bit tricky to run it now. Would you mind to add a short script to experiment with pretrained weight on custom data? It would be really helpful!