How to Fine-tuning pre-trained model with my own data?

[Laodax]

I have downloaded the pre-trained model resnet-200 and want to fine-tuning it. I look "Fine-tuning on a custom dataset" but I still don't know how to start. If I want to use my data(4 classes and each class has 100 image for example), I need to creat my custom data loader or datasets? I have seen "datasets readme" too, but I still have no idea...
Would you explain it clearly? Thank you very much.

[colesbury]

You don't need a custom dataset. Organize your data so that the files look like:

train/<class1>/img1.jpg
train/<class2>/img3.jpg
...

val/<class1>/img4.jpg
etc.

Then run with -data <path to your train and val folders>

[Laodax]

If I retrain the model with 3 classes (tuna,other fish,other), can I use classify.lua to predict an image?
Do I have to modify some code or change something? I want the output will have 3 classes, not 1000 classes. Does it work? Thank you very much.

[aabobakr]

you don't have to modify anything in the classification code, except editing this line to refer to your label names.
local imagenetLabel = require './imagenet'

[Laodax]

I have a question is that if I retrain the model(fine-tuning) with my data, the model will only have my data or still have imagenet data?

[aabobakr]

fine-tuning means that you start with the learned features on the imagenet dataset and adjust these features and maybe the structure of the model to suit your dataset/task instead of starting the learning process on your data from scratch with random weight initialisation.

[Laodax]

So if I want to build a fish classifer(only tuna,shark,dolphin,swordfish) with resnet structure, does it work? Can I use the code here to build my classifer?
Or can I just add 4 calsses to the imagenet dasates, then the output will be 1004 classes. And use the classify.lua to predict an image. Does it work?
Sorry about the easy question, but I really need help.. Thank you very much.

[aabobakr]

It doesn't make sense to have a solution with 1004 classes while you want only to classify between 4 classes. You can simply load a pre-trained ResNet model, remove the last classification layer and add a new layer with 4 classes only, and fine-tune the whole model, this will train your new layer and adjust the weights of the previously learned layers.

Actually, I became curious to give this a try using the code in this repository and it worked well with me. Here are the steps:
1- arrange your data in this manner mentioned by @colesbury
2- download a pre-trained model
3- run this command for fine-tuning the model:

th main.lua -resetClassifier true -nClasses 3 -retrain pretrained/resnet-18.t7 -nEpochs 20 -data ~/Desktop/data/ -save ~/Desktop/ -batchSize 64 -LR 0.0001 -nThreads 5

Notes: I have tried with just 3 classes, you have to decrease the learning rate because you are not training from scratch, play with this value.

for classify.lua,
1- create a file like imagenet.lua containing your class label names
2- refer to the new file by editing this line of code

 --local imagenetLabel = require 'imagenet'
local imagenetLabel = require 'places'

3- as you have less than 5 classes: set N to your number of classes
I did it like that

if N > output:size(1) then
      N = output:size(1) 
   end
local probs, indexes = output:topk(N, true, true)

and done 👍

[Laodax]

Yes, you said the key point. I want to classify between 4 classes. So will this resnet code here achieve my purposes? How can I start it? Would you tell me the steps in detail? By the way I hope use torch to do this.
Thank you very much.

[aabobakr]

@Laodax just updated my answer with the steps to use the code in this repository.

[Laodax]

Oh I really thank you!! I will try it again. Thanks a lot!

[masaff]

Hi,
My dataset is .mat files. Is it possible to start training on this dataset? If yes, can I use a pre-trained (resnet-18) model or I should start train from the scratch?

[colesbury]

@masaff, you'll need to convert your .mat files to .JPEG images. (See https://github.com/facebook/fb.resnet.torch/tree/master/pretrained#fine-tuning-on-a-custom-dataset)

It depends on your data, but you'll likely get better accuracy if you fine-tune an existing model

[masaff]

@colesbury Thanks for your reply.

[panovr]

@colesbury @aabobakr Thanks for the reply, and I can also fine-tuning the pre-trained model with my own data set now following the instructions.

[masaff]

My dataset has 2M images. It's a big dataset but the images( .mat files which have been saved as .jpg files) are kind of weird images. They are grey scale images which are very different than images in imagenet,flickers,.... . Basically there is no clear object or background or scene in them. I'm wondering if I should train a CNN from scratch or I can do fine-tuning on some pre-trained model and If yes which model should I go with? I'm totally new to deep learning sorry if my question is an obvious question:D

[colesbury]

@masaff: try both and see what works best. There's probably less benefit from starting with a pre-trained model if you have lots of data and your images are very different from the ImageNet training set.

[Soumali13]

I tried to train with cifar10 and got the model_best.t7 with resnet-44. Now when I do classify.lua with the test images of cifar10 one by one, it always predicting cat. I have made the following changes in classify.lua(changed the imagenetLabel to use cifar10 label file that i created and also changed the mean(std) and transform in the classify.lua according to the mean(std) used in cifar10 in /datasets/cifar10.lua)

Can someone help me please from the authors

[saira05]

hi,
Can anyone tell me how can we use a random or my own model as a feature extractor? is there any link kindly share

[surajitkundu29]

It doesn't make sense to have a solution with 1004 classes while you want only to classify between 4 classes. You can simply load a pre-trained ResNet model, remove the last classification layer and add a new layer with 4 classes only, and fine-tune the whole model, this will train your new layer and adjust the weights of the previously learned layers.

Actually, I became curious to give this a try using the code in this repository and it worked well with me. Here are the steps:
1- arrange your data in this manner mentioned by @colesbury
2- download a pre-trained model
3- run this command for fine-tuning the model:

th main.lua -resetClassifier true -nClasses 3 -retrain pretrained/resnet-18.t7 -nEpochs 20 -data ~/Desktop/data/ -save ~/Desktop/ -batchSize 64 -LR 0.0001 -nThreads 5

Notes: I have tried with just 3 classes, you have to decrease the learning rate because you are not training from scratch, play with this value.

for classify.lua,
1- create a file like imagenet.lua containing your class label names
2- refer to the new file by editing this line of code

 --local imagenetLabel = require 'imagenet'
local imagenetLabel = require 'places'

3- as you have less than 5 classes: set N to your number of classes
I did it like that

if N > output:size(1) then
      N = output:size(1) 
   end
local probs, indexes = output:topk(N, true, true)

and done 👍

What would be the sequence of the classes in imagenet.lua ? Should it be in alphabetical order or any specific

[AqsaHassan]

I've organized my data in following manner in order to fine tuning ResNet50 using VMMRdb. In order to get started I'm just trying to retain the pretrained model with 8 classes of different car's models

train//img1.jpg
train//img3.jpg
...

val//img4.jpg
etc.
but after organizing my data in above manner when I run the following command
th main.lua -retrain resnet-50.t7 -data /home/aqsa/anaconda3/envs/tf-gpu/fb.resnet.torch-master -resetClassifier true -nClasses 8
unfortunately, it does not give any output nor any error.
I also tried to run dataloader by lua dataloader.lua in terminal but it is also giving error that no module named threads.