YOLOv8 transfer Learning

[pagalscientist]

Search before asking

• I have searched the YOLOv8 issues and discussions and found no similar questions.

Question

Please do not close this thread until i am done!
I am really sorry guys i am writing this message. But i have searched all over the internet and tried almost every solution provided on the internet. Nothing worked for me. Here's the call
I have trained yolov8s-cls.pt model on 2088 classes (Huge dataset). Now i want to add more classes in the dataset,
A Quick recap what i have already tried

================================================
from ultralytics import YOLO
from glob import glob
import os

os.environ["CUDA_VISIBLE_DEVICES"] = '-1'

pretrained_weights = "/detection/weights/detectionmodel.pt"
model = YOLO('yolov8n-cls.pt').load(pretrained_weights)
model.train(data='/detection/dataset/', epochs=40, batch = 384)

Tried passing additional argument also

pre_trained_weights = "/detection/detectionmodel.pt"
data_path = '/detection/dataset/'
model.train(
data= data_path,
epochs=1,
batch=64,
single_cls=False
)

i notice this alert in this cell
Overriding model.yaml nc=2088 with nc=14 (that might be the reason Might be)

I have been struggling really hard. I have upgraded my architecture running in production and now i am unable to add more classes. Help would be really appreciated.

Additional

No response

[github-actions]

👋 Hello @alimuneebml1, thank you for your interest in Ultralytics YOLOv8 🚀! We recommend a visit to the Docs for new users where you can find many Python and CLI usage examples and where many of the most common questions may already be answered.

If this is a 🐛 Bug Report, please provide a minimum reproducible example to help us debug it.

If this is a custom training ❓ Question, please provide as much information as possible, including dataset image examples and training logs, and verify you are following our Tips for Best Training Results.

Join the vibrant Ultralytics Discord 🎧 community for real-time conversations and collaborations. This platform offers a perfect space to inquire, showcase your work, and connect with fellow Ultralytics users.

Install

Pip install the ultralytics package including all requirements in a Python>=3.8 environment with PyTorch>=1.8.

pip install ultralytics

Environments

YOLOv8 may be run in any of the following up-to-date verified environments (with all dependencies including CUDA/CUDNN, Python and PyTorch preinstalled):

• Notebooks with free GPU:
• Google Cloud Deep Learning VM. See GCP Quickstart Guide
• Amazon Deep Learning AMI. See AWS Quickstart Guide
• Docker Image. See Docker Quickstart Guide

Status

If this badge is green, all Ultralytics CI tests are currently passing. CI tests verify correct operation of all YOLOv8 Modes and Tasks on macOS, Windows, and Ubuntu every 24 hours and on every commit.

[glenn-jocher]

@pagalscientist hello! It seems you're looking to perform transfer learning with YOLOv8 to add more classes to your already trained model. The message you're seeing about overriding nc (number of classes) indicates that the model's configuration is being updated to match the new dataset's class count.

For transfer learning, you should ensure that your new dataset includes the original classes plus the additional ones. The model.yaml file should reflect the total number of classes (original + new). When you initiate training with the .train() method, the model should automatically detect the number of classes from the dataset provided.

If you're still encountering issues, double-check your dataset's structure and the model.yaml to ensure they are correctly set up for the total number of classes you intend to train on. If the problem persists, please provide more details about the error messages or behavior you're observing, and we'll do our best to assist you further.

Remember, the Ultralytics Docs can be a helpful resource for understanding the training process and configuration. Keep at it, and don't hesitate to reach out if you need more guidance! 😊👍

[pagalscientist]

Thank you so much for your quick response @glenn-jocher.
Unfortunately i could not find model.yaml file anywhere. I used the following command for the very first training
yolo classify train data=/detection/dataset model=yolov8n-cls.pt epochs=1000 imgsz=640 patience = 150

And a humble request!
Transfer learning documentation for all of the YOLOv8 models on ultralytics would be really appreciated. I have been struggling to find accurate guidelines for quite a long time. And there are a lot of others also. Thanks!

Regards!

[glenn-jocher]

@pagalscientist, glad to assist! The model.yaml is typically included within the YOLOv8 model's directory. If you're unable to locate it, it might be due to using the classify command which is for image classification tasks, not object detection.

For transfer learning in object detection with YOLOv8, you should use the detect command instead. Ensure your dataset is properly annotated for detection with the correct number of classes.

Regarding transfer learning documentation, we appreciate your feedback and understand the importance of clear guidelines. We're continuously working on improving our documentation at Ultralytics Docs. Your request is noted, and we aim to provide more comprehensive resources for our users.

Keep an eye on our updates, and thank you for being part of the YOLO community! 😊🚀

[pagalscientist]

Perfect!

l am talking about the yolov8 classification model. I trained the model previously using shared command above. i have added new class and its error rate went higher, it was predicting 117 images out of 10846 and now it predicts more than 400 wrong images even after training for 7 epochs,

[glenn-jocher]

@pagalscientist, for the YOLOv8 classification model, an increase in error rate after adding new classes and training could be due to several factors, such as insufficient training data for the new classes, class imbalance, or the need for more epochs to properly converge.

When adding new classes, it's crucial to provide a balanced dataset with enough examples for each class. Also, consider training for more epochs and monitor the validation loss to ensure the model is improving and not overfitting.

Keep fine-tuning your model, and make sure your dataset is well-prepared for the new classes. Patience and iteration are key in machine learning. Good luck! 😊👍

[pagalscientist]

@glenn-jocher The model was still training yesterday night, and now it has been trained on 22 epochs. I was expecting good results after training for a few epochs. Because i trained the model for 40 epochs for the very first time. But this time i have transfer learned the model i was expecting good results right after a few epochs. But not.
Second, Dataset is good enough and balanced. Last time i achieved 98.92% accuracy i just added 1 single class. Any suggestions?

[glenn-jocher]

@pagalscientist, if your dataset is balanced and previously yielded high accuracy, it's possible that the model requires more epochs to adjust to the new class. Transfer learning can accelerate the training process, but it doesn't always lead to immediate improvements, especially with a significant change like an additional class.

Consider continuing the training for more epochs while closely monitoring the validation metrics to ensure progress. Sometimes, small learning rate adjustments or additional data augmentation can also help the model generalize better with the new class.

Keep iterating, and with patience, you should see improvements. Good luck! 😊👍

[Avv22]

@pagalscientist, if your dataset is balanced and previously yielded high accuracy, it's possible that the model requires more epochs to adjust to the new class. Transfer learning can accelerate the training process, but it doesn't always lead to immediate improvements, especially with a significant change like an additional class.

Consider continuing the training for more epochs while closely monitoring the validation metrics to ensure progress. Sometimes, small learning rate adjustments or additional data augmentation can also help the model generalize better with the new class.

Keep iterating, and with patience, you should see improvements. Good luck! 😊👍

Thanks Glenn. So for one-class object detection, you also advice to do transfer learning or fine-tuning or training given I want to take advantage of your model as I guess it has a lot of information, what do you think please?