How do I save the images of the yolov8 training predictions?

[Lay-best]

Search before asking

• I have searched the YOLOv8 issues and discussions and found no similar questions.

Question

In the yolov8 interface , when I use the method predict which in the class YOLO , I do not know how to save the output images.And there maybe have not args in the method predict to save the output images.

Additional

No response

[kyoko9000]

you can use results = model("path to your image" , save=True)

[Lay-best]

you can use results = model("path to your image" , save=True)
I got it .Thank you.

[robmarkcole]

save=True would be a good default, that appears to be the behaviour when using CLI & yolo task=detect mode=predict

[dmddmd]

@kyoko9000 where does it save?

And does it also work for stream=True

[prince0310]

@kyoko9000 it will save on runs\detect\predict folder

[dmddmd]

found it .. it was confusing that the runs folder is saved two directories back

[prince0310]

if you don't need further clarification please close the issue

[Harshit26042004]

@kyoko9000 where does it save?

And does it also work for stream=True

is stream=True used for streaming live in web-cam?

[JJ-McClure]

The 'save' option needs to be listed under 'Configuration'.

[Siddharth1698]

Add save = True at end. That is,
!yolo predict model="ultralytics/runs/detect/train/weights/best.pt" save=True

[RelativelyFine]

How do define the location where it will save?

[Laughing-q]

@RelativelyFine By default it'll save to runs/task/predict and the task can be detect/segment/classify(depends on what task you're using). The save path consists of two args -> project/name, and you can modify these two args to customize your save path.

[sangramdhurve]

try using this code it is working for me,it was really good but I want here cropping of detected bounding box and save it into disk, if anyone can help me please

from ultralytics import YOLO
import cv2

model = YOLO("model/best.pt")
model.predict(source = '/home/sangramdh/Downloads/rajsir/test_models/test/2 Wheel Test/', show = True, save=True)
cv2.waitKey(0)

[glenn-jocher]

To save crops you can use save_crop=True

[kittutrans]

Add save = True at end. That is, !yolo predict model="ultralytics/runs/detect/train/weights/best.pt" save=True

Thanks it worked

[Youssefkhaled55]

Ultralytics now use the flag "save=True" to save results. to save the output results by making runs folder
automatically and saving the image in it for example code be like this:
yolo task=detect mode=predict model=yolov8n.pt source=t.jpg conf=0.5 save=True

[Dineth9D]

Here you can send an image through the Yolov8 model and get predictions.

from ultralytics import YOLO

model = YOLO("best.pt")
path = "image.jpg"

results = model.predict(source=path, show=True, save=False, save_txt=False, save_conf=False)

[arjun5arvind]

@kyoko9000 where does it save?
And does it also work for stream=True

is stream=True used for streaming live in web-cam?

is there any way for live web cam usage?

[Dineth9D]

accepts all formats - image/dir/Path/URL/video/PIL/ndarray. 0 for webcam
results = model.predict(source="0")

[JesseDeGelderVW]

Let's say I want to run my model over a large dataset, can I also make it so only detections above, say 0.6 confidence are saved?

[glenn-jocher]

@Camerabiologist you can set conf to any value you'd like during predict:

yolo predict conf=0.6

[glenn-jocher]

@Camerabiologist you can set conf to any value you'd like during predict:

yolo predict conf=0.6

See https://docs.ultralytics.com/

[JesseDeGelderVW]

Thanks @glenn-jocher! My current yolo predict outputs my full dataset including detections and non-detections. I want to filter out the non-detections from the resulting Predict_xx folder.

[glenn-jocher]

@Camerabiologist see https://docs.ultralytics.com/modes/predict for examples

[JiaxinWang123]

@Laughing-q > @RelativelyFine By default it'll save to runs/task/predict and the task can be detect/segment/classify(depends on what task you're using). The save path consists of two args -> project/name, and you can modify these two args to customize your save path.

Could you please give a short example code of customizing predicting save path?

[Laughing-q]

@JiaxinWang123

model = YOLO("yolov8n.pt")
model.predict(source=..., project="xx", name="xxx")

then it'll be save in xx/xxx.

[JiaxinWang123]

@Laughing-q

@JiaxinWang123

model = YOLO("yolov8n.pt")
model.predict(source=..., project="xx", name="xxx")

then it'll be save in xx/xxx.

Great! It worked, thanks!

[dronespov]

@glenn-jocher Aha your advice worked, below is the script I am using to isolate segmented masks, place white mask over segmented object, black out areas not segmented. You my good sir have acquired a new sponsor.

Now, I am need to work on optimizing the scripts and/or dataset (current is 92 jpg's at 1 mb each & txt's are around 50 kb) so training time reduces from x2 (7:50 min per epoch) to 3 min per epoch, this time was prior to implementing the Isolate segmented object script.

Isolate Segmented Objects & Save as Masks

from pathlib import Path

import cv2 as cv
import numpy as np
from ultralytics import YOLO

# Load YOLO model
m = YOLO('path/to/dir/best.pt')

# Set source directory
source = 'path/to/dir/'

# Get YOLO predictions
res = m.predict(source, boxes=False, save=True, imgsz=640, conf=0.6)

# Iterate over detection results
for r in res:
    img = np.copy(r.orig_img)
    img_name = Path(r.path).stem

    # Create a black canvas for the final image
    final_image = np.zeros_like(img)

    # Iterate over each object contour
    for ci, c in enumerate(r):
        label = c.names[c.boxes.cls.tolist().pop()]

        # Create binary mask for the object
        object_mask = np.zeros(img.shape[:2], np.uint8)
        contour = c.masks.xy.pop().astype(np.int32).reshape(-1, 1, 2)
        _ = cv.drawContours(object_mask, [contour], -1, 255, cv.FILLED)

        # Create white mask over the object
        white_mask = np.ones_like(img) * 255
        white_over_object = cv.bitwise_and(white_mask, white_mask, mask=object_mask)

        # Apply the white mask over the object in the final image
        final_image = cv.bitwise_or(final_image, white_over_object)



        # Save the isolated object
        _ = cv.imwrite(f'path/to/dir/{img_name}.jpg', final_image)

Masks to Polygons

import os

import cv2

input_dir = 'path/to/dir'
output_dir = 'path/to/dir'

for j in os.listdir(input_dir):
    image_path = os.path.join(input_dir, j)
    # load the binary mask and get its contours

    # Skip non-image files
    if not j.lower().endswith(('.jpg', '.jpeg', '.png')):
        print(f"Skipping non-image file: {image_path}")
        continue

    mask = cv2.imread(image_path, cv2.IMREAD_GRAYSCALE)
    # Check if the image was successfully loaded

    if mask is None:
        print(f"Error reading image: {image_path}")
        continue
    _, mask = cv2.threshold(mask, 1, 255, cv2.THRESH_BINARY)

    H, W = mask.shape
    contours, hierarchy = cv2.findContours(mask, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)

    # convert the contours to polygons
    polygons = []
    for cnt in contours:
        if cv2.contourArea(cnt) > 200:
            polygon = []
            for point in cnt:
                x, y = point[0]
                polygon.append(x / W)
                polygon.append(y / H)
            polygons.append(polygon)


    # print the polygons
    with open('{}.txt'.format(os.path.join(output_dir, j)[:-4]), 'w') as f:
        for polygon in polygons:
            for p_, p in enumerate(polygon):
                if p_ == len(polygon) - 1:
                    f.write('{}\n'.format(p))
                elif p_ == 0:
                    f.write('0 {} '.format(p))
                else:
                    f.write('{} '.format(p))

        f.close()
else:
        print("No contours found in {image_path}")

Train pre-trained weight

from ultralytics import YOLO
#train pre-trained model
model = YOLO("/path/to/best.pt")
#dataset path
model.train(data='path/to/config.yaml', imgsz=640, epochs=20, device='mps', patience=0)

South.txt

• Andrew K.

[glenn-jocher]

@dronespov Hi Andrew,

I'm thrilled to hear the advice helped! Your scripts look well-organized and appear to serve their intended purposes effectively. Thank you for the support, it's truly appreciated!

To optimize training time:

• Review Data Augmentation: Heavy augmentations can slow down training; ensure they're beneficial for your use case.
• Adjust Batch Size: If possible, increase the batch size to better utilize GPU resources.
• Prune the Model: Consider simplifying the network if high accuracy is maintained.
• Use Preprocessing: Ensure that dataset preprocessing does not become a bottleneck.
• Hardware Utilization: Check if your hardware is being fully utilized during training.

Reducing training time often involves a mix of tweaks to model configurations and efficient resource management. Focus on the most impactful changes to see significant reductions in epoch duration. Keep up the fantastic work!

• Glenn Jocher

[dronespov]

@glenn-jocher Turns out I included an additional directory by accident and that caused training time to x2-3. When I removed the directory, training time reduced.

Im trying to make my training and retraining process more efficient. Do you know if there's a way to

Predict & segment objects in one script then in a second script import the segmented results, isolate the segmented objects, white mask the object, and black out the undetected areas?

Currently the below script Predicts, Segments, Isolates, White & Black Masks in one script. Id like to separate that process into two scripts.

from pathlib import Path

import cv2 as cv
import numpy as np
from ultralytics import YOLO

# Load YOLO model
m = YOLO('path/to/best.pt')

# Set source directory
source = 'path/to/dir'

# Get YOLO predictions
res = m.predict(source, boxes=False, save=True, imgsz=640, conf=0.8)

# Iterate over detection results
for r in res:
    img = np.copy(r.orig_img)
    img_name = Path(r.path).stem

    # Create a black canvas for the final image
    final_image = np.zeros_like(img)

    # Iterate over each object contour
    for ci, c in enumerate(r):
        label = c.names[c.boxes.cls.tolist().pop()]

        # Create binary mask for the object
        object_mask = np.zeros(img.shape[:2], np.uint8)
        contour = c.masks.xy.pop().astype(np.int32).reshape(-1, 1, 2)
        _ = cv.drawContours(object_mask, [contour], -1, 255, cv.FILLED)

        # Create white mask over the object
        white_mask = np.ones_like(img) * 255
        white_over_object = cv.bitwise_and(white_mask, white_mask, mask=object_mask)

        # Apply the white mask over the object in the final image
        final_image = cv.bitwise_or(final_image, white_over_object)



        # Save the isolated object
        _ = cv.imwrite(f'/path/to/{img_name}.jpg', final_image)

• Andrew K.

[glenn-jocher]

@dronespov Hi Andrew,

Glad to hear you resolved the training time issue by adjusting the directory setup.

To streamline your workflow into two separate scripts:

1. The first script would run your YOLO predict and segmentation tasks, saving the intermediate results, such as segmentation masks and their metadata.

2. The second script would then load these intermediate results and proceed with the isolation, masking, and saving of the final processed images.

This separation allows you to handle the prediction and post-processing steps independently, potentially enabling more efficient use of computational resources.

Remember, for the first script, focus on generating and storing all the necessary data for further processing in a structured manner. The second script will then access this stored data to complete the masking and isolation tasks.

Keep refining your process!

• Glenn Jocher

[dronespov]

Hi @glenn-jocher Everything is running well, thanks for all your help.

I am now trying deploy the weight to sagemaker so when I upload an image to my website, it passes through s3,then lambda, then sagemaker, and predicted results back to s3. Do you know of any informative articles or videos that you'd recommend for building this flow?

[glenn-jocher]

@dronespov hi Andrew,

I'm delighted to hear your project is progressing smoothly.

For deploying a model on AWS SageMaker with the workflow you've described, I would suggest exploring the official AWS documentation and tutorials on SageMaker, Lambda, and S3 services. AWS also provides sample notebooks and code repositories that illustrate similar end-to-end processes.

Best of luck with your deployment!

• Glenn Jocher

[dronespov]

@glenn-jocher Do you have any documentation for adding a new class to a pre-trained model? Im trying to add a new class by specifying it in the annotation and config set but when I check the val mosaic, the images are being classified as cat(0)

for example im trying to add dog as the second class. In the labeling process I define dog as 1 then add the images and labels to the new training dataset. When I evaluate the val mosaic, dogs (1) are registering as cats (0)

config.yaml

path: 'path/to/dataset' # dataset root dir
train: data/images/train/ # train images (relative to 'path')
val: data/images/val/ # val images (relative to 'path')

Classes

names:
0: cat
1: dog

[glenn-jocher]

@dronespov hi there,

To add a new class to a pre-trained model, you'll need to update your dataset's annotation files to include the new class index and label, and then modify the config.yaml file to reflect the new class in the names section, as you've done. After these updates, you should retrain the model with the new dataset that includes examples of the new class.

If the new class is being misclassified, it's important to double-check that the annotations are correct and that the config.yaml file is properly set up with the correct class indices. Also, ensure that the new class has sufficient and varied examples in the training dataset for the model to learn effectively.

Retraining from scratch or fine-tuning the pre-trained model with the updated dataset should then recognize and classify the new class correctly.

• Glenn Jocher

[dronespov]

@glenn-jocher Is there any way to add a new class to a pre trained model without a retraining the entire model? Im thinking if it's possible to add a new class with associated dataset then continue retraining previous model to detect the new class.

• Andrew K.

[glenn-jocher]

@dronespov Hi Andrew,

Yes, you can add a new class to a pre-trained model without starting from scratch. You'll need to update your dataset and config.yaml to include the new class, then continue training your model with the expanded dataset. This process is known as incremental learning or fine-tuning.

• Glenn Jocher

[dronespov]

@glenn-jocher Understood, I've tried that approach but unsuccessful. For some reason, my new class id is being identified as a different class id. I did some research and found the annotation tool I use - CVAT - sets label id as default 1-9. I tried changing the ID # for the new class in their website but their website reverts the changed id # to the default id # ( e.g dog id 1 changed to 0 then their website reverts it back to 1). I submitted that as a bug to their team, hopefully will get that fixed soon.

• Andrew K.

[glenn-jocher]

@dronespov Hi Andrew,

It seems like the issue is with the annotation tool's class ID assignment. Good job on reporting the bug to the CVAT team. In the meantime, you might consider manually adjusting the class IDs in your annotation files to match your config.yaml. Once CVAT resolves the issue, you should be able to proceed with fine-tuning your model to recognize the new class.

• Glenn Jocher

[dronespov]

@glenn-jocher Manually changing the class-index helped, thanks for the tip. Do you know if there's a way to From CVAT, export the Segmented Masks and Convert the Segmentation Class to txt with the correct class index?

import os

import cv2

input_dir = 'path/to/SegmentationClass'
output_dir = 'path/to/dir'

for j in os.listdir(input_dir):
    image_path = os.path.join(input_dir, j)
    # load the binary mask and get its contours

    # Skip non-image files
    if not j.lower().endswith(('.jpg', '.jpeg', '.png')):
        print(f"Skipping non-image file: {image_path}")
        continue


    mask = cv2.imread(image_path, cv2.IMREAD_GRAYSCALE)
    # Check if the image was successfully loaded

    if mask is None:
        print(f"Error reading image: {image_path}")
        continue
    _, mask = cv2.threshold(mask, 1, 255, cv2.THRESH_BINARY)

    H, W = mask.shape
    contours, hierarchy = cv2.findContours(mask, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)

    # convert the contours to polygons
    polygons = []
    for cnt in contours:
        if cv2.contourArea(cnt) > 200:
            polygon = []
            for point in cnt:
                x, y = point[0]
                polygon.append(x / W)
                polygon.append(y / H)
            polygons.append(polygon)


    # print the polygons
    with open('{}.txt'.format(os.path.join(output_dir, j)[:-4]), 'w') as f:
        for polygon in polygons:
            for p_, p in enumerate(polygon):
                if p_ == len(polygon) - 1:
                    f.write('{}\n'.format(p))
                elif p_ == 0:
                    f.write('0 {} '.format(p))
                else:
                    f.write('{} '.format(p))

        f.close()
else:
        print("No contours found in {image_path}")

[glenn-jocher]

@dronespov Hi Andrew,

Glad to hear that manually adjusting the class index was helpful. Regarding your question, CVAT does allow you to export annotations in different formats, including masks. However, converting these masks to YOLO format .txt files with the correct class index isn't a built-in feature.

Your script approach for converting masks to .txt files seems like a good start. Ensure that the class index is correctly assigned in the output .txt files, matching the class indices in your config.yaml.

• Glenn Jocher

[dronespov]

@glenn-jocher when train my pretrained model on a new class, the resulted weight only detects the new class, and excludes the previously trained class. Do you have any idea why this is occurring?

• Andrew K.

[glenn-jocher]

@dronespov Hi Andrew,

This typically happens if the new training dataset only contains the new class. Ensure your dataset includes examples of both the new and previously trained classes. Also, verify that the config.yaml reflects all classes correctly.

• Glenn Jocher

[Mehreganbeiki]

Hi, Does anyone know how to save the segmentation prediction without the bounding box?

[dronespov]

Hi, Does anyone know how to save the segmentation prediction without the bounding box?

@Mehreganbeiki

Define boxes parameter as True or False in predict.

example

# Get YOLO predictions
res = m.predict(source, boxes=True, save=True, imgsz=640)

[dronespov]

Hi @glenn-jocher

I figured out how to add a new class to my pretrained model. The complication I'm facing now is when I train the new class, my old class is showing poor prediction results.

For example;

The new class is dog. My previous class (cat) has months of trained progress & is predicting cats well.

My classes are

0: Cat
1: Dog

When I train dog with 30 images & 300 epochs, dog prediction performs well. Although, cat performance on the new weight only registers 1 cat in a photo with 20 cats compared to 15 cats in a photo with 20 cats.

Conf = 0.1 on both prediction tests

Do you know why the cat prediction performance drop is occurring?

[glenn-jocher]

Hi @dronespov 👋,

It sounds like you're experiencing a common issue known as "catastrophic forgetting" during incremental learning. When you train on a new class with a pre-trained model, without including the original classes in the training set, the model may forget the previous knowledge.

To mitigate this, you should continue training with a dataset that includes both the new class (dog) and the previously trained class (cat). This way, the model maintains its ability to predict both classes effectively. Here's a quick example of how your training command might look:

yolo train data=combined_dataset.yaml model=pretrained_weights.pt epochs=300

Make sure combined_dataset.yaml includes examples of both cats and dogs. Keep an eye on the balance of the classes in your dataset to prevent bias towards one class.

Happy training! 🚀

[dronespov]

Hi @glenn-jocher ive tried training the new class with 30 images of Dog and 5 images of cats to ensure cats is maintained but still forgetting occurs. Should I train with more cat images? Is there documentation you could send on training new classes on pre-trained models?

• Andrew K.

[glenn-jocher]

@dronespov hi Andrew 👋,

Yes, increasing the number of cat images can help. It's important to have a balanced dataset for both classes to prevent forgetting. For training on new classes with pre-trained models, ensure that your dataset is representative of all classes you want the model to recognize.

While we don't have specific documentation on incremental learning, the general training guidelines apply. You might want to try techniques like mixed precision training or adjusting learning rates to fine-tune your model further.

Keep experimenting with the balance, and good luck! 🐾🐱

• Glenn Jocher

[dronespov]

@glenn-jocher lets say I have 20 classes and only want to fine tune 1 class out of the 20. How do I fine tune 1 class out of 20 while the other 19 keeping the same progress? Its a big issue that the other classes are catastrophically forgetting.

[glenn-jocher]

@dronespov hi there! 👋

Fine-tuning a single class out of many without affecting the others is indeed tricky due to catastrophic forgetting. One approach is to include a modest amount of data for the 19 classes you want to preserve, alongside a larger portion for the 1 class you're focusing on. This way, the model refreshes its memory on the 19 classes while learning more about the new class.

Here's a quick snippet on how you might adjust your training setup:

# Consider having a balanced dataset
data='dataset.yaml'  # Make sure this includes all 20 classes

# Adjust your training command
yolo train data=data.yaml model=pretrained.pt

Ensure your dataset.yaml has a good mix of all classes. Hope this helps! Keep experimenting and happy training! 🚀

[dronespov]

@glenn-jocher could you suggest other ways to counteract catastrophic forgetting? Maybe dividing classes into multiple weights?

• Andrew K.

[glenn-jocher]

@dronespov hi Andrew K. 👋,

Indeed, catastrophic forgetting is a challenge in continuous or incremental learning. Dividing classes into multiple weights, or "modular training," is a possibility. This involves training separate models for different groups of classes and combining their predictions at inference time. Another method is to employ "experience replay" where you periodically retrain the model on a subset of the old data along with the new data. This helps the model retain knowledge of previous classes.

Using Elastic Weight Consolidation (EWC) is another advanced technique that protects important weights of a model from significant changes during new training sessions.

Here's a quick snippet for experience replay:

# Mix old data with new class data for training sessions
dataset = mix_dataset(old_data, new_data)

Keep experimenting to find the best strategy for your specific use case. Happy training! 🚀

• Glenn

[dronespov]

@glenn-jocher Regarding adding new class and model catastrophically forgetting past training progress, I've. included nearly 50 images & videos of past trained data. Still, Catastrophically forgetting is occurring. It seems as if the model is now creating a new model instead of retraining. How do you suggest I solve this catastrophic forgetting bug?

• Andrew K.

[glenn-jocher]

@dronespov hi Andrew K. 👋,

It sounds like you're facing a common challenge with catastrophic forgetting. A good approach is to ensure a balanced dataset with both old and new classes during the retraining phase. Also, consider using a lower learning rate to fine-tune the model gently without overriding previous knowledge. Here's an example snippet:

yolo train data=your_dataset.yaml model=pretrained_weights.pt epochs=300 lr0=0.001  # Adjust learning rate as needed

Remember, consistency and a bit of trial and error go a long way! Keep experimenting, and you'll get there. 🚀

• Glenn

[dronespov]

@glenn-jocher thank you I'll try this approach. I'm curious, why hasn't ultralytics team solved catastrophic forgetting and provided a solution in either the model or documentation?

[glenn-jocher]

Hi @dronespov 👋,

Great question! Catastrophic forgetting is a complex issue inherent to neural networks, especially as they learn sequentially. We're continuously exploring ways to improve this, but a one-size-fits-all solution is challenging due to varied use cases and dataset specifics.

We appreciate community input like yours as it guides prioritization and development. Keep sharing your feedback and experiences!

Happy experimenting! 🚀

• The Ultralytics Team