guides/kfold-cross-validation/

[giscus[bot]]

guides/kfold-cross-validation/

An in-depth guide demonstrating the implementation of K-Fold Cross Validation with the Ultralytics ecosystem for object detection datasets, leveraging Python, YOLO, and sklearn.

https://docs.ultralytics.com/guides/kfold-cross-validation/

[Guan9Wu]

In the 5th step, there is a bug. As we all known, 'labels' is a generator, it can't be resued. so in the step, 'labels' is empty when runing code for label in labels.

[pderrenger]

@WuG978 thanks for pointing out the issue with the labels generator being reused, which indeed would be empty on the second iteration. A simple fix is to convert the generator to a list right after creation, ensuring it can be reused across different parts of the code. Here's how you can adjust it:

labels = list(sorted(dataset_path.rglob("*labels/*.txt")))

This change ensures that labels retains all its entries for future use. Let me know if this helps or if there's anything else you need! 😊

[Guan9Wu]

Thank for your reply, I got it.

[pderrenger]

Great to hear you got it! If you have any more questions or need further assistance as you work through the K-Fold Cross Validation or any other process, feel free to reach out. Happy coding! 😊🚀

[fazrigading]

Is there any way to implement Stratified K-Fold Cross Validation with YOLOv8?

[pderrenger]

Yes, you can implement Stratified K-Fold Cross Validation with YOLOv8! Stratified K-Fold ensures that each fold has a similar distribution of class labels, which can be particularly useful for imbalanced datasets.

To achieve this, you can use StratifiedKFold from sklearn.model_selection instead of KFold. Here's a brief example of how to modify the K-Fold implementation:

from sklearn.model_selection import StratifiedKFold

# Assuming labels_df is your DataFrame with class counts
ksplit = 5
skf = StratifiedKFold(n_splits=ksplit, shuffle=True, random_state=20)

# Generate stratified splits
kfolds = list(skf.split(labels_df, labels_df.idxmax(axis=1)))

# The rest of the process remains the same

For a detailed guide on K-Fold Cross Validation, please refer to the Ultralytics documentation. If you have any more questions, feel free to ask! 😊

[alexander-sony]

Good guide, consider adding it in a colab notebook.

In section "Train YOLO using K-Fold Data Splits", model should be re-initialized foe every k, i.e. it should be inside the for-loop: for k in range(ksplit).

[glenn-jocher]

@alexander-sony thank you for your feedback and suggestion! 😊

You're absolutely right about re-initializing the model within the loop for each fold to ensure a fresh start for every training iteration. This can help avoid any potential carryover effects from previous folds. Here's the corrected code snippet for the training section:

from ultralytics import YOLO

weights_path = "path/to/weights.pt"
results = {}

# Define your additional arguments here
batch = 16
project = "kfold_demo"
epochs = 100

for k in range(ksplit):
    model = YOLO(weights_path, task="detect")  # Re-initialize the model inside the loop
    dataset_yaml = ds_yamls[k]
    model.train(data=dataset_yaml, epochs=epochs, batch=batch, project=project)  # include any train arguments
    results[k] = model.metrics  # save output metrics for further analysis

Regarding your suggestion to add this guide to a Colab notebook, that's a fantastic idea! It would make the process more interactive and accessible for users. We'll consider creating a Colab version of this guide in the future.

For more details on K-Fold Cross Validation, you can refer to our comprehensive guide.

Thanks again for your valuable input! If you have any more suggestions or questions, feel free to share. Happy coding! 🚀