[Question] How to do validation with custom dataloader

[ternaus]

Search before asking

• I have searched the YOLOv8 issues and discussions and found no similar questions.

Question

Code:

from pathlib import Path

import cv2
import torch
import albumentations as A
from torch.utils.data import Dataset, DataLoader
from albumentations.pytorch.transforms import ToTensorV2
from pycocotools.coco import COCO
from PIL import Image
import numpy as np
from ultralytics.models import YOLO
from ultralytics.models.yolo.detect.val import DetectionValidator


class COCODataset(Dataset):
    def __init__(self, annotation_file: str, img_dir: str, transforms: A.Compose):
        super().__init__()
        self.coco = COCO(annotation_file)
        self.img_dir = Path(img_dir)
        self.transforms = transforms
        self.image_ids = self.coco.getImgIds()

    def __getitem__(self, idx: int) -> dict:
        # Retrieve image ID and metadata
        img_id = self.image_ids[idx]
        img_meta = self.coco.loadImgs(img_id)[0]
        img_path = self.img_dir / img_meta["file_name"]

        # Load image
        image = Image.open(img_path).convert("RGB")
        image = np.array(image)

        # Retrieve annotations
        ann_ids = self.coco.getAnnIds(imgIds=img_id)
        annotations = self.coco.loadAnns(ann_ids)

        # Extract bounding boxes and labels
        boxes = []
        labels = []
        for ann in annotations:
            x, y, w, h = ann["bbox"]
            boxes.append([x, y, w, h])
            labels.append(ann["category_id"])

        # Apply transformations
        if self.transforms:
            augmented = self.transforms(image=image, bboxes=boxes, category_id=labels)
            image = augmented["image"]
            boxes = augmented["bboxes"]
            labels = augmented["category_id"]

        # Normalize boxes [x_center, y_center, width, height, class]
        h, w = image.shape[:2]
        boxes = torch.tensor([
            [(b[0] + b[2] / 2) / w, (b[1] + b[3] / 2) / h, b[2] / w, b[3] / h, l]
            for b, l in zip(boxes, labels)
        ], dtype=torch.float32)

        # Return the processed data
        return {"img": image, "labels": boxes}

    def __len__(self) -> int:
        return len(self.image_ids)


size = 640

# Define transformations
transforms = A.Compose([
    A.LongestMaxSize(max_size=size, p=1),
    A.PadIfNeeded(min_height=size, min_width=size, p=1, position="center", border_mode=cv2.BORDER_CONSTANT, value=(114, 114, 114)),
    A.Normalize(mean=(0, 0, 0), std=(1, 1, 1), p=1),
    ToTensorV2()
], bbox_params=A.BboxParams(format="coco", label_fields=["category_id"]))

# Initialize dataset
dataset = COCODataset(
    annotation_file="datasets/coco/annotations/instances_val2017.json",
    img_dir="datasets/coco/images/val2017",
    transforms=transforms
)

model = YOLO("yolov8n.pt")

val_dataloader = DataLoader(dataset, batch_size=64, shuffle=False, num_workers=16, drop_last=False)

args = dict(model="yolov8n.pt")


class CustomValidator(DetectionValidator):
    def get_dataloader(self) -> DataLoader:
        return val_dataloader


validator = CustomValidator(args)
validator(model=model.model)

Error:

(ultralitics_experiments) (base) ➜  ultralitics_experiments git:(make_detection_work) ✗ python -m src.validation
loading annotations into memory...
Done (t=0.57s)
creating index...
index created!
Ultralytics YOLOv8.2.10 🚀 Python-3.10.14 torch-2.3.0+cu121 CUDA:0 (NVIDIA GeForce RTX 4090, 24214MiB)
YOLOv8n summary (fused): 168 layers, 3151904 parameters, 0 gradients, 8.7 GFLOPs
Traceback (most recent call last):
  File "/home/vladimir/anaconda3/envs/ultralitics_experiments/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/home/vladimir/anaconda3/envs/ultralitics_experiments/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/mnt/evo850/workspace/ultralitics_experiments/src/validation.py", line 93, in <module>
    validator(model=model.model)
  File "/home/vladimir/anaconda3/envs/ultralitics_experiments/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/mnt/evo850/workspace/ultralitics_experiments/ultralytics/engine/validator.py", line 147, in __call__
    raise FileNotFoundError(emojis(f"Dataset '{self.args.data}' for task={self.args.task} not found ❌"))
FileNotFoundError: Dataset 'None' for task=detect not found ❌

Additional

No response

[github-actions]

👋 Hello @ternaus, thank you for your interest in Ultralytics YOLOv8 🚀! We recommend a visit to the Docs for new users where you can find many Python and CLI usage examples and where many of the most common questions may already be answered.

If this is a 🐛 Bug Report, please provide a minimum reproducible example to help us debug it.

If this is a custom training ❓ Question, please provide as much information as possible, including dataset image examples and training logs, and verify you are following our Tips for Best Training Results.

Join the vibrant Ultralytics Discord 🎧 community for real-time conversations and collaborations. This platform offers a perfect space to inquire, showcase your work, and connect with fellow Ultralytics users.

Install

Pip install the ultralytics package including all requirements in a Python>=3.8 environment with PyTorch>=1.8.

pip install ultralytics

Environments

YOLOv8 may be run in any of the following up-to-date verified environments (with all dependencies including CUDA/CUDNN, Python and PyTorch preinstalled):

• Notebooks with free GPU:
• Google Cloud Deep Learning VM. See GCP Quickstart Guide
• Amazon Deep Learning AMI. See AWS Quickstart Guide
• Docker Image. See Docker Quickstart Guide

Status

If this badge is green, all Ultralytics CI tests are currently passing. CI tests verify correct operation of all YOLOv8 Modes and Tasks on macOS, Windows, and Ubuntu every 24 hours and on every commit.

[glenn-jocher]

@ternaus it looks like the issue is related to the data argument not being set in your args dictionary, which is required by the DetectionValidator to locate the dataset configuration. You can fix this by specifying the data argument with the path to your dataset configuration file in the args dictionary. Here's how you can adjust your code:

args = dict(model="yolov8n.pt", data="path/to/your/dataset_config.yaml")

Make sure to replace "path/to/your/dataset_config.yaml" with the actual path to your dataset configuration file. This file should specify details about your dataset such as path, classes, etc.

If you don't have a dataset configuration file, you'll need to create one according to the YOLOv8 dataset format. This typically includes paths to your training and validation images and annotations, as well as class names and other metadata.

After making this adjustment, your custom validator should be able to locate and load the dataset correctly.