[BUG] Issue title...gt_is_difficult_list[gt_class_labels == i]) IndexError: boolean index did not match indexed array along dimension 0; dimension is 0 but corresponding boolean dimension is 1

[Ekta246]

Describe the bug
A clear and concise description of what the bug is.
While evaluating the dataset on efficientdet-d0 with Pascal VOC Tfm evaluator with 1 class and I have a dataloader with 1 class indexed class label.
To Reproduce
Steps to reproduce the behavior:
1.Possibility to make it 0-indexed?
2. Add background class as given in the VOCParser in the parsers in the repo.

Expected behavior
A clear and concise description of what you expected to happen.
Since, I have a annotation .json file which only includes the bounding box coordinates and the respective filename and that I don't have any COCO format annotations. I am not able to evaluate my dataset on the COCOapi.
Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

• OS: [e.g. Windows, Ubuntu 18.04]
• This repository version [e.g. pypi 0.3.1 or branch/commit ref]
• PyTorch version w/ CUDA/cuDNN [e.g. from conda list, 1.7.0 py3.8_cuda11.0.221_cudnn8.0.3_0]

Additional context
Add any other context about the problem here.

[rwightman]

One class works perfectly fine with the TFM eval code, I've tested it. Background class is 0, with one class, your foreground class from the dataset must be 1, the model has '1' class (background class not included in the count). That is it, just like all of the example datasets.

[Ekta246]

Yes, I have not counted background class. Also, I have not included background class in count.
But I have not given the background flag in my code either.

Sorry to label it as bug, but do you got any idea why I get the error even if there is only 1 class with 1 index and 1 count of classes