Here we provided some common troubles faced by the users and their corresponding solutions here. If the contents here do not cover your issue, please create an issue about it.
We've already opened an issue about FAQs, please refer to Frequently Asked Questions for more details.
ImportError: Cannot import 'detrex._C', therefore 'MultiscaleDeformableAttention' is not available
detrex need CUDA runtime to build the MultiScaleDeformableAttention
operator. In most cases, users do not need to specify this environment variable if you have installed cuda correctly. The default path of CUDA runtime is usr/local/cuda
. If you find your CUDA_HOME
is None
. You may solve it as follows:
- If you've already installed CUDA runtime in your environments, specify the environment variable (here we take cuda-11.3 as an example):
export CUDA_HOME=/path/to/cuda-11.3/
- If you do not find the CUDA runtime in your environments, consider install it following the CUDA Toolkit Installation to install CUDA. Then specify the environment variable
CUDA_HOME
. - After setting
CUDA_HOME
, rebuild detrex again by runningpip install -e .
You can also refer to these issues for more details: #98, #85.
assert (boxes1[:, 2:] >= boxes1[:, :2]).all()" in `generalized_box_iou`
This means the model produces illegal box predictions. You may solute this issue as follows:
1. Check the learning rate which should not be too large. The DETR-like models are usually trained using AdamW
with lr=1e-4
.
2. Make sure that your model are initilized correctly. Please check the init_weights()
function in models.
How to not filter empty annotations during training?
There're few ways for you to not filter empty annotations during training.
- modify configs in configs/common/data/coco_detr.py as follows:
dataloader.train = L(build_detection_train_loader)(
dataset=L(get_detection_dataset_dicts)(names="coco_2017_train", filter_empty=False),
...,
)
- modify configs in projects as dino_r50_4scale_24ep.py.
# your config.py
dataloader = get_config("common/data/coco_detr.py").dataloader
# modify dataloader config
# not filter empty annotations during training
dataloader.train.dataset.filter_empty = False
- modify your training scripts to override the config.
cd detrex
python tools/train_net.py --config-file projects/dino/configs/path/to/config.py --num-gpus 8 dataloader.train.dataset.filter_empy=False
You can also refer to these issues for more details: #issue78-comments
RuntimeError: The server socket has failed to listen on any local network address. The server socket has failed to bind to [::]:54980 (errno: 98 - Address already in use).
This means that the process you started earlier did not exit correctly, there's two solutions:
- kill the process you started before totally
- change the running port by setting
--dist-url
python tools/train_net.py \
--config-file path/to/config.py \
--num-gpus 8 \
--dist-url tcp://127.0.0.1:12345 \
How to inference DINO on CPU?
Please refer to this PR #157 for more details
How to train the custom coco-like format dataset?
Please refer to this PR #186 for more details