Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

代码里的faster_rcnn.py运行失败 #202

Closed
cwudis opened this issue Jun 17, 2024 · 8 comments
Closed

代码里的faster_rcnn.py运行失败 #202

cwudis opened this issue Jun 17, 2024 · 8 comments
Assignees
Labels
bug Something isn't working

Comments

@cwudis
Copy link

cwudis commented Jun 17, 2024

欢迎您反馈PaddleRS使用问题。辛苦您提供以下信息,以方便我们快速定位和解决问题:

  1. PaddleRS版本: PaddleRS develop
  2. PaddlePaddle版本: PaddlePaddle 2.6.1
  3. 操作系统信息:Windows
  4. Python版本号:3.8
  5. CUDA/cuDNN版本:CUDA12.0/cuDNN 8.9
  6. 完整的代码:原训练脚本faster_rcnn.py
  7. 详细的错误信息与相关log:
    Traceback (most recent call last):
    File "tutorials/train/object_detection/faster_rcnn.py", line 75, in
    model.train(
    File "E:\cwudis\software\anaconda3\envs\py3810\lib\site-packages\paddlers\tasks\object_detector.py", line 350, in train
    return self._real_train(**args)
    File "E:\cwudis\software\anaconda3\envs\py3810\lib\site-packages\paddlers\tasks\object_detector.py", line 454, in _real_train
    self.train_loop(
    File "E:\cwudis\software\anaconda3\envs\py3810\lib\site-packages\paddlers\tasks\base.py", line 396, in train_loop
    outputs = self.train_step(step, data, net, optimizer)
    File "E:\cwudis\software\anaconda3\envs\py3810\lib\site-packages\paddlers\tasks\base.py", line 705, in train_step
    outputs = self.run(net, data, mode='train')
    File "E:\cwudis\software\anaconda3\envs\py3810\lib\site-packages\paddlers\tasks\object_detector.py", line 157, in run
    net_out = net(inputs)
    File "E:\cwudis\software\anaconda3\envs\py3810\lib\site-packages\paddle\nn\layer\layers.py", line 1429, in call
    return self.forward(*inputs, **kwargs)
    File "E:\cwudis\software\anaconda3\envs\py3810\lib\site-packages\paddlers\models\ppdet\modeling\architectures\meta_arch.py", line 60, in forward
    out = self.get_loss()
    File "E:\cwudis\software\anaconda3\envs\py3810\lib\site-packages\paddlers\models\ppdet\modeling\architectures\faster_rcnn.py", line 112, in get_loss
    rpn_loss, bbox_loss = self._forward()
    File "E:\cwudis\software\anaconda3\envs\py3810\lib\site-packages\paddlers\models\ppdet\modeling\architectures\faster_rcnn.py", line 80, in _forward
    rois, rois_num, rpn_loss = self.rpn_head(body_feats, self.inputs)
    File "E:\cwudis\software\anaconda3\envs\py3810\lib\site-packages\paddle\nn\layer\layers.py", line 1429, in call
    return self.forward(*inputs, **kwargs)
    File "E:\cwudis\software\anaconda3\envs\py3810\lib\site-packages\paddlers\models\ppdet\modeling\proposal_generator\rpn_head.py", line 141, in forward
    rois, rois_num = self._gen_proposal(scores, deltas, anchors, inputs)
    File "E:\cwudis\software\anaconda3\envs\py3810\lib\site-packages\paddlers\models\ppdet\modeling\proposal_generator\rpn_head.py", line 234, in _gen_proposal
    bs_rois_num_collect = paddle.concat(bs_rois_num_collect)
    File "E:\cwudis\software\anaconda3\envs\py3810\lib\site-packages\paddle\tensor\manipulation.py", line 1263, in concat
    return _C_ops.concat(input, axis)
    ValueError: (InvalidArgument) The axis is expected to be in range of [0, 0), but got 0
    [Hint: Expected axis >= -rank && axis < rank == true, but received axis >= -rank && axis < rank:0 != true:1.] (at ..\paddle\phi\infermeta\multiary.cc:1035)
    运行其他模型没问题
@cwudis cwudis added the bug Something isn't working label Jun 17, 2024
@github-actions github-actions bot added triage new issue/PR waiting to be dealed and removed triage new issue/PR waiting to be dealed labels Jun 17, 2024
@cwudis cwudis changed the title [Bug] [Bug] 代码里的faster_rcnn.py运行失败 Jun 17, 2024
@cwudis cwudis changed the title [Bug] 代码里的faster_rcnn.py运行失败 代码里的faster_rcnn.py运行失败 Jun 17, 2024
@Bobholamovic
Copy link
Member

你好,建议在E:\cwudis\software\anaconda3\envs\py3810\lib\site-packages\paddlers\models\ppdet\modeling\proposal_generator\rpn_head.py的第234行添加断点,检查bs_rois_num_collect中各张量的尺寸。

@cwudis
Copy link
Author

cwudis commented Jun 17, 2024

image

@Bobholamovic
Copy link
Member

这有可能是一个bug。

目前可以尝试在把E:\cwudis\software\anaconda3\envs\py3810\lib\site-packages\paddlers\models\ppdet\modeling\proposal_generator\rpn_head.py的第234行从

bs_rois_num_collect = paddle.concat(bs_rois_num_collect)

改成:

if bs_rois_num_collect[0].ndim >= 1:
    bs_rois_num_collect = paddle.concat(bs_rois_num_collect)
else:
    bs_rois_num_collect = paddle.stack(bs_rois_num_collect)

@cwudis
Copy link
Author

cwudis commented Jun 17, 2024

改后,报新的错误
Can not use conditional_random_field. Please install pydensecrf first.
Warning: import ppdet from source directory without installing, run 'python setup.py install' to install ppdet firstly
2024-06-17 16:18:07,737-WARNING: post-quant-hpo is not support in system other than linux
2024-06-17 16:18:07 [INFO] Decompressing ./data/sarship.zip...
2024-06-17 16:18:11 [DEBUG] ./data/sarship.zip decompressed.
2024-06-17 16:18:11 [INFO] Starting to read file list from dataset...
2024-06-17 16:18:12 [INFO] 175 samples in file ./data/sarship/train.txt, including 175 positive samples and 0 negative samples.
creating index...
index created!
2024-06-17 16:18:12 [INFO] Starting to read file list from dataset...
2024-06-17 16:18:12 [INFO] 5 samples in file ./data/sarship/eval.txt, including 5 positive samples and 0 negative samples.
creating index...
index created!
W0617 16:18:13.132372 8608 gpu_resources.cc:119] Please NOTE: device: 0, GPU Compute Capability: 8.9, Driver API Version: 12.5, Runtime API Version: 12.0
W0617 16:18:13.139367 8608 gpu_resources.cc:164] device: 0, cuDNN Version: 8.9.
2024-06-17 16:18:13 [WARNING] The initial batch_transforms will be overwritten.
2024-06-17 16:18:13 [INFO] Loading pretrained model from ./output/faster_rcnn/pretrain\faster_rcnn_r50_fpn_2x_coco.pdparams
2024-06-17 16:18:14 [WARNING] [SKIP] Shape of parameters bbox_head.bbox_score.weight do not match. (pretrained: [1024, 81] vs actual: [1024, 2])
2024-06-17 16:18:14 [WARNING] [SKIP] Shape of parameters bbox_head.bbox_score.bias do not match. (pretrained: [81] vs actual: [2])
2024-06-17 16:18:14 [WARNING] [SKIP] Shape of parameters bbox_head.bbox_delta.weight do not match. (pretrained: [1024, 320] vs actual: [1024, 4])
2024-06-17 16:18:14 [WARNING] [SKIP] Shape of parameters bbox_head.bbox_delta.bias do not match. (pretrained: [320] vs actual: [4])
2024-06-17 16:18:14 [INFO] There are 291/295 variables loaded into FasterRCNN.
Traceback (most recent call last):
File "tutorials/train/object_detection/faster_rcnn.py", line 75, in
model.train(
File "E:\cwudis\software\anaconda3\envs\py3810\lib\site-packages\paddlers\tasks\object_detector.py", line 350, in train
return self._real_train(**args)
File "E:\cwudis\software\anaconda3\envs\py3810\lib\site-packages\paddlers\tasks\object_detector.py", line 454, in _real_train
self.train_loop(
File "E:\cwudis\software\anaconda3\envs\py3810\lib\site-packages\paddlers\tasks\base.py", line 396, in train_loop
outputs = self.train_step(step, data, net, optimizer)
File "E:\cwudis\software\anaconda3\envs\py3810\lib\site-packages\paddlers\tasks\base.py", line 705, in train_step
outputs = self.run(net, data, mode='train')
File "E:\cwudis\software\anaconda3\envs\py3810\lib\site-packages\paddlers\tasks\object_detector.py", line 157, in run
net_out = net(inputs)
File "E:\cwudis\software\anaconda3\envs\py3810\lib\site-packages\paddle\nn\layer\layers.py", line 1429, in call
return self.forward(*inputs, **kwargs)
File "E:\cwudis\software\anaconda3\envs\py3810\lib\site-packages\paddlers\models\ppdet\modeling\architectures\meta_arch.py", line 60, in forward
out = self.get_loss()
File "E:\cwudis\software\anaconda3\envs\py3810\lib\site-packages\paddlers\models\ppdet\modeling\architectures\faster_rcnn.py", line 112, in get_loss
rpn_loss, bbox_loss = self._forward()
File "E:\cwudis\software\anaconda3\envs\py3810\lib\site-packages\paddlers\models\ppdet\modeling\architectures\faster_rcnn.py", line 81, in _forward
bbox_loss, _ = self.bbox_head(body_feats, rois, rois_num,
File "E:\cwudis\software\anaconda3\envs\py3810\lib\site-packages\paddle\nn\layer\layers.py", line 1429, in call
return self.forward(*inputs, **kwargs)
File "E:\cwudis\software\anaconda3\envs\py3810\lib\site-packages\paddlers\models\ppdet\modeling\heads\bbox_head.py", line 267, in forward
rois, rois_num, targets = self.bbox_assigner(rois, rois_num, inputs)
File "E:\cwudis\software\anaconda3\envs\py3810\lib\site-packages\paddlers\models\ppdet\modeling\proposal_generator\target_layer.py", line 166, in call
outs = generate_proposal_target(
File "E:\cwudis\software\anaconda3\envs\py3810\lib\site-packages\paddlers\models\ppdet\modeling\proposal_generator\target.py", line 241, in generate_proposal_target
new_rois_num = paddle.concat(new_rois_num)
File "E:\cwudis\software\anaconda3\envs\py3810\lib\site-packages\paddle\tensor\manipulation.py", line 1263, in concat
return _C_ops.concat(input, axis)
ValueError: (InvalidArgument) The axis is expected to be in range of [0, 0), but got 0
[Hint: Expected axis >= -rank && axis < rank == true, but received axis >= -rank && axis < rank:0 != true:1.] (at ..\paddle\phi\infermeta\multiary.cc:1035)

@Bobholamovic
Copy link
Member

Bobholamovic commented Jun 17, 2024

看起来是类似的错误,可以尝试对E:\cwudis\software\anaconda3\envs\py3810\lib\site-packages\paddlers\models\ppdet\modeling\proposal_generator\target.py的第241行进行修改,使用类似的方法解决。

@cwudis
Copy link
Author

cwudis commented Jun 18, 2024

嗯,改完后,可以运行了,但训练不了几轮就报类似的错误,只不过出错位置不同
分别是:
Can not use conditional_random_field. Please install pydensecrf first.
Warning: import ppdet from source directory without installing, run 'python setup.py install' to install ppdet firstly
2024-06-18 11:46:06,188-WARNING: post-quant-hpo is not support in system other than linux
2024-06-18 11:46:06 [INFO] Decompressing ./data/sarship.zip...
2024-06-18 11:46:10 [DEBUG] ./data/sarship.zip decompressed.
2024-06-18 11:46:10 [INFO] Starting to read file list from dataset...
2024-06-18 11:46:11 [INFO] 175 samples in file ./data/sarship/train.txt, including 175 positive samples and 0 negative samples.
creating index...
index created!
2024-06-18 11:46:11 [INFO] Starting to read file list from dataset...
2024-06-18 11:46:11 [INFO] 5 samples in file ./data/sarship/eval.txt, including 5 positive samples and 0 negative samples.
creating index...
index created!
W0618 11:46:12.038090 17148 gpu_resources.cc:119] Please NOTE: device: 0, GPU Compute Capability: 8.9, Driver API Version: 12.5, Runtime API Version: 12.0
W0618 11:46:12.045085 17148 gpu_resources.cc:164] device: 0, cuDNN Version: 8.9.
2024-06-18 11:46:12 [WARNING] The initial batch_transforms will be overwritten.
2024-06-18 11:46:12 [INFO] Loading pretrained model from ./output/faster_rcnn/pretrain\faster_rcnn_r50_fpn_2x_coco.pdparams
2024-06-18 11:46:13 [WARNING] [SKIP] Shape of parameters bbox_head.bbox_score.weight do not match. (pretrained: [1024, 81] vs actual: [1024, 2])
2024-06-18 11:46:13 [WARNING] [SKIP] Shape of parameters bbox_head.bbox_score.bias do not match. (pretrained: [81] vs actual: [2])
2024-06-18 11:46:13 [WARNING] [SKIP] Shape of parameters bbox_head.bbox_delta.weight do not match. (pretrained: [1024, 320] vs actual: [1024, 4])
2024-06-18 11:46:13 [WARNING] [SKIP] Shape of parameters bbox_head.bbox_delta.bias do not match. (pretrained: [320] vs actual: [4])
2024-06-18 11:46:13 [INFO] There are 291/295 variables loaded into FasterRCNN.
2024-06-18 11:46:35 [INFO] [TRAIN] Epoch=1/10, Step=4/43, loss_rpn_cls=0.306890, loss_rpn_reg=0.725881, loss_bbox_cls=0.375581, loss_bbox_reg=0.163465, loss=1.571817, lr=0.005000, time_each_step=5.33s, eta=0:38:2
2024-06-18 11:46:55 [INFO] [TRAIN] Epoch=1/10, Step=8/43, loss_rpn_cls=0.393064, loss_rpn_reg=0.544364, loss_bbox_cls=0.178163, loss_bbox_reg=0.115977, loss=1.231568, lr=0.005000, time_each_step=5.1s, eta=0:36:2
2024-06-18 11:47:16 [INFO] [TRAIN] Epoch=1/10, Step=12/43, loss_rpn_cls=0.246939, loss_rpn_reg=0.390652, loss_bbox_cls=0.234597, loss_bbox_reg=0.231559, loss=1.103747, lr=0.005000, time_each_step=5.29s, eta=0:37:0
2024-06-18 11:47:38 [INFO] [TRAIN] Epoch=1/10, Step=16/43, loss_rpn_cls=0.168808, loss_rpn_reg=0.383643, loss_bbox_cls=0.244856, loss_bbox_reg=0.240769, loss=1.038077, lr=0.005000, time_each_step=5.5s, eta=0:38:8
2024-06-18 11:47:59 [INFO] [TRAIN] Epoch=1/10, Step=20/43, loss_rpn_cls=0.626304, loss_rpn_reg=0.586001, loss_bbox_cls=0.588068, loss_bbox_reg=0.649747, loss=2.450120, lr=0.005000, time_each_step=5.2s, eta=0:35:43
2024-06-18 11:48:19 [INFO] [TRAIN] Epoch=1/10, Step=24/43, loss_rpn_cls=0.257486, loss_rpn_reg=0.390526, loss_bbox_cls=0.201990, loss_bbox_reg=0.291991, loss=1.141993, lr=0.005000, time_each_step=5.02s, eta=0:34:9
2024-06-18 11:48:39 [INFO] [TRAIN] Epoch=1/10, Step=28/43, loss_rpn_cls=0.244560, loss_rpn_reg=0.372828, loss_bbox_cls=0.268240, loss_bbox_reg=0.297336, loss=1.182964, lr=0.005000, time_each_step=4.94s, eta=0:33:16
2024-06-18 11:49:00 [INFO] [TRAIN] Epoch=1/10, Step=32/43, loss_rpn_cls=0.446006, loss_rpn_reg=0.520212, loss_bbox_cls=0.286503, loss_bbox_reg=0.331737, loss=1.584458, lr=0.005000, time_each_step=5.14s, eta=0:34:14
2024-06-18 11:49:21 [INFO] [TRAIN] Epoch=1/10, Step=36/43, loss_rpn_cls=0.259678, loss_rpn_reg=0.430123, loss_bbox_cls=0.117319, loss_bbox_reg=0.106297, loss=0.913417, lr=0.005000, time_each_step=5.31s, eta=0:35:2
2024-06-18 11:49:42 [INFO] [TRAIN] Epoch=1/10, Step=40/43, loss_rpn_cls=0.153984, loss_rpn_reg=0.413964, loss_bbox_cls=0.138056, loss_bbox_reg=0.192680, loss=0.898685, lr=0.005000, time_each_step=5.26s, eta=0:34:23
2024-06-18 11:49:57 [INFO] [TRAIN] Epoch 1 finished, loss_rpn_cls=0.3493410130572874, loss_rpn_reg=0.4661950642286345, loss_bbox_cls=0.2512764199528583, loss_bbox_reg=0.25913346593463144, loss=1.3259459692378377 .
2024-06-18 11:50:03 [INFO] [TRAIN] Epoch=2/10, Step=1/43, loss_rpn_cls=0.455122, loss_rpn_reg=0.452868, loss_bbox_cls=0.131628, loss_bbox_reg=0.190294, loss=1.229912, lr=0.005000, time_each_step=5.36s, eta=0:34:40
2024-06-18 11:50:25 [INFO] [TRAIN] Epoch=2/10, Step=5/43, loss_rpn_cls=0.189669, loss_rpn_reg=0.528992, loss_bbox_cls=0.200498, loss_bbox_reg=0.215080, loss=1.134238, lr=0.005000, time_each_step=5.37s, eta=0:34:20
2024-06-18 11:50:45 [INFO] [TRAIN] Epoch=2/10, Step=9/43, loss_rpn_cls=0.230574, loss_rpn_reg=0.461590, loss_bbox_cls=0.322907, loss_bbox_reg=0.529938, loss=1.545010, lr=0.005000, time_each_step=5.07s, eta=0:32:7
2024-06-18 11:51:06 [INFO] [TRAIN] Epoch=2/10, Step=13/43, loss_rpn_cls=0.307745, loss_rpn_reg=0.373867, loss_bbox_cls=0.213243, loss_bbox_reg=0.360049, loss=1.254905, lr=0.005000, time_each_step=5.26s, eta=0:32:59
2024-06-18 11:51:27 [INFO] [TRAIN] Epoch=2/10, Step=17/43, loss_rpn_cls=0.179184, loss_rpn_reg=0.222601, loss_bbox_cls=0.190243, loss_bbox_reg=0.339382, loss=0.931410, lr=0.005000, time_each_step=5.31s, eta=0:32:56
2024-06-18 11:51:47 [INFO] [TRAIN] Epoch=2/10, Step=21/43, loss_rpn_cls=0.260561, loss_rpn_reg=0.402224, loss_bbox_cls=0.217972, loss_bbox_reg=0.383740, loss=1.264497, lr=0.005000, time_each_step=4.86s, eta=0:29:49
2024-06-18 11:52:08 [INFO] [TRAIN] Epoch=2/10, Step=25/43, loss_rpn_cls=0.195323, loss_rpn_reg=0.383074, loss_bbox_cls=0.196630, loss_bbox_reg=0.331674, loss=1.106701, lr=0.005000, time_each_step=5.34s, eta=0:32:23
2024-06-18 11:52:28 [INFO] [TRAIN] Epoch=2/10, Step=29/43, loss_rpn_cls=0.125539, loss_rpn_reg=0.265280, loss_bbox_cls=0.186745, loss_bbox_reg=0.291602, loss=0.869166, lr=0.005000, time_each_step=4.85s, eta=0:29:7
2024-06-18 11:52:49 [INFO] [TRAIN] Epoch=2/10, Step=33/43, loss_rpn_cls=0.271513, loss_rpn_reg=0.531270, loss_bbox_cls=0.201685, loss_bbox_reg=0.338219, loss=1.342686, lr=0.005000, time_each_step=5.31s, eta=0:31:31
2024-06-18 11:53:10 [INFO] [TRAIN] Epoch=2/10, Step=37/43, loss_rpn_cls=0.220014, loss_rpn_reg=0.404535, loss_bbox_cls=0.212469, loss_bbox_reg=0.372014, loss=1.209032, lr=0.005000, time_each_step=5.35s, eta=0:31:21
2024-06-18 11:53:31 [INFO] [TRAIN] Epoch=2/10, Step=41/43, loss_rpn_cls=0.180509, loss_rpn_reg=0.373099, loss_bbox_cls=0.190887, loss_bbox_reg=0.308534, loss=1.053029, lr=0.005000, time_each_step=5.22s, eta=0:30:15
2024-06-18 11:53:41 [INFO] [TRAIN] Epoch 2 finished, loss_rpn_cls=0.20816632212941036, loss_rpn_reg=0.3524570853211159, loss_bbox_cls=0.19791206474914108, loss_bbox_reg=0.3229728959674059, loss=1.081508366174476 .
2024-06-18 11:53:52 [INFO] [TRAIN] Epoch=3/10, Step=2/43, loss_rpn_cls=0.098014, loss_rpn_reg=0.304489, loss_bbox_cls=0.149317, loss_bbox_reg=0.220986, loss=0.772805, lr=0.005000, time_each_step=5.13s, eta=0:29:23
2024-06-18 11:54:13 [INFO] [TRAIN] Epoch=3/10, Step=6/43, loss_rpn_cls=0.219997, loss_rpn_reg=0.336605, loss_bbox_cls=0.279411, loss_bbox_reg=0.393751, loss=1.229764, lr=0.005000, time_each_step=5.42s, eta=0:30:42
2024-06-18 11:54:34 [INFO] [TRAIN] Epoch=3/10, Step=10/43, loss_rpn_cls=0.175381, loss_rpn_reg=0.463739, loss_bbox_cls=0.207165, loss_bbox_reg=0.233856, loss=1.080141, lr=0.005000, time_each_step=5.12s, eta=0:28:40
2024-06-18 11:54:55 [INFO] [TRAIN] Epoch=3/10, Step=14/43, loss_rpn_cls=0.111875, loss_rpn_reg=0.213160, loss_bbox_cls=0.212481, loss_bbox_reg=0.443002, loss=0.980518, lr=0.005000, time_each_step=5.31s, eta=0:29:22
2024-06-18 11:55:15 [INFO] [TRAIN] Epoch=3/10, Step=18/43, loss_rpn_cls=0.148508, loss_rpn_reg=0.426484, loss_bbox_cls=0.183255, loss_bbox_reg=0.365108, loss=1.123355, lr=0.005000, time_each_step=4.97s, eta=0:27:10
2024-06-18 11:55:37 [INFO] [TRAIN] Epoch=3/10, Step=22/43, loss_rpn_cls=0.185638, loss_rpn_reg=0.436173, loss_bbox_cls=0.241199, loss_bbox_reg=0.302381, loss=1.165392, lr=0.005000, time_each_step=5.46s, eta=0:29:28
2024-06-18 11:55:58 [INFO] [TRAIN] Epoch=3/10, Step=26/43, loss_rpn_cls=0.263669, loss_rpn_reg=0.462009, loss_bbox_cls=0.236246, loss_bbox_reg=0.434107, loss=1.396031, lr=0.005000, time_each_step=5.28s, eta=0:28:9
2024-06-18 11:56:20 [INFO] [TRAIN] Epoch=3/10, Step=30/43, loss_rpn_cls=0.169363, loss_rpn_reg=0.442410, loss_bbox_cls=0.305235, loss_bbox_reg=0.469653, loss=1.386661, lr=0.005000, time_each_step=5.49s, eta=0:28:54
2024-06-18 11:56:42 [INFO] [TRAIN] Epoch=3/10, Step=34/43, loss_rpn_cls=0.089245, loss_rpn_reg=0.214869, loss_bbox_cls=0.179644, loss_bbox_reg=0.377911, loss=0.861670, lr=0.005000, time_each_step=5.55s, eta=0:28:50
2024-06-18 11:57:04 [INFO] [TRAIN] Epoch=3/10, Step=38/43, loss_rpn_cls=0.064340, loss_rpn_reg=0.148108, loss_bbox_cls=0.098305, loss_bbox_reg=0.276777, loss=0.587531, lr=0.005000, time_each_step=5.4s, eta=0:27:43
2024-06-18 11:57:26 [INFO] [TRAIN] Epoch=3/10, Step=42/43, loss_rpn_cls=0.251964, loss_rpn_reg=0.286586, loss_bbox_cls=0.159892, loss_bbox_reg=0.234308, loss=0.932750, lr=0.005000, time_each_step=5.51s, eta=0:27:54
2024-06-18 11:57:30 [INFO] [TRAIN] Epoch 3 finished, loss_rpn_cls=0.15356651624274809, loss_rpn_reg=0.3208635182574738, loss_bbox_cls=0.19191310381473498, loss_bbox_reg=0.32796219753664596, loss=0.9943053459012231 .
2024-06-18 11:57:46 [INFO] [TRAIN] Epoch=4/10, Step=3/43, loss_rpn_cls=0.068906, loss_rpn_reg=0.271396, loss_bbox_cls=0.146419, loss_bbox_reg=0.230301, loss=0.717021, lr=0.005000, time_each_step=5.17s, eta=0:25:50
2024-06-18 11:58:07 [INFO] [TRAIN] Epoch=4/10, Step=7/43, loss_rpn_cls=0.153264, loss_rpn_reg=0.266419, loss_bbox_cls=0.239140, loss_bbox_reg=0.458539, loss=1.117362, lr=0.005000, time_each_step=5.24s, eta=0:25:51
2024-06-18 11:58:30 [INFO] [TRAIN] Epoch=4/10, Step=11/43, loss_rpn_cls=0.098484, loss_rpn_reg=0.313728, loss_bbox_cls=0.115039, loss_bbox_reg=0.129856, loss=0.657107, lr=0.005000, time_each_step=5.58s, eta=0:27:8
2024-06-18 11:58:51 [INFO] [TRAIN] Epoch=4/10, Step=15/43, loss_rpn_cls=0.114065, loss_rpn_reg=0.279468, loss_bbox_cls=0.148531, loss_bbox_reg=0.252175, loss=0.794240, lr=0.005000, time_each_step=5.39s, eta=0:25:51
2024-06-18 11:59:12 [INFO] [TRAIN] Epoch=4/10, Step=19/43, loss_rpn_cls=0.115953, loss_rpn_reg=0.359139, loss_bbox_cls=0.218196, loss_bbox_reg=0.313900, loss=1.007188, lr=0.005000, time_each_step=5.12s, eta=0:24:12
2024-06-18 11:59:33 [INFO] [TRAIN] Epoch=4/10, Step=23/43, loss_rpn_cls=0.180564, loss_rpn_reg=0.339634, loss_bbox_cls=0.190908, loss_bbox_reg=0.353038, loss=1.064144, lr=0.005000, time_each_step=5.28s, eta=0:24:38
2024-06-18 11:59:54 [INFO] [TRAIN] Epoch=4/10, Step=27/43, loss_rpn_cls=0.063062, loss_rpn_reg=0.131187, loss_bbox_cls=0.148998, loss_bbox_reg=0.401446, loss=0.744693, lr=0.005000, time_each_step=5.31s, eta=0:24:25
2024-06-18 12:00:14 [INFO] [TRAIN] Epoch=4/10, Step=31/43, loss_rpn_cls=0.094141, loss_rpn_reg=0.355829, loss_bbox_cls=0.185503, loss_bbox_reg=0.318990, loss=0.954462, lr=0.005000, time_each_step=5.12s, eta=0:23:11
2024-06-18 12:00:36 [INFO] [TRAIN] Epoch=4/10, Step=35/43, loss_rpn_cls=0.138795, loss_rpn_reg=0.408093, loss_bbox_cls=0.210600, loss_bbox_reg=0.330653, loss=1.088141, lr=0.005000, time_each_step=5.45s, eta=0:24:19
2024-06-18 12:00:58 [INFO] [TRAIN] Epoch=4/10, Step=39/43, loss_rpn_cls=0.188317, loss_rpn_reg=0.421094, loss_bbox_cls=0.326682, loss_bbox_reg=0.350248, loss=1.286341, lr=0.005000, time_each_step=5.53s, eta=0:24:18
2024-06-18 12:01:18 [INFO] [TRAIN] Epoch=4/10, Step=43/43, loss_rpn_cls=0.054785, loss_rpn_reg=0.379295, loss_bbox_cls=0.192715, loss_bbox_reg=0.356704, loss=0.983499, lr=0.005000, time_each_step=5.06s, eta=0:21:54
2024-06-18 12:01:18 [INFO] [TRAIN] Epoch 4 finished, loss_rpn_cls=0.12866939093137897, loss_rpn_reg=0.3106158853963364, loss_bbox_cls=0.1751946992305822, loss_bbox_reg=0.3062816501356835, loss=0.9207616379094679 .
2024-06-18 12:01:41 [INFO] [TRAIN] Epoch=5/10, Step=4/43, loss_rpn_cls=0.108889, loss_rpn_reg=0.425793, loss_bbox_cls=0.246288, loss_bbox_reg=0.444199, loss=1.225168, lr=0.005000, time_each_step=5.51s, eta=0:23:31
2024-06-18 12:02:02 [INFO] [TRAIN] Epoch=5/10, Step=8/43, loss_rpn_cls=0.066840, loss_rpn_reg=0.331812, loss_bbox_cls=0.164916, loss_bbox_reg=0.324023, loss=0.887591, lr=0.005000, time_each_step=5.41s, eta=0:22:43
2024-06-18 12:02:23 [INFO] [TRAIN] Epoch=5/10, Step=12/43, loss_rpn_cls=0.073815, loss_rpn_reg=0.238272, loss_bbox_cls=0.162799, loss_bbox_reg=0.366960, loss=0.841847, lr=0.005000, time_each_step=5.24s, eta=0:21:39
2024-06-18 12:02:44 [INFO] [TRAIN] Epoch=5/10, Step=16/43, loss_rpn_cls=0.088353, loss_rpn_reg=0.301441, loss_bbox_cls=0.213698, loss_bbox_reg=0.474088, loss=1.077579, lr=0.005000, time_each_step=5.28s, eta=0:21:28
2024-06-18 12:03:07 [INFO] [TRAIN] Epoch=5/10, Step=20/43, loss_rpn_cls=0.084279, loss_rpn_reg=0.213426, loss_bbox_cls=0.113013, loss_bbox_reg=0.210423, loss=0.621141, lr=0.005000, time_each_step=5.6s, eta=0:22:25
2024-06-18 12:03:28 [INFO] [TRAIN] Epoch=5/10, Step=24/43, loss_rpn_cls=0.076019, loss_rpn_reg=0.277422, loss_bbox_cls=0.136279, loss_bbox_reg=0.218154, loss=0.707873, lr=0.005000, time_each_step=5.44s, eta=0:21:24
2024-06-18 12:03:50 [INFO] [TRAIN] Epoch=5/10, Step=28/43, loss_rpn_cls=0.163400, loss_rpn_reg=0.292277, loss_bbox_cls=0.193181, loss_bbox_reg=0.260862, loss=0.909721, lr=0.005000, time_each_step=5.4s, eta=0:20:52
2024-06-18 12:04:11 [INFO] [TRAIN] Epoch=5/10, Step=32/43, loss_rpn_cls=0.077981, loss_rpn_reg=0.275515, loss_bbox_cls=0.157151, loss_bbox_reg=0.345409, loss=0.856055, lr=0.005000, time_each_step=5.25s, eta=0:19:56
2024-06-18 12:04:31 [INFO] [TRAIN] Epoch=5/10, Step=36/43, loss_rpn_cls=0.110221, loss_rpn_reg=0.269887, loss_bbox_cls=0.224974, loss_bbox_reg=0.412423, loss=1.017505, lr=0.005000, time_each_step=5.09s, eta=0:19:0
2024-06-18 12:04:53 [INFO] [TRAIN] Epoch=5/10, Step=40/43, loss_rpn_cls=0.105802, loss_rpn_reg=0.306461, loss_bbox_cls=0.154323, loss_bbox_reg=0.272000, loss=0.838586, lr=0.005000, time_each_step=5.36s, eta=0:19:38
2024-06-18 12:05:08 [INFO] [TRAIN] Epoch 5 finished, loss_rpn_cls=0.1140497207295063, loss_rpn_reg=0.28902111185151474, loss_bbox_cls=0.1781560981342959, loss_bbox_reg=0.31857951678508933, loss=0.899806446807329 .
2024-06-18 12:05:08 [WARNING] Detector only supports single card evaluation with batch_size=1 during evaluation, so batch_size is forcibly set to 1.
2024-06-18 12:05:09 [INFO] Start to evaluate (total_samples=5, total_steps=5)...
Traceback (most recent call last):
File "tutorials/train/object_detection/faster_rcnn.py", line 75, in
model.train(
File "E:\cwudis\software\anaconda3\envs\py3810\lib\site-packages\paddlers\tasks\object_detector.py", line 350, in train
return self._real_train(**args)
File "E:\cwudis\software\anaconda3\envs\py3810\lib\site-packages\paddlers\tasks\object_detector.py", line 454, in _real_train
self.train_loop(
File "E:\cwudis\software\anaconda3\envs\py3810\lib\site-packages\paddlers\tasks\base.py", line 449, in train_loop
eval_result = self.evaluate(
File "E:\cwudis\software\anaconda3\envs\py3810\lib\site-packages\paddlers\tasks\object_detector.py", line 687, in evaluate
outputs = self.run(self.net, data, 'eval')
File "E:\cwudis\software\anaconda3\envs\py3810\lib\site-packages\paddlers\tasks\object_detector.py", line 157, in run
File "E:\cwudis\software\anaconda3\envs\py3810\lib\site-packages\paddle\nn\layer\layers.py", line 1429, in call
return self.forward(*inputs, **kwargs)
File "E:\cwudis\software\anaconda3\envs\py3810\lib\site-packages\paddlers\models\ppdet\modeling\architectures\meta_arch.py", line 76, in forward
outs.append(self.get_pred())
File "E:\cwudis\software\anaconda3\envs\py3810\lib\site-packages\paddlers\models\ppdet\modeling\architectures\faster_rcnn.py", line 125, in get_pred
bbox_pred, bbox_num = self._forward()
File "E:\cwudis\software\anaconda3\envs\py3810\lib\site-packages\paddlers\models\ppdet\modeling\architectures\faster_rcnn.py", line 93, in _forward
bboxes, bbox_pred, bbox_num = self.bbox_post_process.get_pred(
File "E:\cwudis\software\anaconda3\envs\py3810\lib\site-packages\paddlers\models\ppdet\modeling\post_process.py", line 134, in get_pred
bbox_num = paddle.concat(bbox_num_list)
File "E:\cwudis\software\anaconda3\envs\py3810\lib\site-packages\paddle\tensor\manipulation.py", line 1263, in concat
return _C_ops.concat(input, axis)
ValueError: (InvalidArgument) The axis is expected to be in range of [0, 0), but got 0
[Hint: Expected axis >= -rank && axis < rank == true, but received axis >= -rank && axis < rank:0 != true:1.] (at ..\paddle\phi\infermeta\multiary.cc:1035)


Traceback (most recent call last):
File "tutorials/train/object_detection/faster_rcnn.py", line 75, in
model.train(
File "E:\cwudis\software\anaconda3\envs\py3810\lib\site-packages\paddlers\tasks\object_detector.py", line 350, in train
return self._real_train(**args)
File "E:\cwudis\software\anaconda3\envs\py3810\lib\site-packages\paddlers\tasks\object_detector.py", line 454, in _real_train
self.train_loop(
File "E:\cwudis\software\anaconda3\envs\py3810\lib\site-packages\paddlers\tasks\base.py", line 449, in train_loop
eval_result = self.evaluate(
File "E:\cwudis\software\anaconda3\envs\py3810\lib\site-packages\paddlers\tasks\object_detector.py", line 687, in evaluate
outputs = self.run(self.net, data, 'eval')
File "E:\cwudis\software\anaconda3\envs\py3810\lib\site-packages\paddlers\tasks\object_detector.py", line 157, in run
net_out = net(inputs)
File "E:\cwudis\software\anaconda3\envs\py3810\lib\site-packages\paddle\nn\layer\layers.py", line 1429, in call
return self.forward(*inputs, **kwargs)
File "E:\cwudis\software\anaconda3\envs\py3810\lib\site-packages\paddlers\models\ppdet\modeling\architectures\meta_arch.py", line 76, in forward
outs.append(self.get_pred())
File "E:\cwudis\software\anaconda3\envs\py3810\lib\site-packages\paddlers\models\ppdet\modeling\architectures\faster_rcnn.py", line 125, in get_pred
bbox_pred, bbox_num = self._forward()
File "E:\cwudis\software\anaconda3\envs\py3810\lib\site-packages\paddlers\models\ppdet\modeling\architectures\faster_rcnn.py", line 93, in _forward
bboxes, bbox_pred, bbox_num = self.bbox_post_process.get_pred(
File "E:\cwudis\software\anaconda3\envs\py3810\lib\site-packages\paddlers\models\ppdet\modeling\post_process.py", line 150, in get_pred
scale = paddle.concat([scale_x, scale_y, scale_x, scale_y])
File "E:\cwudis\software\anaconda3\envs\py3810\lib\site-packages\paddle\tensor\manipulation.py", line 1263, in concat
return _C_ops.concat(input, axis)
ValueError: (InvalidArgument) The axis is expected to be in range of [0, 0), but got 0
[Hint: Expected axis >= -rank && axis < rank == true, but received axis >= -rank && axis < rank:0 != true:1.] (at ..\paddle\phi\infermeta\multiary.cc:1035)

@Bobholamovic
Copy link
Member

类似这样的都是同一种错误,可以尝试根据报错信息,在使用concat的位置按照上面的方式修改~

@cwudis
Copy link
Author

cwudis commented Jun 18, 2024

好的,谢谢~

@cwudis cwudis closed this as completed Jun 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants