Add mean IOU op. #10519

wanghaoshuang · 2018-05-09T08:30:40Z

Performance on GPU P40:

input size	class_num	GPU	CPU
1024 * 2048	100	0.168812ms	13.0831ms
1024 * 2048	50	0.172748ms	13.5145ms
1024 * 2048	20	0.174807ms	14.4619ms
1024 * 2048	10	0.188483ms	16.1516ms
1024 * 2048	1	0.230743ms	12.7893ms
1024 * 2048 *2	100	0.308306ms	26.4576
1024 * 2048 * 2	50	0.326073ms	26.9835ms
1024 * 2048 *2	20	0.28971ms	29.0224ms
1024 * 2048* 2	10	0.267694ms	34.2029ms
1024 * 2048 * 2	1	0.295844ms	25.4808ms

test code:

import paddle.fluid as fluid
import numpy as np
from paddle.fluid import core

num_classes=1
input_size=1024 * 2048 * 2

images = fluid.layers.data(name='image', shape=[1], dtype='int32')
label = fluid.layers.data(name='label', shape=[1], dtype='int32')
fluid.layers.mean_iou(images, label, num_classes)

place = fluid.CUDAPlace(0)
exe = fluid.Executor(place)
exe.run(fluid.default_startup_program())

image_t = core.LoDTensor()
label_t = core.LoDTensor()
image_t.set(np.random.randint(num_classes, size=[input_size]).astype("int32"), place)
label_t.set(np.random.randint(num_classes, size=[input_size]).astype("int32"), place)

with fluid.profiler.profiler('GPU', sorted_key='total'):
    for i in range(500):
        exe.run(feed={"image": image_t, "label":label_t})

wanghaoshuang · 2018-05-10T10:54:05Z

请先不要review，我再优化下GPU kernel.

1. Merge computing in GPU to two kernel. 2. Use wrong array and correct array instead of confusion matrix.

qingqing01 · 2018-05-16T12:34:44Z

paddle/fluid/operators/mean_iou_op.cc

+              "A Tensor representing the"
+              " mean intersection-over-union.");
+    AddOutput("out_wrong", "A Tensor with shape [num_classes]. ");
+    AddOutput("out_correct", "A Tensor with shape [num_classes]. ");


Please follow https://github.com/PaddlePaddle/Paddle/blob/develop/doc/fluid/dev/name_convention.md#opprotomaker-names for all the inputs and outputs name.

Thx. FIxed.

qingqing01 · 2018-05-16T12:36:56Z

paddle/fluid/operators/mean_iou_op.cc

+    AddComment(R"DOC(
+mean-IOU Operator.
+Mean Intersection-Over-Union is a common evaluation metric for semantic image segmentation, which first computes the IOU for each semantic class and then computes the average over classes. IOU is defined as follows: IOU = true_positive / (true_positive + false_positive + false_negative). The predictions are accumulated in a confusion matrix and mean-IOU is then calculated from it.
+


Since we have iou_similarity_op:

https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/fluid/operators/detection/iou_similarity_op.cc#L71

The doc here better to give more details for the difference.

Thx. FIxed.

qingqing01 · 2018-05-16T12:40:20Z

paddle/fluid/operators/mean_iou_op.cc

+        .AsDispensable();
+    AddOutput("out_mean_iou",
+              "A Tensor representing the"
+              " mean intersection-over-union.");


Also need to give the shape.

Thx. FIxed.

qingqing01 · 2018-05-16T12:41:55Z

paddle/fluid/operators/mean_iou_op.cc

+REGISTER_OPERATOR(mean_iou, ops::MeanIoUOp, ops::MeanIoUOpMaker,
+                  paddle::framework::EmptyGradOpMaker);
+REGISTER_OP_CPU_KERNEL(mean_iou, ops::MeanIoUKernel<int>,
+                       ops::MeanIoUKernel<int64_t>);


上面文档描述里是支持int32和int64，这里没有注册int32。

Thx. FIxed.

qingqing01 · 2018-05-16T12:43:55Z

paddle/fluid/operators/mean_iou_op.cu

+
+namespace ops = paddle::operators;
+REGISTER_OP_CUDA_KERNEL(mean_iou, ops::MeanIoUCUDAOpKernel<int>,
+                        ops::MeanIoUKernel<int64_t>);


Thx. FIxed.

qingqing01 · 2018-05-16T12:45:16Z

paddle/fluid/operators/mean_iou_op.cc

+      : OpProtoAndCheckerMaker(proto, op_checker) {
+    AddInput("predictions",
+             "A Tensor of prediction results for semantic labels"
+             " with type int32 or int64.");


Need to give the shape

Thx. FIxed.

qingqing01 · 2018-05-16T12:48:06Z

paddle/fluid/operators/mean_iou_op.cu

+    float* out_mean_iou_data =
+        out_mean_iou->mutable_data<float>(ctx.GetPlace());
+
+    // get eigen tensor


eigen -> Eigen

Thx. FIxed.

qingqing01 · 2018-05-16T12:48:15Z

paddle/fluid/operators/mean_iou_op.cu

+    auto out_wrong_t = EigenTensor<int, 1>::From(*out_wrong);
+    auto out_correct_t = EigenTensor<int, 1>::From(*out_correct);
+
+    // Tmp tensor


Dot not use Tmp

Thx. FIxed.

qingqing01 · 2018-05-16T12:53:57Z

paddle/fluid/operators/mean_iou_op.cc

+        .AsDispensable();
+    AddInput("in_mean_iou",
+             "A list of Tensor that Output(mean_iou) should "
+             "be added to. Empty list is also valid here.")


in_wrongs, in_corrects, in_mean_iou是干啥的？和out_wrong/correct/mean_iou有啥区别？

in_wrongs, in_corrects, in_mean_iou之前当前batch之前累计的数据，加上当前batch的统计结果，就得到：out_wrong/correct/mean_iou

qingqing01 · 2018-05-16T13:09:41Z

paddle/fluid/operators/mean_iou_op.cu

+  for (int i = threadIdx.x; i < num_classes; i += blockDim.x) {
+    atomicAdd(wrong + i, wrong_c[i]);
+    atomicAdd(correct + i, correct_c[i]);
+  }


如果num_classes较小， predictions的shape较大，会导致这个kernel的性能非常低效，其实感觉类似这样的kernel，先CPU即可，后续最好评估下时间。

input size class_num GPU CPU

1024 * 2048 100 0.168812ms 13.0831ms

1024 * 2048 50 0.172748ms 13.5145ms

1024 * 2048 20 0.174807ms 14.4619ms

1024 * 2048 10 0.188483ms 16.1516ms

1024 * 2048 1 0.230743ms 12.7893ms

1024 * 2048 *2 100 0.308306ms 26.4576

1024 * 2048 * 2 50 0.326073ms 26.9835ms

1024 * 2048 *2 20 0.28971ms 29.0224ms

1024 * 2048* 2 10 0.267694ms 34.2029ms

1024 * 2048 * 2 1 0.295844ms 25.4808ms

… mean_iou

qingqing01 · 2018-06-12T12:47:25Z

python/paddle/fluid/layers/nn.py

+    'softmax_with_cross_entropy', 'smooth_l1', 'one_hot',
+    'autoincreased_step_counter', 'reshape', 'lod_reset', 'lrn', 'pad',
+    'label_smooth', 'roi_pool', 'dice_loss', 'image_resize',
+    'image_resize_short', 'resize_bilinear', 'gather', 'random_crop', 'mean_iou'


This is due to he yapf version?

The version of my yapf is 0.22. I'm not sure it is due to he yapf version. Which style is correct?

wanghaoshuang added 3 commits May 9, 2018 09:40

Add mean_iou op.

7732d89

Add unitest for mean iou op.

3bb246c

Add optional collections of confusion matrix and mean_iou.

f161d69

wanghaoshuang requested a review from qingqing01 May 9, 2018 08:31

Fix cuda kernel.

5855926

Refine code.

c7eb88b

1. Merge computing in GPU to two kernel. 2. Use wrong array and correct array instead of confusion matrix.

qingqing01 reviewed May 16, 2018

View reviewed changes

wanghaoshuang added 5 commits June 11, 2018 02:40

Add python api and fix cuda kernel.

13e7b5a

Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

70ae3d2

… mean_iou

Fix comments.

ea505a4

Small fix.

e706336

Small fix.

8c8d004

qingqing01 reviewed Jun 12, 2018

View reviewed changes

qingqing01 approved these changes Jun 14, 2018

View reviewed changes

wanghaoshuang merged commit 6fcdb24 into PaddlePaddle:develop Jun 14, 2018

wanghaoshuang deleted the mean_iou branch May 20, 2022 03:59

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add mean IOU op. #10519

Add mean IOU op. #10519

wanghaoshuang commented May 9, 2018 •

edited

Loading

wanghaoshuang commented May 10, 2018

qingqing01 May 16, 2018

wanghaoshuang Jun 12, 2018

qingqing01 May 16, 2018

wanghaoshuang Jun 12, 2018

qingqing01 May 16, 2018

wanghaoshuang Jun 12, 2018

qingqing01 May 16, 2018

wanghaoshuang Jun 12, 2018

qingqing01 May 16, 2018

wanghaoshuang Jun 12, 2018

qingqing01 May 16, 2018

wanghaoshuang Jun 12, 2018

qingqing01 May 16, 2018

wanghaoshuang Jun 12, 2018

qingqing01 May 16, 2018

wanghaoshuang Jun 12, 2018

qingqing01 May 16, 2018

wanghaoshuang Jun 12, 2018

qingqing01 May 16, 2018

wanghaoshuang Jun 12, 2018 •

edited

Loading

qingqing01 Jun 12, 2018

wanghaoshuang Jun 13, 2018 •

edited

Loading

Add mean IOU op. #10519

Add mean IOU op. #10519

Conversation

wanghaoshuang commented May 9, 2018 • edited Loading

wanghaoshuang commented May 10, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

wanghaoshuang Jun 12, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

wanghaoshuang Jun 13, 2018 • edited Loading

Choose a reason for hiding this comment

wanghaoshuang commented May 9, 2018 •

edited

Loading

wanghaoshuang Jun 12, 2018 •

edited

Loading

wanghaoshuang Jun 13, 2018 •

edited

Loading